A method for generating a pathology image segmentation enhancement based on diversity controlled guided generation

Abstract

The application discloses a kind of based on diversity controlled guidance generation pathological image segmentation enhancement method, it is related to pathological image analysis technical field, the method includes: to pathological image block is carried out multi-scale self-supervised style clustering;In each style cluster, construct semantic prototype feature according to cell category;With nucleolus centroid map and reference image feature after prototype feature dynamic adjustment as condition, by cross attention and residual dense module fusion, construct generation guide condition;Based on the condition drive diffusion model, end-to-end synchronous generation pathological image and corresponding instance level nucleolus label.The application constrains the pathological rationality and diversity range of generation sample by clustering and prototype mechanism, effectively avoids the generation of structure anomaly and invalid sample, simultaneously realizes the efficient collaborative generation of image and label, provides high-quality, diversified training data for pathological image segmentation model, significantly improves the precision and robustness of segmentation model.

Description

Technical Field

[0001] This invention relates to the field of pathological image analysis technology, and in particular to a method for segmenting and enhancing pathological images based on diversity-controlled guided generation. Background Technology

[0002] Pathological image analysis is the core support for medical pathological diagnosis. Pathological image segmentation enhancement technology improves the segmentation accuracy, robustness and automation of key structures such as cell nuclei, cytoplasm and stroma in pathological images through algorithm optimization, data processing or model improvement. Its core goal is to provide objective and quantitative structural analysis basis for disease diagnosis and prognosis assessment, and it has irreplaceable and important value in clinical diagnosis and medical research.

[0003] In the development of pathological image segmentation technology, initial segmentation relied on manual annotation by pathologists. While this met clinical accuracy requirements, it suffered from significant limitations, including being time-consuming, labor-intensive, heavily influenced by subjective experience, and producing inconsistent results. Subsequently, automatic segmentation algorithms based on single-scale features emerged, such as traditional thresholding and edge detection. However, these methods could not adapt to the complexity and diversity of pathological images, struggling to handle structural differences across different tissue types and staining styles, thus limiting their application scenarios. After 2017, Convolutional Neural Network (CNN) models, represented by U-Net, became mainstream. Their unique encoder-decoder structure combined with skip connections significantly improved the segmentation capability for small objects and remains the benchmark model for pathological image segmentation. To further overcome the bottlenecks in segmentation accuracy and robustness, the industry has gradually formed a collaborative framework of "data augmentation-model optimization." This framework optimizes image quality from a data perspective while incorporating strategies such as transfer learning and self-supervised learning to continuously drive the development of segmentation technology.

[0004] Pathological image generation and enhancement technology, as an important branch of data augmentation, is based on generative artificial intelligence. Its core principle is to generate image samples that conform to the distribution of real pathological data through "forward progressive noise addition and inverse learning denoising," thereby addressing the industry pain points of scarce pathological data and high annotation costs. Before the widespread application of diffusion models, Generative Adversarial Networks (GANs) were the mainstream technology for pathological image generation and enhancement. They could handle unpaired data and achieve effects such as virtual staining through style transfer, alleviating the data shortage problem to some extent. With the maturity of the diffusion model framework, its application in the field of medical imaging has gradually expanded. Initially, it mainly generated pathological samples through unconditional or simple conditional diffusion. Thanks to the stability of "forward noise addition and inverse denoising," it can generate images with higher fidelity than GANs, becoming a current research hotspot in pathological image generation and enhancement.

[0005] Currently, mainstream pathological image generation models are mainly divided into two categories: Generative Adversarial Networks (GANs) and diffusion models. GANs primarily consist of two core modules: a generator and a discriminator. The generator employs an encoder-decoder architecture, extracting input image features through convolutional layers and receiving a semantic mask of the pathological structure as conditional input, thereby achieving image generation with controllable structural layout. However, GANs suffer from inherent gradient instability during training, making it difficult for the generator to synthesize image content with sufficient pathological realism. Although the generated image may align with the semantic mask in its overall structure, significant distortions often occur in local details, such as missing nucleoli in cell nuclei, blurred nuclear membrane boundaries, and stroma textures that do not match the actual tissue morphology. These distortions do not meet the standard requirements for structural features in clinical pathological diagnosis, potentially causing segmentation models trained on generated data to learn incorrect visual patterns, thus weakening their segmentation accuracy in real-world scenarios.

[0006] Generative methods based on diffusion models abandon the adversarial training mechanism between the generator and discriminator in GANs, and adopt a progressive learning paradigm of forward noise addition and backward denoising, achieving high-quality image generation while ensuring training stability. The core structure of this type of method typically includes a denoising network based on U-Net or Transformer and a conditional coding module: the denoising network is responsible for progressively learning to recover the original signal from the noisy latent vector during training, while the conditional coding module extracts semantic features from real pathological images as guiding conditions for the generation process, so as to achieve structurally controllable and content-realistic pathological image synthesis.

[0007] Despite some progress in existing technologies, current pathological image generation and enhancement techniques still have significant shortcomings, making it difficult to meet the needs of clinical and research fields for high-quality training data.

[0008] 1. Lack of Pathological Rationality Constraints: The style extraction and generation process of the basic diffusion model fails to deeply integrate with pathological knowledge, resulting in generated images that are "stylistically compliant but structurally abnormal," such as disordered hepatocyte arrangement or blurred tumor gland boundaries. Such structural distortions do not meet the requirements of clinical data for structural realism and reliability, and cannot provide effective support for training models related to pathological diagnosis.

[0009] 2. Lack of diversity control mechanisms: While existing methods can enhance generated diversity through style sampling or conditional editing, the lack of explicit constraints on the range of pathological feature variations easily leads to abnormal samples that exceed clinically reasonable limits. These samples may contain artifacts such as rare staining abnormalities or atypical cell morphologies, which not only fail to provide effective training data for the segmentation model but may also interfere with the model's learning of representations of real pathological features, thus reducing the model's training effectiveness.

[0010] 3. Inefficient Co-generation of Images and Corresponding Labels: In traditional generative models, the co-generation of pathological images and instance-level cell nuclear labels has long faced efficiency bottlenecks. Existing methods either rely on pre-generated fine labels to synthesize corresponding images or require multiple independently trained models to complete the generation of images and labels step by step. Such multi-stage, multi-model processes not only lead to high training complexity but also generate significant time and computational costs, making it difficult to meet the needs of efficient acquisition of pathological image-label pairs in clinical and research scenarios.

[0011] In summary, the shortcomings of existing pathological image generation technologies in terms of pathological rationality, diversity control, and collaborative generation efficiency limit their application in the field of pathological image segmentation and enhancement. There is an urgent need for a technical solution that can balance the authenticity of generated samples, controllable diversity, and generation efficiency to address the current pain points in the industry. Summary of the Invention

[0012] To address these issues, this invention provides a pathological image segmentation enhancement method based on diversity-controlled guided generation. This method solves the problems in existing technologies, such as the lack of pathological rationality constraints leading to abnormal image structure, the lack of diversity control mechanisms resulting in invalid samples, and the low efficiency and high training complexity and computation time cost of co-generating pathological images with instance-level cell nuclear labels.

[0013] To address the aforementioned technical problems, embodiments of the present invention provide a method for segmenting and enhancing pathological images based on diversity-controlled guided generation, the method comprising the following steps:

[0014] S1. Data preprocessing: Obtain the original pathological image and the corresponding cell nuclear centroid map, crop the original pathological image into image blocks of a preset size, and use the cell nuclear centroid map and the original pathological image blocks as basic input data;

[0015] S2. Style Clustering Processing: Multi-scale self-supervised clustering is performed on the image patches using the style clustering module, specifically including:

[0016] S21. Construct a global field of view and a local field of view for each image patch, wherein the global field of view focuses on tissue structure features and the local field of view focuses on the microscopic morphological features of cell nuclei;

[0017] S22. A dual-encoder collaborative architecture is adopted to extract features. The global context features of the global vision are extracted by the teacher model encoder, and the features of the global vision and the local vision are fused by the student model encoder after bilinear interpolation and scale alignment.

[0018] S23. Optimize the feature learning consistency of dual encoders based on the DINO loss function, where the teacher model updates parameters using the exponential moving average algorithm and the student model updates parameters using the gradient descent method.

[0019] S24. Project the student model's output features onto a low-dimensional space, and automatically cluster them using a hierarchical density clustering algorithm. A set of style clusters, and a reference image is selected according to a preset strategy;

[0020] S3. Category Prototype Construction and Feature Adjustment: This involves implementing collaborative constraints on semantics and style through a diversity prototype module, specifically including:

[0021] S31. Split the image patches in each style cluster according to the preset cell category, and extract the region features of each category through the U-Net conditional encoder of the diffusion model;

[0022] S32. Calculate the mean feature value of each category in the corresponding style cluster to obtain the category prototype features;

[0023] S33. Extract category reference features from the reference image, calculate the cosine similarity between the category reference features and the corresponding category prototype features as the fusion weight, and dynamically adjust the reference features through linear interpolation to obtain the adjusted features;

[0024] S4. Feature Fusion and Generation Condition Construction: This module enables dual regulation of style and structure through a diversity control module, specifically including:

[0025] S41. Generate a binary category mask based on the cell category, and obtain a single-category sub-image through element-wise multiplication;

[0026] S42. Extract the image features of a single-category sub-image, use them as the query vector, and use the position features of the cell nucleus centroid image as the key vector and value vector. Perform cross-attention calculation to obtain attention features that fuse related information.

[0027] S43. Concatenate multi-class attention features, process attention features and location features separately through residual dense modules, and add the processing results element by element to obtain the final generation conditions;

[0028] S5. Diffusion Model Training and Image Generation: Based on the U-Net architecture, the diffusion model is trained using a multi-task loss function guided by the final generation conditions. The multi-task loss function includes mean squared error loss and KL divergence loss. After training, pathological images and corresponding instance-level cell nuclear labels that conform to pathological rationality are generated end-to-end synchronously through forward noise addition and backward noise reduction processes.

[0029] Preferably, the preset strategy in step S24 includes three types: strategy one directly selects the original image patch as the reference image; strategy two selects the reference image from the style cluster to which the image patch belongs; strategy three selects the reference image from different style clusters.

[0030] Preferably, the specific process of bilinear interpolation in step S22 is as follows: the scale of the local field of view features is adjusted by the bilinear interpolation operator so that it is consistent with the scale of the global field of view features.

[0031] Preferably, the formula for calculating the category prototype feature in step S32 is:

[0032] ,

[0033] in, For category Prototype features, Style clustering Includes categories The total number of image blocks, This is the feature extraction function of the conditional encoder. Style clustering Medium category The corresponding number Image blocks.

[0034] Preferably, the formula for calculating the cosine similarity in step S33 is:

[0035] ,

[0036] in, To integrate weights, For category Prototype features, Category in reference image Reference features.

[0037] Preferably, the formula for calculating cross-attention in step S42 is:

[0038] ,

[0039] in, Image features for single-class sub-images This refers to the location characteristics of the cell nucleus's centroid. For cross-attention calculation function, These are the output features of the attention mechanism.

[0040] Preferably, the overall loss calculation formula for the multi-task loss function in step S5 is as follows:

[0041] ,

[0042] in, , , These are the weight coefficients for pathological image generation, semantic segmentation map generation, and kernel distance label generation, respectively. , , These are the branch losses corresponding to the three generation tasks, and each branch loss is obtained by adding the mean squared error loss and the KL divergence loss.

[0043] Preferably, in step S1, the preset size is 256×256 pixels and the cropping step size is 192 pixels.

[0044] Preferably, the original pathological images are derived from a dataset of glandular tissue pathological images or a dataset of gastric tissue pathological images.

[0045] This invention also provides an electronic device, which includes a processor, a memory, and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the pathological image segmentation enhancement method based on diversity-controlled guided generation described above.

[0046] As can be seen from the above technical solutions, this invention application has the following beneficial effects:

[0047] (1) The generated samples possess both pathological rationality and controllable diversity: Through the style clustering module and the diversity prototype module, this invention transforms multi-scale pathological prior knowledge such as tissue structure and cell morphology into quantifiable feature constraints (cluster sets and category prototypes), clearly setting the boundaries of pathological feature variations that conform to clinical practice. During the generation process, the dynamic guidance of prototype features ensures that the structure of the synthesized image is realistic and reliable, while strictly limiting diversity within a reasonable pathological range, thus eliminating the generation of structural abnormalities and invalid samples from the source.

[0048] (2) Efficient collaborative generation of images and instance-level labels is achieved: This invention innovatively constructs an end-to-end generation framework based on cell nucleus centroid maps and reference images. Through the cross-attention mechanism and feature fusion in the diversity control module, a single diffusion model can simultaneously output high-quality pathological images and corresponding instance-level cell nucleus labels (such as semantic segmentation maps and nuclear distance maps) in a single forward pass. This completely changes the cumbersome process of traditional multi-stage, multi-model operations, greatly reduces model complexity and computational cost, and significantly improves data generation efficiency.

[0049] (3) Provides high-quality training data for downstream segmentation models, effectively improving their performance: Since the generated samples are structurally consistent with pathological realism and have controlled and meaningful diversity in style, they can provide large-scale, high-quality augmented training data for pathological image segmentation models. This helps the segmentation model learn more robust and generalizable feature representations, thereby directly and effectively improving its segmentation accuracy, robustness, and reliability in real clinical scenarios. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Referring to the drawings will make the features and advantages of the present invention clearer. The drawings are illustrative and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0051] Figure 1 This is a flowchart of a pathological image segmentation and enhancement method based on diversity-controlled guided generation provided by the present invention;

[0052] Figure 2 This is a schematic diagram of the three modules of the diversity-controlled diffusion model in this invention;

[0053] Figure 3 This is a schematic diagram of the category-based subgraph splitting process in this invention;

[0054] Figure 4 This is a schematic diagram of the training and inference process of the diversity-controlled diffusion model in this invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] Example 1 addresses the problems in existing technologies, such as the lack of pathological rationality constraints leading to abnormal image structure, the lack of diversity control mechanisms resulting in invalid samples, and the low efficiency and high training complexity and computational time cost of co-generating pathological images and instance-level cell nuclear labels. For example... Figure 1 As shown, this invention proposes a pathological image segmentation and enhancement method based on diversity-controlled guided generation, which includes:

[0057] S1. Data preprocessing: Obtain the original pathological image and the corresponding cell nuclear centroid map, crop the original pathological image into image blocks of a preset size, and use the cell nuclear centroid map and the original pathological image blocks as the basic input data;

[0058] S2. Style Clustering Processing: Multi-scale self-supervised clustering is performed on image patches using the style clustering module, specifically including:

[0059] S21. Construct a global field of view and a local field of view for each image patch. The global field of view focuses on tissue structure features, while the local field of view focuses on the microscopic morphological features of the cell nucleus.

[0060] S22. A dual-encoder collaborative architecture is adopted to extract features. The global context features of the global vision are extracted by the teacher model encoder, and the features of the global vision and the local vision are fused by the student model encoder after bilinear interpolation and scale alignment.

[0061] S23. Optimize the feature learning consistency of dual encoders based on the DINO loss function, where the teacher model updates parameters using the exponential moving average algorithm and the student model updates parameters using the gradient descent method.

[0062] S24. Project the student model's output features onto a low-dimensional space, and automatically cluster them using a hierarchical density clustering algorithm. A set of style clusters, and a reference image is selected according to a preset strategy;

[0063] S3. Category Prototype Construction and Feature Adjustment: This involves implementing collaborative constraints on semantics and style through a diversity prototype module, specifically including:

[0064] S31. Split the image patches in each style cluster according to the preset cell category, and extract the region features of each category through the U-Net conditional encoder of the diffusion model;

[0065] S32. Calculate the mean feature value of each category in the corresponding style cluster to obtain the category prototype features;

[0066] S33. Extract the category reference features of the reference image, calculate the cosine similarity between the category reference features and the corresponding category prototype features as the fusion weight, and dynamically adjust the reference features through linear interpolation to obtain the adjusted features;

[0067] S4. Feature Fusion and Generation Condition Construction: This module enables dual regulation of style and structure through a diversity control module, specifically including:

[0068] S41. Generate a binary category mask based on the cell category, and obtain a single-category sub-image through element-wise multiplication;

[0069] S42. Extract the image features of a single-category sub-image, use them as the query vector, and use the position features of the cell nucleus centroid image as the key vector and value vector. Perform cross-attention calculation to obtain attention features that fuse related information.

[0070] S43. Concatenate multi-class attention features, process attention features and location features separately through residual dense modules, and add the processing results element by element to obtain the final generation conditions;

[0071] S5. Diffusion Model Training and Image Generation: Based on the U-Net architecture, the diffusion model is trained using a multi-task loss function guided by the final generation conditions. The multi-task loss function includes mean squared error loss and KL divergence loss. After training, the pathological images and corresponding instance-level cell nuclear labels that conform to the pathological rationality are generated end-to-end synchronously through forward noise addition and backward noise reduction processes.

[0072] As can be seen from the above technical solution, this invention proposes a pathological image segmentation enhancement method based on diversity-controlled guided generation. Through multi-stage collaborative processing, it achieves full-process control from data preparation to the generation of high-quality image-label pairs. First, the method standardizes and crops the original pathological images and cell nuclear centroid maps, providing basic data units for subsequent processing and ensuring the standardization and consistency of data input. Next, through multi-scale self-supervised style clustering, it integrates features of the tissue's global structure (global view) and the cell nuclear micromorphology (local view), using a dual encoder and DINO loss for consistency learning, and automatically clusters to form style sets. This step unsupervisedly identifies and summarizes diverse image style patterns that conform to pathological reality from massive amounts of data, laying the foundation for controllable diversity generation. Then, within each style cluster, category prototype features are constructed according to cell type, and based on this, the features of the reference image are dynamically adjusted using cosine similarity. This step transforms abstract pathological knowledge (such as the typical morphology of specific cell types) into computable and constrained semantic feature vectors, achieving dual rationality and controllability of the generated content in terms of semantics and style, ensuring that diversity does not deviate from the clinically reasonable range. Furthermore, a diversity control module was designed. By generating single-class sub-graphs, performing cross-attention calculations (associating image features with spatial location features), and residual dense fusion, a final generation condition integrating precise semantics, spatial layout, and style information was constructed. This step achieved fine-grained structural control at the nucleus level and deep coupling with style information, providing strong guidance for generating images and labels with precise spatial correspondences. Finally, a diffusion model was trained and run based on this strong guidance condition. Multi-task loss was used for synchronous optimization, completing the end-to-end collaborative synthesis of pathological images and their corresponding instance-level nucleus labels (such as segmentation maps) in a single model. This step simplifies the traditional multi-stage, multi-model complex process into an efficient integrated generation paradigm. While ensuring the pathological authenticity and structural accuracy of the generated samples, it greatly improves data production efficiency and provides large-scale, high-quality training resources for downstream segmentation models.

[0073] This invention utilizes an integrated framework of "style clustering + prototype guidance + collaborative generation" to achieve end-to-end image-label generation that combines diversity and pathological validity, providing high-quality training data for downstream pathological image segmentation models. This implementation is highly reproducible, and the following details the specific hardware, software, datasets, and technical aspects.

[0074] (a) Hardware environment

[0075] A high-performance computing server equipped with at least two NVIDIA V100 GPUs, each with no less than 32GB of video memory, is used to meet the complex computational needs of large-scale pathological image data processing, model parameter training, and diffusion processes.

[0076] (ii) Software Environment

[0077] 1. Operating system compatibility with Linux / Unix systems;

[0078] 2. Programming language: Python 3.8;

[0079] 3. Deep learning framework: PyTorch 1.12.1;

[0080] 4. Dependencies: OpenSlide (for pathological image reading and cropping), Scikit-image (for image preprocessing and feature calculation), NumPy (for numerical calculation), SciPy (for clustering algorithms and similarity calculation), and Matplotlib (for result visualization).

[0081] (III) Dataset Selection

[0082] The original pathological images were obtained from public datasets, including approximately 41 glandular tissue pathological images from the CoNSeP dataset and approximately 59 gastric tissue pathological images from the GLySAC dataset. All images contained corresponding cell nucleus annotation information, which was used to generate cell nucleus centroid maps and subsequent model supervised training.

[0083] The pathological image segmentation and enhancement method based on diversity-controlled guided generation described in this invention is implemented sequentially according to the following steps, covering the entire process of data preprocessing, style clustering processing, category prototype construction and feature adjustment, feature fusion and generation condition construction, diffusion model training and image generation:

[0084] (a) S1: Data Preprocessing

[0085] 1. Data acquisition: Extract the original pathological images (resolution not less than 1024×1024 pixels) from the above dataset, and generate the centroid map of the cell nucleus corresponding to each original pathological image based on the cell nucleus annotation information corresponding to the image through the centroid calculation algorithm. The centroid position of each cell nucleus in the centroid map is marked in the form of coordinate points.

[0086] 2. Image cropping: A sliding window cropping strategy is adopted to crop the original pathological image into image blocks of a preset size, where the preset size is 256×256 pixels and the cropping step size is 192 pixels, to ensure that a certain overlap area is retained between adjacent image blocks and to avoid the key pathological structures being cut off.

[0087] 3. Input data determination: The original pathological image block after cropping (denoted as...) ) and the corresponding nuclear centroid diagram As the basic input data of the model, it provides original information support for the subsequent feature extraction and generation process.

[0088] (ii) S2: Style clustering processing (corresponding to) Figure 2 (Middle Style Clustering Module)

[0089] This step performs multi-scale self-supervised clustering on the input image patches using the style clustering module, achieving accurate style classification of pathological images and selection of reference images. This provides compliant style constraints for the subsequent generation process. The specific steps are as follows:

[0090] S21. Multi-view construction:

[0091] For each 256×256 pixel image patch in the training set This allows us to construct two complementary analytical perspectives:

[0092] Global perspective : Acquired through wide field-of-view cropping, focusing on the tissue structure features (such as glandular arrangement, stroma distribution, and other macroscopic structures) in image blocks;

[0093] Local field of view : Obtained through narrow field-of-view cropping, focusing on the microscopic morphological features of the cell nucleus (such as nucleolus size, nuclear membrane boundary, etc.).

[0094] S22. Dual encoder collaborative feature extraction:

[0095] A dual-encoder collaborative architecture consisting of a teacher model encoder and a student model encoder is adopted to extract multi-scale features and achieve fusion:

[0096] 1. Teacher Model Encoder : Specifically extracts global view The global context features are output. The calculation formula is: This feature is used to construct the basic representation of the overall style, where The feature extraction function for the teacher model is implemented using a convolutional neural network.

[0097] 2. Student Model Encoder Used to fuse global and local view features, it first employs a bilinear interpolation operator. The scale of local field-of-view features is adjusted to match the scale of global field-of-view features (the specific process of bilinear interpolation is: scaling the local field-of-view features using the bilinear interpolation operator to match the resolution of the global field-of-view features), and then input into the student model encoder for feature fusion, outputting the fused features. Calculation formula: ,in The original scale representing local visual features.

[0098] S23. Feature Learning Consistency Optimization:

[0099] The DINO loss function is used to optimize the feature learning consistency of the dual encoders, ensuring the consistency of representation between global features and fused features.

[0100] 1. DINO Loss Function Calculation: Calculate the probability distribution of the teacher model output. With the student model output probability distribution The cross-entropy and loss function formula is: .

[0101] 2. Model parameter update strategy:

[0102] Teacher model: Parameters are updated using the Exponential Moving Average (EMA) algorithm to ensure the stability of global feature representation. The parameter update formula is: ,in For the encoder parameters of the teacher model, For the student model encoder parameters, The momentum coefficient (taken as 0.999 in this embodiment);

[0103] Student model: Parameters are updated using gradient descent to ensure real-time performance and flexibility in learning local features. The parameter update formula is as follows: ,in The learning rate (in this example, the value is 1e-4). This is the gradient of the loss function with respect to the student model parameters.

[0104] S24. Hierarchical density clustering and reference image selection:

[0105] 1. Feature Projection: The fused features output by the student model. Projecting onto a low-dimensional feature space (projecting to 256 dimensions in this embodiment) yields a low-dimensional feature vector set. ;

[0106] 2. Hierarchical Density Clustering: The Hierarchical Density Clustering algorithm (HDBSCAN) is used to cluster low-dimensional feature vector sets. This algorithm performs automatic clustering without requiring a pre-defined number of clusters. It automatically clusters features by measuring their density distribution in a low-dimensional space. A collection of style clusters Each style cluster The clustering process formula for image patches containing consistent styles is as follows: ;

[0107] 3. Reference Image Selection: Select reference images from the clustering results according to a preset strategy. The preset strategy selects a function based on style gradient. Implementation includes three specific types:

[0108] Strategy 1 ( ): Directly select the original image patch as the reference image. Preserve the consistency of the initial style;

[0109] Strategy Two ( Clustering by style of image patches Randomly select a reference image. To achieve fine-tuning of the style, including To randomly select operators;

[0110] Strategy 3 ( ): Randomly select reference images from different style clusters to maximize the span of style diversity.

[0111] Furthermore, a style gradient selection function is used. Select reference image The mathematical expression is:

[0112] ,

[0113] in, The final selected reference image; The original input image patch; For the original input image patch Corresponding clustering; This is a strategy pattern parameter, and its value determines the gradient of style diversity. The selection operator represents the random selection of an element from a set.

[0114] (III) S3: Category Prototype Construction and Feature Adjustment (corresponding to) Figure 2 Diversity Prototype Module and Figure 3 )

[0115] This step transforms pathological semantic knowledge into quantifiable feature constraints through a diversity prototype module, dynamically adjusts reference features, and ensures the structural rationality of the generated image. The specific steps are as follows:

[0116] S31. Category-aware subgraph splitting:

[0117] 1. Split the cells into different styles according to preset cell categories (4 categories in this embodiment: background 0, cell 1, cell 2, and cell 3). The image patches in the image are divided into subsets corresponding to different cell categories based on the semantic labels of the image patches;

[0118] 2. U-Net Conditional Encoder using a Diffusion Model Extracting region features from image patches in each category subset for style clustering Medium category The Image blocks The extracted regional features are denoted as .

[0119] S32. Calculation of Category Prototype Features:

[0120] Calculate the clustering of each category in the corresponding style The characteristic mean in Figure 3 Taking category 0 as an example, we obtain the category prototype features. This feature is used to characterize the typical features of this category in a specific style cluster, and the calculation formula is:

[0121] ,

[0122] in, Style clustering Contains categories The total number of image blocks, This is the feature extraction function of the conditional encoder. Style clustering Medium category The corresponding number Image blocks.

[0123] S33. Feature Similarity Calculation and Dynamic Fusion:

[0124] 1. Reference Feature Extraction: Extract the reference image selected in step S24. Medium category Reference features Extraction method is ,in Category in reference image The corresponding area;

[0125] 2. Cosine similarity calculation: Calculate the reference features Corresponding category prototype features The cosine similarity is used as the fusion weight. The calculation formula is:

[0126] ,

[0127] The numerator is the dot product of the two features, and the denominator is the product of the moduli of the two features.

[0128] 3. Linear interpolation adjustment: The reference features are dynamically adjusted using linear interpolation to obtain the adjusted features. This makes the style of the generated content closer to the characteristics of the category prototype. The calculation formula is:

[0129] ,

[0130] Adjusted features This will serve as an important constraint for the subsequent generation process.

[0131] (iv) S4: Feature fusion and generation condition construction (corresponding to) Figure 2 Diversity Control Module

[0132] This step integrates cell nuclear centroid features with reference image features through a diversity control module to achieve dual regulation of style and structure, thus constructing the final generation conditions for the diffusion model. The specific steps are as follows:

[0133] S41. Single-category subgraph generation:

[0134] 1. Binary class mask generation: Based on preset cell classes For reference image Generate the corresponding binary category mask In the mask, the pixel value of the area corresponding to the category is 1, and the pixel value of the non-target area is 0.

[0135] 2. Element-by-element product operation: using the reference image With binary category mask Perform element-wise multiplication to obtain a product that contains only categories. Single-category subgraph of information Non-target areas are filled with black pixels with a value of 0. The calculation formula is as follows:

[0136] ,

[0137] in, This is the element-wise multiplication operator.

[0138] S42. Cross-attention calculation:

[0139] 1. Feature extraction: using the U-Net conditional encoder Extracting single-category sub-graphs Image features Simultaneously, extract the centroid image of the cell nucleus. Location features The extraction formulas are as follows:

[0140] , ;

[0141] 2. Cross-attention calculation: This involves calculating image features... As the query vector (Q), the location features of the cell nucleus centroid Simultaneously serving as both the key vector (K) and value vector (V), cross-attention computation is performed to obtain the output features of the attention mechanism that fuses related information. The calculation formula is:

[0142] ,

[0143] in, This is the function for calculating cross-attention. The standard definition of the cross-attention function is:

[0144] ,

[0145] in, The dimension of the key vector (in this embodiment) ), used to scale calculations to ensure stability.

[0146] S43. Feature stitching and residual density processing:

[0147] 1. Multi-class attention feature concatenation: Consolidates attention features of all preset cell categories. ( This represents the total number of cell types in this embodiment. (This is) concatenated into a unified attention feature. The calculation formula is:

[0148] ,

[0149] in, Indicates a splicing operation;

[0150] 2. Residual Dense Module Processing: Residual Dense Modules (RRDM) are introduced to process attention features separately. and location features This module performs deep processing by constructing feature transfer paths between global and local residuals, thus mitigating the loss of fine-grained information during deep feature extraction.

[0151] 3. Final generation conditions construction: The outputs of the two RRDM modules are summed element-wise to obtain the final generation conditions of the diffusion model. The calculation formula is:

[0152] .

[0153] (V) S5: Diffusion Model Training and Image Generation

[0154] A diffusion model based on the U-Net architecture, with final generation conditions To guide the training, a multi-task loss function is used for optimization, ultimately generating pathological images and corresponding instance-level cell nuclear labels end-to-end. The specific steps are as follows:

[0155] 1. Diffusion model structure configuration: such as Figure 4 As shown, the diffusion model uses U-Net as its core architecture, sets up skip connections between the encoder and decoder of U-Net, and finally generates the conditions. To mitigate feature information loss during upsampling, skip connections in each layer of U-Net are injected; three parallel convolutional branches are extended at the back end of U-Net, each used to output pathological images. Semantic segmentation graph and nuclear distance label .

[0156] 2. Definition of Multi-Task Loss Function: The multi-task loss function is used to optimize model training. The overall loss calculation formula is as follows:

[0157] ,

[0158] in, , , These are the weight coefficients for pathological image generation, semantic segmentation map generation, and kernel distance label generation, respectively, which are set to [values] in this embodiment. , , ; , , These are the branch losses corresponding to the three generation tasks, and each branch loss is obtained by adding the mean squared error (MSE) loss and the KL divergence loss:

[0159] MSE loss: used to calculate the difference between the predicted result and the true label at the pixel level, the formula is:

[0160] ,

[0161] in, The real noise generated by the noise dispatcher For the model at time step Based on conditions , Predicted noise, Noisy image;

[0162] KL divergence loss: Used to evaluate the difference between the predicted noise distribution and the actual noise distribution, the formula is:

[0163] ,

[0164] in, , The standard deviation and mean of the true noise distribution are used to measure the dispersion of the true data. , The standard deviation and mean of the predicted noise distribution are used to measure the instability of the model's predictions.

[0165] 3. Model training process:

[0166] Forward noise addition: adding noise to the input image patch According to the time step of the diffusion process (Values ​​range from 0 to 1000) Gaussian noise is added progressively to obtain a noisy image. ;

[0167] Inverse denoising: The model is based on the final generation conditions. and time step Learning from noisy images through the U-Net architecture The process of removing noise in the image outputs the denoised image and its corresponding label.

[0168] Parameter optimization: Calculate the multi-task loss function using the backpropagation algorithm. The model parameters are updated using the Adam optimizer based on the gradient of the model parameters. The training epochs are set to 1000 epochs and the batch size is set to 8.

[0169] 4. Image and Label Generation: After training, the model enters the inference phase:

[0170] Input conditions: Input a new nuclear centroid map into the model. and selected reference images ;

[0171] Forward denoising and backward denoising: The model automatically performs forward denoising (starting from clean noise) and backward denoising processes, adjusting the denoising based on the generation conditions. Adjusting the structure and style of images;

[0172] End-to-end output: Synchronously generate pathological images that conform to pathological rationality and corresponding instance-level cell nuclear labels, realizing a highly efficient generation paradigm of "one-time input, synchronous output".

[0173] In summary, this invention, through its innovative "clustering-prototype-control" modular design, introduces the constraints of pathological knowledge within the diffusion generation framework, enabling the collaborative generation of high-quality pathological images and labels with controlled diversity, thus providing an efficient and reliable data augmentation solution for pathological image analysis.

[0174] Example 2: This embodiment of the invention provides an electronic device including a processor, a memory, and a bus system. The processor and the memory are connected via the bus system. The memory stores instructions, and the processor executes the instructions stored in the memory to implement the aforementioned pathological image segmentation and enhancement method based on diversity-controlled guided generation. This electronic device can be a server, workstation, or personal computer, etc.

[0175] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0176] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0177] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0178] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for pathology image segmentation enhancement based on diversity controlled guided generation, characterized in that, The method comprises the following steps: S1. Data preprocessing: obtaining an original pathological image and a corresponding cell nucleus center map, and cutting the original pathological image into image blocks of a preset size, taking the cell nucleus center map and the original pathological image block as basic input data; S2. Style clustering processing: performing multi-scale self-supervised clustering on the image blocks through a style clustering module, specifically including: S21. Constructing a global view and a local view for each image block, the global view focusing on tissue structure features, and the local view focusing on cell nucleus microscopic morphological features; S22. Extracting features using a dual-encoder collaborative architecture, extracting global context features of the global view through a teacher model encoder, and after scale alignment through a bilinear interpolation, fusing features of the global view and the local view; S23. Optimizing feature learning consistency of the dual-encoder based on a DINO loss function, wherein the teacher model updates parameters through an exponential moving average algorithm, and the student model updates parameters through a gradient descent method; S24. Projecting the student model output features to a low-dimensional space, and automatically clustering into a set of style clusters using a hierarchical density clustering algorithm, and selecting reference images according to a preset strategy; S24. Projecting the student model output features to a low-dimensional space, and automatically clustering into a set of style clusters using a hierarchical density clustering algorithm, and selecting reference images according to a preset strategy; S3. Category prototype construction and feature adjustment: realizing semantic and style collaborative constraints through a diversity prototype module, specifically including: S31. Splitting the image blocks in each style cluster according to a preset cell category, and extracting regional features of each category through a U-Net conditional encoder of a diffusion model; S32. Calculating a feature mean of each category in the corresponding style cluster to obtain a category prototype feature; S33. Extracting a category reference feature of a reference image, calculating a cosine similarity between the category reference feature and a corresponding category prototype feature as a fusion weight, and dynamically adjusting the reference feature through linear interpolation to obtain an adjusted feature; S4. Feature fusion and generation condition construction: realizing dual regulation of style and structure through a diversity control module, specifically including: S41. Generating a binary category mask according to the cell category, and obtaining a single-category subgraph through element-wise multiplication operation; S42. Extracting image features of the single-category subgraph, taking the position features of the cell nucleus center map as key vectors and value vectors, and performing cross-attention calculation to obtain attention features with fusion correlation information; S43. Splicing multi-category attention features, processing the attention features and position features through a residual dense module respectively, and adding the processing results element by element to obtain a final generation condition; S5. Diffusion model training and image generation: based on a U-Net architecture diffusion model, using the final generation condition as a guide, and using a multi-task loss function to optimize model training, the multi-task loss function including mean square error loss and KL divergence loss; after training, a pathological image and a corresponding instance-level cell nucleus label that meet the rationality of pathology are generated through a forward noise adding and reverse noise removing process in an end-to-end manner.

2. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, The preset strategy in step S24 includes three strategies: strategy one directly selects the original image block as the reference image; strategy two selects the reference image from the style cluster to which the image block belongs; and strategy three selects the reference image from different style clusters.

3. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, The specific process of the bilinear interpolation in step S22 is to adjust the scale of the local view feature through a bilinear interpolation operator so that it is consistent with the scale of the global view feature.

4. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, The calculation formula of the category prototype feature in step S32 is: ， wherein, is a prototype feature of the class, is a total number of image patches in the style cluster containing the class, is a feature extraction function of the conditional encoder, is the i-th image patch in the style cluster corresponding to the class .​​​ 5. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, The calculation formula of the cosine similarity in step S33 is: ， wherein, is a fusion weight, is a class prototype feature, is a reference feature of a class in the reference image.

6. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, The calculation formula of the cross attention in step S42 is: ， wherein, is an image feature of a single class subgraph, is a location feature of a cell nucleus center, is a cross-attention computation function, is an output feature of an attention mechanism.

7. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, The overall loss calculation formula of the multi-task loss function in step S5 is: ， wherein, , , are the weight coefficients corresponding to pathological image generation, semantic segmentation map generation, and kernel distance label generation, respectively, , , are branch losses corresponding to the three generation tasks, each of which is obtained by adding a mean square error loss and a KL divergence loss.

8. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, In step S1, the preset size is 256*256 pixels, and the cutting step is 192 pixels.

9. The diversity controlled guidance based generated pathology image segmentation augmentation method of claim 1, wherein, The original pathological image is derived from an adenoid tissue pathological image dataset or a gastric tissue pathological image dataset.

10. An electronic device, comprising: The electronic device includes a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used for storing instructions, and the processor is used for executing the instructions stored in the memory to realize the pathological image segmentation enhancement method based on the diversity controlled guidance generation in any one of claims 1 to 9.

Images