Diffusion model label generation method combined with morphological features

Abstract

The application relates to a diffusion model label generation method combining morphological features, comprising the following steps: obtaining a training data set; performing morphological feature extraction on an instance graph to obtain a multi-channel morphological feature graph; splicing a multi-channel morphological feature graph, a cell category label, a binary mask and a distance graph in a channel dimension to obtain a multi-channel input tensor; training a diffusion model based on the multi-channel input tensor, a text condition and random noise; and inputting a fixed format text condition and random noise into the trained diffusion model to obtain a cell category label. In the diffusion model training stage, the morphological feature graph of the cell instance is jointly input into the diffusion model together with the category label, the distance graph and the binary mask, the joint guidance of multi-class information is realized, the model can sufficiently learn the correlation between various types of input information, and thus a pathological image label with higher authenticity and conforming to morphological constraints can be generated in the reasoning stage.

Description

Technical Field

[0001] This application relates to the field of computer vision, and more specifically, to a diffusion model label generation method based on joint morphological features. Background Technology

[0002] In the field of pathological image analysis, high-quality labeled datasets for pathological images are generally scarce. Core tasks such as cell nucleus segmentation and cell nucleus classification require a large amount of high-quality image-label pairing data as the foundation for model training. However, high-quality annotation relies on manual work by professional pathologists, which is time-consuming, labor-intensive, and extremely costly, making large-scale annotation difficult and thus limiting the training data for downstream pathological image analysis models. To address this annotation shortage, various label generation methods have been proposed, but existing methods still have many shortcomings, as detailed below: Firstly, random generation or splicing methods: These methods do not consider the biological distribution patterns of cells, resulting in a chaotic spatial arrangement of cells that differs greatly from the distribution of cells in real tissues, leading to extremely poor label authenticity. Although the generated labels and corresponding images can be used to supplement training data and alleviate the label shortage problem to some extent, the lack of real cell biological features makes their effect on improving downstream model training very limited and difficult to meet the needs of practical applications.

[0003] Secondly, traditional generative adversarial networks (GANs) have inherent defects such as unstable training and easy pattern collapse, which further leads to poor diversity of generated labels. In addition, the model has difficulty capturing the fine morphological structure of cells and cannot generate labels with clear boundaries and cell shapes that conform to real biological characteristics, thus limiting the generation effect.

[0004] Third, existing diffusion model methods: Compared with the first two methods, diffusion models have significant advantages in terms of stable training and better generation quality. Some studies have already applied them to the generation of pathological image labels, and this type of method can provide effective data support for the training of downstream cell nucleus classification and segmentation models (such as Hovernet). However, the core focus of existing diffusion models is not the optimization of label morphological quality. Although the morphological features of the generated labels have reached a certain level, there is still room for improvement. This is not because there is obvious morphological distortion, but because the detail accuracy can be further optimized.

[0005] In summary, among the existing label generation methods, random generation or splicing and traditional generative adversarial networks (GANs) have significant shortcomings. While the diffusion model is currently the best performing method, it can provide effective data support for downstream model training and its generation effect is significantly better than the other two methods. However, there is still room for improvement in the morphological quality of the generated labels. Further optimization can be carried out to better adapt to the needs of various downstream tasks such as pathological image segmentation and classification for high-quality paired data. Summary of the Invention

[0006] To overcome at least one deficiency in the prior art, this application provides a diffusion model label generation method based on joint morphological features.

[0007] Firstly, a diffusion model label generation method combining morphological features is provided, including: Obtain the training dataset. The samples in the training dataset include instance images, cell category labels, binary masks, and distance maps. The instance images include multiple cell instances, and the distance maps include the distance value of each pixel in the cell instance. The distance value is the distance between each pixel in the cell instance and the center pixel of the cell. Morphological features are extracted from the instance images to obtain multi-channel morphological feature maps; The multi-channel morphological feature map, cell category label, binary mask and distance map are concatenated along the channel dimension to obtain the multi-channel input tensor; The diffusion model is trained based on a multi-channel input tensor, text conditions, and random noise to obtain the trained diffusion model. During the training process, the diffusion model generates cell category labels based on the input multi-channel input tensor, text conditions, and random noise. By inputting fixed-format text conditions and random noise into the trained diffusion model, cell category labels are obtained.

[0008] In one embodiment, morphological feature extraction is performed on the instance image to obtain a multi-channel morphological feature map, including: Perform connected component analysis on the instance graph to identify each cell instance; Calculate the feature values ​​of multiple morphological features for each cell instance; Feature map filling is performed on the feature values ​​of each morphological feature to obtain multiple single-channel feature maps; All single-channel feature maps are merged to obtain a multi-channel morphological feature map.

[0009] In one embodiment, feature map padding is performed on the feature values ​​of each morphological feature to obtain multiple single-channel feature maps, including: Generate multiple blank single-channel images, each corresponding to a morphological feature; The feature value of a morphological feature of each cell instance is filled into all pixel positions of the corresponding cell instance in the blank single-channel image to obtain a single-channel feature map.

[0010] In one embodiment, the diffusion model employs the imagen basic architecture.

[0011] In one embodiment, the loss functions used during training include L2 loss and KL divergence loss.

[0012] Secondly, a diffusion model label generation device combining morphological features is provided, comprising: The training data acquisition module is used to acquire the training dataset. The samples in the training dataset include instance images, cell category labels, binary masks, and distance maps. The instance images include multiple cell instances, and the distance maps include the distance value of each pixel in the cell instance. The distance value is the distance between each pixel in the cell instance and the center pixel of the cell. The morphological feature extraction module is used to extract morphological features from the instance image to obtain a multi-channel morphological feature map. The stitching module is used to stitch together multi-channel morphological feature maps, cell category labels, binary masks, and distance maps along the channel dimension to obtain a multi-channel input tensor. The training module is used to train the diffusion model based on the multi-channel input tensor, text conditions, and random noise to obtain the trained diffusion model. During the training process, the diffusion model generates cell category labels based on the input multi-channel input tensor, text conditions, and random noise. The inference module is used to input fixed-format text conditions and random noise into the trained diffusion model to obtain cell category labels.

[0013] Compared to existing technologies, this application offers the following advantages: The diffusion model label generation method based on joint morphological features, during the diffusion model training phase, inputs the morphological feature map of cell instances, along with category labels, distance maps, and binary masks, into the diffusion model. This enables the model to learn the relationships between various input information types, thereby generating pathological image labels that conform to morphological constraints and have higher realism during the inference phase. By using joint morphological feature maps, this application achieves more accurate and clinically relevant pathological image label generation based on cell morphological features, further improving label quality. Attached Figure Description

[0014] This application can be better understood by referring to the description given below in conjunction with the accompanying drawings, which, together with the detailed description below, are incorporated in and form part of this specification. In the drawings: Figure 1A flowchart of a diffusion model label generation method based on combined morphological features is shown. Detailed Implementation

[0015] The exemplary embodiments of this application will be described below with reference to the accompanying drawings. For clarity and brevity, not all features of the actual embodiments are described in the specification. However, it should be understood that many embodiment-specific decisions can be made in the development of any such actual embodiment to achieve the developer's specific objectives, and these decisions may vary as the embodiments differ.

[0016] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the device structure closely related to the solution of this application is shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0017] It should be understood that this application is not limited to the described embodiments by virtue of the following description with reference to the accompanying drawings. In this document, embodiments may be combined with each other, features may be substituted or borrowed between different embodiments, and one or more features may be omitted in one embodiment, where feasible.

[0018] This application provides a diffusion model label generation method based on combined morphological features. Figure 1 A flowchart illustrating the diffusion model label generation method based on joint morphological features is shown. See [link / reference]. Figure 1 The method mainly includes the following steps: Step S1: Obtain the training dataset. The samples in the training dataset include instance images, cell category labels, binary masks, and distance maps. The instance images include multiple cell instances, and the distance maps include the distance value of each pixel in the cell instance. The distance value is the distance between each pixel in the cell instance and the center pixel of the cell.

[0019] Step S2: Extract morphological features from the instance image to obtain a multi-channel morphological feature map.

[0020] First, a connected component analysis is performed on the instance graph to identify each cell instance, ensuring that the boundaries of each cell instance are clear and non-overlapping, laying the foundation for subsequent morphological feature calculations.

[0021] Then, the feature values ​​of various morphological features for each cell instance are calculated. Here, morphological features can be flexibly selected according to actual needs, such as area, perimeter, eccentricity, and roundness. All features are calculated based on the geometric morphological characteristics of the cell instance, accurately reflecting the morphological differences of the cells.

[0022] Then, feature map filling is performed on the feature values ​​of each morphological feature to obtain multiple single-channel feature maps.

[0023] Specifically, prepare blank single-channel images with the same number of selected morphological feature types and the same size as the original segmentation labels. Each blank single-channel image corresponds to one selected morphological feature. Obtain the coordinates of pixels in the binary mask of each cell instance. Normalize the feature value of one morphological feature of each cell instance and fill it into all pixel positions in the corresponding cell instance in the blank single-channel image to obtain a single-channel feature map. Ensure that all pixels in the region of each cell instance are filled with the feature value of the corresponding instance. Finally, obtain multiple single-channel feature maps with the same number of selected morphological feature types.

[0024] Then, all single-channel feature maps are merged to obtain a multi-channel morphological feature map. Here, the feature values ​​of each single-channel feature map are linearly normalized and mapped to the [0,1] interval to eliminate the influence of differences in feature scales on subsequent model training. Then, all single-channel feature maps are merged to obtain a multi-channel morphological feature map.

[0025] Step S3 involves concatenating the multi-channel morphological feature map, cell category label, binary mask, and distance map along the channel dimension to obtain a multi-channel input tensor. Here, the multi-channel input tensor serves as the core input for model training, providing morphological and geometric constraint information.

[0026] Step S4: The diffusion model is trained based on the multi-channel input tensor, text conditions, and random noise to obtain the trained diffusion model. During training, the diffusion model generates cell category labels based on the input multi-channel input tensor, text conditions, and random noise. Here, the cell category label is still an image, which includes the category information of each cell instance. The text conditions adopt a fixed format, such as: "[Tissue Type]; [Nuclear Ratio]; [Cell Type]".

[0027] Here, the diffusion model adopts the basic architecture of imagen. Multi-channel input tensors, textual conditions, and random noise are input into the diffusion model to ensure that the model can simultaneously receive morphological, geometric, and textual constraint information in each cascade stage. This enables cascaded training guided by multiple types of information and ensures that constraint information is effectively transmitted in each training stage.

[0028] The model training objective remains unchanged, which is to predict the potential noise of the input. The image loss explicitly adopts L2 loss (which fits the core requirement of diffusion model denoising training and can better constrain the image generation accuracy), and is co-optimized with KL divergence loss corresponding to label generation. Through continuous iterative training, the model implicitly learns the rules of generating segmentation labels under given geometric information and morphological feature constraints, ensuring that the generated labels fit the morphological features of real cells.

[0029] Step S5: Input the fixed-format text conditions and random noise into the trained diffusion model to obtain cell category labels.

[0030] In the diffusion model training phase, this application inputs the morphological feature map of cell instances together with the category label, distance map, and binary mask into the diffusion model to achieve joint guidance of multiple types of information. This enables the model to fully learn the correlation between various types of input information, thereby generating pathological image labels that conform to morphological constraints and have higher realism in the inference phase.

[0031] In one embodiment, the Lizard dataset is selected. During preprocessing, the images and corresponding annotations in the dataset are segmented into 256×256 pixel patches, resulting in 12,279 training patches and 785 test patches. The number of channels for each type of image and annotation is as follows: pathological images have 3 channels, instance images and category labels each have 1 channel, and distance images have 2 channels.

[0032] When extracting morphological features from instance images, connected component analysis is performed on the instance images. The minimum connected component area threshold is set to 5 pixels, and noise regions with an area smaller than this threshold are removed to avoid misidentification of background noise and ensure that the boundaries of each cell instance are clear and do not overlap.

[0033] For each cell nucleus instance, three core morphological features are extracted: area, perimeter, and eccentricity. The area is the total number of pixels corresponding to the cell nucleus instance; the perimeter is the pixel length of the cell instance outline; and the eccentricity is calculated by fitting the ratio of the major and minor axes of an ellipse, with a value of 0-1, which accurately reflects the differences in cell morphology.

[0034] For the same type of morphological features, the maximum and minimum normalization method is used. First, the maximum and minimum values ​​are calculated, and then mapped to the [0,1] interval. The formula is: Normalized feature value = (original feature value - minimum value of the feature) / (maximum value of the feature - minimum value of the feature).

[0035] Prepare three 256×256 blank single-channel images, corresponding to area, perimeter, and eccentricity, respectively; obtain the pixel coordinates of each cell instance, fill the cell region of the corresponding blank image with the normalized corresponding feature value, keep the background region at 0, and finally merge the three single-channel images to obtain a three-channel morphological feature map.

[0036] To further verify the effectiveness of the method in this application, a comparative analysis is conducted between the method in this application and existing methods.

[0037] 1. Comparison Model Settings ① Baseline model: Construct a baseline model that is completely identical to the model structure, training parameters, and cascade training method of this application. The only difference between the two is that the baseline model does not input the three-channel morphological feature map added in this application. All other inputs (category labels, distance map, binary mask, fixed format text) and model configuration are the same as the model of this application. ② Other comparison models: In addition to the baseline model, existing mainstream pathological image label generation models were selected as comparison objects to ensure the comprehensiveness of the comparison experiment and fully verify the advantages of the method in this application.

[0038] 2. Label quality assessment The Feature Distribution Distance (FSD) was adopted as the core evaluation index for label quality. Its core function is to measure the difference in morphological feature distribution between the generated label and the corresponding real label. The lower the value, the closer the feature distributions are, and the higher the morphological realism of the label. The evaluation covered the model of this application, the baseline model, and other comparative models. As shown in Table 1, the FSD value of the labels generated by the model of this application is significantly lower than that of the baseline model and other comparative models, intuitively verifying the advantage of this application in improving the morphological realism of labels.

[0039] Table 1

[0040] 3. Image quality assessment Image quality was evaluated using the FID (Fixed Identification Value) metric. This metric measures the overall similarity and visual consistency between the real and generated pathological images by calculating the mean and covariance matrix distance of their feature vectors. A lower FID value indicates that the generated image is closer to the real image and of higher quality. The specific evaluation logic is as follows: Segmentation labels were generated using the combined morphological feature model and the baseline model, respectively. These two types of generated labels were then input into the same label-conditional image generation method to generate corresponding pathological images. The quality differences between the two types of generated images were then compared. The FID value of the generated image corresponding to the model in this application was significantly better than that of the generated image corresponding to the baseline model, further verifying the high-quality characteristics of the generated labels in this application. Relevant comparative data are shown in Table 2.

[0041] Table 2

[0042] 4. Downstream task assessment The experiment consisted of three comparison groups: the first group was the original control group, which directly trained the HoVerNet cell segmentation model using the original Lizard dataset without any data augmentation; the second and third groups were augmentation experimental groups, which used labels and corresponding synthetic pathological images generated by the model in this application and labels and corresponding synthetic pathological images generated by the baseline model, respectively, with 30% data augmentation performed on each group to construct two augmented training sets, and then used the two augmented training sets to train the HoVerNet cell nucleus segmentation model; the segmentation performance of the model was evaluated on a fixed test set of the Lizard dataset in all three groups of experiments. The core evaluation metrics and their meanings are as follows: ① Dice (dice coefficient): measures the degree of overlap between the model segmented region and the real region; ② AJI (adjusted joint intersection): measures the matching accuracy between the segmented region and the real region, reducing the influence of background interference; ③ DQ (detection quality): measures the accuracy of the model in detecting cell nuclei; ④ SQ (segmentation quality): measures the accuracy of the model in segmenting cell nucleus boundaries; ⑤ PQ (panoramic quality): comprehensively evaluates the accuracy of cell nucleus detection and the accuracy of segmentation boundaries. The higher the values ​​of the above five metrics, the better the model segmentation performance. Table 3 shows that after data augmentation, the overall segmentation performance of the two augmentation experimental groups was better than that of the original control group, achieving an overall improvement in segmentation performance. Among them, compared with the model trained on the augmentation training set corresponding to the baseline model, the HoVerNet model trained on the augmentation training set corresponding to the model of this application showed a slight improvement in three core indicators related to segmentation accuracy and detection accuracy: DQ, SQ, and PQ. This reflects the effect of this application in improving the core performance of the downstream cell nucleus segmentation part, and also shows that using the data generated by this application for data augmentation has certain practicality.

[0043] Table 3

[0044] Based on the same inventive concept as the diffusion model label generation method using joint morphological features, this embodiment also provides a corresponding diffusion model label generation apparatus using joint morphological features, including: The training data acquisition module is used to acquire the training dataset. The samples in the training dataset include instance images, cell category labels, binary masks, and distance maps. The instance images include multiple cell instances, and the distance maps include the distance value of each pixel in the cell instance. The distance value is the distance between each pixel in the cell instance and the center pixel of the cell. The morphological feature extraction module is used to extract morphological features from the instance image to obtain a multi-channel morphological feature map. The stitching module is used to stitch together multi-channel morphological feature maps, cell category labels, binary masks, and distance maps along the channel dimension to obtain a multi-channel input tensor. The training module is used to train the diffusion model based on the multi-channel input tensor, text conditions, and random noise to obtain the trained diffusion model. During the training process, the diffusion model generates cell category labels based on the input multi-channel input tensor, text conditions, and random noise. The inference module is used to input fixed-format text conditions and random noise into the trained diffusion model to obtain cell category labels.

[0045] The diffusion model label generation device for the combined morphological features in this embodiment has the same inventive concept as the diffusion model label generation method for the combined morphological features described above. Therefore, the specific implementation of this device can be found in the embodiment section of the diffusion model label generation method for the combined morphological features described above, and its technical effects correspond to the technical effects of the above method. It will not be repeated here.

[0046] In summary, this application has the following technical effects: 1. Enhance the morphological realism of labels. By explicitly introducing cell instance-level morphological feature maps (specific features can be flexibly selected) as model input information, the diffusion model is effectively guided to generate labels with cell size, shape, and outline that are more consistent with real biological laws. The morphological details of the labels are optimized in a targeted manner to improve the realism of the labels and further narrow the gap between the generated labels and real labels.

[0047] 2. Improve the quality of generated images. The high-quality labels generated in this application can serve as core guiding conditions, effectively improving the output quality of the image generation model based on these labels, guiding it to generate high-quality pathological images that closely resemble real cell morphology and have more precise details, thus optimizing the pathological image generation effect.

[0048] 3. Optimize downstream task performance. The high-quality labels generated in this application, when used with corresponding synthetic pathological images for data augmentation in downstream cell nucleus segmentation models (such as HoVerNet), can optimize some core performance aspects of the model. Compared to the original dataset without data augmentation and the dataset with data augmentation generated by the baseline model, it shows a slight advantage in some core metrics, highlighting the practical application value of this invention.

[0049] 4. The implementation is simple, stable, and easy to integrate. This application does not change the core loss function and basic architecture of the diffusion model, but only improves it by adding morphological feature maps as input. There is no need for large-scale reconstruction of the model, the training process is stable, and it is easy to integrate with existing pathological image processing workflows, making it highly practical.

[0050] The above descriptions are merely various embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A diffusion model label generation method combining morphological features, characterized in that, include: Obtain a training dataset, wherein the samples in the training dataset include an instance image, cell category labels, a binary mask, and a distance map. The instance image includes multiple cell instances, and the distance map includes the distance value of each pixel in the cell instance, wherein the distance value is the distance between each pixel in the cell instance and the center pixel of the cell. Morphological features are extracted from the example image to obtain a multi-channel morphological feature map; The multi-channel morphological feature map, the cell category label, the binary mask, and the distance map are concatenated along the channel dimension to obtain a multi-channel input tensor; The diffusion model is trained based on the multi-channel input tensor, text conditions, and random noise to obtain the trained diffusion model. During training, the diffusion model generates cell category labels based on the input multi-channel input tensor, the text conditions, and the random noise. By inputting fixed-format text conditions and random noise into the trained diffusion model, cell category labels are obtained.

2. The method as described in claim 1, characterized in that, in, Morphological feature extraction is performed on the instance image to obtain a multi-channel morphological feature map, including: Perform connected component analysis on the instance graph to identify each cell instance; Calculate the feature values ​​of multiple morphological features for each cell instance; Feature map filling is performed on the feature values ​​of each morphological feature to obtain multiple single-channel feature maps; All single-channel feature maps are merged to obtain a multi-channel morphological feature map.

3. The method as described in claim 2, characterized in that, in, Feature map filling is performed on the eigenvalues ​​of each morphological feature to obtain multiple single-channel feature maps, including: Generate multiple blank single-channel images, each corresponding to a morphological feature; The feature value of a morphological feature of each cell instance is filled into all pixel positions of the corresponding cell instance in the blank single-channel image to obtain a single-channel feature map.

4. The method as described in claim 1, characterized in that, The diffusion model adopts the imagen basic architecture.

5. The method as described in claim 1, characterized in that, During training, the loss functions used include L2 loss and KL divergence loss.

6. A diffusion model label generation device based on combined morphological features, characterized in that, include: The training data acquisition module is used to acquire a training dataset. The samples in the training dataset include an instance image, cell category labels, a binary mask, and a distance map. The instance image includes multiple cell instances, and the distance map includes the distance value of each pixel in the cell instance. The distance value is the distance between each pixel in the cell instance and the center pixel of the cell. The morphological feature extraction module is used to extract morphological features from the instance image to obtain a multi-channel morphological feature map. The splicing module is used to splice the multi-channel morphological feature map, the cell category label, the binary mask and the distance map in the channel dimension to obtain a multi-channel input tensor; The training module is used to train the diffusion model based on the multi-channel input tensor, text conditions, and random noise to obtain the trained diffusion model. During training, the diffusion model generates cell category labels based on the input multi-channel input tensor, the text conditions, and the random noise. The inference module is used to input fixed-format text conditions and random noise into the trained diffusion model to obtain cell category labels.

Images