A nuclear instance segmentation model and method based on attention and ellipse regularization

Abstract

The application discloses a kind of nuclear instance segmentation model and method based on attention and ellipse regularization, belong to deep learning image processing technical field, model includes from shallow to deep window attention mechanism feature extraction module, long-distance feature dependent attention fusion module and ellipse regularization module, input image is extracted feature by from shallow to deep window attention mechanism feature extraction module, then by long-distance feature dependent attention feature fusion, finally by ellipse regularization module constraint model.The application uses the above instance segmentation model and method, for the small and elliptical shape of cell nucleus is constrained training, strengthens the image feature extraction capability based on bounding box instance segmentation, more effectively separates the adherent cell nucleus, can extract more complete cell nucleus density, morphology and location information.

Description

Technical Field

[0001] This invention relates to the field of deep learning image processing technology, and in particular to a cell nucleus instance segmentation model and method based on attention and elliptic regularization. Background Technology

[0002] The detection and segmentation of cell nuclei are crucial for automating disease diagnosis and developing computer-aided systems for clinical and medical applications. Microscopic images often contain thousands of cell nuclei; automated analysis of pathological slides can reduce observational biases by pathologists, lessen their manual examination workload, and significantly accelerate disease diagnosis. Accurate and rapid segmentation of cell nuclei in pathological tissue images is key to obtaining information on their density and location. Extracting the shape and spatial distribution characteristics of cell nuclei from slide images is a critical factor in the Nottingham scoring system and high-content screening for cell phenotypic variation, requiring continuous observation of individual cell nuclei. Furthermore, accurate identification and classification of cell nuclei help determine the developmental lineage of dividing cells, providing important insights into disease pathogenesis and facilitating the implementation of long-term targeted surveillance and treatment programs as well as disease prognosis. However, measuring tens of thousands of cell nuclei is a significant challenge because the shape and size of individual cell types vary across different regions.

[0003] In recent years, deep learning methods have garnered widespread attention due to their outstanding performance in numerous computer vision tasks, and have become dominant in the fields of computer vision and medical image analysis. This trend has accelerated the advancement of medical processing technologies, creating more opportunities for medical research and clinical applications.

[0004] Currently, there are two main methods for locating cell nuclei. One is the bounding box-less segmentation algorithm. First, the pathological slide is cut into multiple small pieces, and a neural network learns the cell nucleus information in each piece, separating the cell nucleus mask and merging them into a complete slice. This limits the acquisition of global information from large slice images and requires complex post-processing steps. For example, Kumar et al. used a neural network to label cell nuclei and contours, and then used a region growing method to extract cell nucleus instances. Graham et al. used pixel-to-centroid distance maps in the horizontal and vertical directions to represent cell nuclei, and required a complex watershed algorithm to separate cell nucleus instances. To overcome these limitations, many researchers have begun to try using bounding box-based instance segmentation models. It can capture the overall spatial information of the cell nucleus. A notable example is Mask R-CNN, which can extract individual cell nucleus masks from instance bounding boxes without complex post-processing steps. However, when using Mask R-CNN for detection, the cell nucleus is small and elliptical, and its representation in the feature map is not rich enough, leading to false positives or false negatives and low detection accuracy. Summary of the Invention

[0005] The purpose of this invention is to provide a cell nucleus instance segmentation model and method based on attention and elliptic regularization. The model is trained with constraints on the small and elliptical shape of the cell nucleus, which enhances the image feature extraction capability based on bounding box instance segmentation, more effectively separates adhered cell nuclei, and can extract more complete cell nucleus density, morphology and position information.

[0006] To achieve the above objectives, this invention provides a cell nucleus instance segmentation model based on attention and elliptic regularization, comprising:

[0007] The feature extraction module based on the window attention mechanism extracts image features from shallow to deep based on window attention concentration.

[0008] Long-range feature-dependent attention fusion module: uses convolutional channel attention and spatial attention to focus on image details, fuses the extracted image features over a long distance, and mines image feature information;

[0009] Elliptic Regularization Module: Regularizes the model in the detection head section with an elliptical cell nucleus shape.

[0010] This invention also provides a method for segmenting cell nuclei based on attention and elliptic regularization, comprising the following steps:

[0011] S1. Input pathological cell nucleus images into the cell nucleus instance segmentation model, and use the shallow-to-deep window attention mechanism feature extraction module to extract cell nucleus image features at different scales to obtain image features at different scales;

[0012] S2. The long-range feature-dependent attention fusion module fuses and extracts image features at different scales, and further fuses the features through the C2F module. After fusion, the features are transmitted to the segmentation branch and the detection head branch, and finally output the cell nucleus mask, position information, category information and mask coefficient information respectively.

[0013] S3, the elliptic regularization module calculates the intersection-union ratio (IU) of the output cell nucleus location information and the cell nucleus location information of the real label in the form of elliptical bounding boxes, and uses the IU value as the regularization term and positive sample selection for training the network model.

[0014] S4. Train and validate the cell nucleus instance segmentation model multiple times, save the optimal training network model parameters on the validation set as a .pth file, and adjust the network parameters according to the metrics on the validation set to obtain the optimal cell nucleus detection and segmentation results.

[0015] Preferably, in step S1, a feature extraction module using a shallow-to-deep window attention mechanism is used to extract cell nucleus image features at four different scales. The four image features, from largest to smallest, are C2, C3, C4, and C5. Features C2, C3, and C4 are directly transmitted to the long-distance feature-dependent attention fusion module, while feature C5 is further extracted by a pooling layer with a spatial pyramid before being transmitted to the long-distance feature-dependent attention fusion module.

[0016] Preferably, step S2 specifically includes:

[0017] S21, the long-range feature-dependent attention fusion module extracts and fuses features from C2, C3, C4 and C5;

[0018] S22. The image features obtained in step S21 are further fused by the C2F module to enrich the semantic information of the image contour features;

[0019] S23. The C2 image features after feature fusion are output by the segmentation branch as a mask image of the cell nucleus, and the cell nucleus location information, category information and mask coefficient information are output by the detection head branch.

[0020] S24. The cell nucleus location information, category information, and mask coefficient information of the C3 and C4 image features after feature fusion are output by the detection head branch.

[0021] Preferably, the Monte Carlo probability method is used to calculate the crossover-union ratio in step S3.

[0022] Preferably, the training in step S4 specifically includes:

[0023] Input training dataset Where n is the number of images in the training dataset, i is the image index, pred is the predicted cell nucleus information, gt is the true label information of the cell nucleus, and the training loss function is:

[0024]

[0025] Among them, L box L represents the regularization loss for elliptical bounding boxes. class and L mask Both are binary cross-entropy losses with activation functions, L dfl Distributed focus loss is used to refine the bounding boxes of detected objects during network training. k λ represents the positive sample score, λ is the weight coefficient of the loss function, and K is the number of positive samples.

[0026] Therefore, the present invention employs the above-mentioned cell nucleus instance segmentation model and method based on attention and elliptic regularization, which has the following beneficial effects:

[0027] (1) When pathological cell nucleus images are input into the model, the model can perform instance segmentation of elliptical cell nuclei based on the global model. At the same time, the model can quickly detect and count cell nuclei in pathological images.

[0028] (2) The model of the present invention can be embedded into the existing instance segmentation model based on object detection boxes, so that it focuses on the shape of the elliptical cell nucleus and enhances the detection accuracy;

[0029] (3) Constrained training is performed on the small and elliptical shape of the cell nucleus, which enhances the image feature extraction capability based on bounding box instance segmentation, and more effectively separates the adhered cell nuclei, and can extract more complete information on cell nucleus density, morphology and position.

[0030] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the partial structure of the instance segmentation model according to an embodiment of the present invention;

[0032] Figure 2 This is a structural diagram of the feature extraction module of the shallow-to-deep window attention mechanism according to an embodiment of the present invention;

[0033] Figure 3 This is a structural diagram of the long-distance feature-dependent attention fusion module according to an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of elliptic regularization in an embodiment of the present invention;

[0035] Figure 5 This is a visual comparison diagram of the instance segmentation model of this invention with the segmentation results of other neural networks;

[0036] Figure 6 This is a visual comparison diagram of the segmentation and counting effects in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0038] Example

[0039] This invention provides a cell nucleus instance segmentation model based on attention and elliptic regularization, with a partial framework diagram as shown below. Figure 1 As shown, it includes:

[0040] The shallow-to-deep window attention mechanism feature extraction module extracts image features from shallow to deep based on window attention concentration. The specific structure is as follows: Figure 2 As shown.

[0041] Long-range feature-dependent attention fusion module: Uses convolutional channel attention and spatial attention to focus on image details, fusing extracted image features over long distances to mine image feature information. Specific structure as follows: Figure 3 As shown.

[0042] Ellipse Regularization Module: In the detection head part, the model is regularized with an elliptical cell nucleus shape. Specifically, the intersection-union ratio (IU) of the output cell nucleus position information and the cell nucleus position information of the real label is calculated in the form of an elliptical bounding box. The IU value is used as the regularization term for training the network model and the selection of positive samples.

[0043] This invention also provides a method for segmenting cell nuclei based on attention and elliptic regularization, comprising the following steps:

[0044] S1. Pathological cell nucleus images are input into the cell nucleus instance segmentation model. A feature extraction module with a pyramid structure and a shallow-to-deep window attention mechanism is used to extract cell nucleus image features at four different scales. The image features at the four different scales, from largest to smallest, are C2, C3, C4, and C5, with a size ratio of 2 between two adjacent features. Features C2, C3, and C4 are directly transferred to the long-range feature-dependent attention fusion module, while feature C5 undergoes further feature extraction via a pooling layer with a spatial pyramid structure before being transferred to the long-range feature-dependent attention fusion module. The feature extraction formula is as follows:

[0045] F c =f 3×3 (f 3×3 (x c ));

[0046] F c ′=W-MSA(LN(F c ))+F c ;

[0047] F c "=MLP(LN(F c ′))+F c ′;

[0048]

[0049] In the formula, f 3×3 This represents a 3×3 kernel-sized convolutional module. LN and MLP represent normalization layers and multilayer perceptrons, respectively. The feature extraction module, from shallow to deep, uses a window attention mechanism and is divided into two parts, front and back. F c and F c ′ represents an intermediate variable in the first part of this module, F c " is the output variable of the first part of this module. and These are intermediate variables for the later part of this module. x is the output variable for the latter part of this module. c The input image features are represented by W-MSA and SW-MSA, which represent window attention mechanism and sliding window attention mechanism, respectively. The specific attention mechanism formulas for W-MSA and SW-MSA are as follows:

[0050]

[0051] Where b is the relative position offset, Q, K, and V are the query feature map, key feature map, and value feature map, respectively, and d is the dimension of the feature map.

[0052] S2, the long-range feature-dependent attention fusion module fuses and extracts image features at different scales, and further fuses these features through the C2F module. The fused features are then transmitted to the segmentation branch and the detection head branch, ultimately outputting the cell nucleus mask, location information, category information, and mask coefficient information, respectively. Specifically:

[0053] S21, the long-range feature-dependent attention fusion module extracts and fuses features from C2, C3, C4, and C5. The feature extraction formula is as follows:

[0054]

[0055]

[0056] Where I represents the input feature, O c For the output of channel attention, O s The specific formulas for channel attention C and spatial attention S, which are the outputs of spatial attention, are as follows:

[0057] C(I)=σ(MLP(Avgpool(I))+MLP(Maxpool(I)));

[0058] S(O c )=σ(f 7×7 [MLP(Avgpool(O c MLP(Maxpool(O) c ))]);

[0059] Where σ is the activation function, f 7×7 This represents a 7×7 kernel-sized convolutional module. Avgpool is for average pooling, and Maxpool is for max pooling.

[0060] The formula for the fusion module is as follows:

[0061]

[0062] Where w is the input feature map, A m Let be a learnable array of size m, σ be the activation function, ε be a decimal of 0.00001, and W be the output feature map.

[0063] S22. The image features obtained in step S21 are further fused by the C2F module, and the image contour feature information contained in the image becomes richer as the features are fused and stacked.

[0064] S23. The C2 image features after feature fusion are output by the segmentation branch as a mask image of the cell nucleus, and the cell nucleus location information, category information and mask coefficient information are output by the detection head branch.

[0065] S24. The cell nucleus location information, category information, and mask coefficient information of the C3 and C4 image features after feature fusion are output by the detection head branch.

[0066] S3, the elliptic regularization module outputs the cell nucleus location information Box. pred ((x min ,y min ),(x max ,y max Box containing the nucleus location information of the actual label gt ((x min ,y min ),(x max ,ymax The cross-union ratios (CUI) of the bounding boxes are extracted and calculated using the Monte Carlo probability method. These CUI values ​​are then used as regularization terms and positive sample selection criteria for training the network model. Specifically:

[0067] S31. Calculate the Intersection over Union (IoU) score between the bounding boxes of all detected objects and their labels. n (s1,s2,...,s n ),s∈[0,1], for IoU n Sort the data from largest to smallest, and select the η highest-scoring intersection-union ratios to obtain the index. n (i1,i2,...,i n ).

[0068] S32. Calculate the largest rectangle formed by the bounding boxes corresponding to the η intersection-union ratios. Construct a Cartesian coordinate system A using this largest rectangle as the basis, with the origin at the center point of the largest rectangle. According to Box... pred and Box gt Obtain the length, width, and midpoint of the rectangular bounding box, and then use them as the major axis a, minor axis b, and center point c of the ellipse, respectively, to construct the elliptical bounding box.

[0069] S33. Randomly generate a set S of coordinate points in region A, denoted as S[(x1,y1),...,(x...y1)...y2)...y3)...y4)...y5)...y6)...y7)...y8)...y9 ... t ,y t ], 10000≤t, where, if there are points in the point set S that simultaneously satisfy the elliptic formulas of both the prediction kernel and the true kernel, they are labeled as set I; if there are points in the point set S that satisfy one of the two elliptic formulas, they are labeled as U. The ratio of the number of points in the two sets is used as the intersection-union ratio between the prediction and true elliptic bounding boxes, and its formula is: IOU=I / U. The specific elliptic formula is:

[0070]

[0071] In the formula, a pred,gt b represents the width of the bounding box predicted by the network for the cell nucleus and the bounding box of the ground truth label. pred,gt This represents the height of the bounding box predicted by the network for the cell nucleus and the bounding box of the ground truth label.

[0072] Then, the generated random points are substituted into the ellipse formula to calculate the intersection-union ratio (IoU) between the predicted and actual elliptical bounding boxes, thus obtaining the IoU. η (s1,s2,...,s η ).

[0073] S34. Based on the index... n (i1,i2,...,i nIntersection over Union (IoU) of elliptical bounding boxes η (s1,s2,...,s η Replace the original Intersection over Union (IoU) score. n (s1,s2,...,s n The new intersection-union ratio (IoU) is obtained based on the Monte Carlo method. er This achieves elliptic regularization. (From...) Figure 4 As shown, (a) is the global cell nucleus rectangle, (b) is the selection of ellipse regularization scheme, (c)-(e) are the transformation of rectangles into ellipse frames, and (f) is the visualization of the ellipse intersection-union ratio solution method based on Monte Carlo method.

[0074] S4. Train and validate the cell nucleus instance segmentation model multiple times, save the optimal training network model parameters on the validation set as a .pth file, and adjust the network parameters according to the metrics on the validation set to obtain the optimal cell nucleus detection and segmentation results.

[0075] The training specifically includes:

[0076] Input training dataset Where n is the number of images in the training dataset, i is the image index, pred is the predicted cell nucleus information, gt is the true label information of the cell nucleus, and the training loss function is:

[0077]

[0078] Among them, L box L represents the regularization loss for elliptical bounding boxes. class and L mask Both are binary cross-entropy losses with activation functions, L dfl Distributed focus loss is used to refine the bounding boxes of detected objects during network training. k λ represents the positive sample score, λ is the weight coefficient of the loss function, and K is the number of positive samples.

[0079] During model training, this invention is based on the PyTorch framework, uses the initial weights of the pre-trained COCO dataset, and is trained using an NVIDIA GeForce 3090 (24G) GPU. The datasets used are Data Science Bowl 2018 and MoNuSeg.

[0080] The Data Science Bowl 2018 dataset, from Kaggle's Data Science Bowl 2018 competition, contains 670 pathological images with cell nucleus masks annotated by pathologists. The MoNuSeg dataset was released for the MICCAI 2018 multi-organ cell nucleus segmentation challenge. The training set consists of 37 images from multiple organs (breast, kidney, liver, prostate, bladder, colon, and stomach).

[0081] During model training, the input image size was 512×512. The batch sizes of the training data from Data Science Bowl 2018 and MoNuSeg were 12 and 4, respectively. AdamW was used as the optimizer for both models. The training iterations were 600, the initial learning rate was 0.000714, and the momentum was 0.9. Finally, the detection and segmentation performance was measured by the average precision (AP) and the number of images detected per second (FPS).

[0082] To demonstrate the effectiveness of the method described in this invention, a horizontal comparison is made between the method of this invention and existing deep learning-based bounding box instance segmentation methods, namely Mask-RCNN, Cascade Mask R-CNN, QueryInst, Mask-dino, YOLOv5s, YOLOv8s, and ASF-YOLO. The evaluation metrics are Average Precision (AP) and Frames Per Second (FPS). The minimum value of the AP metric is 0, and the maximum value is 1. The closer the value is to 1, the better the performance. The comparison results are shown in Tables 1 and 2. In the tables, AP50 indicates that the overlap between the predicted object box and the ground truth label box needs to reach 50% or more to be considered correct. Similarly, AP75 indicates that the overlap between the predicted object box and the ground truth label box needs to reach 75% or more to be considered correct. AP50:95 represents the average AP calculated within the range of 50% to 95% overlap between the predicted object box and the ground truth label box (with a step size of 5%). FPS is a metric for speed, representing the number of images that can be detected per second.

[0083] Table 1 Comparison of detection results on the MoNuSeg dataset

[0084]

[0085] Table 2 Comparison of Detection Results for the Data Science Bowl 2018 Dataset

[0086]

[0087] Experimental data show that the method of this invention performs better than other bounding box-based instance segmentation methods, while also having excellent detection rate. The AP metrics for detection and segmentation are the best on the MoNuSeg and Data Science Bowl 2018 datasets.

[0088] Regarding instance segmentation performance, this invention compares with other neural network models in terms of segmentation effectiveness. Figure 5 As shown, (a) is the original image, (b) is the ground truth label, and (c) is a comparison between the model of this invention and other results (d), (e), and (f). It can be seen that the instance segmentation model based on detection boxes in this invention has a more accurate segmentation effect. Regarding counting, as... Figure 6 The figure shows a comparison between the present invention and the YOLOv8s-seg model. As can be seen from the figure, the present invention can detect small and densely packed cell nuclei more accurately. Therefore, the method of the present invention can more effectively segment adhered cell nuclei.

[0089] Therefore, the present invention adopts the above-mentioned cell nucleus instance segmentation model and method based on attention and elliptic regularization to constrain the small and elliptical shape of the cell nucleus, thereby enhancing the image feature extraction capability based on bounding box instance segmentation.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for segmenting cell nuclei based on attention and elliptic regularization, characterized in that the steps include... include: S1. Input pathological cell nucleus images into the cell nucleus instance segmentation model, and use the shallow-to-deep window attention mechanism feature extraction module to extract cell nucleus image features at different scales to obtain image features at different scales; S2. The long-range feature-dependent attention fusion module fuses and extracts image features at different scales, and further fuses the fused features through the C2F module. The fused features are then transmitted to the segmentation branch and the detection head branch, and finally output the cell nucleus mask, location information, category information and mask coefficient information respectively. S3, the elliptic regularization module calculates the intersection-union ratio (IU) of the output cell nucleus location information and the cell nucleus location information of the real label in the form of elliptical bounding boxes; S4. Train and validate the cell nucleus instance segmentation model multiple times, save the optimal training network model parameters on the validation set as a .pth file, and adjust the network parameters according to the metrics on the validation set to obtain the optimal cell nucleus detection and segmentation results. Nucleus instance segmentation models based on attention and elliptic regularization include: The feature extraction module based on the window attention mechanism from shallow to deep: extracts image features from shallow to deep based on window attention; Long-distance feature-dependent attention fusion module: Uses convolutional channel attention and spatial attention to focus on image details, and fuses the extracted image features to mine image feature information; Elliptic Regularization Module: Regularizes the model in the detection head section with an elliptical cell nucleus shape.

2. The cell nucleus instance segmentation method based on attention and elliptic regularization according to claim 1, characterized in that: In step S1, the feature extraction module of the shallow-to-deep window attention mechanism is used to extract cell nucleus image features at four different scales. The four image features are C2, C3, C4 and C5 from large to small. The C2, C3 and C4 features are directly transmitted to the long-distance feature-dependent attention fusion module, while the C5 feature is further extracted by the pooling layer with spatial pyramid before being transmitted to the long-distance feature-dependent attention fusion module.

3. The cell nucleus instance segmentation method based on attention and elliptic regularization according to claim 2, characterized in that, Step S2 specifically includes: S21, the long-range feature-dependent attention fusion module extracts and fuses features from C2, C3, C4 and C5; S22. The fused image features obtained in step S21 are further fused by the C2F module to enrich the semantic information of the image contour features; S23. The C2 image features after feature fusion by the C2F module are output by the segmentation branch as a mask image of the cell nucleus, and the cell nucleus location information, category information and mask coefficient information are output by the detection head branch. S24. The C3 and C4 image features after feature fusion by the C2F module are output by the detection head branch, which provides information on the cell nucleus location, category, and mask coefficients.

4. The cell nucleus instance segmentation method based on attention and elliptic regularization according to claim 1, characterized in that: In step S3, the Monte Carlo probability method is used to calculate the crossover-union ratio.

5. The cell nucleus instance segmentation method based on attention and elliptic regularization according to claim 1, characterized in that, The training in step S4 specifically includes: Input training dataset ,in, The number of images in the training dataset, The image number is the sequence number. For predicted cell nuclear information, For the true label information of the cell nucleus, the training loss function is: ； in, This represents the regularization loss for an elliptical bounding box. and Both are binary cross-entropy loss with activation functions. This represents the distributed focus loss, used to refine the bounding boxes of detected objects during network training. For positive sample scores, , , , These represent the corresponding weight coefficients of the loss function. This represents the number of positive samples.

Images