Cell nucleus instance segmentation model and method based on attention and ellipse regularization

Abstract

The invention discloses a nucleus instance segmentation model and method based on attention and ellipse regularization, and belongs to the technical field of deep learning image processing, and the model comprises a shallow-to-deep window attention mechanism feature extraction module, a long-distance feature dependence attention fusion module and an ellipse regularization module. Features of an input image are extracted through a shallow-to-deep window attention mechanism feature extraction module, then long-distance features depend on attention feature fusion, and finally a model is constrained through an ellipse regularization module. By adopting the instance segmentation model and method, constraint training is carried out on the small and elliptical form of the cell nucleus, the image feature extraction capability based on bounding box instance segmentation is enhanced, the adhered cell nucleus is separated more effectively, and more and more complete cell nucleus density, form and position information can be extracted.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning image processing, and in particular to a cell nucleus instance segmentation model and method based on attention and ellipse regularization. Background Art

[0002] The detection and segmentation of cell nuclei are crucial to the automation of disease diagnosis and the development of computer-aided systems for clinical and medical applications. There are often tens of thousands of cell nuclei in microscopic images. Automated analysis of pathological sections can not only reduce the observation bias of pathologists, but also relieve their manual inspection workload and greatly speed up disease diagnosis. Accurate and rapid segmentation of cell nuclei in pathological tissue images is the key to obtaining their density and position information. Extracting the shape and spatial distribution characteristics of cell nuclei from slice images is a key factor in the Nottingham scoring system and high-content screening of cell phenotypic variation, which requires continuous observation of individual cell nuclei. In addition, accurate identification and classification of cell nuclei helps determine the developmental lineage of dividing cells, provides important insights into the pathogenesis of diseases, and helps implement long-term targeted monitoring and treatment plans and disease prognosis. However, measuring tens of thousands of cell nuclei is a huge challenge because the shape and size of a single type of cell instance vary in different regions.

[0003] In recent years, deep learning methods have attracted widespread attention due to their outstanding performance in many computer vision tasks and have dominated the fields of computer vision and medical image analysis. This trend has accelerated the progress of medical processing technology and created more opportunities for medical research and clinical applications.

[0004] There are currently two main methods for locating cell nuclei. One is the frameless segmentation algorithm. First, the pathological slice is cut into multiple small blocks, and the neural network is allowed to learn the cell nucleus information in each block, separate the cell nucleus mask, and merge them into a complete slice. This limits the acquisition of global information of large slice images and requires complex post-processing steps. For example, Kumar et al. used a neural network to mark the cell nucleus and contour, and then used the region growing method to extract the cell nucleus instance. Graham et al. used pixel-to-centroid distance maps in the horizontal and vertical directions to represent the cell nucleus, and required a complex watershed algorithm to separate the cell nucleus instance. To overcome the above limitations, many researchers began to try to use bounding box-based instance segmentation models. It can capture the overall spatial information of the cell nucleus. A notable example is Mask R-CNN, which can extract a single cell nucleus mask through an instance bounding box without complex post-processing steps. However, when using Mask R-CNN for detection, the cell nucleus is small and elliptical, and its representation in the feature map is not rich enough, which will result in false detection or missed detection, and the detection accuracy is low. Summary of the invention

[0005] The purpose of the present invention is to provide a cell nucleus instance segmentation model and method based on attention and ellipse regularization, which performs constraint training on the small and elliptical morphology of cell nuclei, strengthens the image feature extraction capability based on bounding box instance segmentation, more effectively separates adhered cell nuclei, and can extract more and more complete cell nucleus density, morphology and position information.

[0006] To achieve the above object, the present invention provides a cell nucleus instance segmentation model based on attention and ellipse regularization, comprising:

[0007] Shallow-to-deep window attention mechanism feature extraction module: extract image features from shallow to deep based on window attention;

[0008] Long-distance feature-dependent attention fusion module: Use convolutional channel attention and spatial attention to focus on image details, fuse the extracted image features over a long distance, and mine image feature information;

[0009] Elliptical regularization module: Regularize the model with an elliptical cell nucleus shape in the detection head.

[0010] The present invention also provides a method for segmenting a cell nucleus instance based on attention and ellipse regularization, the steps comprising:

[0011] S1. Input the pathological cell nucleus image into the cell nucleus instance segmentation model, and use the shallow-to-deep window attention mechanism feature extraction module to extract the cell nucleus image features at different scales to obtain the image features at different scales.

[0012] S2, the long-range feature-dependent attention fusion module fuses and extracts image features at different scales, and further fuses the features through the C2F module. After fusion, it is transmitted to the segmentation branch and the detection head branch, and finally outputs the mask, position information, category information and mask coefficient information of the cell nucleus respectively;

[0013] S3, the elliptical regularization module calculates the intersection-and-union ratio of the output cell nucleus position information and the cell nucleus position information of the true label in the form of an elliptical bounding box, and uses the intersection-and-union ratio value as the regularization term and positive sample selection for training the network model;

[0014] S4. Perform multiple rounds of training and verification on the cell nucleus instance segmentation model, save the model's optimal training network model parameters in the verification set as a .pth file, and adjust the network parameters according to the indicators on the verification set to obtain the optimal cell nucleus detection and segmentation effect.

[0015] Preferably, in step S1, a shallow-to-deep window attention mechanism feature extraction module is used to extract nucleus image features at four different scales, and the four image features are C2, C3, C4 and C5 from large to small, among which C2, C3 and C4 features are directly transmitted to the long-distance feature-dependent attention fusion module, and the C5 feature is further feature extracted by a pooling layer with a spatial pyramid and then transmitted to the long-distance feature-dependent attention fusion module.

[0016] Preferably, step S2 specifically includes:

[0017] S21, the long-range feature-dependent attention fusion module extracts and fuses features from C2, C3, C4, and C5;

[0018] S22, further fusing the image features obtained in step S21 through the C2F module to enrich the semantic information of the image contour features;

[0019] S23, the segmentation branch outputs the mask image of the cell nucleus from the C2 image features after feature fusion, and the detection head branch outputs the position information, category information and mask coefficient information of the cell nucleus;

[0020] S24, the detection head branch outputs the location information, category information and mask coefficient information of the cell nucleus from the C3 and C4 image features after feature fusion.

[0021] Preferably, in step S3, the Monte Carlo probability method is used to calculate the intersection-over-union ratio.

[0022] Preferably, the training in step S4 specifically includes:

[0023] Input training data set Where n is the number of images in the training data set, i is the sequence number of the image, pred is the predicted cell nucleus information, gt is the true label information of the cell nucleus, and the training loss function is:

[0024]

[0025] Among them, L box represents the elliptical bounding box regularization loss, L class and L mask Both are binary cross entropy losses with activation functions, L dfl Distribution focus loss, used to refine the bounding box of detected objects during network training, Aligned k is the positive sample fraction, λ is the weight coefficient of the loss function corresponding to λ, and K is the number of positive samples.

[0026] Therefore, the present invention adopts the above-mentioned cell nucleus instance segmentation model and method based on attention and ellipse regularization, which has the following beneficial effects:

[0027] (1) When pathological cell nucleus images are input into the model, the oval cell nucleus can be instance-segmented based on the global image. At the same time, the model can quickly detect and count the cell nuclei in the pathological images.

[0028] (2) The model of the present invention can be embedded into the existing instance segmentation model based on object detection box, so that it focuses on the shape of elliptical cell nuclei and enhances the detection accuracy;

[0029] (3) Constrained training is performed on the small and elliptical shape of cell nuclei, which strengthens the image feature extraction capability based on bounding box instance segmentation, more effectively separates adherent cell nuclei, and can extract more complete cell nucleus density, morphology, and position information.

[0030] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a partial structure of an instance segmentation model according to an embodiment of the present invention;

[0032] Figure 2 This is a structural diagram of a feature extraction module of a window attention mechanism from shallow to deep according to an embodiment of the present invention;

[0033] Figure 3 It is a structural diagram of a long-distance feature-dependent attention fusion module according to an embodiment of the present invention;

[0034] Figure 4 Schematic diagram of elliptical regularization according to an embodiment of the present invention;

[0035] Figure 5 A schematic diagram showing a visual comparison of the segmentation effects of the instance segmentation model of an embodiment of the present invention and other neural networks;

[0036] Figure 6 It is a schematic diagram for visually comparing the segmentation and counting effects of an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0038] Example

[0039] The present invention provides a cell nucleus instance segmentation model based on attention and ellipse regularization, and a partial framework diagram is shown in FIG. Figure 1 As shown, including:

[0040] From shallow to deep window attention mechanism feature extraction module: extract image features from shallow to deep based on window attention concentration. The specific structure is as follows Figure 2 shown.

[0041] Long-distance feature-dependent attention fusion module: Use convolutional channel attention and spatial attention to focus on image details, fuse the extracted image features over a long distance, and mine image feature information. The specific structure is as follows: Figure 3 shown.

[0042] Elliptical regularization module: In the detection head part, the model is regularized with an elliptical cell nucleus shape. Specifically, the intersection-and-union ratio of the output cell nucleus position information and the cell nucleus position information of the true label is calculated in the form of an elliptical bounding box, and the intersection-and-union ratio score is used as the regularization term and positive sample selection for training the network model.

[0043] The present invention also provides a method for segmenting a cell nucleus instance based on attention and ellipse regularization, the steps comprising:

[0044] S1. Input the pathological cell nucleus image into the cell nucleus instance segmentation model, and use the shallow-to-deep window attention mechanism feature extraction module with a pyramid structure to extract the cell nucleus image features at four different scales. The image features at four different scales are C2, C3, C4 and C5 from large to small, and the size ratio of two adjacent features is 2. Among them, the C2, C3 and C4 features are directly transmitted to the long-distance feature-dependent attention fusion module, and the C5 feature is further extracted by the pooling layer with a spatial pyramid, and then transmitted to the long-distance feature-dependent attention fusion module. The feature extraction formula is as follows:

[0045] F c =f 3×3 (f 3×3 (x c ));

[0046] F c ′=W-MSA(LN(F c ))+F c ;

[0047] F c =MLP(LN(F c ′))+F c ′;

[0048]

[0049] In the formula, f 3×3 represents a convolutional module with a kernel size of 3×3, LN and MLP represent normalization layers and multi-layer perceptrons, and the window attention mechanism feature extraction module is divided into two parts from shallow to deep. c and F c ′ is the intermediate variable in the front part of the module, F c ″ is the output variable of the front part of the module, and is the intermediate variable of the latter part of the module, is the output variable of the latter part of the module, x c represents the input image features, W-MSA and SW-MSA represent the window attention mechanism and sliding window attention mechanism respectively. The specific attention mechanism formulas of W-MSA and SW-MSA are as follows:

[0050]

[0051] Among them, b is the relative position bias, Q, K, V are the query feature map, key feature map and value feature map respectively, and d is the dimension of the feature map.

[0052] S2, the long-distance feature-dependent attention fusion module fuses and extracts image features at different scales, and further fuses the features through the C2F module. After fusion, it is transmitted to the segmentation branch and the detection head branch, and finally outputs the mask, position information, category information and mask coefficient information of the cell nucleus respectively. Specifically:

[0053] S21, long-distance feature dependency attention fusion module extracts and fuses features of C2, C3, C4 and C5. The feature extraction formula is as follows:

[0054]

[0055]

[0056] Among them, I is the input feature, O c is the output result of channel attention, O s is the output result of spatial attention. The specific formulas of channel attention C and spatial attention S are as follows:

[0057] C(I)=σ(MLP(Avgpool(I))+MLP(Maxpool(I)));

[0058] S(O c )=σ(f 7×7 [MLP(Avgpool(O c )); MLP(Maxpool(O c ))]);

[0059] Where σ is the activation function, f 7×7 It represents a convolution module with a kernel size of 7×7, Avgpool is average pooling, and Maxpool is maximum pooling.

[0060] The fusion module formula is as follows:

[0061]

[0062] Among them, w is the input feature map, A m is a learnable array of size m, σ is the activation function, ε is a decimal of 0.00001, and W is the output feature map.

[0063] S22: The image features obtained in step S21 are further fused through the C2F module, and the image contour feature information contained in the image becomes richer as the features are fused and stacked.

[0064] S23, the segmentation branch outputs the mask image of the cell nucleus from the C2 image features after feature fusion, and the detection head branch outputs the position information, category information and mask coefficient information of the cell nucleus;

[0065] S24, the detection head branch outputs the location information, category information and mask coefficient information of the cell nucleus from the C3 and C4 image features after feature fusion.

[0066] S3, the elliptical regularization module outputs the cell nucleus position information Box pred ((x min ,y min ),(x max ,y max )) and the true label of the cell nucleus position information Box gt ((x min ,y min ),(x max ,ymax )) extracts them, calculates their intersection-over-union ratio using the Monte Carlo probability method in the form of an elliptical bounding box, and uses the intersection-over-union ratio score as the regularization term and positive sample selection for training the network model. Specifically:

[0067] S31. Calculate the intersection and union (IoU) score of all detected objects and the rectangular boxes of the labels. n (s1,s2,...,s n ),s∈[0,1], for IoU n Sort from large to small, select the n intersections with the highest scores to get the index Index n (i1,i2,...,i n ).

[0068] S32, calculate the largest rectangular box surrounded by the bounding boxes corresponding to the n intersection and union ratios, and use this largest rectangular box as the base to construct a plane rectangular coordinate system A, with the origin of the coordinate system being the center point of the largest rectangular box. pred and Box gt The length, width and midpoint of the rectangular bounding box are obtained, and then used as the major axis a, minor axis b and center point c of the ellipse, respectively, to construct the elliptical bounding box.

[0069] S33. Randomly generate a set of coordinate points S in area A, expressed as S[(x1,y1),...,(x t ,y t )], 10000≤t, where if there are points in the point set S that meet the ellipse formula of both the predicted kernel and the true kernel, it is marked as set I, and if there are points in the point set S that meet one of the two ellipse formulas, it is marked as U. The ratio of the number of points in the two sets is used as the intersection-over-union ratio between the predicted and true ellipse bounding boxes, and the formula is: IOU = I / U. The specific ellipse formula is:

[0070]

[0071] In the formula, a pred,gt represents the width of the bounding box of the network predicted cell nucleus and the bounding box of the true label, b pred,gt represents the height of the bounding box of the network predicted cell nucleus and the bounding box of the true label.

[0072] Then, substitute the generated random points into the ellipse formula and calculate the intersection-over-union ratio between the predicted and true ellipse bounding boxes to get the ellipse bounding box intersection-over-union ratio IoU η (s1,s2,...,s η ).

[0073] S34. According to the index n (i1,i2,...,i n) The ellipse bounding box intersection and union score IoU η (s1,s2,...,s η ) replaces the original intersection-over-union score IoU n (s1,s2,...,s n ), and the new intersection-over-union ratio IoU is obtained based on the Monte Carlo method er , to achieve elliptic regularization. Figure 4 As shown, (a) is the global cell nucleus rectangular box, (b) is the selection of the ellipse regularization scheme, (c)-(e) are the conversion of the rectangular box into an ellipse box, and (f) is the visualization of the ellipse intersection and union ratio solution method based on the Monte Carlo method.

[0074] S4. Perform multiple rounds of training and verification on the cell nucleus instance segmentation model, save the model's optimal training network model parameters in the verification set as a .pth file, and adjust the network parameters according to the indicators on the verification set to obtain the optimal cell nucleus detection and segmentation effect.

[0075] The training specifically includes:

[0076] Input training data set Where n is the number of images in the training data set, i is the sequence number of the image, pred is the predicted cell nucleus information, gt is the true label information of the cell nucleus, and the training loss function is:

[0077]

[0078] Among them, L box represents the elliptical bounding box regularization loss, L class and L mask Both are binary cross entropy losses with activation functions, L dfl Distribution focus loss, used to refine the bounding box of detected objects during network training, Aligned k is the positive sample fraction, λ is the weight coefficient of the loss function corresponding to λ, and K is the number of positive samples.

[0079] When training the model of the present invention, based on the PyTorch framework, the initial weights of the pre-trained COCO data set are adopted, and the NVIDIA GeForce 3090 (24G) GPU is used for training. The data sets used are: Data Science Bowl 2018 and MoNuSeg.

[0080] Data Science Bowl 2018 is a competition dataset from Kaggle's Data Science Bowl 2018, which contains 670 pathological images with nucleus masks annotated by pathology experts. MoNuSeg dataset was released on the multi-organ nucleus segmentation challenge of MICCAI 2018. The training set consists of 37 images from multiple organs (breast, kidney, liver, prostate, bladder, colon, and stomach).

[0081] When training the model, the input image size is 512×512. The batch sizes of the training data for Data Science Bowl2018 and MoNuSeg are 12 and 4 respectively. AdamW is used as the optimizer to train the model. The number of training times is 600, the initial learning rate is 0.000714, and the momentum is 0.9. Finally, the detection and segmentation effects are measured by the AP average precision index and the FPS number of detection images per second.

[0082] In order to demonstrate the effectiveness of the method of the present invention, the method of the present invention is compared horizontally with the existing instance segmentation methods of bounding boxes based on deep learning, namely Mask-RCNN, Cascade Mask R-CNN, QueryInst, Mask-dino, Yolov5s, Yolov8s, and ASF-YOLO. The evaluation indicators are average precision AP and frames per second FPS. The minimum value of the AP indicator is 0 and the maximum value is 1. The closer the value is to 1, the better the effect. The comparison results are shown in Tables 1 and 2. In the table, AP50 indicates that the overlap between the predicted object box and the true label box needs to reach 50% or more, and the prediction result will be considered correct. Similarly, AP75 indicates that the overlap between the predicted object box and the true label box needs to reach 75% or more, and the prediction result will be considered correct. AP50:95 indicates the AP average value calculated from the overlap between the predicted object box and the true label box from 50% to 95% (step size 5%). FPS is an indicator for measuring speed, indicating the number of images that can be detected per second.

[0083] Table 1 Comparison of detection results of MoNuSeg dataset

[0084]

[0085] Table 2 Comparison of test results on Data Science Bowl 2018 dataset

[0086]

[0087] Experimental data show that the proposed method has better instance segmentation performance than other bounding box-based methods, and also has excellent detection rate. The AP indicators of detection and segmentation are optimal on the MoNuSeg and Data Science Bowl 2018 datasets.

[0088] In terms of instance segmentation effect, the segmentation effect of the present invention is compared with that of other neural network models. Figure 5 As shown in the figure, (a) is the original image, (b) is the real label, and (c) is the comparison between the model of the present invention and other results (d), (e), and (f). It can be seen that the instance segmentation model based on the detection frame of the present invention has a more accurate segmentation effect. Figure 6 As shown in FIG. 1 , the comparison result of the present invention and the YOLOv8s-seg model is shown. It can be seen from the figure that the present invention can more accurately detect small and dense cell nuclei. It can be seen that the method of the present invention can more effectively segment the adherent cell nuclei.

[0089] Therefore, the present invention adopts the above-mentioned cell nucleus instance segmentation model and method based on attention and ellipse regularization to constrain the small and elliptical shape of the cell nucleus, thereby enhancing the image feature extraction capability based on bounding box instance segmentation.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A cell nucleus instance segmentation model based on attention and ellipse regularization, characterized in that: include: Shallow-to-deep window attention mechanism feature extraction module: extract image features from shallow to deep based on window attention; Long-distance feature-dependent attention fusion module: Use convolutional channel attention and spatial attention to focus on image details, fuse the extracted image features over a long distance, and mine image feature information; Elliptical regularization module: Regularize the model with an elliptical cell nucleus shape in the detection head.

2. A method for cell nucleus instance segmentation based on attention and ellipse regularization, using the cell nucleus instance segmentation model based on attention and ellipse regularization according to claim 1, characterized in that the steps include: S1. Input the pathological cell nucleus image into the cell nucleus instance segmentation model, and use the shallow-to-deep window attention mechanism feature extraction module to extract the cell nucleus image features at different scales to obtain the image features at different scales. S2, the long-range feature-dependent attention fusion module fuses and extracts image features at different scales, and further fuses the features through the C2F module. After fusion, it is transmitted to the segmentation branch and the detection head branch, and finally outputs the mask, position information, category information and mask coefficient information of the cell nucleus respectively; S3, the elliptical regularization module calculates the intersection-and-union ratio of the output cell nucleus position information and the cell nucleus position information of the true label in the form of an elliptical bounding box, and uses the intersection-and-union ratio value as the regularization term and positive sample selection for training the network model; S4. Perform multiple rounds of training and verification on the cell nucleus instance segmentation model, save the model's optimal training network model parameters in the verification set as a .pth file, and adjust the network parameters according to the indicators on the verification set to obtain the optimal cell nucleus detection and segmentation effect.

3. The method for cell nucleus instance segmentation based on attention and ellipse regularization according to claim 2, characterized in that: In step S1, a shallow-to-deep window attention mechanism feature extraction module is used to extract nucleus image features at four different scales. The four image features are C2, C3, C4 and C5 from large to small. Among them, C2, C3 and C4 features are directly transmitted to the long-distance feature-dependent attention fusion module, and the C5 feature is further extracted by the pooling layer with a spatial pyramid and then transmitted to the long-distance feature-dependent attention fusion module.

4. The method for cell nucleus instance segmentation based on attention and ellipse regularization according to claim 3, characterized in that: Step S2 specifically includes: S21, the long-range feature-dependent attention fusion module extracts and fuses features from C2, C3, C4, and C5; S22, further fusing the image features obtained in step S21 through the C2F module to enrich the semantic information of the image contour features; S23, the segmentation branch outputs the mask image of the cell nucleus from the C2 image features after feature fusion, and the detection head branch outputs the position information, category information and mask coefficient information of the cell nucleus; S24, the detection head branch outputs the location information, category information and mask coefficient information of the cell nucleus from the C3 and C4 image features after feature fusion.

5. The method for cell nucleus instance segmentation based on attention and ellipse regularization according to claim 2, characterized in that: In step S3, the Monte Carlo probability method is used to calculate the intersection-over-union ratio.

6. The method for cell nucleus instance segmentation based on attention and ellipse regularization according to claim 2, characterized in that: The training in step S4 specifically includes: Input training data set Where n is the number of images in the training data set, i is the sequence number of the image, pred is the predicted cell nucleus information, gt is the true label information of the cell nucleus, and the training loss function is: Among them, L box represents the elliptical bounding box regularization loss, L class and L mask Both are binary cross entropy losses with activation functions, L dfl Distribution focus loss, used to refine the bounding box of detected objects during network training, Aligned k is the positive sample fraction, λ is the weight coefficient of the loss function corresponding to λ, and K is the number of positive samples.

Images