Leukocyte segmentation method based on improved Unet
By improving the Unet neural network model, introducing compressed path networks and extended path networks, and using dimensional space attention mechanisms, the existing leukocyte segmentation methods are solved, and the problem of low accuracy and poor robustness in processing complex images is achieved, achieving higher segmentation accuracy and robustness.
Patent Information
- Application Number
- CN202411609200.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-12
- Publication Date
- 2025-04-18
AI Technical Summary
The existing leukocyte segmentation methods have problems of low segmentation accuracy and poor robustness when processing staining impurities, low contrast images, and images under different staining conditions.
Improve the Unet neural network model, enhance the ability to capture and fusion of context information by introducing compressed path networks and extended path networks, and using dimensional space attention mechanisms during the upsampling process.
It improves the accuracy and robustness of leukocyte segmentation, can process complex leukocyte images more effectively, and reduces missegment and missegmentation, especially in the case of complex backgrounds or weak noise interference.
Smart Images
Figure CN119851268A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the fields of computer technology and medical image analysis technology, and in particular to a leukocyte segmentation method based on improved Unet. Background Art
[0002] In modern medicine and biological research, white blood cells (i.e., WBCs) play a vital role in the human immune system. They not only play a key role in fighting infections and diseases, but also serve as biomarkers for a variety of diseases. Traditional WBC identification methods mainly rely on manual operations and empirical judgments, which are time-consuming and easily affected by human factors. With the rapid development of computer vision and machine learning technologies, medical image processing methods based on image processing and deep learning have gradually become a research hotspot. In medical image analysis, segmentation is very important because its purpose is to extract specific features from the image. Medical image segmentation is the division of an image into smaller parts with relatively uniform features that can be used for complex image understanding. Image segmentation is a virtual mid-level visual processing performed by neurons located between low-level and high-level cortical areas. These methods can achieve effective WBC segmentation by training on a large amount of WBC image data. However, WBC segmentation still faces problems such as staining impurities, low image contrast, and appearance changes under different staining conditions.
[0003] Existing WBC segmentation methods are mainly divided into supervised learning and unsupervised learning. Unsupervised learning is training without any labeled data, and the model discovers hidden patterns or structures through data. Among them, the most effective method is unsupervised aggregation. Relatively speaking, supervised learning is based on existing data sets and uses the known relationship between input and output results for training to obtain an optimal model. This type of method is mainly divided into regression and classification problems, among which segmentation problems are usually regarded as classification problems. Traditional classifiers include logistic regression (LR), k-nearest neighbor algorithm (k-NN), random forest (RF, etc. However, due to the weak image representation ability of traditional classifiers, it is difficult to obtain high classification accuracy on large-scale image datasets. Summary of the invention
[0004] The present invention aims to provide a leukocyte segmentation method based on improved Unet, which realizes image segmentation by improving the U-net network model, so that the segmentation accuracy is higher, and can have a better performance in the segmentation effect while maintaining the main framework structure of the UNet network.
[0005] The technical solution of the present invention is as follows:
[0006] The leukocyte segmentation method based on improved Unet comprises the following steps:
[0007] A. Obtain leukocyte microscopic images from a public data set, preprocess the leukocyte microscopic images, and obtain a data set;
[0008] B. Constructing an improved Unet neural network model, wherein the improved Unet neural network model includes: a compression path network and an expansion path network;
[0009] The compression path network includes multiple groups of context information transmission modules;
[0010] C. Train the improved Unet neural network model to obtain a trained Unet neural network model;
[0011] D. The white blood cell microscope image to be segmented is introduced into the trained Unet neural network model for processing. The process is as follows:
[0012] In the compression path network, the input result is processed by 3*3 convolution, ReLU function, and maximum pooling, and then processed by multiple groups of context information transfer modules in sequence. The input result and the processing results of each group of context information transfer modules are respectively input into the expansion path network to obtain the segmented white blood cell structure image.
[0013] The public dataset used in the preprocessing process in step A is a Kaggle public dataset, which is data collected using the CellaVision DM96 analyzer of the Barcelona Hospital Clinic Core Laboratory, and contains 17,092 single normal cell images, a total of eight categories: neutrophils, eosinophils, basophils, lymphocytes, monocytes, immature granulocytes (promyelocytes, myelocytes and metamyelocytes), red blood cells and platelets or thrombocytes. The preprocessing process in step S1 is as follows:
[0014] a. Download the Kaggle public dataset, add annotation information annotated by clinical pathology experts to each sample in the Kaggle public dataset; perform calculations on each annotation information to generate a new mask image;
[0015] b. Crop the mask image so that the determined cells are located in the center of the image. At the same time, scale the image resolution to 256×256 during cropping to create a data set.
[0016] The compression path network includes four groups of context information transmission modules. The processing process in the compression path network is as follows:
[0017] The input result is first processed by 3*3 convolution, BN function, and ReLU function to obtain the function processing result; the function processing result is processed by maximum pooling, and then input into the first context information transfer module for processing to obtain the first information processing result; the first information processing result is input into the second context information transfer module for processing to obtain the second information processing result; the second information processing result is input into the third context information transfer module for processing to obtain the third information processing result; the third information processing result is input into the fourth context information transfer module for processing to obtain the fourth information processing result;
[0018] The function processing result, the first information processing result, the second information processing result, the third information processing result and the fourth information processing result are respectively input into the extended path network to process and obtain the segmented white blood cell structure image.
[0019] The first context information transmission module, the second context information transmission module, the third context information transmission module, and the fourth context information transmission module have the same structure, and the processing process is as follows:
[0020] The input result is processed by 3*3 convolution, BN function, and ReLU function in sequence, and then processed by the efficient channel attention module. Then, it is processed by 3*3 convolution, BN function, and ReLU function. The result is multiplied and fused with the input result to obtain the output result.
[0021] The processing process in the efficient channel attention module is as follows:
[0022] Each channel of the input result is processed by global average pooling, and then these processed channels are processed by one-dimensional convolution. The one-dimensional convolution results are processed by ReLU function and Sigmoid function in turn, and the obtained results are multiplied and fused with the input results to obtain the output result.
[0023] The processing results in the extended path network are as follows:
[0024] The fourth information processing result is processed in sequence by 3*3 convolution, ReLU function, first dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and then concatenated with the third information processing result by Concat function to obtain a first concatenation result;
[0025] The first splicing result is processed by 3*3 convolution, ReLU function, second dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and then spliced with the second information processing result by Concat function to obtain the second splicing result;
[0026] The second splicing result is processed by 3*3 convolution, ReLU function, third-dimensional spatial attention module, 3*3 convolution, ReLU function, upsampling, and then spliced with the first information processing result by Concat function to obtain the third splicing result;
[0027] The third splicing result is processed by 3*3 convolution, ReLU function, fourth dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and concatenated with the function processing result by Concat function to obtain the fourth splicing result;
[0028] The fourth splicing result is processed by 3*3 convolution, ReLU function, fourth-dimensional spatial attention module, 3*3 convolution, and ReLU function in turn to obtain the output result.
[0029] The first dimensional spatial attention module, the second dimensional spatial attention module, the third dimensional spatial attention module, and the fourth dimensional spatial attention module have the same structure, and the processing process is as follows:
[0030] The input result is divided into three branches; the first branch is divided into two paths. The first path is processed by MaxPool to make its width dimension become 1, and the second path is processed by AvgPool to make its width dimension become 1. Then the two paths of features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the first branch;
[0031] The first branch is divided into two paths. The first path is processed by MaxPool to make its width dimension become 1, and the second path is processed by AvgPool to make its width dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the first branch;
[0032] The second branch is divided into two paths. The first path is processed by MaxPool to make its length dimension become 1, and the second path is processed by AvgPool to make its length dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the second branch;
[0033] The third branch is divided into two paths. The first path is processed by MaxPool to make the height dimension become 1, and the third path is processed by AvgPool to make the height dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the third branch;
[0034] The first branch results, the second branch results, and the third branch results are averaged and aggregated to obtain the output result.
[0035] The beneficial effects of the present invention are as follows:
[0036] 1. The leukocyte segmentation method of the present invention uses a neural network formed by improving the UNet network model. By introducing a dimensional space attention improvement mechanism in the upsampling process, the global features are promoted from the two dimensions of space and channel, and the ability to capture contextual information is effectively improved. The weights of different features can be dynamically adjusted to ensure that the key features are fully paid attention to, improve the robustness of the segmentation, achieve the effective fusion of features at different resolution levels, and avoid the loss of information in the feature map during the downsampling process. The features processed by the method of the present invention contain both low-level fusion edge information and high-level semantic information, so that the leukocyte segmentation results are more refined.
[0037] 2. The method provided by the present invention shows high performance in segmenting lung nodules on the Kaggle public dataset, can effectively improve the segmentation accuracy, and is suitable for complex white blood cell segmentation tasks.
[0038] 3. The segmentation method of the present invention can effectively realize white blood cell detection, improve the work efficiency of doctors, and can segment cell areas more accurately, while reducing the situations of mis-segmentation and missed segmentation, especially in the case of complex background or weak noise interference, showing stronger robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 The overall structural diagram of the improved Unet neural network model;
[0040] Figure 2 It is a structural diagram of the context information transmission module;
[0041] Figure 3 Schematic diagram of the structure of the efficient channel attention module;
[0042] Figure 4 It is a schematic diagram of the structure of the dimensional space attention module;
[0043] Figure 5 A visual comparison chart of multiple methods on the Kaggle public dataset. DETAILED DESCRIPTION
[0044] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0045] Example 1
[0046] The leukocyte segmentation method based on the improved Unet of this embodiment includes the following steps:
[0047] The leukocyte segmentation method based on the improved Unet includes the following steps:
[0048] A. Obtain leukocyte microscopic images from a public data set, preprocess the leukocyte microscopic images, and obtain a data set;
[0049] The public dataset is a Kaggle public dataset, which is collected using the CellaVision DM96 analyzer in the core laboratory of the Barcelona Hospital Clinic. It contains 17,092 single normal cell images, totaling eight categories: neutrophils, eosinophils, basophils, lymphocytes, monocytes, immature granulocytes (promyelocytes, myelocytes and metamyelocytes), red blood cells and platelets or thrombocytes. The preprocessing process in step S1 is as follows:
[0050] a. Download the Kaggle public dataset, add annotation information annotated by clinical pathology experts to each sample in the Kaggle public dataset; perform calculations on each annotation information to generate a new mask image;
[0051] b. Crop the mask image so that the determined cells are located in the center of the image. At the same time, scale the image resolution to 256×256 during cropping to create a data set.
[0052] B. Constructing an improved Unet neural network model, wherein the improved Unet neural network model includes: a compression path network and an expansion path network;
[0053] The compression path network includes multiple groups of context information transmission modules;
[0054] C. Train the improved Unet neural network model to obtain a trained Unet neural network model;
[0055] The training process is as follows: divide the data set samples obtained in step A into a training set, a validation set and a test set, wherein the training set is used to train the improved Unet neural network model, the validation set is used to adjust the parameters of the improved Unet neural network model, and the test set is used to test whether the improved Unet neural network model is accurate.
[0056] D. The white blood cell microscope image to be segmented is introduced into the trained Unet neural network model for processing. The process is as follows:
[0057] In the compression path network, the input result is processed by 3*3 convolution, ReLU function, and maximum pooling, and then processed by multiple groups of context information transfer modules in sequence. The input result and the processing results of each group of context information transfer modules are respectively input into the expansion path network to obtain the segmented white blood cell structure image.
[0058] The compression path network includes four groups of context information transmission modules. The processing process in the compression path network is as follows:
[0059] The input result is first processed by 3*3 convolution, BN function, and ReLU function to obtain the function processing result; the function processing result is processed by maximum pooling, and then input into the first context information transfer module for processing to obtain the first information processing result; the first information processing result is input into the second context information transfer module for processing to obtain the second information processing result; the second information processing result is input into the third context information transfer module for processing to obtain the third information processing result; the third information processing result is input into the fourth context information transfer module for processing to obtain the fourth information processing result;
[0060] The function processing result, the first information processing result, the second information processing result, the third information processing result and the fourth information processing result are respectively input into the extended path network to process and obtain the segmented white blood cell structure image.
[0061] The first context information transmission module, the second context information transmission module, the third context information transmission module, and the fourth context information transmission module have the same structure. Figure 2 As shown, the processing process is as follows:
[0062] The input result is processed by 3*3 convolution, BN function, and ReLU function in sequence, and then processed by the efficient channel attention module. Then, it is processed by 3*3 convolution, BN function, and ReLU function. The result is multiplied and fused with the input result to obtain the output result.
[0063] The processing process in the efficient channel attention module is as follows:
[0064] Each channel of the input result is processed by global average pooling, and then these processed channels are processed by one-dimensional convolution. The one-dimensional convolution results are processed by ReLU function and Sigmoid function in turn, and the obtained results are multiplied and fused with the input results to obtain the output result.
[0065] The processing results in the extended path network are as follows:
[0066] The fourth information processing result is processed in sequence by 3*3 convolution, ReLU function, first dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and then concatenated with the third information processing result by Concat function to obtain a first concatenation result;
[0067] The first splicing result is processed by 3*3 convolution, ReLU function, second dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and then spliced with the second information processing result by Concat function to obtain the second splicing result;
[0068] The second splicing result is processed by 3*3 convolution, ReLU function, third-dimensional spatial attention module, 3*3 convolution, ReLU function, upsampling, and then spliced with the first information processing result by Concat function to obtain the third splicing result;
[0069] The third splicing result is processed by 3*3 convolution, ReLU function, fourth dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and concatenated with the function processing result by Concat function to obtain the fourth splicing result;
[0070] The fourth splicing result is processed by 3*3 convolution, ReLU function, fourth-dimensional spatial attention module, 3*3 convolution, and ReLU function in turn to obtain the output result.
[0071] The first dimensional spatial attention module, the second dimensional spatial attention module, the third dimensional spatial attention module, and the fourth dimensional spatial attention module have the same structure, and the processing process is as follows:
[0072] The input result is divided into three branches; the first branch is divided into two paths. The first path is processed by MaxPool to make its width dimension become 1, and the second path is processed by AvgPool to make its width dimension become 1. Then the two paths of features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the first branch;
[0073] The first branch is divided into two paths. The first path is processed by MaxPool to make its width dimension become 1, and the second path is processed by AvgPool to make its width dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the first branch;
[0074] The second branch is divided into two paths. The first path is processed by MaxPool to make its length dimension become 1, and the second path is processed by AvgPool to make its length dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the second branch;
[0075] The third branch is divided into two paths. The first path is processed by MaxPool to make the height dimension become 1, and the third path is processed by AvgPool to make the height dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the third branch;
[0076] The first branch results, the second branch results, and the third branch results are averaged and aggregated to obtain the output result.
[0077] Example 2
[0078] In order to reflect the effectiveness of the present invention in comparing different methods in white blood cell (WBC) segmentation, the present invention uses the kaggle public data set to conduct relevant experiments and draw relevant tables. Table 1 shows the three accuracy metrics obtained by seven deep learning-based methods on two data sets. In order to more intuitively show the differences between the methods, the present invention visualizes the effect of cell segmentation. During the test, it is compared with multiple related networks, and the evaluation indicators FLOPS, Params, IoU, Dic and F1-score are used for comparative analysis, as shown in Table 1.
[0079] Table 1 Comparison results of the DRA-Net network model of the present invention and multiple related networks
[0080]
[0081] As can be seen from Table 1, (1) the IoU, Dice and F1-score of the neural network model constructed in Example 1 of the present invention are 90.95%, 95.08% and 94.53% respectively. Compared with the Unet network model, the IoU of the DRA-Net network model of the present invention is 86.55% compared with that of U-Net, and the algorithm proposed in this paper is improved by 4.4%.
[0082] (2) Through experiments, it is found that in terms of the number of model parameters (Params), the U-net network model has the least number of parameters, and the number of parameters of the neural network model constructed in Example 1 of the present invention is almost the same as that of the U-net network model. In terms of the amount of calculation (FLOPs), the Swin-Unet network model has the least amount of calculation, and the amount of calculation of the neural network model constructed in Example 1 of the present invention and the U-net network model is almost the same. This shows that the neural network model constructed in Example 1 of the present invention and the U-net network model have achieved relatively excellent segmentation performance while having similar model complexity.
[0083] (3) From Figure 5It can be seen that in terms of the overall segmentation results, compared with other models, the segmentation of the neural network model constructed in Example 1 of the present invention is clearer and more accurate, especially in the segmentation of different types of cells. The boundaries are blurred and the shape is closer to the real label.
[0084] Robustness to noise: While other models may have segmentation recovery or incomplete edges, the segmentation results of the neural network model constructed in Example 1 of the present invention are more complete and show better robustness.
[0085] Accuracy of cell classification: The neural network model constructed in Example 1 of the present invention has a high degree of matching the colors and shapes of most cell types, indicating that it can better perform regional division and identify different cell types.
[0086] Consistency: For different types of cells, the neural network model constructed in Example 1 of the present invention can maintain consistent segmentation quality, while other models show differences in different cell types. These characteristics make the present invention advantageous in this application and can provide more accurate and reliable cell segmentation results.
[0087] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.
Claims
1. A leukocyte segmentation method based on improved Unet, characterized in that: The following steps are involved: A. Obtain leukocyte microscopic images from a public data set, preprocess the leukocyte microscopic images, and obtain a data set; B. Constructing an improved Unet neural network model, wherein the improved Unet neural network model includes: a compression path network and an expansion path network; The compression path network includes multiple groups of context information transmission modules; C. Use the data set to train the improved Unet neural network model to obtain a trained Unet neural network model; D. The white blood cell microscope image to be segmented is introduced into the trained Unet neural network model for processing. The process is as follows: In the compression path network, the input result is processed by 3*3 convolution, ReLU function, and maximum pooling, and then processed by multiple groups of context information transfer modules in sequence. The input result and the processing results of each group of context information transfer modules are respectively input into the expansion path network to obtain the segmented white blood cell structure image.
2. The leukocyte segmentation method based on improved Unet according to claim 1, characterized in that: The public dataset used in the preprocessing process in step A is the Kaggle public dataset, and the preprocessing process is as follows: a. Download the Kaggle public dataset and add annotation information annotated by clinical pathology experts to each sample in the Kaggle public dataset; Perform calculations on each annotation information to generate a new mask image; b. Crop the mask image so that the determined cells are located in the center of the image. At the same time, scale the image resolution to 256×256 during cropping to create a data set.
3. The leukocyte segmentation method based on improved Unet according to claim 1, characterized in that: The compression path network includes four groups of context information transmission modules. The processing process in the compression path network is as follows: The input result is first processed by 3*3 convolution, BN function, and ReLU function to obtain the function processing result; the function processing result is processed by maximum pooling, and then input into the first context information transfer module for processing to obtain the first information processing result; the first information processing result is input into the second context information transfer module for processing to obtain the second information processing result; The second information processing result is input into the third context information transmission module for processing to obtain a third information processing result; The third information processing result is input into the fourth context information transmission module for processing to obtain a fourth information processing result; The function processing result, the first information processing result, the second information processing result, the third information processing result and the fourth information processing result are respectively input into the extended path network to process and obtain the segmented white blood cell structure image.
4. The leukocyte segmentation method based on improved Unet as claimed in claim 3, characterized in that: The first context information transmission module, the second context information transmission module, the third context information transmission module, and the fourth context information transmission module have the same structure, and the processing process is as follows: The input result is processed by 3*3 convolution, BN function, and ReLU function in sequence, and then processed by the efficient channel attention module. Then, it is processed by 3*3 convolution, BN function, and ReLU function. The result is multiplied and fused with the input result to obtain the output result.
5. The leukocyte segmentation method based on improved Unet according to claim 4, characterized in that: The processing process in the efficient channel attention module is as follows: Each channel of the input result is processed by global average pooling, and then these processed channels are processed by one-dimensional convolution. The one-dimensional convolution results are processed by ReLU function and Sigmoid function in turn, and the obtained results are multiplied and fused with the input results to obtain the output result.
6. The leukocyte segmentation method based on improved Unet according to claim 1, characterized in that: The processing results in the extended path network are as follows: The fourth information processing result is processed in sequence by 3*3 convolution, ReLU function, first dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and then concatenated with the third information processing result by Concat function to obtain a first concatenation result; The first splicing result is processed by 3*3 convolution, ReLU function, second dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and then spliced with the second information processing result by Concat function to obtain the second splicing result; The second splicing result is processed by 3*3 convolution, ReLU function, third-dimensional spatial attention module, 3*3 convolution, ReLU function, upsampling, and then spliced with the first information processing result by Concat function to obtain the third splicing result; The third splicing result is processed by 3*3 convolution, ReLU function, fourth dimension spatial attention module, 3*3 convolution, ReLU function, upsampling, and concatenated with the function processing result by Concat function to obtain the fourth splicing result; The fourth splicing result is processed by 3*3 convolution, ReLU function, fourth-dimensional spatial attention module, 3*3 convolution, and ReLU function in turn to obtain the output result.
7. The leukocyte segmentation method based on improved Unet according to claim 6, characterized in that: The first dimensional spatial attention module, the second dimensional spatial attention module, the third dimensional spatial attention module, and the fourth dimensional spatial attention module have the same structure, and the processing process is as follows: The input result is divided into three branches; the first branch is divided into two paths. The first path is processed by MaxPool to make its width dimension become 1, and the second path is processed by AvgPool to make its width dimension become 1. Then the two paths of features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the first branch; The first branch is divided into two paths. The first path is processed by MaxPool to make its width dimension become 1, and the second path is processed by AvgPool to make its width dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the first branch; The second branch is divided into two paths. The first path is processed by MaxPool to make its length dimension become 1, and the second path is processed by AvgPool to make its length dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the second branch; The third branch is divided into two paths. The first path is processed by MaxPool to make the height dimension become 1, and the third path is processed by AvgPool to make the height dimension become 1. Then the two features are added and fused, and then processed by convolution and Sigmoid function in turn, and then added and fused with the input result to obtain the result of the third branch; The first branch results, the second branch results, and the third branch results are averaged and aggregated to obtain the output result.