Blood hierarchical image segmentation method based on multi-channel alternating attention network

Abstract

The application discloses a blood stratified image segmentation method based on a multi-channel alternating attention network. The method comprises the following steps: collecting original blood stratified image data of a blood collection tube; performing data preprocessing; building a blood stratified image segmentation network MCC-UNET of the blood collection tube based on an improved multi-channel alternating attention module; inputting the preprocessed blood stratified image data of the blood collection tube into the MCC-UNET, training network parameters, and obtaining an optimal segmentation network model; for the obtained optimal segmentation network model, after new images are collected and standardized, the new images are input into the optimal segmentation network model for blood stratified image segmentation, and a segmentation result of the blood stratified image of the blood collection tube is obtained. The application proposes a new deep neural network structure for blood stratified image segmentation of the blood collection tube, solves the segmentation precision problem of small-area white blood cells and platelet layers in the blood stratified image, and improves the segmentation precision.

Description

Technical Field

[0001] This invention relates to the fields of computer vision technology and image processing, specifically to a blood layer image segmentation method based on a multi-communication channel attention network. Background Technology

[0002] In modern medical diagnosis, blood tests are an important auxiliary diagnostic tool, enabling the early detection and prevention of some diseases. Different blood components are suitable for the prevention and diagnosis of different diseases. For example, red blood cell layers can be used for molecular diagnosis of tumors, white blood cell and platelet layers can be used for whole-genome sequencing and genetic disease detection, and plasma layers can be used for the detection of infectious diseases such as hepatitis B. Due to the large volume of blood testing tasks and the fact that some blood components constitute a small proportion and require high accuracy, designing an automatic segmentation method for layered blood images is of great significance for automated medical testing and assisting doctors in diagnosis.

[0003] Segmentation of blood stratified images in blood collection tubes is a highly challenging problem in automated blood pipetting systems. Currently, the separation of centrifuged stratified blood mainly relies on professional physicians for sampling and separation. However, manual testing is inefficient and requires long-term training and experience to achieve more accurate separation. Furthermore, the proportion of white blood cells and platelets in the centrifuged layer is relatively small compared to other layers, which poses an even greater challenge to the separation and pipetting of stratified blood components.

[0004] Currently, blood layer image segmentation methods are mainly divided into traditional image segmentation algorithms and deep learning-based segmentation methods. Traditional image segmentation methods do not require dataset annotation during the segmentation process, but the segmentation accuracy is low, and the segmentation edges are coarse, failing to meet the basic requirements of automated blood separation. Existing research (CN201610390622.9 2016.06.03 Blood Layer Recognition Method) has low segmentation efficiency and relies heavily on multiple cameras for manual operation, lacking complete automation performance. In contrast, segmentation methods based on machine learning and deep convolutional neural networks have become mainstream in fields such as medical and automated industries. Compared to traditional segmentation methods, deep convolutional neural network methods obtain more accurate and objective segmentation results. Existing research (CN202210279022.0 2022.7.12 A Blood Collection Tube Visual Recognition System) still relies on multiple camera acquisitions and repeated adjustments to the light source and contrast of the acquired images, and its efficiency and automation level are still insufficient to meet the needs of automated segmentation. Summary of the Invention

[0005] The purpose of this invention is to address the problems of low accuracy and potential insufficient purity of blood component separation products in current blood pipetting systems. It provides a blood layer image segmentation method based on a multi-channel communicative attention network, achieving high-precision end-to-end segmentation of blood layer images from input to output, with excellent results. Experiments demonstrate that the multi-channel communicative attention semantic segmentation network is a leader in the field of layered blood segmentation in blood pipetting systems.

[0006] The objective of this invention is achieved through at least one of the following technical solutions.

[0007] A blood layer image segmentation method based on a multi-communication channel attention network includes the following steps:

[0008] S1. Collect raw blood collection tube blood layer image data;

[0009] S2. Perform data preprocessing on the collected raw blood collection tube blood layer images;

[0010] S3. Construct a blood collection tube layered image segmentation network MCC-UNET based on an improved multi-channel communication attention module;

[0011] S4. Import the preprocessed blood collection tube layered image data into MCC-UNET, use the loss functions FocalLoss and Dice Loss to train the network parameters, and obtain the optimal segmentation network model;

[0012] S5. For the obtained optimal segmentation network model, after acquiring new images and performing standardization processing, input them into the optimal segmentation network model to perform blood layer image segmentation, and obtain the segmentation result of the blood collection tube blood layer image.

[0013] Further, in step S1, color blood layer images from different lighting environments are collected as original blood collection tube blood layer images, and the original blood collection tube blood layer images are manually segmented using mask annotation.

[0014] During testing, manually labeled segmentation results were used as ground truth to evaluate the segmentation performance of the optimal segmentation network model.

[0015] 3. The blood layer image segmentation method based on a multi-channel communication attention network according to claim 1, characterized in that, in step S2, the data preprocessing is as follows:

[0016] The original blood collection tube images were preprocessed by size standardization, partial cropping, horizontal flipping, partial rotation, and data enhancement operations to improve image brightness and contrast. The preprocessed original blood collection tube images were then divided into training, validation, and test sets.

[0017] The training set is used to train the parameters of the blood vessel layered image segmentation network; the validation set is used to verify the performance of the optimal segmentation network model during training, adjust the hyperparameters of the optimal segmentation network model, and conduct a preliminary evaluation of the capability of the optimal segmentation network model; the test set is used to objectively evaluate the performance of the trained blood vessel layered image segmentation network.

[0018] Furthermore, in step S3, the blood collection tube blood layering image segmentation network MCC-UNET based on the improved multi-channel communication attention module includes a feature extraction sub-network, an enhanced feature extraction sub-network, and a pixel classification sub-network connected in sequence.

[0019] The feature extraction subnetwork includes a first downsampling layer, a second downsampling layer, a first multi-channel communication attention downsampling layer, and a second multi-channel communication attention downsampling layer connected in series. The first multi-channel communication attention downsampling layer and the second multi-channel communication attention downsampling layer are obtained by connecting an improved multi-channel communication attention module in series before the max pooling layer of the downsampling layer.

[0020] The enhanced feature extraction subnetwork includes a first multi-channel communication attention upsampling layer, a second multi-channel communication attention upsampling layer, a first upsampling layer, and a second upsampling layer connected in series. The first multi-channel communication attention upsampling layer and the second multi-channel communication attention upsampling layer are obtained by connecting an improved multi-channel communication attention module in series before the activation function layer of the upsampling layer.

[0021] In addition, there is a ShortCut connection between the feature extraction subnetwork and the enhanced feature extraction subnetwork, as detailed below:

[0022] The first downsampling layer is connected to the second upsampling layer;

[0023] The second downsampling layer is connected to the first upsampling layer;

[0024] The first multi-channel communication attention downsampling layer is connected to the second multi-channel communication attention upsampling layer;

[0025] The second multi-channel communication attention downsampling layer is connected to the first multi-channel communication attention upsampling layer.

[0026] Furthermore, a feature extraction subnetwork is used to extract semantic features from the image, with the input being the original image F of blood collection tube stratification. ori The output is the feature map F.down F down The output of the second multi-channel AC attention downsampling layer is fed into the enhanced feature extraction subnetwork, where F... ori ∈R H×W×3 F down ∈R H / 16×W / 16×C1 R represents the overall size of the image, H×W is the original input image size, and C1 is the number of channels in the feature map after four downsampling operations.

[0027] The input to the enhanced feature extraction subnetwork is the output feature map F of the feature extraction subnetwork. down The output is the feature map F. up Feature map F up The output feature map of the second upsampling layer is used to strengthen the feature extraction subnetwork to reconstruct the semantic features of the image, resulting in the feature map F. up The output is fed into the pixel classification subnetwork, where F up ∈R H×W×C2 H×W is the original input image size, and C2 is the number of channels in the feature map after four upsampling steps;

[0028] In the ShortCut connection between the feature extraction subnetwork and the enhanced feature extraction subnetwork, the first downsampling layer, the second downsampling layer, the first multi-channel communication attention downsampling layer, and the second multi-channel communication attention downsampling layer respectively input the downsampling feature maps extracted by the first downsampling layer, the second upsampling layer, the second multi-channel communication attention upsampling layer, and the first multi-channel communication attention upsampling layer to supplement the features, thereby reducing feature loss during the upsampling process and enabling the network to achieve better performance.

[0029] Furthermore, an improved multi-channel communication attention module is used in the first multi-channel communication attention upsampling layer, the second multi-channel communication attention upsampling layer, the first multi-channel communication attention downsampling layer, and the second multi-channel communication attention downsampling layer.

[0030] The improved multichannel communication attention module includes an improved multichannel communication channel attention module and an improved multichannel communication grouping spatial attention module;

[0031] An improved multi-channel communication attention module and an improved multi-channel communication grouping spatial attention module are used in the first multi-channel communication attention upsampling layer, the second multi-channel communication attention upsampling layer, the first multi-channel communication attention downsampling layer, and the second multi-channel communication attention downsampling layer, making the image segmentation model pay more attention to the features of the small regions to be segmented in the image and eliminating local noise and interference in the image.

[0032] Furthermore, the improved multi-channel AC attention module adopts the idea of adjacent-channel AC, and the specific process is as follows:

[0033] S7.1 For the input feature map F of the multi-channel AC attention module, perform spatial global max pooling Maxpool and global average pooling Avgpool on the input feature map F respectively to obtain the max pooling weight g max and the average pooling weight g avg , where, F ∈ R H×W×C , g max ∈ R 1×1×C , g avg ∈ R 1×1×C , H×W and 1×1 represent the spatial dimensions of the corresponding feature maps, and C represents the channel dimension of the corresponding feature maps;

[0034] S7.2 Perform 1D convolution on the current channel and its adjacent k - 1 channels to achieve the AC among the k channels of the max pooling weight g max and the AC among the k channels of the average pooling weight g avg [[ID=2 — 4]]to obtain the global max-channel feature weight W max and the global average-channel feature weight W avg . By performing 1D convolution to achieve the AC among adjacent channels, the influence of the current channel and its adjacent channels on the current channel feature weight is strengthened, which pays more attention to the local small-region features and improves the problem of the loss of small-region target features in the existing segmentation network. Among them, W max ∈ R 1×1×C , W avg ∈ R 1×1×C , C represents the spatial dimension of the corresponding feature weight, the convolution kernel of the 1D convolution is 1*k, k < C and k is a positive integer. In order to keep the weight dimension C unchanged before and after convolution, pad the weight g with the surrounding method of the size of , and round down k / 2;

[0035] S7.3 For the generated feature weights W max and W avg , after adding them in the channel dimension, pass through the Sigmoid activation function to obtain the channel attention weight M c of the input feature map F, and perform dot multiplication with each channel feature in the input feature map F to obtain the channel attention feature map F′, which is used as the input feature map of the multi-channel AC grouped spatial attention module. Among them, M c ∈ R 1 ×1×C , F′ ∈ R H×W×C .

[0036] Furthermore, the improved multi-channel communication group spatial attention module adopts the idea of ​​group spatial attention and randomly performs global max pooling or global average pooling on the obtained group features on the channels. The specific process is as follows:

[0037] S8.1 For the input feature map F′ of multi-channel communication spatial attention, the input feature map F′ is subjected to a channel-based grouping operation to obtain multiple sub-feature maps F with the same dimensions. i ′={f1′,f2′…f i ′,…f N ′},f i ′∈R H×W×C / N Where N represents the number of sub-feature maps after grouping, i = [1, 2…N], f i ′ represents the i-th sub-feature map, H×W represents the spatial dimension of the corresponding feature map, and C represents the channel dimension of the corresponding feature map;

[0038] S8.2 For each sub-feature map f i Global max pooling or global average pooling is randomly performed on the channels to obtain the global maximum spatial feature weights g′. imax Or global average spatial feature weights g′ iavg Where H×W represents the spatial dimension of the corresponding feature weights, and g′ imax ∈R H×W×1 , g′ iavg ∈R H×W×1 ;

[0039] S8.3 For the generated N spatial feature weights, perform Concat concatenation along the channel dimension to obtain the spatial feature comprehensive weight g′, where H×W represents the spatial dimension of the corresponding feature weight, and g′∈R. H×W×N N is the number of sub-feature maps after grouping, which here represents the channel dimension of the corresponding feature weight;

[0040] S8.4 For the obtained spatial feature synthesis weight g′, g′ is fed into a 3×3 convolutional layer for dimensionality reduction. To ensure that the weight g′ remains unchanged in spatial dimension, Pad=1 is used to pad the feature weight g′ around its edges. After passing through the Sigmoid activation function, the spatial attention weight M after feature fusion is obtained. s The output F is obtained by multiplying the output F with each spatial feature in the input feature map F′. f Where H×W represents the spatial dimension of the corresponding feature weights, C represents the spatial dimension of the corresponding feature map, and M represents the spatial dimension of the feature map. s ∈R H×W×1 F f ∈R H×W×C .

[0041] Furthermore, the input to the pixel classification subnetwork is the output feature map F of the enhanced feature extraction subnetwork. up It includes three fully connected layers connected in series and a sigmoid activation function layer, used to enhance the feature map F output by the feature extraction network. up The image pixels are classified and a mask is generated to obtain the final image segmentation result.

[0042] Furthermore, in step S4, the model training method is as follows:

[0043] The network parameters are trained using Focal Loss and Dice Loss to obtain the optimal segmentation network model. The Focal Loss and Dice Loss functions are shown below:

[0044]

[0045]

[0046] Among them, F Loss For Focal Loss, D Loss For Dice Loss; X represents the predicted segmentation pixels, and Y represents the actual segmentation pixels in the image; α and γ are predefined constants, where α is set according to the ratio of positive to negative samples, and γ is an adjustable factor that is set according to the difficulty of segmentation, provided γ > 0; p prediction To predict the probability that a sample belongs to this category.

[0047] The present invention has the following advantages over the prior art:

[0048] 1. The image processing network designed in this invention enhances the network's feature extraction capability through an improved multi-channel communication attention module, strengthens the network's segmentation performance for some smaller feature regions, and more accurately classifies and segments blood collection tube images, thereby improving the accuracy of the final network segmentation results.

[0049] 2. The multi-channel communication attention module designed in this invention replaces the fully connected layers in the channel feature weight reconstruction process with 1D convolutional layers of neighboring channels, enhancing the communication performance between the current channel and neighboring channels, eliminating the influence of distant channels, and reducing the impact of redundant information. This enables more accurate capture of boundary details between different blood components in blood collection tube layered images, improving the accuracy of blood layered image segmentation.

[0050] 3. The multi-channel spatial attention module designed in this invention adopts the idea of ​​grouped spatial attention, dividing the feature map into N different groups in the channel domain to obtain different sub-features. For a specific semantic group, the corresponding semantic features are generated at the correct spatial location in the original image. The spatial attention module designed in this invention performs random global max pooling or global average pooling on the grouped sub-feature maps respectively, and then concatenates them in the channel dimension. Through convolution and feature fusion, different spatial semantic feature weights are obtained. The idea of ​​grouped spatial attention enhances the learning of spatial semantic features, reduces noise and interference, and further improves the accuracy of blood collection tube layer image segmentation.

[0051] 4. The blood collection tube layered image segmentation method designed in this invention based on a multi-channel communicative attention semantic segmentation network enhances some semantic detail features compared to traditional image segmentation methods and traditional deep learning methods. In the field of blood layered image segmentation, it can more accurately grasp the detailed features of the image and obtain higher-definition and more accurate blood collection tube layered image segments. Attached Figure Description

[0052] Figure 1 This is a flowchart of blood collection vessel layer image segmentation in an embodiment of the present invention.

[0053] Figure 2 This is the specific structure of the blood collection vessel layered image segmentation network MCC-UNET in this embodiment of the invention.

[0054] Figure 3 The following is a specific structure of the attention downsampling layer and the attention upsampling layer in the embodiments of the present invention, wherein a is the attention downsampling layer and b is the attention upsampling layer.

[0055] Figure 4 The diagram shows the structure of the improved multi-channel communication attention module in this embodiment of the invention, where a is the overall structure of the improved multi-channel communication attention module, b is the structure of the multi-channel communication channel attention module, and c is the structure of the multi-channel communication spatial attention module.

[0056] Figure 5 This is a schematic diagram of 1D convolution in an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and specifically analyzed step by step below with reference to the accompanying drawings. The described embodiments are some embodiments of the present invention, but not all embodiments.

[0058] In this embodiment of the invention, MIOU (Mean Intersection over Union) is the mean intersection over union ratio, PRE (Precision) is the precision, REC (Recall) is the recall rate, and MCC (Multi-Channel Communication) is the hybrid multi-scale attention module.

[0059] Example 1:

[0060] The flowchart of blood collection tube layer image segmentation in this invention is as follows: Figure 1 As shown. In this embodiment, the dataset uses a unified machine-collected color blood layered image dataset under different modes and lighting conditions. This dataset contains 1183 color images, divided into training, validation, and test sets, with a resolution of 2448*2048. Expert-manually annotated segmentation masks are provided. During testing, the expert-manually annotated segmentation results are used as ground truth to evaluate the segmentation performance of the proposed framework. Specifically, the following steps are included:

[0061] 1. Image preprocessing and data augmentation

[0062] The color training images undergo image preprocessing to match the input of the hybrid attention segmentation network model. Since the input images for the hybrid attention segmentation network model are 512*512 square images, the images in the dataset need to be scaled and standardized. Simultaneously, to ensure the reliability of the segmentation results, this embodiment of the invention performs certain data augmentation on the test set, employing measures such as partial cropping, horizontal flipping, partial rotation, image brightness adjustment, and resolution adjustment to ensure the credibility of the network segmentation results.

[0063] 2. MCC-UNET, a network for segmenting layered images of blood collection tubes.

[0064] The blood collection tube layered image segmentation network MCC-UNET consists of a feature extraction network, an enhanced feature extraction network, and a pixel classification network cascaded together. The feature extraction network employs four downsampling operations, with each downsampling process involving three standard convolutions for feature extraction, using the ReLU activation function. The latter two downsampling operations incorporate an improved multi-channel attention module to obtain more refined attention feature maps. For example... Figure 2 As shown, in addition to regular connections, ShortCut connections are used between the feature extraction network and the enhanced feature extraction network to combine low-level detailed features at different scales with high-level semantic features.

[0065] The feature extraction network and the enhanced feature extraction network constitute the main body of the multi-channel communicative attention semantic segmentation network, and its network structure can refer to the UNET network. The feature extraction network can be used to extract features from the layered blood images of blood collection tubes and output the extracted features of the main network image. The feature extraction operations include, but are not limited to, convolution and pooling. The enhanced feature extraction network can be used to upsample the extracted features of the main network image to obtain the upsampled features of the main network image. The upsampling operations include, but are not limited to, bilinear interpolation and deconvolution.

[0066] For easier understanding, please refer to Figure 2 The backbone segmentation network based on the UNET network structure includes multiple downsampling layers and multiple upsampling layers. The image feature processing process of the blood collection vessel layer image segmentation network MCC-UNET is as follows:

[0067] The blood collection tube images to be segmented are input into the first downsampling layer. After convolution and pooling operations, the first downsampling layer P1 inputs the extracted features of the first main network image into the second downsampling layer P2. The second downsampling layer P2 performs convolution and pooling operations on the extracted features of the first main network image and inputs the extracted features of the second main network image into the third attention downsampling layer P3. And so on. The last attention downsampling layer inputs the extracted features of the main network image into the last attention upsampling layer. The last upsampling layer P5 performs bilinear interpolation based on the extracted features of the last main network image to obtain the upsampling features of the last main network image and inputs them into the previous upsampling layer P6. And so on, until the first upsampling layer P8 recovers the upsampling features of the first image.

[0068] In the feature extraction network described above, the internal features of the downsampling layer are also connected to the input of the upsampling layer in the same layer to form a ShortCut short connection structure. The upsampling layer concatenates the features along the channel dimension using the image features of the downsampling layer to semantically supplement the currently recovered upsampling features. The upsampling layer performs a convolution operation on the image features obtained after channel concatenation and inputs the obtained image features into the previous upsampling layer.

[0069] In this embodiment of the invention, with Figure 3 In Figure a, the third attention downsampling layer P3 in the feature extraction network and Figure 3 Taking the third attention upsampling layer P6 in Figure b as an example, the structure of the downsampling layer and the upsampling layer will be explained:

[0070] like Figure 3As shown in Figure a, its attention downsampling layer consists of a feature extraction layer M1, an improved multichannel communication attention module M2, a pooling layer M3, and an activation function M4.

[0071] The feature extraction layer consists of three convolutional layers with a kernel size of 3*3, a stride of 1, and padding of 1, plus a ReLU activation function layer. Its output feature map is input into the subsequent MCC attention module.

[0072] The MCC attention module consists of a multi-channel communication channel attention module and a multi-channel communication spatial attention module, and is input to the max pooling layer;

[0073] The pooling layer is a max pooling downsampling layer with a 2*2 pooling kernel. The attention-refined features are pooled and downsampled before being input into the ReLU activation function layer.

[0074] The ReLU activation function layer uses the ReLU function as the activation function to perform activation function operations on the image features output by the feature fusion layer, thereby obtaining a downsampled output feature map.

[0075] like Figure 3 As shown in Figure b, its attention upsampling layer consists of a bilinear interpolation layer M5, a concatenate layer M6, a feature reconstruction layer M7, an attention module layer M8, and a ReLU activation function layer.

[0076] The feature reconstruction layer consists of two convolutional layers with a kernel size of 1×1 and a stride of 1, and a ReLU activation function layer.

[0077] The aforementioned backbone segmentation network based on the UNET network structure has the following problems: the detailed features of the image may be lost with continuous convolution and pooling. Although ShortCut connections are used for feature supplementation and feature fusion, it is still difficult to achieve sufficient semantic supplementation, resulting in a decrease in segmentation accuracy.

[0078] To address the aforementioned problems, this invention provides an improved attention module based on multi-channel communication, such as... Figure 4 As shown in this example, the improved attention MCC module is used in the semantic segmentation network based on multi-communication channel attention.

[0079] The improved attention MCC module consists of improved multi-channel communication channel attention and improved multi-channel communication group spatial attention. This improved attention MCC module is used to obtain the network's attention refinement features based on the network's downsampling or upsampling features.

[0080] 3. Improve the structure of the attention MCC module

[0081] In this embodiment of the invention, the improved attention MCC module obtains the network's refined attention features based on the network's downsampling or upsampling features, and inputs the obtained refined attention image features into the next layer of the downsampling or upsampling network. The main network obtains the blood collection tube layered blood image segmentation result based on the refined attention image features.

[0082] In this embodiment of the invention, the working process of the attention module is explained using the third attention downsampling layer P3 in the feature extraction network as an example:

[0083] The improved multi-channel communication attention (MCC) module, as shown in Figure 4a, is composed of an improved multi-channel communication channel attention module N2 and an improved multi-channel communication grouping spatial attention module N3 connected in series.

[0084] As shown in Figure 4b, the multi-channel communication attention N2 of the module adopts the idea of ​​neighboring channel communication, and the specific process is as follows:

[0085] For the input feature map F∈R of the multi-channel communication channel attention module 128×128×256 The input feature map F is subjected to N6 global max pooling and N7 global average pooling to obtain the feature weights g. max ∈R 1 ×1×256 and g avg ∈R 1×1×256 ;

[0086] The feature weights g are achieved by performing two 1D convolutions N8 and N10 on the current channel and its seven neighboring channels. max Communication between the 7 channels and g avg The communication between the 7 channels yielded the new feature weights W after the channel communication. max ∈R 1×1×256 and W avg ∈R 1×1×256 It achieves channel communication between adjacent channels through 1D convolution, strengthens the influence of the current channel and its adjacent channels on the feature weights of the current channel, pays more attention to local small region features, and improves the problem of loss of small region target features in existing segmentation networks; the convolution kernel of 1D convolution is 1×7 in size, and in order to keep the weight dimension unchanged before and after convolution, the weight g is padded with a size of 3 around the perimeter before convolution;

[0087] For the generated feature weights W max and W avgAfter summing along the channel dimension, the input feature map F∈R is obtained by applying the Sigmoid activation function. 128×128×256 Channel attention weights M c ∈R 1×1×256 and with the input feature map F∈R 128×128×256 Dot product of each channel feature in R to obtain the channel attention feature map F′∈R 128×128×256 .

[0088] As shown in Figure 4c, the multi-channel communication group spatial attention module adopts the idea of ​​group spatial attention and randomly performs global max pooling or global average pooling on the obtained grouped features in each channel. The specific process is as follows:

[0089] For the input feature map F′∈R of multi-channel communication spatial attention 128×128×256 The input feature map F′ is subjected to a channel-based grouping operation N18, as shown in N20 in the figure, to obtain multiple sub-feature maps F with the same dimensions. i ′={f1′,f2′…f i ′,…f N ′},f i ′∈R H×W×C / N Where N represents the number of sub-feature maps after grouping, in this embodiment N = 8, i = [1, 2…8], f i ′ represents the i-th sub-feature map.

[0090] For each sub-feature map f i The spatial feature weights g′ are obtained by randomly performing global max pooling or global average pooling on the channels. imax ∈R 128×128×1 or g′ iavg ∈R 128×128×1 .

[0091] The eight generated spatial feature weights are concatenated along the channel dimension to obtain the comprehensive spatial feature weight g′∈R. 128×128×8 .

[0092] The obtained spatial feature synthesis weights g′ are fed into a 3×3 convolutional layer N22 for dimensionality reduction, and then activated by a sigmoid function to obtain the fused spatial attention weights M. s ∈R 128×128×1 and the input feature map F′∈R 128×128×256 The output F of the improved multichannel communication attention module is obtained by multiplying each spatial feature in the matrix. f ∈R 128×128×256 .

[0093] 4. Experimental training process and training parameters of the blood collection tube blood layer image segmentation network MCC-UNET.

[0094] In this embodiment, during the experimental phase, the blood collection tube blood layer image segmentation network shown in the figure was subjected to ablation experiments and comparative experiments with UNET network, PSPNET network and Deeplabv3+ network.

[0095] The experiments were conducted in a PyTorch environment. All the networks used the Adam optimizer, with an initial learning rate of 0.0001 and a decay rate of 0.93. The Dice Loss auxiliary Focal Loss function was used as the loss function for network training.

[0096]

[0097]

[0098] Among them, F Loss For Focal Loss, D Loss This represents Dice Loss. X is the model's predicted segmentation result, and Y is the actual segmentation result of the image. α and γ are predefined constants. p prediction To predict the probability that a sample belongs to this category.

[0099] To evaluate network performance, the experiment used the Mean Intersection Over Union (MIOU), Precision (PRE), and Recall (REC) as evaluation metrics, and the experimental results are shown in Table 1 below.

[0100] Table 1. Comparison of Experimental Results between Example 1 and the Old Network

[0101]

[0102] Wherein, UNET represents the UNET network, MCC-UNET represents the UNET network that uses the multi-scale hybrid attention module proposed in this invention, Deeplabv3+ represents the Deeplabv3+ network, and PSPNET represents the PSPNET network.

[0103] The specific formulas for the performance indicators are shown below:

[0104]

[0105]

[0106]

[0107]

[0108] Wherein, TP: positive sample, positive prediction result; TN: negative sample, negative prediction result; FP: negative sample, positive prediction result; FN: positive sample, negative prediction result.

[0109] Based on the data in Tables 1 and 2 above, it can be seen that in the blood collection tube layered image dataset, as... Figure 2 The MCC-UNET network for segmenting blood collection tube images, as shown, demonstrates superior network performance in both ablation experiments and comparison experiments with older networks, achieving higher image segmentation accuracy. The experiments ultimately yielded a stable and high-performance semantic segmentation network model based on multi-channel communicative attention, which was used to segment blood collection tube images.

[0110] Example 2:

[0111] Compared to Embodiment 1, this embodiment of the invention employs only an improved multi-channel AC attention module and a standard spatial attention module for training on a color blood layered image dataset.

[0112] Practice, verification, and testing. The specific steps are similar to those in Example 1, with the following differences:

[0113] Compared to the MCC attention module described in Example 1, this example uses the MCC-Channel attention module, which combines the multi-channel communication attention module from the MCC attention module with the traditional spatial attention module. The working process of the attention module is explained using the third attention downsampling layer in the feature extraction network as an example:

[0114] The module's multi-channel communication attention mechanism adopts the concept of proximity channel communication, and the specific process is as follows:

[0115] For the input feature map F2∈R of the multi-channel communication channel attention module 128×128×256 The input feature map F is subjected to spatial global max pooling and global average pooling (Avgpool) to obtain the feature weights g. max2 ∈R 1 ×1×256 and g avg2 ∈R 1×1×256 ;

[0116] For the current channel and its seven neighboring channels, perform two 1D convolutions (where the kernel of the 1D convolution is 1×7, and to keep the dimension of the weights unchanged before and after the convolution, apply a padding of size 3 around the weights g before the convolution) to realize the feature weights g. max2 Communication between the 7 channels and g avg2 The communication between the 7 channels yielded the new feature weights W after the channel communication.max2 ∈R 1×1×256 and W avg2 ∈R 1×1×256 It achieves channel communication between adjacent channels through 1D convolution, strengthens the influence of the current channel and its adjacent channels on the feature weights of the current channel, pays more attention to local small region features, and improves the problem of loss of small region target features in existing segmentation networks;

[0117] For the generated feature weights W max2 and W avg2 After summing along the channel dimension, the input feature map F2∈R is obtained by passing it through the Sigmoid activation function. 128×128×256 Channel attention weights M c2 ∈R 1×1×256 and the input feature map F2∈R 128 ×128×256 Dot product is performed on each channel feature in R to obtain the channel attention feature map F2′∈R. 128×128×256 .

[0118] The spatial attention module adopts the traditional spatial attention concept, and the specific process is as follows:

[0119] For the input feature map F2′∈R of multi-channel communication spatial attention 128×128×256 The input feature map F2′ is subjected to global max pooling or global average pooling on each channel to obtain the spatial feature weights g′. max2 ∈R 128×128×1 or g′ avg2 ∈R 128×128×1 .

[0120] The two generated spatial feature weights are concatenated along the channel dimension to obtain the comprehensive spatial feature weight g′∈R. 128×128×2 .

[0121] The obtained spatial feature synthesis weights g′ are fed into a 3×3 convolutional layer for dimensionality reduction, and then activated by a Sigmoid activation function to obtain the spatial attention weights M after feature fusion. s ∈R 128×128×1 and compared with the input feature map F2′∈R 128×128×256 The dot product of each spatial feature in the matrix is ​​performed to obtain the output F of the MCC-Channel attention module. f2 ∈R 128×128×256 .

[0122] The other structures, parameters, datasets, and training processes are the same as in Example 1. Experiments were conducted in a PyTorch environment. All networks used the Adam optimizer, with initial learning rates of 0.0001 and decay rates of 0.93. The Dice Loss-assisted Focal Loss function was used as the loss function for network training. The Mean Intersection over Union (MIOU), Precision (PRE), and Recall (REC) were used as evaluation metrics, and the experimental results are shown in Table 2 below.

[0123] Table 1. Comparison of Experimental Results between Example 2 and the Old Network

[0124]

[0125] Wherein, UNET represents the UNET network, MCC-Channel-UNET represents the UNET network that uses the MCC-Channel attention module described in this embodiment, Deeplabv3+ represents the Deeplabv3+ network, and PSPNET represents the PSPNET network.

[0126] Example 3:

[0127] Compared to Embodiment 1, this embodiment of the invention employs only an improved multi-channel communication spatial attention module and a traditional channel attention module for training, validation, and testing on a color blood layered image dataset. The specific steps are similar to Embodiment 1, with the following differences:

[0128] Compared to the MCC attention module described in Example 1, this example uses the MCC-Spatial attention module, which combines the multi-channel communication attention module from the MCC attention module with the traditional spatial attention module. The working process of the attention module is explained using the third attention downsampling layer in the feature extraction network as an example:

[0129] The module's channel attention adopts a traditional channel attention module, and the specific process is as follows:

[0130] For the input feature map F3∈R of the multi-channel communication channel attention module 128×128×256 The input feature map F is subjected to spatial global max pooling and global average pooling (Avgpool) to obtain the feature weights g. max3 ∈R 1 ×1×256 and g avg3 ∈R 1×1×256 ;

[0131] The two feature weights g are obtained max3 and g avg3The feature weights are reconstructed by inputting a common three-layer fully connected layer, resulting in new feature weights W after channel exchange. max3 ∈R 1×1×256 and W avg3 ∈R 1×1×256 ;

[0132] For the generated feature weights W max3 and W avg3 After summing along the channel dimension, the input feature map F3∈R is obtained by passing it through the Sigmoid activation function. 128×128×256 Channel attention weights M c3 ∈R 1×1×256 and the input feature map F3∈R 128 ×128×256 Dot product is performed on each channel feature in R to obtain the channel attention feature map F3′∈R. 128×128×256 .

[0133] The multi-channel communication spatial attention module of the module adopts the idea of ​​grouped spatial attention, and randomly performs global max pooling or global average pooling on the obtained grouped features on the channels. The specific process is as follows:

[0134] For the input feature map F3′∈R of multi-channel communication spatial attention 128×128×256 The input feature map F′ is grouped based on channels to obtain multiple sub-feature maps F′ with the same dimensions. 3i ={f′ 31 ,f′ 32 …f′ 3i ,…f′ 3N}, f′ 3i ∈R H×W×C / N Where N represents the number of sub-feature maps after grouping, in this embodiment N = 8, i = [1, 2…8], f′ 3i Let i represent the i-th sub-feature map.

[0135] For each sub-feature map f′ 3i Global max pooling or global average pooling is randomly performed on the channels to obtain the spatial feature weights g′. 3imax ∈R 128×128×1 or g′ 3iavg ∈R 128×128×1 .

[0136] The eight generated spatial feature weights are concatenated along the channel dimension to obtain the comprehensive spatial feature weight g′. 3i ∈R 128×128×8 .

[0137] The obtained spatial feature comprehensive weight g′ 3i , will g′ 3iThe data is fed into a 3×3 convolutional layer for dimensionality reduction, and then activated by a sigmoid function to obtain the spatial attention weights M after feature fusion. s3 ∈R 128×128×1 and the input feature map F′3∈R 128×128×256 The output F of the improved multichannel communication attention module is obtained by multiplying each spatial feature in the matrix. f3 ∈R 128×128×256 .

[0138] The other structures, parameters, datasets, and training processes are the same as in Example 1. Experiments were conducted in a PyTorch environment. All networks used the Adam optimizer, with an initial learning rate of 0.0001 and a decay rate of 0.93. The Dice Loss-assisted Focal Loss function was used as the loss function for network training. The Mean Intersection over Union (MIOU), Precision (PRE), and Recall (REC) were used as evaluation metrics, and the experimental results are shown in Table 3 below.

[0139] Table 3. Comparison of Experimental Results between Example 3 and the Old Network

[0140]

[0141] Wherein, UNET represents the UNET network, MCC-Spatial-UNET represents the UNET network that uses the MCC-Spatial attention module described in this embodiment, Deeplabv3+ represents the Deeplabv3+ network, and PSPNET represents the PSPNET network.

[0142] Based on the above three embodiments, the comparative experimental results among the three embodiments are shown in Table 4 below.

[0143] Table 4. Comparative experimental conclusions among the three examples

[0144]

[0145] Experiments have demonstrated that the blood collection tube layered image segmentation method based on a multi-channel communicative attention semantic segmentation network proposed in this invention has advanced performance in the field of blood collection tube layered blood image processing. Compared with traditional image processing methods that struggle to segment layered blood images with high accuracy, the blood collection tube layered image segmentation method based on a multi-channel communicative attention semantic segmentation network proposed in this invention achieves higher segmentation accuracy.

[0146] Experiments have demonstrated that, compared to the visual recognition system for blood collection tubes proposed in patent CN202210279022.0 and the blood layer recognition method proposed in patent CN201610390622.9, the blood layer image segmentation method for blood collection tubes based on a multi-channel communicative attention semantic segmentation network proposed in this invention has higher accuracy and more complete automation characteristics. It does not require cumbersome steps and is simpler and more convenient to operate, making it advanced in this field.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A blood layered image segmentation method based on a multi-channel alternating attention network, characterized in that, The method comprises the following steps: S1, collecting original blood collection tube blood layering image data; S2, data preprocessing is performed on the collected original blood collection tube blood layering image; S3, a blood collection tube blood layering image segmentation network MCC-UNET based on an improved multi-channel cross-attention module is built; the blood collection tube blood layering image segmentation network MCC-UNET based on the improved multi-channel cross-attention module comprises a feature extraction subnetwork, a strengthened feature extraction subnetwork and a pixel classification subnetwork connected in sequence; The feature extraction subnetwork comprises a first downsampling layer, a second downsampling layer, a first multi-channel cross-attention downsampling layer and a second multi-channel cross-attention downsampling layer connected in sequence, wherein the first multi-channel cross-attention downsampling layer and the second multi-channel cross-attention downsampling layer are obtained by connecting an improved multi-channel cross-attention module in series before a max-pooling layer of the downsampling layer; The strengthened feature extraction subnetwork comprises a first multi-channel cross-attention upsampling layer, a second multi-channel cross-attention upsampling layer, a first upsampling layer and a second upsampling layer connected in sequence, wherein the first multi-channel cross-attention upsampling layer and the second multi-channel cross-attention upsampling layer are obtained by connecting an improved multi-channel cross-attention module in series before an activation function layer of the upsampling layer; In addition, there is a ShortCut short connection mode between the feature extraction subnetwork and the strengthened feature extraction subnetwork, and the specific connection mode is as follows: The first downsampling layer is connected to the second upsampling layer; The second downsampling layer is connected to the first upsampling layer; The first multi-channel cross-attention downsampling layer is connected to the second multi-channel cross-attention upsampling layer; The second multi-channel cross-attention downsampling layer is connected to the first multi-channel cross-attention upsampling layer; S4, the preprocessed blood collection tube blood layering image data is imported into the MCC-UNET, the network parameters are trained using a loss function Focal Loss and a Dice Loss, and an optimal segmentation network model is obtained; S5, for the obtained optimal segmentation network model, a new image is collected, standardized processing is performed, and then the optimal segmentation network model is inputted to perform blood layering image segmentation, so that a segmentation result of the blood collection tube blood layering image is obtained.

2. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to claim 1, characterized in that, In step S1, color blood layering images from different light environments are collected as original blood collection tube blood layering images, and the original blood collection tube blood layering images are manually segmented and mask annotated.

3. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to claim 1, characterized in that, In step S2, the data preprocessing is specifically as follows: The original blood collection tube blood layering images are preprocessed through size standardization and partial cropping, horizontal flipping, partial rotation, image brightness and image contrast data enhancement operations, and the preprocessed original blood collection tube blood layering images are divided into a training set, a verification set and a test set; The training set is used to train the parameters in the blood vessel blood layering image segmentation network; the verification set is used to verify the performance of the optimal segmentation network model in the training, adjust the hyperparameters of the optimal segmentation network model and preliminarily evaluate the ability of the optimal segmentation network model; and the test set is used to objectively evaluate the performance of the trained blood vessel blood layering image segmentation network.

4. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to claim 1, characterized in that, The feature extraction subnetwork is used for extracting semantic features of the image, and the input is the blood vessel blood layering original image F ori , and the output is a feature map F down , F down is the output of the second multi-channel alternating attention down-sampling layer, and the output of the feature extraction subnetwork is input into the enhanced feature extraction subnetwork, wherein F ori ∈R H×W×3 , F down ∈R H / 16×W / 16×C1 , R represents the overall size of the image, HxW is the size of the original input image, and C1 is the number of channels of the feature map after four times of down-sampling; The input of the enhanced feature extraction subnetwork is the output feature map F of the feature extraction subnetwork down , and the output is the feature map F up , the feature map F up is the output feature map of the second up-sampling layer, the enhanced feature extraction subnetwork is used for reconstructing image semantic features, and the obtained feature map F up is output to the pixel classification subnetwork, wherein, F up ∈R H×W×C2 , HxW is the size of the original input image, and C2 is the channel number of the feature map after four times of up-sampling; In the ShortCut short connection between the feature extraction sub-network and the enhanced feature extraction sub-network, the first down-sampling layer, the second down-sampling layer, the first multi-channel alternating attention down-sampling layer and the second multi-channel alternating attention down-sampling layer respectively input the down-sampling feature maps extracted by the layer into the second up-sampling layer, the first up-sampling layer, the second multi-channel alternating attention up-sampling layer and the first multi-channel alternating attention up-sampling layer respectively, to supplement the features, thereby reducing the feature loss in the up-sampling process and making the network obtain better performance.

5. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to claim 1, characterized in that, The improved multi-channel alternating attention module is used in the first multi-channel alternating attention up-sampling layer, the second multi-channel alternating attention up-sampling layer, the first multi-channel alternating attention down-sampling layer and the second multi-channel alternating attention down-sampling layer; The improved multi-channel alternating attention module includes an improved multi-channel alternating channel attention module and an improved multi-channel alternating group spatial attention module; The improved multi-channel alternating channel attention module and the improved multi-channel alternating group spatial attention module are used in the first multi-channel alternating attention up-sampling layer, the second multi-channel alternating attention up-sampling layer, the first multi-channel alternating attention down-sampling layer and the second multi-channel alternating attention down-sampling layer, so that the image segmentation model pays more attention to the small region features that need to be segmented in the image and eliminates the local noise and interference of the image.

6. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to claim 5, characterized in that, The improved multi-channel alternating channel attention module adopts the idea of alternating channels of adjacent channels, and the specific process is as follows: S7.1 For the input feature map F of the multi-channel alternating attention module, the input feature map F is respectively subjected to spatial global maximum pooling Maxpool and global average pooling Avgpool to obtain maximum pooling weights g max and average pooling weights g avg , wherein F ∈ R H×W×C , g max ∈ R 1×1×C , g avg ∈ R 1×1×C , H × W and 1 × 1 represent the spatial dimensions of the corresponding feature map, and C represents the channel dimension of the corresponding feature map; S7.2 1D convolution on the current channel and its adjacent k-1 channels to realize the maximum pooling weight g mac the exchange between the k channels and the average pooling weight g avg the exchange between the k channels, and obtain the global maximum channel feature weight W after the channel exchange mac and the global average channel feature weight W avg , realize the channel exchange between the adjacent channels through 1D convolution, strengthen the influence of the current channel and its adjacent channels on the current channel feature weight, pay more attention to the local small area features, and improve the problem of small area target feature loss of the existing segmentation network, wherein, W max ∈R 1×1×C , W avg ∈R 1×1×C , C represents the spatial dimension of the corresponding feature weight, the convolution kernel of the 1D convolution is 1*k, k<C and k is a positive integer, in order to keep the weight dimension C unchanged before and after the convolution, the weight g is padded around the edge in the size of k / 2 is rounded down; S7.3 For the generated feature weight W max and W avg After adding in the channel dimension, pass through the Sigmoid activation function, get the channel attention weight M of the input feature map F c , and point multiply with each channel feature in the input feature map F, get the channel attention feature map F′ as the input feature map of the multi-channel communication group spatial attention module, where M c ∈R 1×1×C , F′∈R H×W×C .

7. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to claim 5, characterized in that, The improved multi-channel alternating group spatial attention module adopts the idea of group spatial attention, and randomly performs global maximum pooling or global average pooling on the obtained group features in the channel, and the specific process is as follows: S8.1 For the input feature map F' of multi-channel attention space, the input feature map F' is subjected to channel-based grouping operation to obtain a plurality of sub-feature maps F' of the same dimension i = {f'1, f'2…f' i ,…f' N}, f i '∈R H×W×C / N , wherein N represents the number of sub-feature maps after grouping, i = [1, 2…N], f i ' represents the i-th sub-feature map, HxW represents the spatial dimension of the corresponding feature map, and C represents the channel dimension of the corresponding feature map; S8.2 For each sub-feature map f i global max pooling or global average pooling is performed randomly on the channel to obtain the global maximum spatial feature weight g′ imax or the global average spatial feature weight g′ iavg , wherein HxW represents the spatial dimension of the corresponding feature weight, g′ imax ∈R H×W×1 , g′ iavg ∈R H×W×1 ; S8.3 For the generated N spatial feature weights, Concat splicing is performed in the channel dimension to obtain a spatial feature comprehensive weight g', wherein HxW represents the spatial dimension of the corresponding feature weight, g'∈R H×W×N , N is the number of sub-feature maps after grouping, and herein represents the channel dimension of the corresponding feature weight. S8.4 For the obtained spatial feature comprehensive weight g', g' is sent to a convolution layer with a convolution kernel of 3*3 for dimension reduction operation, wherein, in order to ensure that the weight g' is unchanged in the spatial dimension, the feature weight g' is padded around by Pad=1, and then the spatial attention weight M after feature fusion is obtained through a Sigmoid activation function s , and is multiplied with each spatial feature in the input feature map F' to obtain the output F of the improved multi-channel cross-attention module f , wherein H*W represents the spatial dimension of the corresponding feature weight, C represents the spatial dimension of the corresponding feature map, M s ∈R H×W×1 , F f ∈R H×W×C .

8. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to claim 3, characterized in that, The input of the pixel classification subnetwork is the output feature map F of the enhanced feature extraction subnetwork up , including three fully connected layers and a sigmoid activation function layer connected in series, for converting the feature map F output by the enhanced feature extraction network up to perform image pixel classification and mask generation, and finally obtain the image segmentation result.

9. The blood hierarchical image segmentation method based on the multi-channel alternating attention network according to any one of claims 1-8, characterized in that, In step S4, the model training method is as follows: The network parameters are trained using the loss functions Focal Loss and Dice Loss to obtain the optimal segmentation network model, and the loss functions Focal Loss and Dice Loss are as follows: F Loss = -a(1 - p prediction ) γ log(p prediction ) # (1) wherein F Loss is Focal Loss, D Loss is Dice Loss; X is the model predicted segmentation pixel point, Y is the image actual segmentation pixel point; a and g are set constants, wherein a is set according to the proportion of positive and negative samples, and g is an adjustable factor, which is set according to the difficulty of segmentation under the premise of g>0; p prediction is the probability that the predicted sample belongs to this kind of sample.

Images