A medical image segmentation method based on a multi-scale cross-layer attention fusion network

Abstract

The application discloses a medical image segmentation method based on a multi-scale cross-layer attention fusion network and belongs to the technical field of image processing. The method comprises the following steps: S100, collecting a lung computer tomography image with honeycomb lung disease and processing the lung computer tomography image to generate a honeycomb lung dataset; S200, constructing a multi-scale cross-layer attention fusion network model by using a UNeXt network framework; S300, based on the honeycomb lung dataset, jointly training the multi-scale cross-layer attention fusion network model by using a binary classification cross-entropy loss function BCE loss and an image segmentation task loss function Dice loss; and S400, inputting a medical image to be segmented into the trained multi-scale cross-layer attention fusion network model to obtain a segmentation result. The application can obtain more semantic information to improve the target segmentation precision and the calculation efficiency in medical images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a medical image segmentation method based on a multi-scale cross-layer attention fusion network. Background Technology

[0002] Honeycomb lung is the end-stage manifestation of various lung diseases, characterized by lung tissue destruction and fibrosis, containing numerous thick-walled cystic cavities and complete loss of pulmonary acinar structure. The etiology of honeycomb lung is nonspecific and can be caused by various diseases, such as idiopathic interstitial pneumonia, diffuse alveolar damage, asbestosis, interstitial granuloma, and eosinophilic granuloma. A more typical cause of honeycomb lung is interstitial lung disease (ILD), whose prevalence is increasing annually due to environmental degradation. This disease has a long course, high mortality rate, poor clinical recovery, and a 5-year survival rate of less than 50%.

[0003] The pathological manifestations of interstitial lung disease are early alveolar wall inflammation, intermediate diffuse interstitial fibrosis, and late alveolar wall fibrosis. Chest X-rays, especially high-resolution CT scans, can reveal bilateral, dense, fine reticular nodules, particularly in the lower lobes. Currently, early diagnosis of honeycomb lung often relies on computed tomography (CT), but CT images contain indistinct areas that are difficult to see with the naked eye, and the assessment of honeycomb lesions is heavily influenced by the physician's subjective experience, resulting in lengthy diagnostic times. Currently, few algorithms have been proposed for the automatic segmentation of honeycomb lesions in CT images. Therefore, an effective solution is to develop an algorithm for automatically segmenting honeycomb lesions in CT images to help physicians accurately locate the lesion area for further diagnosis.

[0004] In existing technologies, machine learning and deep learning methods based on convolutional neural networks have contributed to computer-aided diagnosis to some extent. Traditional machine learning methods include thresholding, edge detection, clustering, and variable model-based segmentation. However, traditional machine learning-based medical image segmentation methods are only effective when the target and background are significantly different, and they are sensitive to noise and prone to oversegmentation. With the rapid development of deep learning, many deep learning-based methods have been developed for medical image segmentation, especially convolutional neural networks, which can automatically learn feature representations of the target, thus producing faster and more accurate segmentation results. U-Net is one of the most widely used networks in medical image segmentation. U-Net employs an encoder-decoder structure, incorporating shrinking and expanding paths composed of repeated convolutions, and fusing features from corresponding layers in the encoder and decoder through skip connections. Due to U-Net's excellent performance in medical image segmentation, many U-Net-based segmentation methods have been proposed, including U-Net++, R2U-Net, and TransU-Net. U-Net++ uses dense connections in U-Net to provide the decoder with correct semantic and coarse-level information. Furthermore, U-Net++ collects segmentation maps in the last layer of the sub-network and upsamples them to the same size as the input image for depth supervision. While this network can obtain information at different scales through dense connections, it ignores the channel and spatial information between pixels in medical images. R2U-Net is another variant of U-Net, which mainly uses the structures in residual networks and recurrent networks to improve U-Net. This network uses residual connections to solve the gradient vanishing problem and uses recurrent convolutional blocks to learn contextual information. However, due to the limitations of convolution, this network cannot obtain the global contextual information of the image. TransU-Net adds VisionTransformer to U-Net, overcoming the limitation that convolutional operations can only extract local information and effectively learns global spatial dependencies in the encoder part. However, this network directly adds VIT to the encoder part of U-Net, which creates a semantic gap between convolutional blocks and VIT. Moreover, the simple stacking of Transformer blocks significantly increases the number of network parameters and reduces computational efficiency.

[0005] Therefore, how to provide a medical image segmentation method based on a multi-scale cross-layer attention fusion network that can obtain more semantic information to improve the accuracy and computational efficiency of target segmentation is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a medical image segmentation method based on a multi-scale cross-layer attention fusion network to solve the technical problems existing in the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A medical image segmentation method based on a multi-scale cross-layer attention fusion network includes the following steps:

[0009] S100: Collect computed tomography (CT) images of the lungs with celluloid lung disease, and process the CT images of the lungs to generate a celluloid lung dataset;

[0010] S200: Constructing a multi-scale, cross-layer attention fusion network model using the UNeXt network framework;

[0011] S300: Based on the aforementioned cellular lung dataset, a multi-scale cross-layer attention fusion network model is constructed by jointly training the binary cross-entropy loss function (BCE loss) and the image segmentation task loss function (Dice loss).

[0012] S400: Input the medical image to be segmented into the trained multi-scale cross-layer attention fusion network model to obtain the segmentation result.

[0013] Preferably, step S100 involves: collecting computed tomography (CT) images of the lungs with cellulitis, and processing the CT images to generate a cellulitis dataset, including:

[0014] S110: Acquire computed tomography images of lungs with honeycomb lung disease from the same CT scanner;

[0015] S120: Adjusts the resolution of lung computed tomography images and uses a mask to remove the patient's private information from the images;

[0016] S130: The RGB color mask of the lung computed tomography image is then converted into a binary image mask to obtain a binarized mask image and generate a cellular lung dataset.

[0017] S140: Perform color transformation and add random noise to obtain the expanded cellular lung dataset.

[0018] Preferably, the multi-scale cross-layer attention fusion network model includes: a multi-scale input module, a cross-layer attention fusion module, and a bidirectional attention gating module.

[0019] Preferably, the multi-scale input module is introduced into the encoder of the UNeXt network framework. The multi-scale input module is used to adjust the size of the lung computed tomography image and input it into the corresponding encoder layer for fusion with the corresponding feature map.

[0020] Preferably, the corresponding feature map refers to the feature map of the encoder part at each layer that is the same size as the input image. After fusion, the negative impact of losing image details after multiple convolution operations can be mitigated.

[0021] Preferably, the cross-layer attention fusion module includes a spatial attention submodule and a channel attention submodule. The spatial attention submodule receives shallow features from adjacent convolutional layers in the encoder and receives deep features from the channel attention submodule. The feature map output by the channel attention submodule is upsampled and concatenated with the result of the spatial attention submodule. The concatenated result is then subjected to a double convolution. A residual connection is used to add the original feature map to the feature map generated after the double convolution. The final result is obtained through a non-linear activation function. The calculation process is as follows:

[0022] Out = ReLU(C2(CAT(SA(F)) x ),C1(CA(F x+1 ))))+F x );

[0023] In the formula, F x For shallow features, F x+1 For deep features, SA represents the spatial attention module, CA represents the channel attention module, CAT represents the connection operation, and C1 and C2 are 2*2 transposed convolution and double convolution blocks used for upsampling, respectively.

[0024] Preferably, shallow features are the features output by the higher layer of two adjacent coding layers; deep features are the features output by the lower layer of two adjacent coding layers.

[0025] Preferably, the spatial attention module obtains the maximum and average values ​​of the channels respectively, and calculates the attention value at the spatial level. The calculation process is as follows:

[0026] S out =F x *σ(C(Cmax(F x )+Cave(F x )));

[0027] In the formula, Sout is the output of the spatial attention module, σ is the sigmoid activation function, Cmax and Cave represent the maximum and average values ​​obtained along the channel direction, respectively, and C represents the convolutional layer.

[0028] Preferably, the calculation process of the channel attention module is as follows:

[0029]

[0030] In the formula, Cout is the output of the CA module, σ is the Sigmoid activation function, and C represents the convolutional layer. This represents a non-linear activation function, where AP and MP represent adaptive average pooling and adaptive max pooling, respectively.

[0031] Preferably, the bidirectional attention gating module includes forward attention calculation and reverse attention calculation;

[0032] The positive attention calculation involves performing a Sigmoid activation function on the feature map input from the decoder layer of the UNeXt network framework, multiplying each pixel by the output of the corresponding cross-layer attention fusion module, and feeding the result into a convolutional block consisting of four convolutional layers. The kernel sizes of each convolutional layer in the convolutional block are 1×1, 3×3, 3×3, and 1×1, respectively. Each convolutional layer is followed by a batch normalization operation and a non-linear activation function.

[0033] The reverse attention calculation is performed by performing the reverse operation after calculating the Sigmoid activation function on the feature map;

[0034] The results obtained from the two attention calculations are concatenated and then input into a convolutional layer for feature fusion. The calculation process is as follows:

[0035]

[0036] C3 and C4 are convolutional blocks consisting of four 1×1 and 3×3 convolutional layers, and two 3×3 convolutional layers, respectively. CAT represents the connection operation, and σ is the sigmoid activation function. This indicates the reverse operation.

[0037] Preferably, S300: a multi-scale cross-layer attention fusion network model jointly trained using the binary cross-entropy loss function (BCE loss) and the image segmentation task loss function (Dice loss), comprising:

[0038] The formula for calculating the binary cross-entropy loss function (BCE loss) is as follows:

[0039]

[0040] The Dice loss function for image segmentation tasks is calculated using the following formula:

[0041]

[0042] In the formula, n is the training batch size, N is the number of pixels, and p i It is the predicted segmentation result, t i These are real labels;

[0043] The total loss function is calculated as follows:

[0044] L=αL BCE +βL Dice ;

[0045] In the formula, α and β are hyperparameters that adjust the two loss weights.

[0046] Preferably, the method further includes evaluating the multi-scale cross-layer attention fusion network, specifically through the following steps:

[0047] The Intersection over Union (IoU) ratio between the predicted and real regions obtained by the multi-scale cross-layer attention fusion network is calculated using the following formula:

[0048]

[0049] The Dice function calculates the similarity between two sets, with values ​​ranging from [0,1]. The formula is as follows:

[0050]

[0051] The two sets mentioned above refer to the Dice function, which can calculate the similarity between any two sets.

[0052] The proportion (SPE) of all correctly predicted negative samples out of all actual negative samples is calculated using the following formula:

[0053]

[0054] The recall rate (SEN) of correctly retrieved positive samples out of all actual positive samples is calculated using the following formula:

[0055]

[0056] The ratio of correct results to all correct results is calculated using the following formula:

[0057]

[0058] In the formula, TP represents the number of true positives, TN represents the number of true negatives, FP represents the number of false positives, and FN represents the number of false negatives. These numbers are obtained by comparing the true labels with the obtained results pixel by pixel.

[0059] As can be seen from the above technical solution, compared with the prior art, this invention discloses a medical image segmentation method based on a multi-scale cross-layer attention fusion network. Using UNeXt as the backbone network, a multi-scale cross-layer attention fusion network model for segmenting cellular lung lesions in CT images is constructed. This model incorporates a multi-scale input module, a CAF module, and a BAG module to obtain as much semantic information as possible, thereby improving target segmentation accuracy and computational efficiency. Specific beneficial effects are as follows:

[0060] 1) Introduce a multi-scale input module into the encoder to reduce spatial information loss after each pooling operation;

[0061] 2) A bidirectional attention gating module was constructed and added to the skip connection, which can better fuse semantic information in the encoder and decoder and reduce the semantic gap;

[0062] 3) A cross-layer multi-attention fusion module was constructed, which can better extract and fuse semantic information in different coding layers, and further learn the spatial relationship between image channels and features.

[0063] 4) Collect computed tomography images of the lungs with celluloid lung disease, and process the computed tomography images of the lungs to generate a celluloid lung dataset, which is an enhanced dataset that can prevent overfitting of the multi-scale cross-layer attention fusion network and improve robustness. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0065] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0066] Figure 2 This is a schematic diagram of the overall structure of MCAFNet (Multi-Scale Cross-Layer Attention Fusion Network Model) of the present invention;

[0067] Figure 3 This is a schematic diagram of the dual convolution module structure of the present invention;

[0068] Figure 4 This is a schematic diagram of the third multi-scale input aggregation module of the present invention;

[0069] Figure 5(a) is a schematic diagram of the cross-layer attention fusion module structure of the present invention;

[0070] Figure 5(b) is a schematic diagram of the channel attention submodule structure of the present invention;

[0071] Figure 5(c) is a schematic diagram of the spatial attention submodule structure of the present invention;

[0072] Figure 6 This is a schematic diagram of the bidirectional attention gating module structure of the present invention;

[0073] Figure 7(a) shows the original computed tomography image of the lung with cellulopathy after de-privacy processing;

[0074] Figure 7(b) shows a labeled computed tomographic image of the lungs;

[0075] Figure 7(c) shows the binarized mask image;

[0076] Figure 8 A comparison chart of ablation experiment results with different module combinations;

[0077] Figure 9 The effect of different values ​​of α and β in the loss function on the reduction of loss;

[0078] Figure 10 This is a comparison of the segmentation performance of the Multi-Scale Cross-Layer Attention Fusion Network (MCAFNet) of this invention with other segmentation models;

[0079] Figure 11 This is a qualitative visualization comparison of the Multi-Scale Cross-Layer Attention Fusion Network (MCAFNet) of this invention with other models;

[0080] Figure 12 This is a comparison chart of the polyp segmentation visualization results of the Multi-Scale Cross-Layer Attention Fusion Network (MCAFNet) of this invention with other models. Detailed Implementation

[0081] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0082] Accurate segmentation of cellular lung lesions from lung CT images plays a crucial role in the diagnosis and treatment of various lung diseases, but currently, algorithms for automatically segmenting cellular lung lesions are still lacking. Considering the shape specificity of cellular lung lesions and their similarity to surrounding vascular shadows, the purpose of this invention is to provide a medical image segmentation method based on a multi-scale cross-layer attention fusion network. Based on the extended UNeXt network framework of this application, a multi-scale cross-layer attention fusion network model is constructed. First, a multi-scale aggregation (MIA) module is introduced into the input part of the multi-scale cross-layer attention fusion network model to reduce spatial information loss during downsampling. Second, a cross-layer attention fusion (CAF) module is added to obtain multi-scale features by fusing channel and spatial information from feature maps of different layers. Finally, a bidirectional attention gate (BAG) module is constructed in the skip connections to enhance the model's focus on the segmentation target by filtering background information. Experimental results show that the proposed medical image segmentation method based on a multi-scale cross-layer attention fusion network achieves IoU, mIoU, and mDice of 0.895, 0.921, and 0.949, respectively, on the cellular lung segmentation dataset, demonstrating superior performance compared to existing medical image segmentation algorithms. Furthermore, experiments on other datasets demonstrate the generalization and robustness of the proposed model.

[0083] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0084] Example 1

[0085] This invention discloses a medical image segmentation method based on a multi-scale cross-layer attention fusion network. See [link to relevant documentation]. Figure 1 As shown, it includes the following steps:

[0086] S100: Collect computed tomography (CT) images of the lungs with celluloid lung disease, and process the CT images to generate a celluloid lung dataset.

[0087] S200: Constructing a multi-scale, cross-layer attention fusion network model using the UNeXt network framework;

[0088] S300: A multi-scale cross-layer attention fusion network model based on the cellular lung dataset, which is jointly trained using the binary cross-entropy loss function (BCE loss) and the image segmentation task loss function (Dice loss).

[0089] S400: Input the medical image to be segmented into the trained multi-scale cross-layer attention fusion network model to obtain the segmentation result.

[0090] Specifically, UNeX is the first fast medical image segmentation network based on convolution and MLP, and it is an encoder-decoder architecture. This invention uses UNeXt as the backbone network and proposes a multi-scale cross-layer attention fusion network model for segmenting cellular lung lesions in CT images, named MCAFNet. It introduces a multi-scale input (MIA) module, a cross-layer attention fusion (CAF) module, and a bidirectional attention gating (BAG) module to obtain as much semantic information as possible to improve target segmentation accuracy and computational efficiency.

[0091] In one specific embodiment, see Figure 2 The diagram shows the overall structure of MCAFNet (Multi-Scale Cross-Layer Attention Fusion Network Model) constructed in this invention. MCAFNet adopts an encoder-decoder architecture. The first three layers of the encoder use double convolutional blocks for feature extraction, and the last two layers use Tokenized MLP for feature extraction. The decoder uses the same modules for the corresponding layers of the encoder. The proposed Cross-Layer Attention Fusion (CAF) module and Bidirectional Attention Gating (BAG) module are used in skip connections to combine with skip connections to achieve information reuse and semantic information transfer.

[0092] Specifically, in the input section, a multi-scale input aggregation (MIA) module is introduced to reduce the loss of spatial information during encoder downsampling. This module resizes the original input CT image and then inputs it into the corresponding encoder layer for fusion with its feature map. The number of encoder channels is set from high to low to 32, 64, 128, 256, and 512, respectively. The first three encoder units use two 3×3 convolutions for feature extraction. Each convolutional layer is followed by a batch normalization (BN) and a non-linear activation function (ReLU), referred to as a double convolution module, as shown below. Figure 3 As shown. After repeated experiments, the last two encoder units retained the Tokenized MLP from UNeXt. The Shifted MLP first labels the channel axes of the convolutional features before serialization, which helps the MLP focus only on certain locations of the convolutional features, thus better extracting local information. The corresponding layers of the decoder are the same as those of the encoder. Finally, a 3×3 convolution is used for pixel-by-pixel classification, and the image resolution is adjusted to match the input size to obtain the final segmentation result.

[0093] Specifically, this invention significantly improves upon skip connections. Previous skip connections simply concatenated feature maps from the encoder with the corresponding layers of the decoder, creating significant semantic gaps between features and hindering feature reuse. Therefore, this invention proposes a Cross-Layer Attention Fusion (CAF) module, which inputs feature maps from two adjacent layers of the encoder to obtain the output. This output, along with upsampled features from the decoder, is then fed into a constructed Bidirectional Attention Gating (BAG) module for further feature filtering, focusing the model on the target region. Finally, the output of the BAG module is fed into the corresponding layer of the decoder.

[0094] In one specific embodiment, many deep learning models have achieved end-to-end segmentation. These models mostly use a single original image as input, which is then processed by the network to obtain the final segmentation result. However, with continuous convolution and pooling operations, detailed image information may be lost and difficult to recover in the decoder, thus affecting segmentation accuracy. Especially for segmentation tasks involving lesions like cellular lung lesions, whose features in CT images are similar to some pulmonary vascular shadows, the encoder downsampling operation may lose some lesion feature information, causing the network to be unable to distinguish between lesions and vascular shadows. To address this issue, this invention also proposes a multi-scale input aggregation (MIA) module.

[0095] Specifically, in the Multi-Scale Input Aggregation (MIA) module, the original image is first resized into four images with resolutions of 1 / 2, 1 / 4, 1 / 8, and 1 / 16. Since the number of channels in each feature map layer is different, each image is processed through a different number of convolutional layers to adjust to the corresponding layer's channel count. These images are then concatenated with the feature maps from the encoder and fed into a double convolutional module for further feature extraction and fusion. The final input is then fed into the next encoding layer. For example, the module structure of the third multi-scale input aggregation module is shown below. Figure 4 As shown in the diagram, this module uses three convolutional blocks to progressively increase the number of image channels to 32, 64, and 128, then concatenates them with feature maps from the corresponding layers. At this point, the number of image channels doubles to 256. This is then fed into a dual convolutional module, adjusting the number of channels to 128 and further fusing feature information from the input image and feature maps. Introducing the MIA module helps reduce feature loss caused by convolution and downsampling operations in the encoder. Simultaneously, it enhances local features of cellular lung lesions, further improving segmentation accuracy.

[0096] In one specific embodiment, after deep learning methods were introduced into the field of medical image segmentation, convolution and pooling operations have become essential basic structures for every model. These operations can simultaneously extract features, reduce image resolution, and increase the number of channels. Therefore, in adjacent layers, the feature map of the shallower layer will contain more spatial information, while the feature map of the deeper layer will contain more channel information. To fuse these two feature maps and obtain more information, a novel cross-layer attention fusion (CAF) module is proposed, see [link to details]. Figures 5(a)-5(c) As shown, the shallower feature F in two adjacent layers X The data is fed into the spatial attention module, where deeper features F are processed. X+1 The input is fed into the Channel Attention (CA) module. In the diagram, AMP represents Adaptive Max Pooling, AAP represents Adaptive Average Pooling, CAverage represents Channel Average Pooling, and CMax represents Channel Max Pooling. This is represented as splicing. This indicates pixel-by-pixel addition. This indicates multiplication. Since the resolution of FX+1 is half that of FX, the feature map output by CA needs to be upsampled and concatenated with the result of spatial attention output; subsequently, the concatenated result is subjected to a double convolution operation. Considering the negative impact of attention calculation and convolution operation on the original features, and in order to prevent gradient vanishing and accelerate convergence, residual connections are used to multiply the original feature map F. X The result is added to the sum of the previous and current results, and finally passed through a non-linear activation function to obtain the final result. The specific process can be represented as follows:

[0097] Out = ReLU(C2(CAT(SA(F)) x ),C1(CA(F x+1 ))))+F x )#(1)

[0098] In the formula, SA represents the spatial attention module, CA represents the channel attention module, CAT represents the connection operation, and C1 and C2 are the 2*2 transposed convolution and double convolution modules used for upsampling, respectively.

[0099] Specifically, shallow features are the features output by the higher layer of two adjacent coding layers; deep features are the features output by the lower layer of two adjacent coding layers.

[0100] In spatial attention, the maximum and average values ​​of each channel are obtained, and then the attention value is calculated at the spatial level to make the network focus more on the target region. The calculation process of SA is as follows:

[0101] S out =F x *σ(C(Cmax(F x )+Cave(Fx )))#(2)

[0102] In the formula, Sout is the output of the SA module, σ is the Sigmoid activation function, Cmax and Cave represent the maximum and average values ​​obtained along the channel direction, respectively, and C is a convolutional layer with a kernel size of 7*7.

[0103] Channel attention involves searching for noteworthy feature maps within a channel and then performing further feature extraction operations on them. The calculation process for CA is as follows:

[0104]

[0105] C out This is the output of the CA module, where σ is the Sigmoid activation function, and C represents a convolutional layer with a kernel size of 1*1. This represents a non-linear activation function, where AP and MP represent adaptive average pooling and adaptive max pooling, respectively.

[0106] Specifically, the feature map refers to the feature map of the same size as the input image at each layer of the encoder. Fusion can mitigate the negative impact of losing image details after multiple convolution operations.

[0107] To improve segmentation accuracy and reduce false detections of small objects with highly variable shapes, Attention U-Net introduced an attention gate for image analysis for the first time in UNet. In the segmentation of cellular lung lesions, distinguishing between the target and the background is particularly important. Therefore, a BAG-Bidirectional Attention Gate module was constructed to calculate the attention weights for the target and background separately. The specific structure is shown in [link to BAG module]. Figure 6 First, the feature map S input from the decoder layer is... i Perform Sigmoid activation function calculation, then correlate each pixel with the corresponding CAF module output F. i The results are multiplied and fed into a convolutional block consisting of four layers. The kernel sizes of each layer in this block are 1×1, 3×3, 3×3, and 1×1, respectively. Each convolutional layer is followed by a batch normalization operation and a non-linear activation function. The reverse attention calculation method is essentially the same, except that after performing a sigmoid operation on the feature map, it is reversed. Finally, the results from the two attention calculations are concatenated and fed into a convolutional layer to fuse the features. Equation 4 briefly illustrates the BAG calculation process:

[0108]

[0109] In the formula, C3 and C4 are convolutional blocks consisting of four 1×1 and 3×3 convolutional layers, and a convolutional block consisting of two 3×3 convolutional layers, respectively. CAT represents the connection operation, and σ is the sigmoid activation function. This indicates the reverse operation.

[0110] Forward attention focuses the model on the target segmentation region, while backward attention highlights the background region of the image. By gradually erasing the background, the segmentation result is obtained. The simultaneous action of both attention methods results in a more refined segmentation, especially in highlighting the edges of lesions during the two attention calculations.

[0111] Because cellular lung lesions are very similar to surrounding vascular shadows in CT images, it is necessary to give more consideration to edge pixels during segmentation to improve segmentation accuracy. Therefore, the network is trained by combining the binary cross-entropy loss function and the Dice loss. The BCE loss and Dice loss are described as follows:

[0112]

[0113]

[0114] Where n is the training batch size, N is the number of pixels, and p i It is the predicted segmentation result, t i It is ground truth.

[0115] The total loss function is constructed by fusing the binary cross-entropy loss function and the Dice loss to calculate the loss value between the predicted segmentation result and the true label. Equation 7 describes the total loss function:

[0116] L=αL BCE +βL Dice #(7)

[0117] α and β are hyperparameters used to adjust the two loss weights. After repeated experiments, α was set to 0.5 and β was set to 1.

[0118] In one specific embodiment, the construction of the honeycomb lung dataset consists of two steps: data collection and data processing. Computed tomography (CT) images of the lungs of different patients with honeycomb lung were collected from Shanxi Bethune Hospital. Informed consent was obtained from all patients, and the data passed medical ethics review. To avoid significant differences between data sets, all patient data were obtained from the same CT scanner. A total of 2349 original images were collected, with image sizes including 1896×878, 1896×790, 1684×1006, and 1684×878. To protect privacy, all images were resized to 512×512 resolution, and patient privacy information was removed using a mask. Subsequently, the RGB color mask was converted to a binary image mask, resulting in a binarized masked image and the honeycomb lung dataset. To enhance the dataset, prevent network overfitting, and improve model robustness, data augmentation methods such as color transformation and the addition of random noise were performed, resulting in a three-fold increase in the dataset size. See [link to relevant documentation]. Figures 7(a)-7(c) The images show the original CT scan of the lungs with cellular lung disease after de-anonymization, the labeled CT scan of the lungs, and the binarized masked image.

[0119] In one embodiment, the evaluation of a multi-scale, cross-layer attention fusion network is also included.

[0120] Specifically, this embodiment selects six metrics to evaluate the segmentation accuracy of the multi-scale cross-layer attention fusion network model (MCAFNet) of this invention: intersection overunion (IoU), mean intersection overunion, mean dice score (Dice), specificity (SPE), sensitivity (SEN), and accuracy (Acc). Each metric is described in detail below:

[0121] IoU calculates the ratio of the intersection to the union of the predicted and ground truth regions. IoU exhibits scale invariance and non-negativity during calculation. A higher IoU value indicates more accurate segmentation. The formula is as follows: mean IoU is the average IoU across all categories.

[0122]

[0123] Dice is a comprehensive evaluation index used to calculate the similarity between two sets. Its value ranges from [0,1], and the calculation method is shown in Formula 9. mean Dice represents the average Dice across all categories.

[0124]

[0125] SPE (Segmentation Prediction) is a commonly used metric in medical image segmentation to measure segmentation accuracy. It represents the proportion of correctly predicted negative samples out of all actual negative samples, focusing only on the type of negative sample. SPE is expressed as:

[0126]

[0127] SEN, the opposite of SPE, represents the ability to correctly recall positive samples from all actual positive samples; it is also known as recall rate. It focuses only on the positive sample type and can be expressed as:

[0128]

[0129] ACC represents the ratio of the number of correct decisions to the total number of decisions. The number of correct decisions is (TP + TN), so ACC can be represented as:

[0130]

[0131] In the above formula, TP, TN, FP, and FN represent the number of true positives, true negatives, false positives, and false negatives, respectively. These numbers are obtained by comparing the true labels with the obtained results pixel by pixel.

[0132] Example 2

[0133] This paper investigates the impact of three modules in the multi-scale cross-layer attention fusion network model (MCAFNet) of this invention on the overall cellular lung lesion segmentation task. First, a network with all sub-modules removed, using only convolutional layers in the encoder and decoder, is termed Base. After testing the segmentation results of Base, the three modules are added separately for further testing. Table 1 shows the results of ablation experiments verifying the effectiveness of each part of the model. When only one of the three modules is used, the CAF module shows the greatest improvement in model accuracy. As more modules are used, the segmentation metrics gradually increase. When all three modules are applied to the segmentation task, the model achieves optimal performance, with IoU and mDice reaching 0.895 and 0.949, respectively. The ablation experiments on the modules demonstrate the effectiveness of the three-sub-module design. Figure 8 The segmentation visualization results of the ablation experiment are presented.

[0134] Table 1. Validation of the effectiveness of each part of the ablation experiment model.

[0135]

[0136] In one specific embodiment, to verify the impact of the hyperparameters α and β in the loss function on segmentation accuracy and loss, five sets of loss functions with different values ​​were selected for training and testing on the cellular lung dataset. Table 2 shows the results of the loss function ablation experiment, which shows that the model achieves the best segmentation effect when α is 0.5 and β is 1.0. Figure 9 This is a visualization of the impact of different values ​​of α and β in the loss function on the decrease in loss value during training. It can be observed that when α is 0.5 and β is 1.0, the loss value decreases to around 0.1 the fastest and eventually reaches its lowest value. Therefore, the experimental results prove that the hyperparameters set in this invention have the best effect.

[0137] Table 2 shows the impact of hyperparameter selection in the loss function on segmentation accuracy in ablation experiments.

[0138]

[0139] Example 3

[0140] To verify the effectiveness of the medical image segmentation method based on a multi-scale cross-layer attention fusion network provided in this invention in cellular lung segmentation, as well as the model's generalization and robustness, this embodiment 3 selects five state-of-the-art methods for comparison with the method of this invention. These five state-of-the-art methods include UNet, PraNet, UNeXt, ConvUNeXt, and M... 2 Snet.

[0141] In the cell lung segmentation experiment, all models were trained on the constructed training set and the results were obtained on the same test set.

[0142] Specifically, a quantitative comparison of six different methods using six metrics was first conducted on a cellular lung dataset. Table 3 shows that the method proposed in this invention outperforms all models used for comparison. In particular, the method of this invention achieves a 2.2% performance improvement compared to ConvUNeXt in the mIoU metric, which is crucial for cellular lung lesion segmentation. The multi-scale cross-layer attention fusion network model (MCAFNet) constructed in this invention can effectively capture multi-scale information across layers to handle lesions of different sizes, thereby more accurately segmenting target boundaries and small target regions.

[0143] Table 3. Quantitative results of honeycomb lung segmentation

[0144]

[0145] In one specific embodiment, such as Figure 10The qualitative comparison between the models is shown, with different rows representing the segmentation results of five different honeycomb lung lesions. Each column represents a typical segmentation model, and the bottom shows a magnified view of the detailed segmentation results. The results show that the segmentation results of the method proposed in this invention are closest to the true labels and outperform other methods in lesion segmentation at different scales. Specifically, in the small lesions in the middle and lower right corner of the first row, MCAFNet can accurately segment them, while UNet fails to identify these lesions. Other networks can identify them, but cannot accurately segment the lesion edges. In the third row, the method proposed in this invention can clearly characterize the boundaries of large target areas and separate the two parts of the lesion on the right. In contrast, other networks show some adhesion in both parts of the lesion. UNet and PraNet also have unidentified areas in the middle of the lesion. From the detailed magnified view in the last row, it can be seen that the method proposed in this invention performs excellently in characterizing the boundaries of small lesions. This is attributed to the CAF module proposed in this invention, which can capture semantic information at different scales across layers, thus making the identification of small-scale lesions more accurate.

[0146] In one specific embodiment, to verify the generalization ability of the proposed model, this embodiment selected a public dataset for pneumonia infection region segmentation and three public datasets for polyp segmentation for training and testing. The pneumonia dataset is the Covid-19CT lung and infection region segmentation dataset, and the polyp segmentation datasets are Kvasir, CVC-ClinicDB, and CVC-300, respectively.

[0147] In the pneumonia segmentation experiment, the original CT images in the dataset were sliced ​​and reconstructed into training and test sets for comparative testing.

[0148] In the polyp segmentation experiment, following the experimental method described in "D.-P.Fan,G.-P.Ji,T.Zhou,G.Chen,H.Fu,J.Shen,L.Shao,Pranet:Parallel reverse attention network for polyp segmentation,Medical Image Computing and Computer Assisted Intervention–MICCAI 2020:23rd International Conference,Lima,Peru,October4–8,2020,Proceedings,Part VI 23,Springer,2020,pp.263-273", 900 images from Kvasir and 550 images from CVC-ClinicDB were used as the training set to train all models. The remaining images from both datasets, along with CVC-300, were used as the test set for testing.

[0149] In one specific embodiment, to verify the generalization and robustness of the proposed method, it was applied to another medical image segmentation task, namely, COVID-19 lung infection region segmentation, using lung CT images as input. Table 4 shows the quantitative comparison results of the models, which show that among all tested models, the method proposed in this invention achieved the highest average Dice score and average IoU score, at 0.914 and 0.877, respectively. Furthermore, Figure 11 Qualitative visualizations of MCAFNet and other models are presented, with the vertical axis representing different lung CT images and the horizontal axis representing the segmentation results of different networks. Observations reveal that MCAFNet has a significant advantage in segmenting smaller infected areas. Particularly in the enlarged results in the last row, only the segmentation result of the method presented in this invention is similar to the true label, and it exhibits better edge segmentation and feature representation capabilities compared to other models.

[0150] Overall, the experimental results show that MCAFNet has certain robustness and generalization in different segmentation domains of lung CT images.

[0151] Table 4. Quantitative infection region segmentation results of the COVID-19 dataset.

[0152]

[0153] Specifically, Table 5 provides quantitative comparison results between the method of this invention and other methods on the CVC-ClinicDB dataset. The results show that the method of this invention achieves a leading position on most evaluation metrics and also shows significant improvements compared to models targeting polyp segmentation. For example, compared to PraNet, it shows improvements of 3.1%, 3.5%, 0.9%, 2.6%, and 4.1% on mDice, mIoU, SPE, SEN, and ACC, respectively.

[0154] Table 5. Quantitative polyp segmentation results from the CVC-ClinicDB dataset.

[0155]

[0156] Specifically, Table 6 shows the comparative experimental results of the method of this invention with other models on the CVC-300 dataset. It can be seen that, compared to the new model, the method of this invention achieves the best results in four metrics, especially the improvement in SPE. This indicates that the added reverse attention mechanism can effectively identify the background and help improve the model's ability to classify negative examples. Meanwhile, compared with M... 2 Compared to SNet, the improvement in mIoU also shows that MCAFNet can more accurately locate lesions, which is due to the extraction of multi-scale information across layers.

[0157] Table 6. Quantitative polyp segmentation results from the CVC-300 dataset.

[0158]

[0159] Specifically, Table 7 presents quantitative comparisons with other methods on the Kvasir dataset. These results further validate the effectiveness and generalization of the method described in this invention across various medical image segmentation tasks.

[0160] Table 7. Quantitative polyp segmentation results from the Kvasir dataset.

[0161]

[0162] Specifically, such as Figure 12 These are visualizations of the polyp segmentation results from various models. Because the pixel values ​​of the segmentation target and background are close and there is shadow interference in the polyp image, polyp segmentation is more difficult. It can be seen that UNet and PraNet fail to accurately identify the segmentation target in many scenarios, leading to the incorrect identification of the background as the segmentation target. In contrast, the MCAFNet proposed in this invention can accurately locate the target, and the segmentation results are closer to the true label than all the compared methods.

[0163] More specifically, this invention first proposes a multi-scale input aggregation (MIA) module to provide the model with input images of different scales to reduce information loss during downsampling. To fuse semantic information from different layers and further improve the model's segmentation accuracy for highly variable lesions, a cross-layer attention fusion (CAF) module is proposed, which fully extracts channel information from lower-layer feature maps and spatial information from higher-layer feature maps. Furthermore, a bidirectional attention gating (BAG) module is proposed and incorporated into skip connections to reduce the semantic gap between corresponding layers of the encoder and decoder, and forward and reverse attention are used to focus on the relevant semantic information of the segmentation target and background, respectively. Experiments on the cellular lung dataset demonstrate that MCAFNet outperforms other methods, achieving satisfactory results. Experiments on a pneumonia dataset and three polyp datasets also demonstrate that the method of this invention has certain generalization capabilities and can be used in different tasks of medical image segmentation.

[0164] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0165] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A medical image segmentation method based on a multi-scale cross-layer attention fusion network, characterized in that, Includes the following steps: S100: Collect computed tomography (CT) images of the lungs with celluloid lung disease, and process the CT images of the lungs to generate a celluloid lung dataset; S200: Constructing a multi-scale, cross-layer attention fusion network model using the UNeXt network framework; S300: Based on the aforementioned cellular lung dataset, a multi-scale cross-layer attention fusion network model is constructed by jointly training the binary cross-entropy loss function (BCE loss) and the image segmentation task loss function (Dice loss). S400: Input the medical image to be segmented into the trained multi-scale cross-layer attention fusion network model to obtain the segmentation result; The multi-scale cross-layer attention fusion network model includes: a multi-scale input module, a cross-layer attention fusion module, and a bidirectional attention gating module; The multi-scale input module is introduced into the encoder of the UNeXt network framework. The multi-scale input module is used to adjust the size of the lung computed tomography image and input it into the corresponding encoder layer for fusion with the corresponding feature map. The cross-layer attention fusion module includes a spatial attention submodule and a channel attention submodule. It is used to feed shallow features from adjacent convolutional layers in the encoder into the spatial attention module and deep features into the channel attention module. The feature map output by the channel attention module is upsampled and concatenated with the result of the spatial attention output. The concatenated result is then subjected to a double convolution. A residual connection is used to add the original feature map to the feature map generated after the double convolution. The final result is obtained through a non-linear activation function. The calculation process is as follows: ； In the formula, These are shallow features. For deep features, SA represents spatial attention module, CA represents channel attention module, and CAT represents connection operation. and 2 for upsampling 2. Transposed convolution and double convolution blocks.

2. The medical image segmentation method based on a multi-scale cross-layer attention fusion network according to claim 1, characterized in that, S100: Collecting computed tomography (CT) images of the lungs with cellulitis, and processing the CT images to generate a cellulitis dataset, including: S110: Acquire computed tomography images of lungs with honeycomb lung disease from the same CT scanner; S120: Adjusts the resolution of lung computed tomography images and uses a mask to remove the patient's private information from the images; S130: The RGB color mask of the lung computed tomography image is then converted into a binary image mask to obtain a binarized mask image and generate a cellular lung dataset. S140: Perform color transformation and add random noise to obtain the expanded cellular lung dataset.

3. The medical image segmentation method based on a multi-scale cross-layer attention fusion network according to claim 1, characterized in that, The spatial attention module obtains the maximum and average values ​​of the channels respectively, and calculates the attention value at the spatial level. The calculation process is as follows: ； In the formula, Sout is the output of the spatial attention module, σ is the sigmoid activation function, Cmax and Cave represent the maximum and average values ​​obtained along the channel direction, respectively, and C represents the convolutional layer.

4. The medical image segmentation method based on a multi-scale cross-layer attention fusion network according to claim 1, characterized in that, The calculation process of the channel attention module is as follows: ； In the formula, Cout is the output of the CA module, σ is the Sigmoid activation function, and C represents the convolutional layer. This represents a non-linear activation function, where AP and MP represent adaptive average pooling and adaptive max pooling, respectively.

5. The medical image segmentation method based on a multi-scale cross-layer attention fusion network according to claim 1, characterized in that, The bidirectional attention gating module includes forward attention calculation and reverse attention calculation; The positive attention calculation involves performing a Sigmoid activation function on the feature map input from the decoder layer of the UNeXt network framework, multiplying each pixel by the output of the corresponding cross-layer attention fusion module, and feeding the result into a convolutional block consisting of four convolutional layers. The kernel sizes of each convolutional layer in the convolutional block are 1×1, 3×3, 3×3, and 1×1, respectively. Each convolutional layer is followed by a batch normalization operation and a non-linear activation function. The reverse attention calculation is performed by performing the reverse operation after calculating the Sigmoid activation function on the feature map; The results obtained from the two attention calculations are concatenated and then input into a convolutional layer for feature fusion. The calculation process is as follows: ； A multi-scale, cross-layer attention fusion network model, jointly trained with CE loss and the Dice loss function for image segmentation tasks, includes: The formula for calculating the binary cross-entropy loss function (BCE loss) is as follows: The Dice loss function for image segmentation tasks is calculated using the following formula: In the formula, n is the training batch size, and N is the number of pixels. It is the predicted segmentation result. These are real labels; The total loss function is calculated as follows: In the formula, α and β are hyperparameters that adjust the two loss weights.

6. The medical image segmentation method based on a multi-scale cross-layer attention fusion network according to claim 1, characterized in that, It also includes evaluating the multi-scale cross-layer attention fusion network, specifically through the following steps: The Intersection over Union (IoU) ratio between the predicted and real regions obtained by the multi-scale cross-layer attention fusion network is calculated using the following formula: The Dice function calculates the similarity between two sets, with values ​​ranging from [0,1]. The formula is as follows: The proportion (SPE) of all correctly predicted negative samples out of all actual negative samples is calculated using the following formula: The recall rate (SEN) of correctly retrieved positive samples out of all actual positive samples is calculated using the following formula: The ratio of correct results to all correct results is calculated using the following formula: In the formula, TP represents the number of true positives, TN represents the number of true negatives, FP represents the number of false positives, and FN represents the number of false negatives.

Images