A cellular lung lesion segmentation method combining U-shaped convolution and PPCT

Abstract

The application discloses a honeycomb lung lesion segmentation method combining U-shaped convolution and PPCT, and belongs to the technical field of medical image segmentation. The method provided by the application obtains a honeycomb lung lesion segmentation result by using a CUPTNet, and the encoding-decoding structure of the CUPTNet network is optimized by using GCT and ConvMixer, so that the ability of the model to extract global context in the encoding stage is improved, the correlation between modeling feature channels is improved, and these information is efficiently mixed. In addition, a pyramid pooling channel Transformer module is used to improve a simple skip connection, and the module can effectively utilize semantic information in multi-scale channel features, so as to provide sufficient useful information for decoding work. Compared with the prior art, the method provided by the application has better honeycomb lung lesion segmentation effect.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of medical image segmentation, in particular to a honeycomb lung lesion segmentation method combining U-shaped convolution and PPCT. BACKGROUND

[0002] Accurate lesion segmentation plays a crucial role in the clinical diagnosis of honeycomb lung. However, due to the diversity of honeycomb lung shape, distribution and texture, honeycomb lung segmentation is a challenging task.

[0003] At present, there are non-negligible shortcomings when using pure CNN methods and pure Transformer methods for lesion segmentation. For CNN, it may have difficulty in modeling long-distance context. Due to the locality of convolution operation, CNN is difficult to capture global information, which seriously affects the improvement of lesion segmentation accuracy. Although the Transformer can effectively extract global features, it is very sensitive to the position information in the input sequence and difficult to obtain local features, so the model is prone to overfitting or generate inaccurate segmentation results.

[0004] Therefore, it is an urgent problem for those skilled in the art to propose a honeycomb lung lesion segmentation method combining U-shaped convolution and PPCT to further improve the performance of honeycomb lung lesion segmentation. SUMMARY

[0005] Therefore, the application provides a honeycomb lung lesion segmentation method combining U-shaped convolution and PPCT, which can accurately segment the lesion area from the honeycomb sign CT image.

[0006] In order to achieve the above purpose, the application adopts the following technical solutions:

[0007] A honeycomb lung lesion segmentation method combining U-shaped convolution and PPCT comprises the following steps:

[0008] Obtain a honeycomb lung CT image data set, preprocess the honeycomb lung CT image data set to obtain an expanded experimental data set, and divide the experimental data set into a training set and a test set;

[0009] Construct a CUPTNet model combining U-shaped convolution and PPCT, and the skip connection in the CUPTNet model uses a pooling pyramid channel Transformer module;

[0010] Train the CUPTNet model based on the training set to determine an optimal CUPTNet model;

[0011] Input the test set into the optimal CUPTNet model to output a honeycomb lung lesion segmentation result.

[0012] Preferably, the CUPTNet model includes a U-shaped encoder-decoder section, which includes an encoding path and a decoding path;

[0013] The encoding path consists of five layers from top to bottom, and each layer is connected to the other layers through a GCT module. The GCT module is connected to the pooling pyramid channel Transformer module through a PatchEmbed. The first four layers of the encoding path each include two convolutional layers and one downsampling layer. The fifth layer introduces a ConvMixer module and is connected to the decoding path.

[0014] The decoding path is set to correspond to the first four layers of the encoding path. Each layer of the decoding path includes two convolutional layers and one upsampling layer. The features from the pooling pyramid channel Transformer module and the ConvMixer module are fused and decoded layer by layer and then fed into a 1×1 convolution. The Sigmoid activation function is used for image segmentation.

[0015] Preferably, the pooled pyramid channel Transformer module includes a pyramid pooling feature perception submodule, a channel self-attention submodule, and a reverse residual feedforward network;

[0016] The pyramid pooling feature perception submodule uses a depthwise convolution to extract local information from the feature map output by the GCT module, and then uses average pooling layers with different pooling rates to generate a pyramid feature map P1. i The pyramid feature map P1 i Expand and connect to obtain the output P of the pyramid pooling feature perception submodule. i The output P of the pyramid pooling feature sensing submodule i The sequence is reshaped into a patch sequence for feature labeling, with the corresponding patch being T. i , where i = 1, 2, 3, 4.

[0017] Preferably, the channel self-attention submodule addresses patch T. i Cascading yields cascading patch T ∑ Using the cascaded patch T ∑ Obtain the Key and Value, and use the patch T. i The Query and Key are obtained, and a similarity matrix Sm is generated from them through a cross-attention mechanism. i The similarity matrix Sm is obtained through Value. i The weighted result is obtained by performing a weighted calculation, and the attention mechanism feature output CAOut is obtained based on the weighted result. i .

[0018] Preferably, the inverse residual feedforward network is represented by the following formula:

[0019] IRFFN(CAOut i )=Conv(DWConv(Conv(CAOut i ))+Conv(CAOut i ));

[0020] wherein Conv(·) represents a 1x1 normal convolution, and DWConv(·) represents a 3x3 depthwise convolution;

[0021] The attention mechanism feature output CAOut i is connected in residual with IRFFN(CAOut i ) to obtain the output Out i of the channel Transformer.

[0022] Preferably, the GCT module works as follows:

[0023] For a given feature Figure X , first, the global information is aggregated by global average pooling to obtain a channel vector Z;

[0024] The channel vector Z is normalized to obtain

[0025] The normalized global context is converted and activated by a Gaussian function to obtain an attention map g;

[0026] After the attention map g is multiplied by the feature Figure X , the output of the GCT module is obtained.

[0027] Preferably, the ConvMixer module is composed of L ConvMixer layers, and each ConvMixer layer is composed of a depthwise convolution and a pointwise convolution, followed by an activation function GELU and normalization, to finally obtain the output of the ConvMixer layer.

[0028] According to the above technical solution, compared with the prior art, the application provides a honeycomb lung lesion segmentation method combining U-shaped convolution and PPCT, which has the following beneficial effects:

[0029] 1. GCT and ConvMixer are used to improve the ability of the model to extract global context in the encoding stage, model the correlation between feature channels, and efficiently mix these information. Specifically, ConvMixer is introduced at the U-shaped bottleneck to replace the normal convolution block, so that more spatial and channel features are contained in the feature map, and the feature map is directly sent to the decoder part to avoid loss of useful information.

[0030] 2. Using PPCT to replace the hop connection in the traditional U-shaped network, effectively fusing multi-scale features from the encoder and reducing the semantic gap between the encoder and decoder. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0032] Figure 1 Fig. 1 is a schematic diagram of the overall structure of the CUPTNet model;

[0033] Figure 2 Fig. 2 is a schematic diagram of the structure of the GCT module;

[0034] Figure 3 Fig. 3 is a schematic diagram of the structure of the pyramid pooling channel Transformer module;

[0035] Figure 4 Fig. 4 is a schematic diagram of the structure of the pyramid pooling feature perception sub-module;

[0036] Figure 5 Fig. 5 is a schematic diagram of the channel self-attention mechanism flow;

[0037] Figure 6 Fig. 6 is a schematic diagram of the structure of the reverse residual feedforward network;

[0038] Figure 7 Fig. 7 is a schematic diagram of the structure of the ConvMixer module;

[0039] Figure 8 Fig. 8 is a qualitative comparison diagram of the polyp cell lung dataset, wherein (a) is the input image, (b) is the Ground Truth, (c) is UNet, (d) is Attention-UNet, (e) is DeepLabV1, (f) is DeepLabV3, (g) is DilatedSegNet, (h) is MALUNet, (i) is PolypPVT, (j) is UCTransNet, (k) is SETR, (l) is TransUNet, and (m) is CUPTNet;

[0040] Figure 9 Fig. 9 is a schematic diagram of the loss value and IoU score of CUPTNet on the polyp cell lung dataset after 200 iterations;

[0041] Figure 10To ablate the qualitative results of the experiment. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0043] The embodiment of the present application discloses a honeycomb lung lesion segmentation method combining U-shaped convolution and PPCT, comprising the following steps:

[0044] 1. Obtain a honeycomb lung CT image data set, preprocess the honeycomb lung CT image data set, obtain an expanded experimental data set, and divide the experimental data set into a training set and a test set.

[0045] In the embodiment, the honeycomb lung CT image data set is collected with the assistance of Shanxi Baiqianen Hospital, and the image size of the data set is 1896x878, 1896x790, 1684x1006, 1683x1006, 4684x1054, 1684x878, etc., containing 9240 honeycomb lung CT images. It contains two types of images, normal lung images and honeycomb lung images, and each type of image is annotated by multiple experienced chest radiologists. In the embodiment, 2350 honeycomb lung images are selected and cropped to a size of 256x256, then normalized, and the data set is expanded to 7050 images by adjusting the saturation, brightness and contrast of the images. The training / test ratio of the experimental data set is 8:2, that is, 5640 images are randomly selected as training images, and the remaining images are used as test images.

[0046] 2. Construct a CUPTNet model combining U-shaped convolution and PPCT, and the skip connection in the CUPTNet model uses a pooling pyramid channel Transformer module.

[0047] The overall structure of the CUPTNet model is as follows Figure 1As shown, CUPTNet consists of two main parts: a U-shaped encoder-decoder part composed of CNNs and a pyramid pooling channel Transformer module (PPCT). In the encoder, GCT is used to realize contextual feature excitation. In the bottleneck part of the model, ConvMixer is used to mix the spatial and channel positions of features. In the skip connections, the proposed pyramid pooling channel Transformer module is used. Taking advantage of the Transformer's ability to model long dependencies, multi-scale features from the encoder are deeply fused and rich contextual information is captured. Finally, the extracted feature information is sent to the decoder to predict lesions.

[0048] The encoder-decoder part of the U-shaped network is mainly used to extract basic features and decode the feature map. The encoder consists of five layers from top to bottom. The first four layers have two convolutional blocks and one downsampling operation. The convolutional blocks mainly consist of convolution, ReLU activation function, and normalization. GCT is added after downsampling to activate contextual features. The hierarchical structure of the encoder allows it to efficiently capture both low-level and high-dimensional features. Downsampling uses a 2×2 max pooling operation. In the fifth layer of the encoder, at the U-shaped bottleneck, a ConvMixer is introduced to replace the ordinary convolutional blocks, allowing the feature map to contain more spatial and channel features, which are then directly fed into the decoder to avoid losing useful information. The decoder part of the model consists of two convolutional blocks and one upsampling operation. It fuses the features from PPCT and ConvMixer and decodes them step by step. Finally, the feature map is fed into a 1×1 convolution to reduce the number of channels, and the Sigmoid activation function is used to predict the final pixel-by-pixel segmentation map.

[0049] like Figure 2 As shown, the GCT module consists of three processes: global context aggregation, normalization, and Gaussian context activation. Given a feature map... First, global information is aggregated using global average pooling to obtain the channel vector Z, as shown in the following formula:

[0050]

[0051] Where W and H are the width and height of the feature map, respectively, and C is the number of channels in the feature map.

[0052] To stabilize the global context distribution, the channel vector Z is normalized to obtain...

[0053]

[0054] Finally, the Gaussian function is used to normalize the global context. The attention map g is obtained by performing transformation and activation operations. The Gaussian function formula is:

[0055]

[0056] where c is the standard deviation, and in this embodiment, c is set to 2.

[0057] Attention map g and feature Figure X After multiplication, the feature map E output by the GCT module i As follows:

[0058]

[0059] The pyramid pooling channel Transformer module (PPCT) structure is shown in Figure 3 The pyramid pooling feature perception (PPFP) module is designed to be embedded in the multi-head self-attention (MHSA) of the Transformer to retain as much key information as possible, reduce the length of the sequence while learning powerful feature representations; at the same time, the channel self-attention is used instead of the original self-attention, which can effectively utilize the semantic information in the multi-scale channel features and reduce the semantic gap between the encoder and the decoder, and the inverse residual feedforward network (IRFFN) is used as the feedforward network of the Transformer to encode the channel feature relationship of the output and obtain useful information therefrom.

[0060] Referring to Figure 4 In this embodiment, the PPFP first uses a 3x3 deep convolution to extract local information in the feature map E i output by the four GCT modules in the encoder (i=1, 2, 3, 4) (hereinafter referred to as X instead of E i as input) for description. Then, four average pooling layers with different pooling rates are applied to X to generate pyramid feature maps Pl i

[0061] Pl1 = AvgPool1 (DWConv (X) + X)

[0062] Pl2 = AvgPool2 (DWConv (X) + X)

[0063] Pl3 = AvgPool3 (DWConv (X) + X)

[0064] Pl4 = AvgPool4 (DWConv (X) + X)

[0065] where (i=1, 2, 3, 4), HxW is the feature map resolution, C represents the feature channel dimension, and AvgPool i ​(i = 1, 2, 3, 4) represent average pooling operations with different pooling ratios, DWConv(·) is a 3x3 depthwise convolution, Pl i (i = 1, 2, 3, 4) represent the generated pyramid feature maps.

[0066] Then Pl i is unfolded and concatenated to get the output P i of PPFP:

[0067] P i = LN(Concat(Pl1, Pl2, Pl3, Pl4))

[0068] where LN is layer normalization operation.

[0069] The output P i of PPFP is reshaped to patch sequence for feature tokenization, so that patches at different scales can be mapped to the same feature region, and the corresponding patches T i (i = 1, 2, 3, 4) have sizes Pa、 and the number of channels is the same as that of the original feature map.

[0070] Unlike the original self-attention mechanism, channel self-attention is an attention operation along the channel axis rather than the patch axis, which can achieve smooth gradient propagation by mapping and normalizing each similarity matrix.

[0071] First, the patches T i (i = 1, 2, 3, 4) are concatenated to get the concatenated patch T Σ :

[0072] T Σ = Concat(T1, T2, T3, T4);

[0073] Then use the concatenated patch T Σ as Key and Value, and the patches T i (i = 1, 2, 3, 4) as Query:

[0074] Query i = T i W Q

[0075] Key = T Σ W K

[0076] Value = T Σ W V

[0077] where Corresponding weights of Query, Key, Value, respectively, C i Channel dimensions of the feature map E i Channel dimensions of the feature map E

[0078] The detailed process is shown in Figure 5 The similarity matrix Sm i is generated by the cross-attention mechanism, and then weighted by

[0079]

[0080] Where ξ is instance normalization, and τ is the Softmax function.

[0081] Finally, the feature output of the attention mechanism is

[0082]

[0083] Where n is the number of attention heads, and in this embodiment, n = 4.

[0084] The expansion layer and projection layer of the inverse residual feed-forward network (IRFFN) are composed of convolutions, and a 3x3 depth convolution is also used to extract local information. Inspired by the inverse residual block, the IRFFN introduces a residual connection and modifies the connection position to improve the gradient cross-layer propagation ability. Figure 6 The schematic diagram of the IRFFN is shown in FIG. 6, which can be represented by the following formula:

[0085] IRFFN(CAOut i ) = Conv(DWConv(Conv(CAOut i ))+Conv(CAOut i ))

[0086] Where Conv(·) represents a 1x1 ordinary convolution, and DWConv(·) represents a 3x3 depth convolution. In addition, for convenience of representation, the activation function GELU and normalization of the IRFFN are omitted in the above formula.

[0087] The output CAOut i of the attention mechanism is connected in residual to IRFFN(CAOut i ), and the output Out i of the channel Transformer is obtained, and the formula is as follows:

[0088] Out​​i = IRFFN(CAOut i )+ CAOut i

[0089] Finally, the output Out i of the formula is repeated L times to construct PPCT, that is, PPCT is composed of L layers of channel Transformer. In order to balance the model performance and the scale of the computing cost, according to experience, the number of layers L is set to 4 in the embodiment.

[0090] It is worth noting that the number of inputs and outputs of each layer of channel Transformer corresponds to the number of encoders and decoders, so each layer of channel Transformer has 4 inputs and 4 outputs. After the 4 feature maps of the encoder are input into PPCT for self-attention calculation, the 4 output features of PPCT are sent to the 4 decoders respectively. That is, the 4 outputs Out i of the Lth layer of channel Transformer are reconstructed by upsampling and convolution operation respectively, and are fused with the corresponding level of decoder feature.

[0091] The ConvMixer module can efficiently mix the spatial and channel information of the features Figure X , and can better maintain the integrity of the global and detailed information in the feature map. As shown in Figure 7 , the ConvMixer layer is composed of a deep convolution and a pointwise convolution (i.e. kernel size 1x1), and the group channel number of the deep convolution kernel is the number of channels of X. After each convolution, there is an activation function GELU and normalization, and L ConvMixer layers form a ConvMixer module. The ConvMixer layer can be represented by the following formula:

[0092] Y = PW(DW(X) + X);

[0093] Where X and Y are the input and output feature maps of the ConvMixer layer, DWConv(·) is a 7x7 deep convolution, and PWConv(·) is a 1x1 pointwise convolution. In addition, for convenience of representation, the activation function GELU and the normalization of the ConvMixer layer are omitted in the above formula.

[0094] Since the resolution of the feature map in L layers remains the same, the output Y of the Lth layer will be directly sent into the decoder after upsampling.

[0095] 3. Training the CUPTNet model based on the training set to determine the optimal CUPTNet model.

[0096] The training was performed on an RTX 2080Ti GPU, the running environment was Ubuntu 18.04, and the programming environment was realized by Python 3.8 under the PyTorch framework. In this embodiment, all models used the same parameters. In the training process with 200 iterations, the random gradient descent (SGD) optimizer was used to optimize the back propagation of the network, the initial learning rate was set to 0.001, and the momentum was set to 0.9. The cross-entropy loss function was used as the loss function in the initial stage of the training process, as follows:

[0097]

[0098] wherein y i is the actual label value of each pixel, is the predicted label value of each pixel, and n is the sample quantity.

[0099] The segmentation performance of the CUPTNet was quantitatively evaluated by four indicators: IoU, mIoU, Dice, and precision. IoU, which stands for Intersection over Union, is used to evaluate the similarity between the predicted value and the actual value. This indicator is used to evaluate the segmentation ability of the model, and the value range is [0, 1], the best result is 1, and the worst result is 0. mIoU (Mean Intersection over Union) represents the average IoU value of each class on the dataset. The Dice coefficient is usually used to calculate the similarity of two samples, and the value range is [0, 1], the higher the score, the better. Precision refers to the percentage of pixels with positive predictions, and the value in the range of [0, 1] is closer to 1, indicating that the prediction result is better. The formula is as follows:

[0100]

[0101] wherein TP is the number of true positives, FP is the number of false positives, FN is the number of true negatives, and k is the number of segmentation categories.

[0102] 4. The test set was input into the optimal CUPTNet model, and the honeycomb lung lesion segmentation result was output.

[0103] 5. The performance of the CUPTNet model was verified by experiments in this embodiment.

[0104] Specifically, to evaluate the effectiveness and robustness of CUPTNet, we compared it with recent techniques, including UNet, Attention-UNet, DeepLabV1, DeepLabV3, DilatedSegNet, MALUNet, PolypPVT, UCTransNet, SETR, and TransUNet. To be fair, we unified the experimental settings of these methods, and the results of these segmentation methods on the honeycomb lung dataset are shown in Table 1, where the best results are in bold.

[0105] Table 1 Evaluation of the segmentation performance of the models on the honeycomb lung dataset using three indicators

[0106]

[0107] As can be seen from Table 1, CUPTNet outperforms other methods on the honeycomb lung dataset. Specifically, CUPTNet achieves the highest score on the IoU indicator, which is 3.26% higher than UNet, 2.51% higher than Attention-UNet, 8.47% higher than DeepLabV1, 4.77% higher than DeepLabV3, 3.37% higher than DilatedSegNet, and 24.72% higher than MALUNet, compared with CNN models. Compared with CNN-Transformer combined models, it is 22.58% higher than PolypPVT, 2.41% higher than UCTransNet, 12.66% higher than SETR, and 4.04% higher than TransUNet. Similarly, CUPTNet also shows certain advantages on indicators mIoU and Dice.

[0108] The visualization of the segmentation results is shown in Figure 8 , where the obvious erroneous predictions are marked with red boxes. It can be seen that the honeycomb lung lesions predicted by CUPTNet are more accurate. The honeycomb lung dataset contains lesions with different appearances and shapes, so it is difficult to obtain segmentation results with complete shape information. However, CUPTNet can segment honeycomb lung lesions with more accurate shape features on the honeycomb lung dataset, especially the contours of the honeycomb lung lesions in its prediction map are closer to the Ground Truth, because CUPTNet has strong feature capturing ability and long-range semantic information modeling ability. Through qualitative comparison, this method outperforms other methods in segmentation performance.

[0109] Figure 9 The learning ability of CUPTNet on the honeycomb lung dataset is shown in , it can be seen that the model loss value converges quickly in the first fifty iterations, while the IoU score also rises quickly. Then the loss value and the IoU score both maintain a stable change trend until the end of the training phase.

[0110] To test the effectiveness of each module in CUPTNet, the ablation experiment was performed on the cellular lung dataset. PPCT replaced the skip connection of the traditional U-shaped network, combined the multi-scale features of the input channel self-attention, and extracted the global and local useful information from it. In this embodiment, the PPCT module is replaced by a skip connection to obtain the model md1. Then, to test the effectiveness of ConvMixer, the ConvMixer in the bottleneck part of the model is removed and replaced by a normal convolution, and the model is recorded as md2. In addition, based on the CUPTNet model, the GCT in the encoder part is eliminated, and the model md3 is obtained under the condition that other conditions remain unchanged. Table 2 is the quantitative results of the ablation experiment, and the best result is in bold.

[0111] Table 2 Four index scores corresponding to the ablation experiment performed on the cellular lung dataset

[0112]

[0113] In Figure 10 A qualitative comparison was made, in which the green line is the true label and the red line is the segmentation result. After eliminating the GCT, the segmentation performance of md3 decreased, with a decrease of 0.92%, 0.47%, 0.44% and 1.14% in IoU, mIoU, Dice and precision, respectively. After eliminating the ConvMixer, the IoU and Dice scores of md2 decreased by 1.87% and 1.08%, respectively, and the performance decreased significantly. Finally, after eliminating the PPCT, the segmentation performance decreased most obviously, and compared with CUPTNet, the IoU, mIoU, Dice and precision of md1 decreased by 2.25%, 1.15%, 1.45% and 1.94%, respectively. From Figure 10 It can also be seen that the prediction map of md3 is not as fine as that of CUPTNet, some noise appears on the prediction map of md2, and the prediction map of md1 is the worst, not only significant noise appears, but also the lesion edge cannot be well fitted with the label. From the above qualitative and quantitative analysis, it can be clearly concluded that the addition of GCT, ConvMixer and PPCT modules enhances the segmentation performance of the model and completes more accurate segmentation.

[0114] Further, to discuss the influence of parameters on the segmentation accuracy of the model, this embodiment performs experiments on the cellular lung dataset in four aspects: GCT version selection, encoder-decoder layer number, ConvMixer depth convolution kernel size and pooling rate in the PPFP module.

[0115] The GCT includes two versions of a non-participation version B0 and a participation version B1. The non-participation version B0 does not introduce any parameters, and the participation version B1 adaptively learns the standard deviation in the Gaussian function. To select a suitable version, models md4 and md5 are designed by using B0 and B1 respectively under the condition that other conditions remain unchanged. The experimental results are shown in Table 3. It can be seen that md4 has better performance, and therefore B0 is used as the attention module in this embodiment.

[0116] Table 3 Influence of different GCT versions on model accuracy

[0117]

[0118] In addition, in order to select a suitable number of encoder-decoder layers, the embodiment performs relevant experiments. As shown in Table 4, the number of layers is set to 3, 4 and 5, and the number of PPCT self-attention heads, inputs and outputs is also set to 3, 4 and 5. md6, md7 and md8 correspond to the number of encoder-decoder layers of 3, 4 and 5 respectively. As can be seen from Table 4, when the number of encoder-decoder layers in the model increases from 3 to 4, the performance of the model improves significantly, and IoU and Dice increase by 2.24% and 1.29% respectively. When the number of layers increases from 4 to 5, the performance of the model decreases, and IoU and Dice decrease by 0.74% and 0.62% respectively. Therefore, when the number of encoder-decoder layers is set to 4, the performance of the model is optimal.

[0119] Table 4 Influence of different numbers of encoder-decoder layers on model accuracy

[0120]

[0121] In order to select a suitable ConvMixer depth convolution kernel size, as shown in Table 5, the embodiment tests the kernel size by setting it to 3x3, 5x5, 7x7, 9x9 and 11x11. As can be seen from Table 5, the performance of md11 using a 7x7 convolution kernel size is the best, and the IoU and Dice scores are 2.14% and 1.32% higher than those of md9 based on a 3x3 convolution kernel, 1.92% and 1.25% higher than those of md10 based on a 5x5 convolution kernel, 0.66% and 0.31% higher than those of md11 based on a 9x9 convolution kernel, and 1.64% and 1.00% higher than those of md12 based on an 11x11 convolution kernel, which is a significant effect. Therefore, the convolution kernel of ConvMixer is finally set to 7x7.

[0122] Table 5 Influence of different convolution kernel sizes on model accuracy

[0123]

[0124] Finally, the pooling rate in the PPFP module is discussed in this embodiment. Four different pooling rates R i (i = 1, 2, 3, 4) are designed according to experience, which are {12, 16, 20, 24}, {6, 8, 10, 12}, {3, 4, 5, 6} and {1, 2, 3, 4} respectively. From Table 6, it can be seen that when the pooling rate is R3, the performance of the model reaches the optimal.

[0125] Table 6 Influence of different pooling rates on model accuracy

[0126]

[0127] This embodiment combines CNN and Transformer to propose a pyramid pooling channel Transformer-based model CUPTNet for honeycomb lung lesion segmentation. The model uses GCT and ConvMixer to improve the ability of the model to extract global context in the encoding stage, model the correlation between feature channels and efficiently mix these information. PPCT is used to replace the traditional U-shaped network skip connection, effectively fusing multi-scale features from the encoder and reducing the semantic gap between the encoder and decoder.

[0128] A large number of experiments are conducted on the honeycomb lung dataset to verify the advancement of the proposed model. The experimental results show that CUPTNet achieves IoU, mIoU, Dice and precision scores of 0.8899, 0.9439, 0.9369 and 0.9131 respectively. Ablation experiments prove the effectiveness of each module, for example, after eliminating the PPCT module, the IoU, mIoU, Dice and precision scores of the model decrease by 2.25%, 1.15%, 1.45% and 1.94% respectively. Finally, the influence of parameters on the segmentation accuracy of the model is analyzed through experiments, and CUPTNet achieves the best performance in segmentation performance.

[0129] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between each embodiment can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0130] The foregoing description of the disclosed embodiments enables a person skilled in the art to make or use the application. Modifications of these embodiments will occur to persons of skill in the art, and that the appended claims are intended to cover all such modifications that do not depart from the true spirit and scope of the application. Therefore, the application is not limited to the embodiments shown but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for segmenting honeycomb lung lesions combining U-shaped convolution and PPCT, characterized in that, Includes the following steps: A cellular lung CT image dataset is obtained, and the cellular lung CT image dataset is preprocessed to obtain an expanded experimental dataset. The experimental dataset is then divided into a training set and a test set. A CUPTNet model combining U-shaped convolution and PPCT is constructed, wherein the skip connections in the CUPTNet model use the pooled pyramid channel Transformer module; Train the CUPTNet model based on the training set and determine the optimal CUPTNet model; The test set is input into the optimal CUPTNet model, which outputs the celluloid lung lesion segmentation results. The CUPTNet model includes a U-shaped encoder-decoder part, which includes an encoding path and a decoding path; The encoding path consists of five layers from top to bottom, and each layer is connected to the other layers through a GCT module. The GCT module is connected to the pooling pyramid channel Transformer module through a Patch Embed. The first four layers of the encoding path each include two convolutional layers and one downsampling layer. The fifth layer introduces a ConvMixer module and is connected to the decoding path. The decoding path is set to correspond to the first four layers of the encoding path. Each layer of the decoding path includes two convolutional layers and one upsampling layer. The features from the pooling pyramid channel Transformer module and the ConvMixer module are fused and decoded layer by layer and then fed into a 1×1 convolution. The Sigmoid activation function is used for image segmentation. The pooled pyramid channel Transformer module includes a pyramid pooling feature perception submodule, a channel self-attention submodule, and a reverse residual feedforward network; The pyramid pooling feature perception submodule uses a depthwise convolution to extract local information from the feature map output by the GCT module, and then uses average pooling layers with different pooling rates to generate a pyramid feature map P1. i The pyramid feature map P1 i Expand and connect to obtain the output P of the pyramid pooling feature perception submodule. i The output P of the pyramid pooling feature sensing submodule i The sequence is reshaped into a patch sequence for feature labeling, with the corresponding patch being T. i Where i = 1, 2, 3, 4; The channel self-attention submodule for patch T i Cascading yields cascading patch T ∑ Using the cascaded patch t ∑ Obtain the Key and Value, and use the patch T. i The Query and Key are obtained, and a similarity matrix Sm is generated from them through a cross-attention mechanism. i The similarity matrix Sm is obtained through Value. i The weighted result is obtained by performing a weighted calculation, and the attention mechanism feature output CAOut is obtained based on the weighted result. i .

2. The method for segmenting cellular lung lesions combining U-shaped convolution and PPCT according to claim 1, characterized in that, The reverse residual feedforward network is represented by the following formula: IRFFN(CAOut i )=Conv(DWConv(Conv(CAOut i ))+Conv(CAOut i )); Where Conv(·) represents a 1×1 ordinary convolution, and DWConv(·) represents a 3×3 depthwise convolution; Output the attention mechanism features in CAOut i With IRFFN(CAOut) i Perform residual connections to obtain the output of the channel Transformer. i .

3. The method for segmenting cellular lung lesions combining U-shaped convolution and PPCT according to claim 1, characterized in that, The GCT module operates as follows: For a given feature map X, the channel vector Z is first obtained by aggregating global information through global average pooling. Normalizing the channel vector Z yields Use the Gaussian function to normalize the global context. The attention map g is obtained by performing transformation and activation operations; The output of the GCT module is obtained by multiplying the attention map g with the feature map X.

4. The method for segmenting cellular lung lesions combining U-shaped convolution and PPCT according to claim 1, characterized in that, The ConvMixer module consists of L ConvMixer layers, each composed of depthwise convolution and pointwise convolution. Each convolution is followed by the GELU activation function and normalization, ultimately yielding the output of the ConvMixer layer.

Images