Deep learning based lung ct image segmentation method and imaging method

Abstract

The application discloses a lung CT image segmentation method based on deep learning, comprising the following steps: obtaining an existing lung CT image dataset and preprocessing the same to obtain a training dataset; constructing a lung CT image segmentation original model and training the same to obtain a lung CT image segmentation model; and segmenting an actual lung CT image by using the lung CT image segmentation model. The application further discloses an imaging method comprising the lung CT image segmentation method based on deep learning. The application can realize more accurate image segmentation results, can accelerate the convergence speed, can also show strong segmentation effects on small-scale datasets, can realize more accurate segmentation results, and can avoid feature collapse and generate diversified features. Therefore, the application is high in reliability, good in accuracy, and objective and scientific.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, specifically relating to a lung CT image segmentation and imaging method based on deep learning. Background Technology

[0002] With the development of economy and technology and the improvement of people's living standards, people are paying more and more attention to health; therefore, medical image processing technology is becoming more and more important.

[0003] In both clinical applications and experimental research, lung CT images play a crucial auxiliary role, effectively assisting clinicians in diagnosing lung diseases and researchers in conducting their studies. Therefore, the segmentation process of lung CT images is particularly important.

[0004] Currently, segmentation methods for lung CT images often employ deep learning models to independently learn feature representations and utilize the learned high-dimensional abstractions to complete the segmentation task. For example, image segmentation methods based on Transformer technology are commonly used to segment lung CT images.

[0005] The Transformer was initially applied to natural language processing and achieved significant success; since the advent of Vitality Intervention (ViT), it has achieved competitive performance on a range of computer vision tasks. However, as research has progressed, some shortcomings of the Transformer have gradually become apparent. The most significant of these is that the Transformer typically requires large-scale datasets for training to exhibit better performance. However, medical image datasets are generally small, and datasets of lung CT images are even fewer. Therefore, existing lung CT image segmentation methods based on Transformer technology have relatively poor accuracy and low reliability. Summary of the Invention

[0006] One of the objectives of this invention is to provide a lung CT image segmentation method based on deep learning that is highly reliable, accurate, and objectively scientific.

[0007] The second objective of this invention is to provide an imaging method that includes the aforementioned deep learning-based lung CT image segmentation method.

[0008] The lung CT image segmentation method based on deep learning provided by this invention includes the following steps:

[0009] S1. Obtain the existing lung CT image dataset;

[0010] S2. Preprocess the data obtained in step S1 to obtain the training dataset;

[0011] S3. Based on the convolutional encoder, the Transformer encoder, the fusion module and the corresponding decoder, construct the original model for lung CT image segmentation;

[0012] S4. Using the training dataset obtained in step S2, train the original lung CT image segmentation model constructed in step S3 to obtain the lung CT image segmentation model.

[0013] S5. Using the lung CT image segmentation model obtained in step S4, segment the actual lung CT image.

[0014] Step S2 involves preprocessing the data obtained in step S1, specifically including the following steps:

[0015] Preprocessing includes image cropping, resampling, and normalization operations;

[0016] Image cropping involves setting a pixel threshold to obtain the bounding box of the desired target.

[0017] The obtained bounding box is then resampled.

[0018] After resampling, the image is normalized using Z-score:

[0019]

[0020] In the formula z i The normalized grayscale value; x i σ is the original grayscale value; μ is the mean of the image grayscale; σ is the standard deviation of the image grayscale.

[0021] Step S3, which involves constructing the original lung CT image segmentation model based on a convolutional encoder, a Transformer encoder, a fusion module, and a corresponding decoder, specifically includes the following steps:

[0022] The acquired image data is input into the convolutional encoder module to learn local features, and simultaneously input into the Transformer encoder module to learn global features.

[0023] The acquired local and global features are fused using a fusion module to obtain the encoder's output.

[0024] Based on the convolutional encoder module, a corresponding decoder is constructed; and the acquired features are input into the decoder for decoding.

[0025] The features output by the decoder are processed using the softmax function to obtain the final probabilities of features of different categories.

[0026] The convolutional encoder module is specifically composed of several convolutional modules, each of which includes two Conv-IN-LeakyReLU modules;

[0027] Each Conv-IN-LeakyReLU module consists of a convolutional layer, an InstanceNorm layer, and a LeakyReLU layer, which are sequentially connected.

[0028] The processing function of the convolutional encoder module is Y. o =LeakyReLU(IN(Conv(X))), where Y o X represents the output features of the encoder module, X represents the input features, Conv() is the convolutional layer processing function, IN() is the InstanceNorm layer processing function, and LeakyReLU() is the LeakyReLU layer processing function.

[0029] The convolutional layer is a three-dimensional convolutional layer;

[0030] The processing formula for the InstanceNorm layer is as follows: Where Y is the processed feature, X is the input feature, μ is the mean of the input feature, σ is the variance of the input feature, ε is the error coefficient, γ is the first learning parameter, and β is the second learning parameter;

[0031] The processing formula for the LeakyReLU layer is: Where y i The processed features are those after the activation function; x i Let be the i-th value in the feature map; a is a fixed constant.

[0032] The first Conv-IN-LeakyReLU module in each convolutional module is used for downsampling operations, with a kernel size of 3 and a kernel stride of 2.

[0033] The Transformer encoder module specifically includes a patch bedding module, several consecutive Transformer blocks, and several downsampling modules; each adjacent Transformer block is connected to a downsampling module.

[0034] The downsampling module uses trilinear interpolation and then uses a convolution with a kernel size of 1 to adjust the number of channels in the feature map, thereby ensuring that the number of channels in the feature map is doubled.

[0035] The patchembedding module is used to divide an image into a series of non-overlapping patches and project each patch into a high-dimensional space;

[0036] Each Transformer block includes a first normalization layer, a ShiftDW layer, a second normalization layer, and a multilayer perceptron layer. The input features are normalized by the first normalization layer and then processed by the ShiftDW layer. The processing result of the ShiftDW layer is concatenated with the input features to obtain the second input features. The second input features are input to the second normalization layer for processing and then processed by the multilayer perceptron layer. The processing result of the multilayer perceptron layer is concatenated with the second input features and used as the output features of the current Transformer block.

[0037] The calculation formula for the Transformer block is:

[0038]

[0039]

[0040] In the formula This represents the output of the l-th layer ShiftDW module; ShiftDW() is the ShiftDW layer processing function; z l This represents the output of the l-th Transformer layer; BN() is the processing function for the first and second normalized layers; MLP() is the processing function for the multilayer perceptron layer, and the multilayer perceptron has 2 layers; the calculation formula for MLP() is:

[0041] MLP=Linear2(GELU(Linear1(x)))

[0042] In the formula, MLP is the processing function; x is the input feature; Linear1 and Linear2 are fully connected modules; GELU() is the processing function of the Gaussian error linear unit, and x is the input feature, erf() is the intermediate processing function, and

[0043] The ShiftDW layer replaces the multi-head self-attention module in the Transformer block. Specifically, the ShiftDW layer includes a series of sequential shift operations, a first pointwise convolution operation, a depthwise convolution operation, and a second pointwise convolution operation. The calculation formula for the ShiftDW layer is:

[0044] ShiftDW(z)=PWConv2(DWConv(PWConv1(Shift(x))))

[0045] In the formula, ShiftDW(z) represents the output feature of the ShiftDW layer; x represents the input feature; PWConv1() is the first pointwise convolution operation function with a kernel size of 1; DWConv() is the depthwise convolution operation function; PWConv2() is the second pointwise convolution operation function with a kernel size of 1; Shift() is the shift operation function, and the calculation formula for the shift operation is:

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053] In the formula The z[] represents the feature after the shift operation; z[] represents the input feature; D represents the spatial depth; H represents the spatial width; W represents the spatial height; γ represents the scaling factor; and C represents the number of channels.

[0054] The process of fusing the acquired local and global features using a fusion module to obtain the encoder output specifically includes the following steps:

[0055] The fusion module includes a global attention module and a local attention module;

[0056] The global attention module uses global average pooling to obtain the weights of different channels, and then uses pointwise convolution as the channel context aggregator; the local attention module uses pointwise convolution as the context aggregator, while maintaining the same resolution as the original feature map, thus preserving more spatial information.

[0057] The calculation formula for the fusion module is:

[0058] z a =Cat(z) c ,z t )

[0059] z = Conv(z) a )

[0060] z l =Local(z)

[0061] z g=Global(z)

[0062] z o =z l +z g

[0063] In the formula z a This is the concatenation of the output features of the convolutional encoder and the output features of the Transformer encoder; Cat() is the concatenation function in the channel dimension; z c z represents the output features of the convolutional encoder. t z represents the output features of the Transformer encoder; z represents the input features; Conv() is the convolution operation function, and the kernel size is C×2C×1×1×1; z l The features output by the local attention module; z g z represents the output feature of the global attention module. o The output characteristics of the fusion module;

[0064] The formula for calculating the global attention module is:

[0065] Global(z)=BN2(Conv2(ReLU(BN1(Conv1(GAP(z))))))

[0066] In the formula, Global(z) represents the output feature of the global attention module; z represents the input feature; and GAP() is the global average pooling function. D is the spatial depth, H is the spatial width, W is the spatial height, and z [:,i,j,k] The input features are defined by Conv1(), which is the processing function of the first convolutional layer, and the kernel size is [value missing]. C represents the number of channels in the input feature, r is the scaling factor; BN1() is the first batch normalization function; ReLU() is the activation function; Conv2() is the second convolutional layer processing function, and the kernel size is... BN2() is the second batch normalization function;

[0067] The formula for calculating the local attention module is:

[0068] Local(z)=BN2(Conv2(ReLU(BN1(Conv1(z)))))

[0069] In the formula, Local(z) represents the feature output by the local attention module.

[0070] The construction of the decoder based on the convolutional encoder module specifically includes the following steps:

[0071] The decoder includes several convolutional blocks, the same number as the convolutional encoder module. Each convolutional module of the decoder includes two Conv-IN-LeakyReLU modules, and an upsampling module is also included between two adjacent Conv-IN-LeakyReLU modules. The upsampling module is implemented using deconvolution. Each stage of the decoder connects the features of the corresponding parallel encoder through skip connections, and then enters the next stage of the decoder through upsampling.

[0072] The training described in step S4 specifically includes the following steps:

[0073] The output of the original lung CT image segmentation model and the true segmentation labels are input into the loss function, and the original lung CT image segmentation model is optimized based on the result of the loss function.

[0074] The loss function L is calculated as L = αL Dice +βL CE L Dice For Dice's loss, L CE Here, α is the weight value of the Dice loss, and β is the weight value of the cross-entropy loss.

[0075] Dice loss L Dice The calculation formula is Where TP is the probability of a true positive result, FP is the probability of a false positive result, and FN is the probability of a false negative result;

[0076] Cross-entropy loss L CE The calculation formula is Where N is the number of samples, y i For authentic labels, This refers to the prediction results output by the model.

[0077] The present invention also provides an imaging method including the aforementioned deep learning-based lung CT image segmentation method, specifically including the following steps:

[0078] A. Obtain the original lung CT image to be segmented;

[0079] B. Using the deep learning-based lung CT image segmentation method described above, the original lung CT image to be segmented obtained in step A is segmented.

[0080] C. Mark and perform secondary imaging on the original lung CT image obtained in step B using the image segmentation results obtained in step A, and output a lung CT image with the lung CT image segmentation results to complete the imaging process.

[0081] The lung CT image segmentation and imaging method provided by this invention fully leverages the advantages of convolution and Transformer to achieve more accurate image segmentation results. The parallel encoder structure designed in this invention can accelerate its convergence speed while retaining the powerful global modeling capabilities of Transformer, enabling it to exhibit strong segmentation performance even on small datasets. The fusion module of this invention can effectively fuse features from different branches, thereby achieving more accurate segmentation results. The Transformer module of this invention uses shift operations to replace the original multi-head self-attention operation, which can reduce computational complexity and achieve faster inference. Moreover, it adopts enhanced shortcut connections to avoid feature collapse and generate diverse features. Therefore, this invention has high reliability, good accuracy, and is objective and scientific. Attached Figure Description

[0082] Figure 1 This is a schematic diagram of the segmentation method of the present invention.

[0083] Figure 2 This is a schematic diagram of the Transformer encoder module of the segmentation method of the present invention.

[0084] Figure 3 This is a schematic diagram of the fusion module of the segmentation method of the present invention.

[0085] Figure 4 This is a schematic diagram of the imaging method of the present invention. Detailed Implementation

[0086] like Figure 1 The diagram shown is a flowchart of the segmentation method of the present invention: The lung CT image segmentation method based on deep learning provided by the present invention includes the following steps:

[0087] S1. Obtain the existing lung CT image dataset;

[0088] S2. Preprocess the data obtained in step S1 to obtain the training dataset; specifically, this includes the following steps:

[0089] Preprocessing includes image cropping, resampling, and normalization operations;

[0090] Image cropping involves setting a pixel threshold to obtain the bounding box of the desired target.

[0091] The obtained bounding box is then resampled.

[0092] After resampling, the image is normalized using Z-score:

[0093]

[0094] In the formula z i The normalized grayscale value; x i σ represents the original grayscale value; μ represents the mean of the image grayscale value; σ represents the standard deviation of the image grayscale value.

[0095] S3. Based on the convolutional encoder, Transformer encoder, fusion module, and corresponding decoder, construct the original lung CT image segmentation model; specifically including the following steps:

[0096] The acquired image data is input into the convolutional encoder module to learn local features, and simultaneously input into the Transformer encoder module to learn global features.

[0097] The acquired local and global features are fused using a fusion module to obtain the encoder's output.

[0098] Based on the convolutional encoder module, a corresponding decoder is constructed; and the acquired features are input into the decoder for decoding.

[0099] The features output by the decoder are processed using the softmax function to obtain the final probabilities of features of different categories;

[0100] In specific implementation, the convolutional encoder module is composed of several convolutional modules, and each convolutional module includes two Conv-IN-LeakyReLU modules;

[0101] Each Conv-IN-LeakyReLU module consists of a convolutional layer, an InstanceNorm layer, and a LeakyReLU layer, which are sequentially connected.

[0102] The processing function of the convolutional encoder module is Y. o =LeakyReLU(IN(Conv(X))), where Y o X represents the output features of the encoder module, X represents the input features, Conv() is the convolutional layer processing function, IN() is the InstanceNorm layer processing function, and LeakyReLU() is the LeakyReLU layer processing function.

[0103] The convolutional layer is a three-dimensional convolutional layer;

[0104] The processing formula for the InstanceNorm layer is as follows: Where Y is the processed feature, X is the input feature, μ is the mean of the input feature, σ is the variance of the input feature, ε is the error coefficient, γ is the first learning parameter, and β is the second learning parameter;

[0105] The processing formula for the LeakyReLU layer is: Where y i The processed features are those after the activation function; x i Let be the i-th value in the feature map; a is a fixed constant.

[0106] The first Conv-IN-LeakyReLU module in each convolutional module is used for downsampling operations, with a kernel size of 3 and a kernel stride of 2.

[0107] Transformer encoder module (such as...) Figure 2 As shown in the figure, it specifically includes a patchbedding module, several consecutive Transformer blocks, and several downsampling modules; each adjacent Transformer block is connected to a downsampling module.

[0108] The downsampling module uses trilinear interpolation and then uses a convolution with a kernel size of 1 to adjust the number of channels in the feature map, thereby ensuring that the number of channels in the feature map is doubled.

[0109] The patchembedding module is used to divide an image into a series of non-overlapping patches and project each patch into a high-dimensional space;

[0110] Each Transformer block includes a first normalization layer, a ShiftDW layer, a second normalization layer, and a multilayer perceptron layer. The input features are normalized by the first normalization layer and then processed by the ShiftDW layer. The processing result of the ShiftDW layer is concatenated with the input features to obtain the second input features. The second input features are input to the second normalization layer for processing and then processed by the multilayer perceptron layer. The processing result of the multilayer perceptron layer is concatenated with the second input features and used as the output features of the current Transformer block.

[0111] The calculation formula for the Transformer block is:

[0112]

[0113]

[0114] In the formula This is the output of the ShiftDW module; ShiftDW() is the ShiftDW layer processing function; z lThis represents the output of the l-th Transformer layer; BN() is the processing function for the first and second normalized layers; MLP() is the processing function for the multilayer perceptron layer, and the multilayer perceptron has 2 layers; the calculation formula for MLP() is:

[0115] MLP=Linear2(GELU(Linear1(x)))

[0116] In the formula, MLP is the processing function; Linear1() is the first fully connected operation; x is the input feature; Linear2() is the second fully connected operation; GELU() is the processing function of the Gaussian error linear unit, and x is the input feature, erf() is the intermediate processing function, and

[0117] The ShiftDW layer replaces the multi-head self-attention module in the Transformer block. Specifically, the ShiftDW layer includes a series of sequential shift operations, a first pointwise convolution operation, a depthwise convolution operation, and a second pointwise convolution operation. The calculation formula for the ShiftDW layer is:

[0118] ShiftDW(z)=PWConv2(DWConv(PWConv1(Shift(x))))

[0119] In the formula, ShiftDW(z) represents the output feature of the ShiftDW layer; x represents the input feature; PWConv1() is the first pointwise convolution operation function with a kernel size of 1; DWConv() is the depthwise convolution operation function; PWConv2() is the second pointwise convolution operation function with a kernel size of 1; Shift() is the shift operation function, and the calculation formula for the shift operation is:

[0120]

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127] In the formula The features obtained after the shift operation are: z[], ...

[0128] The process of fusing the encoder's output involves the following steps:

[0129] The fusion module includes a global attention module and a local attention module (such as...). Figure 3 (as shown);

[0130] The global attention module uses global average pooling to obtain the weights of different channels, and then uses pointwise convolution as the channel context aggregator; the local attention module uses pointwise convolution as the context aggregator, while maintaining the same resolution as the original feature map, thus preserving more spatial information.

[0131] The calculation formula for the fusion module is:

[0132] z a =Cat(z) c ,z t )

[0133] z = Conv(z) a )

[0134] z l =Local(z)

[0135] z g =Global(z)

[0136] z o =z l +z g

[0137] In the formula z a This is the concatenation of the output features of the convolutional encoder and the output features of the Transformer encoder; Cat() is the concatenation function in the channel dimension; z c z represents the output features of the convolutional encoder. t z represents the output features of the Transformer encoder; z represents the input features; Conv() is the convolution operation function, and the kernel size is C×2C×1×1×1; z l The features output by the local attention module; z g z represents the output feature of the global attention module. o The output characteristics of the fusion module;

[0138] The formula for calculating the global attention module is:

[0139] Global(z)=BN2(Conv2(ReLU(BN1(Conv1(GAP(z))))))

[0140] In the formula, Global(z) represents the output feature of the global attention module; z represents the input feature; and GAP() is the global average pooling function. D is the spatial depth, H is the spatial width, W is the spatial height, and z [:,i,j,k] The input features are defined by Conv1(), which is the processing function of the first convolutional layer, and the kernel size is [value missing]. C represents the number of channels in the input feature, r is the scaling factor; BN1() is the first batch normalization function; ReLU() is the activation function; Conv2() is the second convolutional layer processing function, and the kernel size is... BN2() is the second batch normalization function;

[0141] The formula for calculating the local attention module is:

[0142] Local(z)=BN2(Conv2(ReLU(BN1(Conv1(z)))))

[0143] In the formula, Local(z) represents the feature output by the local attention module;

[0144] Building a decoder specifically includes the following steps:

[0145] The decoder includes several convolutional blocks, the same number as the convolutional encoder module. Each convolutional module of the decoder includes two Conv-IN-LeakyReLU modules, and an upsampling module is also included between two adjacent Conv-IN-LeakyReLU modules. The upsampling module is implemented using deconvolution. Each stage of the decoder connects the features of the corresponding parallel encoder through skip connections, and then enters the next stage of the decoder through upsampling.

[0146] S4. Using the training dataset obtained in step S2, train the original lung CT image segmentation model constructed in step S3 to obtain the lung CT image segmentation model; specifically including the following steps:

[0147] The output of the original lung CT image segmentation model and the true segmentation labels are input into the loss function, and the original lung CT image segmentation model is optimized based on the result of the loss function.

[0148] The loss function L is calculated as L = αL Dice +βL CE L Dice For Dice's loss, L CEHere, α is the weight value of the Dice loss, and β is the weight value of the cross-entropy loss.

[0149] Dice loss L Dice The calculation formula is Where TP is the probability of a true positive result, FP is the probability of a false positive result, and FN is the probability of a false negative result;

[0150] Cross-entropy loss L CE The calculation formula is Where N is the number of samples, y i For authentic labels, The predicted results output by the model;

[0151] S5. Using the lung CT image segmentation model obtained in step S4, segment the actual lung CT image.

[0152] The effectiveness of the segmentation method of the present invention will be demonstrated through the following experiments.

[0153] UNETR: Its encoder is a hybrid architecture of convolution and Transformer, while the decoder is a convolutional architecture. In the encoder, it first uses Transformer to extract features, and then uses convolution as skip connections to obtain features at different scales.

[0154] nnFormer proposes a novel strategy combining Transformer and convolution. In the encoder, it uses a 3D SwingTransformer for feature extraction, but between adjacent SwingTransformer blocks, it uses convolution to downsample the feature map to obtain multi-scale feature representations. In the decoder, it also uses a 3D SwingTransformer and employs deconvolution to restore the image resolution.

[0155] nnUNet: This method is a pure convolutional segmentation network with the same network architecture as UNet. nnUNet can adaptively adjust the model depth and some other model hyperparameters based on the dataset.

[0156] In the experiment, Dice, Precision, Sensitivity, HD95, and ASSD evaluation metrics were used to assess the performance of the method. Among these, the Dice metric is commonly used to evaluate the quality of medical image segmentation; a value close to 1 indicates a more accurate segmentation result. Its calculation formula is as follows:

[0157]

[0158] HD95 and ASSD represent the distance between two sets; the smaller the value, the better the segmentation performance. The calculation formula is as follows:

[0159]

[0160]

[0161] In the formula and These represent the boundary points of G and M, respectively. It is the cardinality of the G boundary. Is point x and The shortest distance between all points. The formulas for calculating Precision and Sensitivity are as follows:

[0162]

[0163]

[0164] In the formula, TP, FN, and FP represent the probability of a true positive, the probability of a false negative, and the probability of a false positive, respectively.

[0165] The data used in this experiment was a lung cancer dataset. The proposed method and the control method were evaluated on the same test set, and the experimental results are shown in Table 1. The numbers in parentheses represent the standard deviation.

[0166] Table 1. Comparison of segmentation performance between the segmentation method of the present invention and the comparative method.

[0167] method Dice Precision Recall HD95 ASSD UNETR 0.533(0.239) 0.588(0.284) 0.595(0.241) 103.8(70.50) 22.90(21.85) nnFormer 0.664(0.205) 0.724(0.237) 0.685(0.229) 36.50(45.54) 8.06(10.83) nnUNet 0.641(0.228) 0.644(0.274) 0.737(0.204) 74.15(78.63) 17.40(22.15) This invention 0.714(0.169) 0.803(0.166) 0.701(0.230) 27.77(43.96) 5.59(9.64)

[0168] As shown in Table 1, the method proposed in this invention achieves the best performance among all methods on Dice and ASSD. Model evaluation on the test set yielded average Dice, Precision, Sensitivity, HD95, and ASSD values ​​of 0.520 (0.341), 0.686 (0.306), 0.530 (0.351), 33.44 (50.35), and 13.60 (28.50) across five experiments, respectively.

[0169] In the comparison, nnFormer and nnUNet also achieved good segmentation results. The segmentation results of nnUNet demonstrate that using only convolution for feature extraction in the encoder is insufficient to model the global features of the image, resulting in low segmentation performance. The segmentation results of nnFormer and CFC show that hybrid architectures can effectively model the global features of the image, but different combinations lead to different segmentation performance. On the lung cancer segmentation dataset, parallel architectures can more effectively combine Transformer and convolution, leveraging their advantages more effectively.

[0170] like Figure 4 The diagram shown is a flowchart of the imaging method of the present invention: The imaging method provided by the present invention, which includes the deep learning-based lung CT image segmentation method, specifically includes the following steps:

[0171] A. Obtain the original lung CT image to be segmented;

[0172] B. Using the deep learning-based lung CT image segmentation method described above, the original lung CT image to be segmented obtained in step A is segmented.

[0173] C. Mark and perform secondary imaging on the original lung CT image obtained in step B using the image segmentation results obtained in step A, and output a lung CT image with the lung CT image segmentation results to complete the imaging process.

[0174] like Figure 4 The diagram shown is a flowchart of the imaging method of the present invention: The imaging method provided by the present invention, which includes the deep learning-based lung CT image segmentation method, specifically includes the following steps:

[0175] A. Obtain the original lung CT image to be segmented;

[0176] B. Using the deep learning-based lung CT image segmentation method described above, the original lung CT image to be segmented obtained in step A is segmented.

[0177] C. Mark and perform secondary imaging on the original lung CT image obtained in step B using the image segmentation results obtained in step A, and output a lung CT image with the lung CT image segmentation results to complete the imaging process.

[0178] In practical implementation, the imaging method of this invention can be used in existing lung CT image acquisition devices, such as CT scanners. Specifically, the imaging method of this invention is integrated into an existing CT image acquisition device. Then, the original lung CT image is acquired using existing technology. Next, the imaging method described above is used to perform secondary imaging on the original lung CT image, resulting in a lung CT image with lung CT image segmentation results. This lung CT image with imaging results is then directly output. In this way, medical professionals (including clinicians, radiologists, or laboratory personnel) can obtain lung CT images with segmentation results, greatly facilitating existing users.

Claims

1. A deep learning-based lung CT image segmentation method, comprising the following steps: S1. Obtain an existing dataset of lung CT images; S2. The data obtained in step S1 is preprocessed to obtain the training dataset; S3. Based on the convolutional encoder, Transformer encoder, fusion module and corresponding decoder, construct the original model for lung CT image segmentation; wherein The convolutional encoder module is specifically composed of several convolutional modules, each of which includes two Conv-IN-LeakyReLU modules; Each Conv-IN-LeakyReLU module consists of a convolutional layer, an InstanceNorm layer, and a LeakyReLU layer, which are sequentially connected. The processing function of the convolutional encoder module is wherein is the output feature of the encoder module, is the input feature, is the convolutional layer processing function; is the InstanceNorm layer processing function; is the LeakyReLU layer processing function; The convolutional layer is a three-dimensional convolutional layer; The processing formula of the InstanceNorm layer is wherein is a processing result feature, is an input feature, is a mean value of the input feature, is a variance of the input feature, is an error coefficient, is a first learning parameter; is a second learning parameter; The processing formula of the LeakyReLU layer is wherein is the processed feature after the activation function; is the i-th value in the feature map; is a fixed constant; The first Conv-IN-LeakyReLU module in each convolutional module is used for downsampling operations, with a kernel size of 3 and a kernel stride of 2. The Transformer encoder module specifically includes a patchbedding module, several consecutive Transformer blocks, and several downsampling modules; each adjacent Transformer block is connected to a downsampling module. The downsampling module uses trilinear interpolation and then uses a convolution with a kernel size of 1 to adjust the number of channels in the feature map, thereby ensuring that the number of channels in the feature map is doubled. The patch embedding module is used to divide an image into a series of non-overlapping patches and project each patch into a high-dimensional space; Each Transformer block includes a first normalization layer, a ShiftDW layer, a second normalization layer, and a multilayer perceptron layer. The input features are normalized by the first normalization layer and then processed by the ShiftDW layer. The processing result of the ShiftDW layer is concatenated with the input features to obtain the second input features. The second input features are input to the second normalization layer for processing and then processed by the multilayer perceptron layer. The processing result of the multilayer perceptron layer is concatenated with the second input features and used as the output features of the current Transformer block. The calculation formula for the Transformer block is: wherein is the output of the ShiftDW module; is the ShiftDW layer processing function; is the output of the l-th Transformer; is the processing function of the first normalization layer and the second normalization layer; is the multi-layer perceptron layer processing function, and the number of layers of the multi-layer perceptron is 2 layers; The calculation formula of is: wherein is a processing function; is a first fully connected operation; is an input feature; is a second fully connected operation; is a processing function for a Gaussian error linear unit, and , is an input feature, is an intermediate processing function and ; S4. Using the training dataset obtained in step S2, train the original lung CT image segmentation model constructed in step S3 to obtain the lung CT image segmentation model. S5. Using the lung CT image segmentation model obtained in step S4, segment the actual lung CT image.

2. The lung CT image segmentation method based on deep learning according to claim 1, characterized in that... Step S2 involves preprocessing the data obtained in step S1, specifically including the following steps: Preprocessing includes image cropping, resampling, and normalization operations; Image cropping involves setting a pixel threshold to obtain the bounding box of the desired target. The obtained bounding box is then resampled. After resampling, the image is normalized using Z-score: In the formula The grayscale value is the normalized value. This is the original grayscale value; The mean value of the image grayscale; denoted as the standard deviation of the image grayscale.

3. The lung CT image segmentation method based on deep learning according to claim 2, characterized in that... Step S3, which involves constructing the original lung CT image segmentation model based on a convolutional encoder, a Transformer encoder, a fusion module, and a corresponding decoder, specifically includes the following steps: The acquired image data is input into the convolutional encoder module to learn local features, and simultaneously input into the Transformer encoder module to learn global features. The acquired local and global features are fused using a fusion module to obtain the encoder's output. Based on the convolutional encoder module, a corresponding decoder is constructed; and the acquired features are input into the decoder for decoding. The features output by the decoder are processed using the softmax function to obtain the final probabilities of features of different categories.

4. The lung CT image segmentation method based on deep learning according to claim 3, characterized in that... The ShiftDW layer replaces the multi-head self-attention module in the Transformer block. Specifically, the ShiftDW layer includes a series of sequential shift operations, a first pointwise convolution operation, a depthwise convolution operation, and a second pointwise convolution operation. The calculation formula for the ShiftDW layer is: In the formula The output features of the ShiftDW layer; For input features; This is the first pointwise convolution operation function, with a kernel size of 1; This is the depthwise convolution operation function; This is the second pointwise convolution operation function, with a kernel size of 1; Let be the shift operation function, and let be the formula for calculating the shift operation: In the formula The features obtained after the shift operation; For input features; For spatial depth; For space width; For spatial height; It is a scaling factor; This represents the number of channels.

5. The lung CT image segmentation method based on deep learning according to claim 4, characterized in that... The process of fusing the acquired local and global features using a fusion module to obtain the encoder output specifically includes the following steps: The fusion module includes a global attention module and a local attention module; The global attention module uses global average pooling to obtain the weights of different channels, and then uses pointwise convolution as the channel context aggregator; the local attention module uses pointwise convolution as the context aggregator, while maintaining the same resolution as the original feature map, thus preserving more spatial information. The calculation formula for the fusion module is: In the formula This is the feature obtained by concatenating the output features of the convolutional encoder and the output features of the Transformer encoder. This is a processing function for concatenation at the channel level; The output features of the convolutional encoder; The output features of the Transformer encoder; For input features; Let be the convolution operation function, and let the convolution kernel size be . ; Features output by the local attention module; The output features of the global attention module; The output characteristics of the fusion module; The formula for calculating the global attention module is: In the formula The output features of the global attention module; For input features; This is the global average pooling function, and , For spatial depth, For space width, For spatial height, For input features; This is the processing function for the first convolutional layer, and the kernel size is... , The number of channels for the input feature. This is the scaling factor; This is the first batch normalization function; For activation functions; This is the processing function for the second convolutional layer, and the kernel size is... ; This is the second batch normalization function; The formula for calculating the local attention module is: In the formula These are the features output by the local attention module.

6. The lung CT image segmentation method based on deep learning according to claim 5, characterized in that... The construction of the decoder based on the convolutional encoder module specifically includes the following steps: The decoder includes several convolutional blocks, the same number as the convolutional encoder module. Each convolutional module of the decoder includes two Conv-IN-LeakyReLU modules, and an upsampling module is also included between two adjacent Conv-IN-LeakyReLU modules. The upsampling module is implemented using deconvolution. Each stage of the decoder connects the features of the corresponding parallel encoder through skip connections, and then enters the next stage of the decoder through upsampling.

7. The lung CT image segmentation method based on deep learning according to claim 6, characterized in that... The training described in step S4 specifically includes the following steps: The output of the original lung CT image segmentation model and the true segmentation labels are input into the loss function, and the original lung CT image segmentation model is optimized based on the result of the loss function. The formula for calculating the loss function L is as follows: ,in For Dice's loss, For cross-entropy loss, These are the weights of the Dice loss. These are the weight values ​​for the cross-entropy loss; Dice loss The calculation formula is ,in This represents the probability that the result is a true positive. This represents the probability of a false positive result. This represents the probability of a false negative result. Cross-entropy loss The calculation formula is ,in For the sample size, For authentic labels, This refers to the prediction results output by the model.

8. An imaging method comprising the lung CT image segmentation method based on deep learning as described in any one of claims 1 to 7, specifically comprising the following steps: A. Obtain the original lung CT image to be segmented; B. Using the deep learning-based lung CT image segmentation method according to any one of claims 1 to 7, perform image segmentation on the original lung CT image to be segmented obtained in step A; C. The image segmentation results obtained in step B are marked and re-imagined on the original lung CT image in step A, and the lung CT image with the lung CT image segmentation results is output to complete the imaging.

Images