A Pulmonary Embolism Segmentation Method Based on LSCU-net

Abstract

The present invention discloses a pulmonary embolism segmentation method based on LSCU-net, which solves the problem that the use of only the U-net model to segment the lesion area cannot meet the requirements for segmentation accuracy, segmentation speed, segmentation accuracy, etc. due to the diversity of the shapes of the lesion areas and the differences caused by different organ structures, and belongs to the field of deep learning semantic segmentation. The present invention includes: proposing a pulmonary embolism segmentation method based on LSCU-net. The present invention improves the U-net by reasonably designing and adding a CBAM attention module and a Bi-LSTM bidirectional long short-term memory module, which can realize the fusion of feature information at different scales and the extraction of sequence information between pulmonary embolism CTPA slices. The present invention has the characteristics of small model volume, low cost, simple structure, etc., and can be used to realize the automatic segmentation of the pulmonary embolism lesion area. In addition, it also has great application potential in the segmentation of lesion areas in other medical images.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical images, and particularly relates to a pulmonary embolism segmentation method based on LSCU-net. Background Art

[0002] Pulmonary embolism (PE) refers to a group of syndromes in which endogenous emboli from the venous system or the right heart block the pulmonary artery or its branches, causing disorders of pulmonary circulation and respiratory function. The emboli include thrombus, fat cells, amniotic fluid, air, tumor cells, bacteria, etc. The clinical symptoms mainly include dyspnea, abdominal pain, syncope or shock, hemoptysis, etc. Pulmonary embolism includes pulmonary thromboembolism (PTE), fat embolism syndrome, amniotic fluid embolism, air embolism, tumor embolism, etc. Among them, PTE is the most common type of pulmonary embolism.

[0003] According to foreign epidemiological survey data, the missed diagnosis rate of pulmonary embolism disease is about 70%, and the probability of death of patients without a clear diagnosis before death is as high as over 40%. Precise and early identification and screening of pulmonary embolism is the basis for the diagnosis and treatment of pulmonary embolism, which can greatly reduce the death rate caused by pulmonary embolism. Therefore, rapid and accurate diagnosis of pulmonary embolism is particularly important clinically. However, at present, in clinical practice, the discrimination and evaluation of medical images are mostly completed by physicians. Radiologists need to visually judge each two-dimensional pulmonary CTPA slice image, carefully observe every detail of the pulmonary medical image to determine whether the patient has pulmonary embolism. However, since CTPA scans will generate hundreds of two-dimensional sequence images, diagnosing pulmonary embolism manually is a very time-consuming and laborious task, and the characteristics of pulmonary embolism are similar to those of other surrounding abdominal tissues, which is prone to confusion, and the accuracy of judgment is also affected by human factors such as the experience and concentration of physicians.

[0004] The limitations of the prior art in solving the problem of pulmonary embolism segmentation are as follows:

[0005] 1. There are some difficulties in the pulmonary embolism segmentation of the neural network model U-net commonly used in the field of medical image segmentation. The main reasons are:

[0006] 1.1 The volume of pulmonary embolism is relatively small compared to the volume of the lungs, and the extremely unbalanced positive and negative samples will seriously affect the learning and prediction effects of the model.

[0007] 1.2 U-net generally can only learn the spatial information within the CTPA slice and it is difficult to effectively utilize the sequence information between CTPA slices.

[0008] 2. Currently, the existing U-net-based pulmonary embolism segmentation methods have problems such as inaccurate segmentation and slow speed, and cannot well meet the clinical needs for rapid and accurate diagnosis of pulmonary embolism.

[0009] 3. Existing methods rely on a large number of labeled datasets for model training, while the labeled data of pulmonary embolism are relatively scarce, increasing the training difficulty of the model.

[0010] 4. The existing model structures are relatively complex and computationally intensive, resulting in poor real-time segmentation effects.

[0011] 5. The design of effective modules is lacking, such as attention modules, bidirectional LSTM modules, etc., which cannot effectively fuse feature information of different scales and cannot effectively extract the sequence information between CTPA slices.

[0012] 6. The lack of effective model optimization strategies makes it difficult to improve the model effects.

[0013] 7. The lack of methods for interpretive analysis of the model makes it impossible to effectively evaluate the model effects.

[0014] 8. The existing models generally have the phenomenon of overfitting, resulting in weak generalization ability.

[0015] 9. The lack of effective large-scale model training and tuning methods limits the improvement space of the model effects.

[0016] In summary, the existing technologies have obvious limitations in pulmonary embolism segmentation, and it is necessary to further design effective model structures, modules and training strategies to improve the segmentation effect and speed, reduce the dependence on data annotation, and enhance the generalization ability of the model. This is the main technical problem to be solved by the present invention. Summary of the Invention

[0017] The purpose of the present invention is to provide a pulmonary embolism segmentation method based on LSCU-net to solve the technical problems existing in the above-mentioned existing technologies.

[0018] To achieve the above purpose, the technical solution of the present invention is:

[0019] A pulmonary embolism segmentation method based on LSCU-net is provided, including the following steps:

[0020] (1) Obtain CTPA images of pulmonary embolism and obtain a pulmonary embolism dataset through image preprocessing, and divide it into a training set and a test set;

[0021] (2) Build an improved LSCU-net neural network model;

[0022] (3) Use the training set as training samples to train the improved LSCU-net neural network obtained in step 2;

[0023] (4) Use the trained LSCU-net to perform segmentation testing on the pulmonary embolism samples in the test set.

[0024] A further setting of the present invention is that in step (1), the data set is provided by professional doctors in the hospital. Image preprocessing means first using the slicing method for CTPA images to process the three-dimensional dicom format CTPA images into two-dimensional jpg format CTPA images, and then using the cropping method to improve the quality of the two-dimensional CTPA images.

[0025] A further setting of the present invention is that in step (1), the pulmonary embolism data set marked by doctors is randomly divided into a training set and a test set according to a ratio of 9:1.

[0026] A further setting of the present invention is that in step (2), the U-net neural network is improved, and the specific improvement content is as follows:

[0027] (1) Introduce the convolutional module CBAM-Conv with the lightweight attention module CBAM with residual in the encorder module. The specific structure is: use CBAM-Conv in the encoder structure area for feature extraction. This module is composed of a layer of CBAM and two basic convolutional blocks in series. The CBAM layer is obtained by connecting the channel attention module CA and the spatial attention module SA in series, which is used to change the learning strategy of the neural network model and effectively fuse the feature information of different scales and different levels, strengthen the attention of the neural network to the information of the region of interest. After passing through the CBAM layer, the image feature data will pass through the channel attention module CA and the spatial attention module SA in sequence, and distinguish the importance of features by adjusting the weights, focusing on the features of the key region from the spatial dimension and the channel dimension, and suppressing unimportant features such as the background;

[0028] The basic convolutional block is specifically composed of a 3×3 convolutional layer, a batch normalization Bn layer and an activation function ReLU in series. When the image feature data passes through the basic convolutional block, feature extraction of the image features will be performed, the information of the input picture will be extracted, the batch normalization will change the data distribution to avoid overfitting of the model, and the activation judgment will increase the nonlinearity to avoid overfitting of the model.

[0029] (2) The Bi-LSTM module is a bidirectional long short-term memory module, which is a combination of a forward LSTM and a backward LSTM. The structure of a unidirectional LSTM module specifically consists of a forget gate, an input gate, and an output gate. After the image feature data is input into the LSTM module, it will first perform feature fusion add in the feature dimension with the input data in the previous time state, and then the image feature data is input into the forget gate. After passing through a sigmoid activation function, it is multiplied by the CELL cell state used to learn long-term dependence information in the LSTM module to forget the CELL cell state and discard invalid information. Subsequently, the image feature data enters the input gate. After passing through the sigmoid and tanh activation functions respectively, they are multiplied, and then feature fusion add is performed with the CELL cell state in the feature dimension to remember the important information in the image feature data. Finally, the image feature data is input into the output gate. After the image feature data and the CELL cell state pass through the sigmoid and tanh activation functions respectively and are multiplied, the finally output image feature data is obtained;

[0030] The Bi-LSTM module can transfer the state of the cells in the previous moment in both directions to the state of the cells in the next moment, achieving the purpose of learning and memorizing sequence information, and at the same time avoiding the vanishing gradient of the neural network. The L2Norm module realizes the feature normalization of image data through L2 regularization, simplifies the model, and avoids overfitting of the model.

[0031] A further setting of the present invention is: in step (3), the training set is input into the LSCU-net for training after data augmentation. The loss function is FTLoss + BCEloss, the optimizer is SGD, the learning rate adjustment strategy is the cosine annealing descent method, the initial learning rate is le-4, the training period is 100 epochs, and the batch-size is 8.

[0032] A further setting of the present invention is: in step (4), the training set is input into the LSCU-net for effect verification, and the segmentation result is obtained and the evaluation indexes (Jac, Dice Score, Map, Accuracy) are calculated.

[0033] Since the present invention adopts the above technical solutions, it has the following beneficial effects:

[0034] (1) The volume of pulmonary embolism is relatively small, and the distribution of positive and negative samples is extremely unbalanced, which will lead to poor learning effects of existing pulmonary embolism segmentation models based on U-net. By introducing a CBAM attention module with residual in the encoder module of U-net, the present invention can effectively improve the attention of the model to negative samples and improve the detection effect of the model on pulmonary embolism lesions.

[0035] (2) The existing U-net model can only learn the spatial information within a single CTPA slice and cannot effectively utilize the sequential information between CTPA slices. By introducing a bidirectional LSTM (Long Short-Term Memory) module into the bottleneck module of the U-net in the present invention, the sequential information between CTPA slices can be learned, which helps to identify the persistence and dynamic changes of lesions.

[0036] (3) The existing U-net model has limitations in feature extraction and model efficiency. By replacing the U-net module in the present invention, the optimization of the model structure is achieved, so that the model has improvements in both feature extraction effect and running efficiency.

[0037] (4) The existing pulmonary embolism segmentation methods rely on professional doctors for image analysis, which is highly subjective and time-consuming and laborious. The present invention uses a deep learning model to achieve automatic segmentation, which can improve the detection efficiency and reduce the workload of doctors.

[0038] (5) The existing methods are highly dependent on the dataset and the effects are unstable. By means of data augmentation and effect verification, etc. in the present invention, the stability and generalization ability of the model effects are improved.

[0039] (6) The existing methods lack the modeling of the correlation between pulmonary embolism lesions. By considering the sequential correlation information between lesions through the bidirectional LSTM module in the present invention, it helps to identify the overall structure of the lesions.

[0040] (7) The existing U-net model has a large number of parameters and is complex. By means of lightweight design in the present invention, the volume and complexity of the model are reduced.

[0041] Generally speaking, through the improvement of the model structure, this method realizes more accurate automatic segmentation of pulmonary embolism lesions, improves the detection efficiency and effect, reduces the workload of doctors, and has important clinical application value. Description of the Drawings

[0042] Figure 1 It is a schematic structural diagram of the LSCU-net model.

[0043] Figure 2 It is a schematic structural diagram of the module CBAM-Conv.

[0044] Figure 3 It is a schematic structural diagram of the module CBAM.

[0045] Figure 4 It is a schematic structural diagram of the module Conv-block.

[0046] Figure 5 It is a schematic structural diagram of the module SA.

[0047] Figure 6 It is a schematic structural diagram of module CA.

[0048] Figure 7 It is a schematic structural diagram of module LSTM.

[0049] Figure 8 It is a schematic structural diagram of module Bi-LSTM. Specific implementation solutions

[0050] The following combines the drawings of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

[0051] When the prior art deals with the problem of pulmonary embolism CTPA image segmentation, the following difficulties exist:

[0052] (1) The diversity of the shapes of pulmonary embolism lesion areas and the differences in organ structures make it difficult to segment the lesion areas only using the traditional U-net model. The U-net model mainly segments by learning local spatial information and cannot effectively utilize feature information of different scales and different levels.

[0053] (2) The low contrast between pulmonary embolism lesions and surrounding tissues is prone to confusion, and there is a risk of misdiagnosis relying solely on the experience of physicians.

[0054] (3) The traditional U-net model is large in volume, complex in structure, and large in calculation amount, and the real-time segmentation speed is difficult to meet the clinical needs.

[0055] The prior art has the following deficiencies in improving the pulmonary embolism segmentation effect:

[0056] (1) Only using the U-net model as the segmentation network cannot effectively utilize feature information of different scales and different levels, resulting in poor segmentation effect.

[0057] (2) Not using the attention mechanism or memory module cannot effectively focus on the information of the region of interest and cannot effectively extract sequence information, resulting in inaccurate segmentation.

[0058] (3) The U-net model is large in volume and large in calculation amount, and the real-time segmentation speed is difficult to meet the clinical needs.

[0059] The present invention aims to solve the above-mentioned problems existing in the prior art in the segmentation of pulmonary embolism CTPA images, and provides a pulmonary embolism segmentation method based on LSCU-net. By designing a CBAM attention module and a Bi-LSTM bidirectional long short-term memory module, the fusion of feature information at different scales and the effective extraction of sequence information are realized, the segmentation effect is improved, the misdiagnosis risk is reduced, and the real-time and accurate segmentation of pulmonary embolism lesions is achieved.

[0060] To this end, the present invention provides a pulmonary embolism segmentation method based on LSCU-net to solve the following technical problems:

[0061] 1. Difficulties existing in the traditional neural network model U-net in the pulmonary embolism segmentation task

[0062] 1.1. The U-net model cannot effectively utilize feature information at different scales, resulting in poor segmentation effects;

[0063] 1.2. The U-net model cannot effectively extract the sequence information between pulmonary embolism CTPA slices and cannot achieve accurate segmentation of the lesion area;

[0064] 1.3. The U-net model has a large volume and a large amount of calculation, and the real-time segmentation effect is poor.

[0065] 2. Problems existing in the segmentation of pulmonary embolism lesion areas only using the U-net model in the prior art

[0066] 2.1. Only using the U-net model cannot effectively solve the difference problems caused by the diversity of lesion area shapes and the differences in organ structures;

[0067] 2.2. The U-net model cannot achieve semantic associations between lesion areas and cannot effectively utilize the context information between lesion areas.

[0068] 3. A new model needs to be designed to solve the limitations of the above U-net model

[0069] 3.1. It can effectively fuse between feature information at different scales and enhance the model's attention to lesion area information;

[0070] 3.2. It can effectively extract and utilize the sequence information between pulmonary embolism CTPA slices to achieve accurate segmentation of the lesion area;

[0071] 3.3. The model has a small volume and high calculation efficiency, and can achieve real-time segmentation of pulmonary embolism lesion areas.

[0072] In summary, it is necessary to design a new LSCU-net-based pulmonary embolism segmentation method. By introducing the CBAM attention module and the Bi-LSTM bidirectional long short-term memory module, the fusion of feature information of different scales and the extraction of sequence information of the lesion area can be achieved, thereby effectively solving the limitations of the U-net model in pulmonary embolism segmentation.

[0073] To this end, the present invention discloses a pulmonary embolism segmentation method based on LSCU-net. This method effectively segments pulmonary embolism CTPA images by introducing a convolutional module (CBAM-Conv) with a residual lightweight attention module (CBAM) into the original U-net model and a bidirectional long short-term memory module (Bi-LSTM) in the Bottleneck layer. This method has the following technical solutions:

[0074] 1. Improved segmentation accuracy

[0075] By introducing CBAM-Conv into the encoder module, we can effectively fuse feature information at different scales, focusing more on regions of interest and improving segmentation accuracy. Furthermore, the Bi-LSTM effectively extracts sequence information between CTPA slices, avoiding information loss and further improving segmentation accuracy.

[0076] 2. Improved segmentation speed

[0077] Compared to U-net, the LSCU-net model is smaller, simpler, and requires less computation, resulting in faster overall execution. Furthermore, the optimized model utilizes a residual module for network optimization, which avoids the vanishing gradient problem, improves training stability, and reduces model training time.

[0078] 3. Improved segmentation efficiency

[0079] The LSCU-net model, by introducing CBAM-Conv and Bi-LSTM modules, integrates feature information at different scales, reducing the model's overreliance on single-scale information and improving information utilization. Furthermore, the Bi-LSTM module effectively extracts sequence information between CTPA slices, reducing information redundancy and computational complexity. These designs enhance the overall operational efficiency of the LSCU-net model.

[0080] 4. Improved user experience

[0081] Compared to U-net, the LSCU-net model significantly reduces computational effort and improves performance while maintaining high segmentation accuracy. This enables real-time pulmonary embolism segmentation, effectively assisting doctors in making rapid and accurate diagnoses and improving user experience.

[0082] In summary, by introducing the CBAM-Conv and Bi-LSTM modules into the U-net model, the present invention realizes the effective segmentation of CTPA images of pulmonary embolism, and has the advantages of small computational amount, fast speed, high efficiency and good user experience. This provides the possibility for the automatic segmentation of clinical pulmonary embolism lesion areas and also has a wide range of application prospects.

[0083] The present invention will be further described below in conjunction with embodiments.

[0084] As Figure 1 shown, the present invention also provides an improved U-net model, LSCU-net.

[0085] The improvements include: using the convolutional module CBAM-Conv with the lightweight attention module CBAM with residuals in the encorder module to replace the original convolutional module Conv, and using the bidirectional long short-term memory module Bi-LSTM in the bottleneck layer to replace the original module, where:

[0086] (1) In the convolutional module CBAM-Conv:

[0087] The CBAM-Conv module (as Figure 2 ) contains one CBAM layer and two convolutional layers Conv-Block (as Figure 4 ).

[0088] As Figure 3 shown, the CBAM layer of the lightweight and plug-and-play attention module is composed of a residual, a channel attention module (Channel Attention, CA) as Figure 5 and a spatial attention module (Spatial Attention, SA) as Figure 6 combined, which can effectively fuse feature information of different scales and different levels, and at the same time improve the network's feature extraction ability.

[0089] The formula for the CA module is:

[0090] Mc(F) = sigmoid(MLP(AvgPool(F) + MaxPool(F))) (1)

[0091] where F is the input of the module, sigmoid is the activation function, MLP is the shared multi-layer perceptron, AvgPool is the global average pooling operation, and MaxPool is the max pooling operation.

[0092] The formula for the SA module is:

[0093] Ms(Fc) = sigmoid(f7×7(AvgPool(Fc); MaxPool(Fc)))(2)

[0094] where Fc represents the input, sigmoid is the activation function, and AvgPool is the global average pooling operation.

[0095] MaxPool is the max pooling operation.

[0096] (2) In Bi-LSTM: As Figure 8 shown, Bi-LSTM is a bidirectional long short-term memory module, which is a combination of a forward LSTM and a backward LSTM. And a single LSTM module, as Figure 7 shown, can transfer the state of the cell at the previous moment to the state of the cell at the next moment to achieve the purpose of learning and memorizing sequence information, while avoiding the vanishing gradient.

[0097] It can be seen that the improved U-net proposed by the present invention combines spatial attention and channel attention, and uses a bidirectional long short-term memory module to replace the original U-net bottleneck layer, enabling deeper fusion of feature information at different levels and scales, improving the model's feature extraction ability, and increasing the model's ability to extract sequence information between pictures.

Claims

1. A pulmonary embolism segmentation method based on LSCU-net, characterized in that, It includes the following steps: Step s1: Obtain CTPA images of pulmonary embolism and get a pulmonary embolism dataset through image preprocessing, and divide it into a training set and a test set; Step s2: Build an improved LSCU-net neural network model: Step s21: Introduce a convolutional module CBAM-Conv with a residual lightweight attention module CBAM into the encorder module of the LSCU-net neural network model; Step s22: Introduce a bidirectional long short-term memory module Bi-LSTM into the bottleneck layer of the LSCU-net neural network model to strengthen the utilization of sequential information between pulmonary embolism slices by U-net; Step s3: Use the training set as training samples to train the improved LSCU-net neural network obtained in Step 2. After data augmentation of the training set, it is input into LSCU-net for training, and the loss function is FTLoss + BCEloss; Step s4: Use the trained LSCU-net to perform segmentation testing on pulmonary embolism samples in the test set; Improve the U-net neural network. The specific improvement content is as follows: (1) Introduce a convolutional module CBAM-Conv with a residual lightweight attention module CBAM into the encorder module of the LSCU-net neural network model, specifically as follows: Use the CBAM-Conv module in the encoder structure area for feature extraction. This CBAM-Conv module is composed of a CBAM layer and two basic convolutional blocks in series; CBAM layer: Obtained by connecting a channel attention module CA and a spatial attention module SA in series, which is used to change the learning strategy of the neural network model and effectively fuse feature information of different scales and levels, strengthen the attention of the neural network to the information of the region of interest. After passing through the CBAM layer, the image feature data will pass through the channel attention module CA and the spatial attention module SA in turn, and the weights are adjusted to distinguish the importance of features, focusing on the features of the key region from the spatial dimension and the channel dimension, and suppressing unimportant features such as the background; Basic convolutional block: Specifically includes a 3×3 convolutional layer, a batch normalization Bn layer, and an activation function ReLU. When the image feature data passes through the basic convolutional block, it will perform feature extraction of the image features in turn, extract the information of the input picture, batch normalization changes the data distribution to avoid overfitting of the model, and activation determination increases non-linearity to avoid overfitting of the model; (2) Introduce a bidirectional long short-term memory module Bi-LSTM into the bottleneck layer of the LSCU-net neural network model to strengthen the utilization of sequential information between pulmonary embolism slices by U-net. The specific structure is as follows: After using the CBAM-Conv module and 2×2 max pooling multiple times in the encorder module to extract the features of the pulmonary embolism dataset, the obtained tensor data is introduced into the Bi-LSTM module and output through an L2Norm; In Bi-LSTM: The Bi-LSTM module is a bidirectional long short-term memory module, which is a combination of a forward LSTM and a backward LSTM. The structure of a unidirectional LSTM module specifically consists of an input gate, a forget gate, and an output gate. After the image feature data is input into the LSTM module, it will first perform feature fusion (add) in the feature dimension with the input data in the previous time state. Then, the image feature data is input into the forget gate. After passing through a sigmoid activation function, it is multiplied element-wise with the CELL state in the LSTM module that is used to learn long-term dependence information to forget the CELL state and discard invalid information. Subsequently, the image feature data enters the input gate. After passing through the sigmoid and tanh activation functions respectively and then performing element-wise multiplication, it is then fused with the CELL state in the feature dimension (add) to remember the important information in the image feature data. Finally, the image feature data is input into the output gate. After the image feature data and the CELL state pass through the sigmoid and tanh activation functions respectively and then perform element-wise multiplication, the finally output image feature data is obtained; The Bi-LSTM module transfers the state of the cells in the previous bidirectional moment to the cells in the next moment to achieve the purpose of learning and memorizing sequence information, while avoiding the vanishing gradient of the neural network. The L2Norm module realizes the feature normalization of the image feature data through L2 regularization, simplifies the model, and avoids overfitting of the model.

2. The pulmonary embolism segmentation method based on LSCU-net according to claim 1, wherein In step (1), image preprocessing means first slicing the CTPA image to process the three-dimensional dicom format CTPA image into a two-dimensional jpg format CTPA image, and then using the cropping method to improve the quality of the two-dimensional CTPA image.

3. The pulmonary embolism segmentation method based on LSCU-net according to claim 1, characterized in that, The channel attention module CA is expressed as: (1) where F is the input of the module, sigmoid is the activation function, MLP is a shared multi-layer perceptron, AvgPool is the global average pooling operation, and MaxPool is the max pooling operation; The spatial attention module SA is expressed as: (2) where Fc represents the input, sigmoid is the activation function, AvgPool is the global average pooling operation, and MaxPool is the max pooling operation.

4. A pulmonary embolism segmentation method based on LSCU-net according to claim 1, characterized in that, In step (3), the optimizer is SGD, the learning rate adjustment strategy is the cosine annealing descent method, the initial learning rate is 1e-4, the number of training epochs is 100, and the batch-size is 8.

5. A pulmonary embolism segmentation method based on LSCU-net according to claim 1, characterized in that, In step (1), the pulmonary embolism dataset marked by doctors is randomly divided into a training set and a test set in a ratio of 9:1.

Images