A CT image recognition and segmentation method for pulmonary embolism based on SCUNet++ neural network

Abstract

The present invention discloses a method for identifying and segmenting pulmonary embolism CT images based on a SCUnet++ neural network, comprising the following steps: S1, constructing a SCUnet++ neural network model for identifying and segmenting pulmonary embolism CT images; S2, establishing a pulmonary embolism CT image dataset and dividing it into a training set and a test set; S3, training the SCUnet++ neural network model using the training set, and optimizing the parameters of the SCUnet++ neural network model based on the characteristics of pulmonary embolism CT images to obtain a final segmentation network model; S4, testing and evaluating the final segmentation network model using the test set, and implementing automatic and intelligent pulmonary embolism CT image recognition and segmentation functions through the final segmentation network model. This method can achieve automatic and intelligent pulmonary embolism CT image recognition and segmentation functions with high segmentation accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision deep learning and medical image processing, and specifically to a pulmonary embolism CT image recognition and segmentation method based on the SCUNet++ neural network. Background Art

[0002] Pulmonary embolism (PE) is a serious and potentially fatal disease. It occurs when emboli from the systemic circulation detach and block the pulmonary artery and its branches, leading to partial or complete occlusion of the pulmonary artery and pulmonary circulatory impairment. Without active treatment, the mortality rate for PE approaches 30%. However, with timely and strategic intervention, this rate can be reduced to 2% to 11%. Currently, the mainstay of PE diagnosis is CT pulmonary angiography (CTPA), which requires radiologists to meticulously trace every branch of the artery to screen for PE. In CT angiography, contrast agents dissolve in the blood, making the blood vessels appear bright. However, emboli do not absorb this agent, resulting in pulmonary embolism appearing as dark areas within the pulmonary arteries on CT scans. Voxel values ​​typically range from -50 HU to 100 HU (where HU represents the brightness of the image).

[0003] When performing CTPA examinations, doctors often manually segment PE in CTPA images to accurately grade the severity of the disease and assess its risk. This is a time-consuming and labor-intensive process, and because clinical symptoms and signs are often nonspecific, many patients are prone to misdiagnosis or missed diagnosis, resulting in unsatisfactory treatment options. Furthermore, because CTPA consists of hundreds of images, each representing a single lung slice, identifying PE with high clinical accuracy is both time-consuming and difficult. Therefore, diagnosing PE is a complex task, with numerous potential causes for misdiagnosis, such as a high false-positive rate. Furthermore, imaging issues such as respiratory motion, blood flow, partial volume and step artifacts, lymph nodes, and vascular bifurcations can also affect diagnostic results.

[0004] Therefore, the medical community currently needs an accurate automatic or semi-automatic pulmonary embolism identification and segmentation method to help radiologists accurately detect and diagnose PE and reduce CTPA reading time to improve detection efficiency. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention proposes a pulmonary embolism CT image recognition and segmentation method based on the SCUNet++ neural network, which can realize automatic and intelligent pulmonary embolism CT image recognition and segmentation functions with high segmentation accuracy and efficiency.

[0006] In order to solve the above technical problems, the technical solution of the present invention is:

[0007] A pulmonary embolism CT image recognition and segmentation method based on the SCUNet++ neural network, the method comprising the following steps:

[0008] Step a: Construct a SCUNet++ neural network model for identifying and segmenting pulmonary embolism CT images. The SCUNet++ neural network model consists of an encoder module, a bottleneck network module, a decoder module, and a multi-fusion dense skip connection module. The SCUNet++ neural network model consists of a Swin Transformer module, an encoder module, a bottleneck network module, a decoder module, and a multi-fusion dense skip connection module. It can realize automatic and intelligent recognition and segmentation of pulmonary embolism CT images with high segmentation accuracy and efficiency.

[0009] The Swin Transformer module consists of two consecutive swin transformer blocks, each of which consists of an LN layer, a multi-head self-attention module, a residual connection, and a two-layer nonlinear MLP with GELU. The multi-head self-attention module of the first swin transformer block is a window-based multi-head self-attention module (W-MSA), and the multi-head self-attention module of the second swin transformer block is a shifted window-based multi-head self-attention module (SW-MSA).

[0010] The encoder module consists of a linear embedding layer, three patch merging layers, and three Swin Transformer modules.

[0011] The bottleneck network module consists of a convolutional module with a 1×1Conv-3×3Conv-1×1Conv structure, with a BN-ReLU function before each Conv. Since using only the Swin Transformer module to construct a bottleneck network module would be insufficient in extracting local spatial features of the image, we use convolutional modules to construct a bottleneck for learning deep feature representations to compensate for the network's inability to extract local spatial features.

[0012] The decoder module consists of three Swin Transformer modules and four patch expansion layers.

[0013] The multi-fusion dense skip connection module designs multiple similar multi-fusion dense skip connections to aggregate features of different semantic scales on the decoder subnetwork, generating highly flexible features for fusion. This allows the multi-scale features in the encoder to be fused with the upsampled features. The multiple multi-fusion dense skip connections designed in the SCUNet++ neural network enable the aggregation of features of different semantic scales on the decoder subnetwork, effectively extracting local spatial features and overall global features of pulmonary embolism images, and combining deep and shallow semantic information for accurate segmentation. The model strengthens connectivity by introducing nested and dense multi-fusion dense skip connections, ultimately resulting in a dense design with both long and short connections, filling the hollow spaces in the center of the model. This structure effectively reduces the semantic gap between the encoder and decoder, while capturing features at different levels and with receptive fields of varying sizes, yielding richer multi-scale information. Therefore, the SCUNet++ network model is able to effectively extract local spatial features and overall global features of pulmonary embolism images, combining deep and shallow semantic information and performing a splicing process to reduce the spatial information loss caused by downsampling.

[0014] Step b: Establish a pulmonary embolism CT image dataset, which is divided into a training set and a test set.

[0015] Step c: Use the training set to train the SCUNet++ neural network model. Combined with the characteristics of pulmonary embolism CT images, the parameters of the SCUNet++ neural network model are tuned to achieve the best recognition and segmentation effect.

[0016] Step d: Use the test set to test and evaluate the obtained network model, and finally realize the automatic and intelligent pulmonary embolism CT image recognition and segmentation function. If there is no pulmonary embolism disease, a normal CT image is output; if there is pulmonary embolism disease, the pulmonary embolism CT image recognition and segmentation result is output.

[0017] Preferably, the Swin Transformer module in step a is converted into the following formula:

[0018]

[0019]

[0020]

[0021]

[0022] where z′ L represents the output of the L-th multi-head self-attention module, z L Represents the output of the L-th multilayer perceptron.

[0023] Preferably, the encoder module of the SCUNet++ neural network model in step a consists of a linear embedding layer, three patch merging layers, and three Swin Transformer modules. The encoder first converts the input pulmonary embolism CT image into a sequence embedding, then projects the feature dimension to C dimensions through the linear embedding layer. Feature representation learning and downsampling are performed three times in the encoder module. Each time, feature representation learning is performed through a Swin Transformer module, and then two times downsampling is performed through the patch merging layer to reduce the number of representations and increase the feature dimension to twice the original number.

[0024] The patch merging layer divides the input patch into four parts and concatenates them together through the patch merging layer. This process reduces the feature resolution by a factor of two. Furthermore, since the concatenation operation results in a fourfold increase in feature dimensionality, a linear layer is added to the concatenated features to obtain features with twice the original depth.

[0025] Preferably, the decoder module of the SCUNet++ neural network model in step a consists of three SwinTransformer modules and four patch expansion layers. Corresponding to the encoder, the symmetric decoder is constructed based on the SwinTransformer module. In contrast to the patch merging layer used in the encoder, a patch expansion layer is used in the decoder to upsample the extracted deep features to feature maps. The patch expansion layer reshapes the feature maps of adjacent dimensions into higher resolution feature maps by twice upsampling, and correspondingly reduces the feature dimension to half of the original dimension.

[0026] The patch expansion layer uses a linear layer to increase the feature dimension of the input higher resolution feature map to 2 times the original one. Then, the rearrange operation is used to expand the resolution of the input feature to 2 times the input feature resolution and reduce the feature dimension to one quarter of the input feature dimension.

[0027] As a preferred dataset, a total of 8,100 pulmonary embolism CT images are included, with the pixels of the images uniformly sized at 512*512. 7,000 of these images are used as a training set, and 1,100 as a test set. The training set includes 2,000 normal samples and 5,000 pulmonary embolism samples, while the test set includes 550 normal samples and 550 pulmonary embolism samples.

[0028] As a preferred approach, we used the training set to train the SCUNet++ neural network model. Taking into account the characteristics of pulmonary embolism CT images, we optimized the SCUNet++ neural network model parameters to achieve optimal recognition and segmentation results. We used the PyTorch framework to build the SCUNet++ network model and an NVIDIA GeForce RTX 3090 GPU for training. During training, we used a batch size of 12, the Adam gradient optimization algorithm, and a learning rate of 0.0001. We trained for 150 epochs and 23 hours.

[0029] Preferably, in step d, the network model obtained by testing and evaluating the network model using a test set is used to ultimately achieve automatic and intelligent pulmonary embolism CT image recognition and segmentation functions.

[0030] To evaluate the model's performance on image segmentation, we use the Dice similarity coefficient. To verify the performance of segmentation, we use the Dice similarity coefficient (DSC) and 95% Hausdorff distance (Hausdorff95, Hd95), which are the most commonly used evaluation indicators for image segmentation.

[0031] The final value of HD95 is multiplied by 0.95 to eliminate the unreasonable distances of some outliers and maintain the overall distance stability. The HD95 value represents the relative distance between the boundary values ​​of the two samples. The smaller the distance value, the closer the boundary between the predicted image and the segmentation image is, and the more accurate the segmentation.

[0032] DSC represents the degree of overlap between the segmentation result and the label, while HD95 represents the maximum distance between the network's predicted region and the label. Therefore, the higher the DSC and the lower the HD95, the better the semantic segmentation model's performance. DSC is sensitive to the interior of the segmented region and the true region, while HD95 focuses primarily on the segmentation boundary. Therefore, combining these two metrics allows us to objectively and quantitatively evaluate the model's segmentation performance. The training result with the best segmentation performance on the test set is used as the final network model. Ultimately, simply inputting CT images of pulmonary embolism can achieve automatic and intelligent recognition and segmentation of CT images of pulmonary embolism.

[0033] The present invention has the following characteristics and beneficial effects:

[0034] Traditional medical image segmentation methods primarily rely on computer graphics and machine learning techniques. Among these, the more established methods include: threshold-based segmentation, regional image segmentation using similar region features, boundary segmentation using edge detection operators, and contour segmentation based on curve evolution. However, because traditional medical image segmentation methods often rely on low-level visual features or pixel-based features, and the shapes and contours of organs vary widely, in practice, as the complexity of the segmented objects increases, the evaluation value of the segmentation results fluctuates or even decreases, making it difficult to fully guarantee image segmentation accuracy. However, with continuous breakthroughs in theoretical research related to deep learning and convolutional neural networks, image segmentation models based on convolutional neural networks have continued to develop in recent years, accumulating numerous advanced research results that significantly improve performance compared to traditional medical image segmentation methods. Medical image segmentation tasks are gradually shifting towards image segmentation methods using convolutional neural networks. However, to date, research on the diagnosis of pulmonary embolism (PE) has primarily focused on binary classification or detection of PE, and there is still a lack of research methods for automatic PE segmentation.

[0035] Therefore, in this invention, based on the theories of pattern recognition and medical image processing, combined with the actual characteristics of pulmonary embolism CT image data and the actual process and diagnostic methods of radiologists in diagnosing pulmonary embolism, targeted research on pulmonary embolism segmentation algorithms was carried out. Ultimately, a recognition and segmentation neural network model (SCUNet++) suitable for pulmonary embolism CT images was proposed, consisting of a Swin Transformer module, an encoder module, a bottleneck network module, a decoder module, and a multi-fusion dense skip connection module. The core Swin Transformer module has more efficient computing. By partitioning and locally sensing the input data, the Swin Transformer module can reduce computational complexity and memory usage, thereby making training and inference more efficient.

[0036] Longer context: The traditional Transformer module is limited by the length of the sequence and cannot process sequences longer than a certain length. The Swin Transformer module can process longer sequences by using local perception windows and cross-group attention. Better scalability: The Swin Transformer module can expand the model size by increasing the number of modules and channels without negatively affecting the model performance. Stronger generalization ability: The Swin Transformer module can improve the generalization ability of the model by using cross-layer connections and deep cross-network connections, thereby better adapting to various tasks and datasets.

[0037] The proposed automatic segmentation method for pulmonary embolism CT image recognition and segmentation (SCUNet++) requires only a pulmonary embolism CT image as input to automatically and intelligently identify and segment the image. If the patient does not have pulmonary embolism, a normal CT image is output; if the patient does have pulmonary embolism, a pulmonary embolism CT image recognition and segmentation result is output. This method makes it possible to establish a precision medicine-assisted diagnosis model for pulmonary embolism and assist doctors in diagnosis and treatment, which is of great significance in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is the overall structure diagram of the SCUNet++ neural network model of the present invention.

[0040] Figure 2 Schematic diagram of the Swin Transformer module of the present invention.

[0041] Figure 3 Schematic diagram of the multi-fusion dense skip connection module of the present invention.

[0042] Figure 4 Schematic diagram of a pulmonary embolism CT image dataset according to the present invention.

[0043] Figure 5 This is a schematic diagram of the pulmonary embolism CT image recognition and segmentation results of the present invention. DETAILED DESCRIPTION

[0044] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0045] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0046] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0047] The present invention provides a pulmonary embolism CT image recognition and segmentation method based on the SCUNet++ neural network, the method comprising the following steps:

[0048] Step a: Construct a SCUNet++ neural network model for identifying and segmenting pulmonary embolism CT images, such as Figure 1 As shown in Figure 1, the SCUNet++ neural network model consists of an encoder module, a bottleneck network module, a decoder module, and a multi-fusion dense skip connection module. It can achieve automatic and intelligent recognition and segmentation of pulmonary embolism CT images with high segmentation accuracy and efficiency.

[0049] Specifically, such as Figure 2As shown in the figure, the Swin Transformer module consists of two consecutive Swin transformer blocks. Each Swin transformer block consists of an LN layer, a multi-head self-attention module, a residual connection, and a two-layer nonlinear MLP with GELU. The multi-head self-attention module of the first Swin transformer block is a window-based multi-head self-attention module (W-MSA), and the multi-head self-attention module of the second Swin transformer block is a shifted window-based multi-head self-attention module (SW-MSA). The Swin Transformer module is converted into the following formula:

[0050]

[0051]

[0052]

[0053]

[0054] where z′ L represents the output of the L-th multi-head self-attention module, z L Represents the output of the L-th multilayer perceptron.

[0055] Furthermore, the encoder module consists of a linear embedding layer, three patch merging layers, and three SwinTransformer modules. The encoder first converts the input pulmonary embolism CT image into a sequence embedding, and then projects the feature dimension to the C dimension through the linear embedding layer. It then performs feature representation learning and downsampling three times in the encoder module. Each time, it first passes through a Swin Transformer module for feature representation learning, and then performs two-fold downsampling through the patch merging layer to reduce the number of representations and increase the feature dimension to twice the original. The patch merging layer divides the input patch into four parts and connects them together through the patch merging layer. Such processing will reduce the feature resolution by two times. In addition, since the result of the splicing operation is that the feature dimension is increased fourfold, a linear layer is added to the spliced ​​features to unify the feature dimension to twice the original dimension.

[0056] Furthermore, the bottleneck network module is composed of convolutional modules, which adopt a 1×1Conv-3×3Conv-1×1Conv structure, with a BN-ReLU function before each Conv. Since using only the Swin Transformer module to build a bottleneck network module would be insufficient in extracting local spatial features of the image, we use convolutional modules to build a bottleneck for learning deep feature representations to compensate for the network's inadequate local spatial feature extraction.

[0057] Furthermore, the decoder module consists of three Swin Transformer modules and four patch expansion layers. Corresponding to the encoder, the symmetric decoder is built based on the Swin Transformer module. In contrast to the patch merging layer used in the encoder, the patch expansion layer is used in the decoder to upsample the extracted deep features. The patch expansion layer reshapes the feature maps of adjacent dimensions into higher resolution feature maps by upsampling twice, and correspondingly reduces the feature dimension to half of the original dimension. The patch expansion layer uses a linear layer on the input features to increase the feature dimension to twice the original. Then, the resolution of the input feature is expanded to twice the input feature resolution through the rearrange operation, and the feature dimension is reduced to one-quarter of the input feature dimension.

[0058] Furthermore, the multi-fusion dense skip connection module, such as Figure 3 As shown, multiple similar multi-fusion dense skip connections are designed to aggregate features of different semantic scales on the decoder subnetwork, generating highly flexible features for fusion. Multi-scale features from the encoder can then be fused with upsampled features. The multiple multi-fusion dense skip connections designed in the SCUNet++ neural network enable aggregation of features of different semantic scales on the decoder subnetwork, effectively extracting both local spatial and global features of pulmonary embolism images, and combining deep and shallow semantic information for accurate segmentation. The model strengthens connectivity by introducing nested and dense multi-fusion dense skip connections, ultimately resulting in a dense design with both long and short connections, filling the hollow spaces in the center of the model. This structure effectively reduces the semantic gap between the encoder and decoder, while capturing features at different levels and with varying receptive fields, yielding richer multi-scale information. Therefore, the SCUNet++ network model effectively extracts both local spatial and global features of pulmonary embolism images, combining deep and shallow semantic information and performing a splicing process to minimize spatial information loss caused by downsampling.

[0059] Step b: Create Figure 4The pulmonary embolism CT image dataset shown is divided into a training set and a test set. In this example, the pulmonary embolism CT image dataset consists of computed tomography angiography images of pulmonary embolism from 35 different patients, with labels completed by two radiologists. The dataset contains a total of 8,100 pulmonary embolism CT images, with a uniform pixel size of 512*512. Of these, 7,000 images are used as a training set, and 1,100 images are used as a test set. The training set contains 2,000 normal samples and 5,000 pulmonary embolism samples, while the test set includes 550 normal samples and 550 pulmonary embolism samples.

[0060] Step c: Use the training set to train the SCUNet++ neural network model. Taking into account the characteristics of pulmonary embolism CT images, the SCUNet++ neural network model parameters were optimized to achieve optimal recognition and segmentation results. We used the PyTorch framework to build the SCUNet++ network model and an NVIDIA GeForce RTX 3090 GPU for training. During training, the batch size was 12, the Adam gradient optimization algorithm was used, and the learning rate was 0.0001. The number of epochs was 150, and the training time was 23 hours.

[0061] Step d: Use the test set to test and evaluate the resulting network model, ultimately achieving automatic, intelligent recognition and segmentation of pulmonary embolism CT images. To evaluate the model's image segmentation performance, in this embodiment, the Dice similarity coefficient (DSC) and the 95% Hausdorff distance (Hd95), both of which are commonly used in image segmentation, were used as evaluation metrics.

[0062] Specifically, the predicted segmentation mask is represented as X, the ground-truth label is represented as Y, and the calculation formula of DSC is:

[0063]

[0064] Where X and Y represent the segmentation pixel value and label pixel value respectively, |X| and |Y| are the areas of the segmentation result and the label, and |X∩Y| is the intersection of X and Y, that is, the area where the segmentation result and the label overlap.

[0065] The calculation formula of DSC is:

[0066]

[0067] Where X and Y represent the segmentation pixel value and label pixel value respectively, d(x, y) represents the Euclidean distance between pixel values ​​X and Y, and d H (X, Y) represents the value of 95% Hausdorff distance.

[0068] The final value of HD95 is multiplied by 0.95 to eliminate the unreasonable distances of some outliers and maintain the overall distance stability. The HD95 value represents the relative distance between the boundary values ​​of the two samples. The smaller the distance value, the closer the boundary between the predicted image and the segmentation image is, and the more accurate the segmentation.

[0069] DSC represents the degree of overlap between the segmentation result and the label, while HD95 represents the maximum distance between the network-predicted region and the label. Therefore, the higher the DSC and the lower the HD95, the better the semantic segmentation model's performance. DSC is sensitive to the interior of the segmented region and the true region, while HD95 focuses primarily on the boundaries of the segmentation result. Therefore, combining these two metrics allows us to objectively and quantitatively evaluate the model's segmentation performance. The training result with the best segmentation performance on the test set is used as the final network model. Ultimately, we only need to input a pulmonary embolism CT image. If the patient does not have pulmonary embolism, a normal CT image will be output. If the patient has pulmonary embolism, a pulmonary embolism CT image recognition and segmentation result will be output, ultimately achieving automatic and intelligent pulmonary embolism CT image recognition and segmentation capabilities.

[0070] In the invention, focusing on the theories of pattern recognition and medical image processing, combined with the actual characteristics of pulmonary embolism CT image data and the actual process and diagnosis methods of radiologists in diagnosing pulmonary embolism, targeted research on the pulmonary embolism segmentation algorithm was carried out, and finally a recognition and segmentation neural network model (SCUNet++) suitable for pulmonary embolism CT images was proposed, which is composed of a Swin Transformer module, an encoder module, a bottleneck network module, a decoder module, and a multi-fusion dense jump connection module. The core Swin Transformer module has more efficient calculation. The Swin Transformer module can reduce the computational complexity and memory usage by dividing and locally perceiving the input data, thereby more efficiently training and inferring. Longer context. The traditional Transformer module is limited by the length of the sequence and cannot process sequences exceeding a certain length. The Swin Transformer module can process longer sequences by using local perception windows and cross-grouping attention. Better scalability. The Swin Transformer module can expand the model scale by increasing the number of modules and channels without negatively affecting the model performance. Stronger generalization ability: The Swin Transformer module can improve the generalization ability of the model by using cross-layer connections and deep cross-network connections, thereby better adapting to various tasks and data sets.

[0071] In addition, we experimentally compared SCUNet++ with existing image segmentation models. The SCUNet++ neural network model achieved the best segmentation accuracy of 80.61 and an HD95 score of 7.10 for pulmonary embolism images. This is because the transformer, based on its attention mechanism, captures global connections in one step while also focusing on the local connections between elements. Therefore, compared to segmentation models that only use convolution, SCUNet++'s DSC improved from 71.47 to 80.61, and its HD95 decreased from 7.69 to 7.10. Furthermore, compared to Swin-Unet, our proposed SCUNet++ improved its DSC on the pulmonary embolism dataset from 58.30 to 80.61, and its HD95 decreased from 20.00 to 7.10. This is because SCUNet++ deepens its network structure and adds a multi-scale feature fusion module to each Swin-Transformer module to fuse features from multiple scales, thereby improving segmentation performance.

[0072] The automatic segmentation method for pulmonary embolism CT image recognition and segmentation (SCUNet++) proposed in this invention only needs to input pulmonary embolism CT image to realize automatic and intelligent pulmonary embolism CT image recognition and segmentation function. If there is no pulmonary embolism disease, a normal CT image will be output; if there is pulmonary embolism disease, a negative CT image will be output. Figure 5 The results of CT image segmentation for pulmonary embolism are shown, and the actual segmentation effect is better than that of existing segmentation models. This invention makes it possible to establish a precise medical auxiliary judgment model for pulmonary embolism and assist doctors in diagnosis and treatment, which is of great significance in practical applications.

[0073] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. It will be apparent to those skilled in the art that various changes, modifications, substitutions, and variations of these embodiments, including components, without departing from the principles and spirit of the present invention are still within the scope of protection of the present invention.

Claims

1. A CT image recognition and segmentation method for pulmonary embolism based on SCUNet++ neural network, characterized in that: The steps include: S1. Constructing a SCUNet++ neural network model for identifying and segmenting pulmonary embolism CT images, wherein the SCUNet++ neural network model includes an encoder module, a bottleneck network module, a decoder module, and a multi-fusion dense skip connection module; The encoder module consists of a linear embedding layer, three patch merging layers, and three Swin Transformer modules; The bottleneck network module is composed of a convolution module; The decoder module consists of three Swin Transformer modules and four patch expansion layers; The multi-fusion dense skip connection module consists of three Swin Transformer modules and three patch expansion layers. It uses nested, dense multi-fusion dense skip connections to strengthen the connection. The final design is a dense connection with both long and short connections, filling the hollow position in the middle of the model. The Swin Transformer module consists of two consecutive Swin transformer blocks, each of which consists of an LN layer, a multi-head self-attention module, a residual connection, and a two-layer nonlinear MLP with GELU. The multi-head self-attention module of the first Swin transformer block is a window-based multi-head self-attention module, and the multi-head self-attention module of the second Swin transformer block is a shifted window-based multi-head self-attention module. The Swin Transformer module is converted into the following formula: where z' L represents the output of the L-th multi-head self-attention module, z L Represents the output of the L-th multilayer perceptron; S2. Establish a pulmonary embolism CT image dataset and divide it into a training set and a test set; S3. Use the training set to train the SCUNet++ neural network model. Combined with the characteristics of pulmonary embolism CT images, the parameters of the SCUNet++ neural network model are tuned to obtain the final segmentation network model. S4. Use the test set to test and evaluate the final segmentation network model, and realize automatic and intelligent pulmonary embolism CT image recognition and segmentation function through the final segmentation network model.

2. The method for pulmonary embolism CT image recognition and segmentation based on the SCUNet++ neural network according to claim 1, characterized in that: The convolution module constituting the bottleneck network module adopts a 1×1Conv-3×3Conv-1×1Conv structure, and a BN-ReLU function is used before each Conv.

3. The method for pulmonary embolism CT image recognition and segmentation based on the SCUNet++ neural network according to claim 2, characterized in that: In step S1, the method for extracting depth features through the encoder module is as follows: First, the input pulmonary embolism CT image is converted into a sequence embedding. Then, the feature dimension is projected to C dimension through the linear embedding layer. Then, feature representation learning and downsampling are performed three times in the encoder module. Each time, feature representation learning is performed through a SwinTransformer module, and then two times downsampling is performed through the patch merging layer to reduce the number of representations and increase the feature dimension to twice the original. The patch merging layer divides the input patch into four parts and connects them together through the patch merging layer, thereby reducing the feature resolution by two times. Since the result of the splicing operation increases the feature dimension by four times, a linear layer is added to the spliced ​​features to unify the feature dimension to a deep feature with twice the original dimension.

4. The method for pulmonary embolism CT image recognition and segmentation based on the SCUNet++ neural network according to claim 3, characterized in that: In step S1, the decoder module corresponds to the encoder module, and uses the patch expansion layer to upsample the extracted deep features to obtain feature maps. The patch expansion layer reshapes the feature maps of adjacent dimensions into higher resolution feature maps by upsampling twice, and correspondingly reduces the feature dimension to half of the original dimension; The patch expansion layer uses a linear layer to increase the feature dimension of the higher resolution feature map to twice the original one, and then expands the resolution of the input feature to twice the input feature resolution through a rearrange operation and reduces the feature dimension to one quarter of the input feature dimension.

5. The method for CT image recognition and segmentation of pulmonary embolism based on SCUNet++ neural network according to claim 1, characterized in that: The dataset contains 8,100 pulmonary embolism CT images, with the pixels of the images uniformly sized at 512*512. 7,000 of them are used as training sets and 1,100 as test sets. The training set contains 2,000 normal samples and 5,000 pulmonary embolism samples, and the test set includes 550 normal samples and 550 pulmonary embolism samples.

6. The method for pulmonary embolism CT image recognition and segmentation based on the SCUNet++ neural network according to claim 1, characterized in that: In step S3, NVIDIA GeForce RTX 3090 GPU is used for training. During the training process, the batch size is 12, the gradient optimization algorithm used is Adam, the learning rate is 0.0001, the number of epochs is 150, and the training time is 23 hours.

7. The method for pulmonary embolism CT image recognition and segmentation based on the SCUNet++ neural network according to claim 1, characterized in that: In step S4, the Dice similarity coefficient is used to verify the performance of the segmentation, and the Dice similarity coefficient and the 95% Hausdorff distance are used as evaluation indicators.

Images