A method for pulmonary embolism segmentation based on multi-scale context perception

Abstract

The application particularly relates to a lung embolism segmentation method based on multi-scale context perception, which comprises a multi-level feature extraction layer, a context fusion layer and a mask reconstruction module. The multi-level feature extraction layer adopts Res2Net50 as an encoder backbone and embeds a SimAM attention mechanism to enhance the extraction of small target features; the context fusion layer comprises an ASCF module capable of realizing cross-slice feature fusion and adaptive multi-scale receptive field adjustment; and the mask reconstruction module adopts a double convolution attention layer and integrates a triple attention mechanism to realize cross-dimension feature enhancement and context complementation. The application explicitly models the interlayer continuity of CTPA images through a multi-slice fusion mechanism to enhance the context perception of small targets, realizes the precise matching of feature extraction and embolus size distribution by using an adaptive multi-scale and attention mechanism, and forces the model to balance the attention to large and small emboli by means of a dynamic region optimization loss function, so as to finally realize the end-to-end segmentation from the original CTPA image to the precise segmentation mask.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a pulmonary embolism segmentation method based on multi-scale context awareness. Background Technology

[0002] Pulmonary embolism (PE) is a common and dangerous vascular disease with a high mortality rate, but timely diagnosis and treatment can significantly improve patient prognosis. Currently, CT pulmonary angiography (CTPA) is the gold standard for diagnosing PE, but its clinical practice faces serious challenges: hundreds of CTPA images are generated for each patient, which are reviewed frame by frame by radiologists to locate and delineate the pulmonary embolism area. This process is highly dependent on the doctor's experience, time-consuming, and subjective, easily leading to missed diagnoses and delayed treatment, especially for patients with atypical symptoms or small emboli.

[0003] Although deep learning-based automatic segmentation methods offer potential solutions to the aforementioned problems and have achieved some success in the segmentation of large emboli, existing technologies still face the following core technical challenges in practical clinical applications, especially in the crucial early diagnostic phase: (1) The sensitivity of detecting microemboli (subsegmental and smaller vessels) is seriously insufficient: the existing segmentation model has a high rate of missed detection for microemboli. However, microemboli are often a precursor to fatal large-scale pulmonary embolism, and their missed detection means missing the critical window for early intervention, which is directly related to the patient's survival rate.

[0004] (2) Lack of modeling ability for the anisotropic characteristics of CTPA images: CTPA data has significant anisotropy in spatial dimension (high resolution within slices, low resolution between slices). Traditional 2D segmentation networks lose context of small emboli because they cannot utilize continuous information between slices; while 3D networks face problems such as large number of parameters, high computational cost, easy overfitting, and poor adaptability to scan data of different thicknesses.

[0005] (3) Mismatch between feature extraction and multi-scale perception mechanism: The receptive field settings of the general multi-scale modules of existing networks (such as the standard ASPP) do not match the actual physical size distribution of pulmonary embolism. An excessively large receptive field can easily cause the features of small emboli to be submerged during the fusion process, and at the same time, it fails to dynamically focus on key areas. In addition, the feature fusion mechanism has not been fully optimized, which limits the robustness of the model to segmentation in complex scenarios where large and small emboli coexist.

[0006] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] This invention provides a multi-scale context-aware pulmonary embolism segmentation method, a computer-readable storage medium, and a computer program product, which can effectively overcome the defects existing in the prior art.

[0008] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0009] According to a first aspect of the present invention, a multi-scale context-aware pulmonary embolism segmentation method is provided, the method comprising: The convolutional downscaling module in the pulmonary embolism segmentation model is used to perform preliminary feature extraction on the first slice image, the second slice image, and the third slice image to obtain the corresponding first feature map; wherein, the first slice image and the third slice image are the previous adjacent slice image and the next adjacent slice image of the second slice image, respectively. Based on the multi-level feature extraction layer in the feature extraction module, the first feature map is enhanced step by step according to the energy function to obtain the second feature map corresponding to each level of feature extraction layer; Using the context fusion layer in the feature extraction module, the second feature maps corresponding to each level of the feature extraction layer are fused hierarchically to obtain the first fused feature map corresponding to each level of the second feature map; and the first feature map is fused hierarchically to obtain the second fused feature map. Based on the mask reconstruction module, scale recovery is performed using the first fused feature map and the second fused feature map to obtain a probability map with the same resolution as the second slice image; wherein, the pixel value of the probability map is used to represent the probability that the corresponding pixel belongs to a thrombus; The probability map is divided into binary segments based on the probability threshold to obtain the pulmonary embolism segmentation results.

[0010] In some exemplary embodiments, the multi-level feature extraction layer in the feature extraction module performs progressive feature enhancement on the first feature map according to an energy function to obtain the second feature map corresponding to each level of the feature extraction layer, including: The first feature map is transformed using the first-level feature extraction layer, and the spatial position weights are calculated based on the energy function of the first feature map. The feature transformation results are then weighted and modulated to obtain the first-level feature map. The second-level feature extraction layer is used to perform feature transformation on the first-level feature map, and the features are weighted and modulated based on the energy function of the first-level feature map to obtain the second-level feature map; The third-level feature extraction layer is used to perform feature transformation on the second-level feature map, and the features are weighted and modulated based on the energy function of the second-level feature map to obtain the third-level feature map; The fourth-level feature extraction layer is used to perform feature transformation on the third-level feature map, and the features are weighted and modulated based on the energy function of the third-level feature map to obtain the fourth-level feature map. The second feature map corresponding to each feature extraction layer is obtained based on the feature maps at each level.

[0011] In some exemplary embodiments, the step of using the context fusion layer in the feature extraction module to perform hierarchical fusion of the second feature maps corresponding to each level of feature extraction layer to obtain the first fused feature map corresponding to each level of the second feature map includes: The feature maps corresponding to the first, second, third, and third slice images in the first, second, third, and fourth level feature maps are concatenated along the channel dimensions to obtain the concatenated features; Global average pooling is performed on the splicing features according to the channels to obtain channel-level global information; The fusion weights are obtained by learning channel-level global information through two fully connected layers; The fusion weights and concatenated features are multiplied channel by channel, and the three-channel information is fused and the dimensionality is reduced through a convolutional layer to obtain the first fused feature map corresponding to each level of the second feature map.

[0012] In some exemplary embodiments, the mask-based reconstruction module performs scale recovery using a first fused feature map and a second fused feature map to obtain a probability map with the same resolution as the second slice image, including: The dual convolutional attention layer in the mask reconstruction module is used to perform feature transformation on the fused feature map corresponding to the fourth-level feature map to obtain the first-level reconstructed features; wherein, the dual convolutional attention layer is embedded with triple attention units; The first dilated convolutional layer is used to perform feature integration and scale transformation on the fused feature map corresponding to the first-level reconstructed feature map and the third-level feature map to obtain the second-level reconstructed feature map. The second dilated convolutional layer is used to perform feature integration and scale transformation on the fused feature map corresponding to the second-level reconstructed feature and the second-level feature map to obtain the third-level reconstructed feature. The third dilated convolutional layer is used to perform feature integration and scale transformation on the fused feature map corresponding to the third-level reconstructed feature and the first-level feature map to obtain the fourth-level reconstructed feature. Based on the second fusion feature map and the fourth-level reconstruction feature, a probability map with the same resolution as the second slice image is obtained.

[0013] In some exemplary embodiments, obtaining a probability map with the same resolution as the second slice image based on the second fused feature map and the fourth-level reconstructed features includes: The second fused feature map and the fourth-level reconstructed feature map are spliced ​​together to obtain the spliced ​​feature map. By utilizing an adaptive multi-scale context modeling layer, the weight allocation of the concatenated features is adaptively adjusted to obtain a multi-scale feature representation; The feature transformation process of the multi-scale feature representation is performed based on the fourth dilated convolutional layer to obtain a probability map with the same resolution as the second slice image.

[0014] In some exemplary embodiments, the first-level feature extraction layer is provided with a first preset number of residual blocks based on the parameter free attention mechanism, the second-level feature extraction layer is provided with a second preset number of residual blocks based on the parameter free attention mechanism, the third-level feature extraction layer is provided with a third preset number of residual blocks based on the parameter free attention mechanism, and the fourth-level feature extraction layer is provided with a first preset number of residual blocks based on the parameter free attention mechanism; wherein, the third preset number is greater than the second preset number, and the second preset number is greater than the first preset number.

[0015] In some exemplary embodiments, the method further includes: Obtain a sample image dataset; the sample image dataset contains multiple raw pulmonary angiography images; The sample image dataset is preprocessed to construct the model training dataset, and the model training dataset is input into the initial pulmonary embolism segmentation model to obtain the class imbalance loss and adaptive region-specific loss of the initial pulmonary embolism segmentation model. The class imbalance loss and the adaptive region-specific loss are weighted and matched to calculate the composite loss function; The initial pulmonary embolism segmentation model is trained with the goal of minimizing the composite loss function until a preset termination condition is met, thus obtaining the trained initial pulmonary embolism segmentation model.

[0016] In some exemplary embodiments, the preprocessing of the sample image dataset to construct the model training dataset includes: The pixel values ​​of the original pulmonary angiography image are converted to Huntsfield unit values ​​to obtain a Huntsfield unit image; The Huntsfield unit image is windowed, and the pulmonary blood vessel mask is extracted from the windowed image using automatic segmentation technology to obtain the pulmonary blood vessel mask image. The pulmonary vascular mask image and the original pulmonary artery angiography image are subjected to mask constraint processing to obtain a mask constraint image; The mask constraint image is center-cropped, and then divided into training, testing, and validation sets according to a preset ratio.

[0017] According to a second aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the storage medium is located to execute the above-described multi-scale context-aware pulmonary embolism segmentation method.

[0018] According to a third aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the above-described multi-scale context-aware pulmonary embolism segmentation method.

[0019] According to a fourth aspect of the present invention, an electronic device is provided, comprising: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to implement the above-described multi-scale context-aware pulmonary embolism segmentation method by executing the executable instructions.

[0020] The embodiments of this invention provide a multi-scale context-aware pulmonary embolism segmentation method. By adding SimAM attention to the Res2Net encoder, the network can automatically focus on critical regions without introducing additional parameters, thus improving the feature representation ability of small targets. An ASCF module is introduced to fuse adjacent slice features through multi-scale channel attention, explicitly modeling the continuity between CTPA layers. The traditional ASPP module is improved to obtain the Ada_ASPP module, which can adaptively adapt the inflation rate and dynamic learning mechanism to optimize multi-scale feature extraction. A DCAM module is added to the bottleneck layer of the U-shaped network, combined with a cascaded convolutional structure of Triplet Attention, to enhance feature representation ability. A dynamic region adaptive loss function is added, which improves the segmentation performance of small targets by statistically analyzing FN / FP by region and dynamically adjusting the weights.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0023] Figure 1 The flowchart illustrating an exemplary embodiment of the present invention is shown below, illustrating a method for pulmonary embolism segmentation based on multi-scale context awareness. Figure 2 This schematic diagram illustrates the network framework of a pulmonary embolism segmentation model according to an exemplary embodiment of the present invention. Figure 3 This diagram illustrates a comparison between a conventional Res2Net residual module (left) and a residual module with a multi-level feature extraction layer in this invention (right), representing an exemplary embodiment of the present invention. Figure 4 This schematic diagram illustrates the framework of an exemplary embodiment of the present invention, ASCF. Figure 5 This schematic diagram illustrates the architecture of a dual convolutional attention layer, an exemplary embodiment of the present invention. Figure 6 The schematic diagram illustrates the architecture of Ada_ASPP, an exemplary embodiment of the present invention. Figure 7 This schematic diagram illustrates a visualization of the calculation process of an adaptive region-specific loss term according to an exemplary embodiment of the present invention. Figure 8 The diagram illustrates the composition of an electronic device according to an exemplary embodiment of the present invention. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0025] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0026] Early and accurate diagnosis of pulmonary embolism (PE) is crucial for its clinical prognosis. CT pulmonary angiography (CTPA) is currently the primary imaging modality for diagnosing PE; however, its analysis heavily relies on radiologists reviewing hundreds of images frame by frame, resulting in problems such as high time consumption, strong subjectivity, and a high risk of missed diagnoses (especially for small emboli). Therefore, the development of automated, high-precision pulmonary embolism segmentation methods is an urgent clinical need.

[0027] The closest existing technology to this invention is a pulmonary embolism segmentation network based on U-Net, especially some variants that incorporate attention mechanisms or multi-scale designs. Specifically: The dual-U-Net cascaded network proposed by Liu et al. ("CAM‐Wnet: An effective solution for accurate pulmonary embolism segmentation." Medical Physics) employs two U-Nets for coarse and fine segmentation, introducing CA residual blocks (CARB) and pyramid pooling modules (PPM) in the encoder to fuse multi-scale features. While this two-stage processing improves segmentation accuracy, it still has some shortcomings: a large number of network parameters, resulting in high training costs; insufficient utilization of the spatial continuity between adjacent CTPA slices, leading to inadequate inter-slice context modeling for small emboli; and fixed receptive fields in the multi-scale modules, potentially diluting the features of small emboli.

[0028] The pulmonary embolism segmentation method proposed by Xu et al. (Liver segmentation in CT based on ResUNet with 3D probabilistic and geometric post-process. IEEE International Conference on Signal and Image Processing) embeds ResNet modules into U-Net and performs feature extraction through residual connections, enhancing the encoder's feature extraction capability. It performs robustly in general medical segmentation tasks, but faces some challenges with pulmonary embolism data: it fails to specifically optimize for the anisotropy of CTPA, and inter-slice information is not effectively utilized; the loss function uses Dice and cross-entropy loss, making it difficult to dynamically balance the precision and recall of small emboli.

[0029] The Swin-Unet (Unet-like puretransformer for medical image segmentation. European conference on computer vision. Cham: Springer Nature Switzerland) medical segmentation network employing the Swin Transformer captures long-range dependencies and achieves state-of-the-art performance on some publicly available datasets. However, it suffers from significant drawbacks in pulmonary embolism segmentation: the Transformer structure requires extensive training data and is prone to overfitting on limited medical labeled data; the windowing mechanism may disrupt the continuity of small targets, especially on anisotropic CTPA data.

[0030] Fan et al. proposed a pulmonary embolism detection method based on a threshold adjustment segmentation network (TSNet) (Scientific Reports). This study proposes a pulmonary embolism segmentation method for 3D CT PA images, mainly consisting of a threshold adjustment module (TAD), a geometry-topology axial feature module (GT-AFM), and a channel splitting exchange attention module (CSSA). TSNet aims to reduce the false positive rate and improve detection sensitivity by fusing adaptive threshold adjustment with geometric topology features. However, this method has the following limitations: 1) The sensitivity to small emboli is still insufficient, especially under low false positive rate conditions, the sensitivity decreases significantly; 2) The network is a full 3D structure, with high computational complexity and high resource consumption for training and inference. The attention-based U-Net-based pulmonary embolism segmentation and quantification framework proposed by Nitha et al. (Engineering Applications of Artificial Intelligence) employs a lightweight attention-based U-Net for pulmonary embolism segmentation, followed by contour extraction and quantification analysis (e.g., area, perimeter, diagonal length) in post-processing. While this method performs well in segmentation, its main limitations are: 1) It is only applicable to 2D slices, failing to fully utilize 3D contextual information and limiting its ability to continuously model emboli across layers; 2) The attention mechanism is relatively basic, failing to enhance the features of small emboli; 3) Quantification analysis relies on post-processing and lacks end-to-end optimization with the segmentation network, potentially introducing errors.

[0031] To address the shortcomings and deficiencies of existing technologies, this example implementation provides a multi-scale context-aware pulmonary embolism segmentation method. (Reference) Figure 1 As shown, it can specifically include: Step S10: Use the convolutional downscaling module in the pulmonary embolism segmentation model to perform preliminary feature extraction on the first slice image, the second slice image, and the third slice image to obtain the corresponding first feature map; wherein, the first slice image and the third slice image are the previous adjacent slice image and the next adjacent slice image of the second slice image, respectively. Step S12: Based on the multi-level feature extraction layers in the feature extraction module, the first feature map is enhanced step by step according to the energy function to obtain the second feature map corresponding to each level of feature extraction layer; Step S14: Using the context fusion layer in the feature extraction module, the second feature maps corresponding to each level of the feature extraction layer are fused hierarchically to obtain the first fused feature map corresponding to each level of the second feature map; and the first feature map is fused hierarchically to obtain the second fused feature map. Step S16: Based on the mask reconstruction module, scale recovery is performed using the first fusion feature map and the second fusion feature map to obtain a probability map with the same resolution as the second slice image; wherein, the pixel value of the probability map is used to represent the probability that the corresponding pixel belongs to a thrombus. Step S18: Perform binary segmentation on the probability map based on the probability threshold to obtain the pulmonary embolism segmentation result.

[0032] The following will describe in more detail each step of a multi-scale context-aware pulmonary embolism segmentation method in this exemplary embodiment, with reference to the accompanying drawings and embodiments.

[0033] For example, refer to Figure 2 As shown, Figure 2 A schematic diagram of the network framework for a pulmonary embolism segmentation model.

[0034] In step S10, the above-mentioned convolutional downscaling module is... Figure 2 It consists of three parallel ConvDown network layers.

[0035] The first slice image, the second slice image, and the third slice image mentioned above are respectively Figure 2 Middle layer slice image Central layer slice image and the next layer slice image .

[0036] Specifically, using the center slice image Centered on the previous slice image and the next layer slice image Construct a three-channel input block. The three slice images are input into the corresponding ConvDown network layers respectively; the corresponding ConvDown network layers are used to perform preliminary feature extraction on the three slice images to obtain the first feature map corresponding to each slice image.

[0037] For example, in step S12, the multi-level feature extraction layer in the feature extraction module performs progressive feature enhancement on the first feature map according to the energy function to obtain the second feature map corresponding to each level of the feature extraction layer, including: Step S121: The first feature map is transformed using the first-level feature extraction layer, and the spatial position weight is calculated based on the energy function of the first feature map. The feature transformation result is then weighted and modulated to obtain the first-level feature map. Step S122: The first-level feature map is transformed using the second-level feature extraction layer, and the features are weighted and modulated based on the energy function of the first-level feature map to obtain the second-level feature map. Step S123: The second-level feature map is transformed using the third-level feature extraction layer, and the features are weighted and modulated based on the energy function of the second-level feature map to obtain the third-level feature map. Step S124: The fourth-level feature extraction layer is used to perform feature transformation processing on the third-level feature map, and the features are weighted and modulated based on the energy function of the third-level feature map to obtain the fourth-level feature map. Step S125: Obtain the second feature map corresponding to each feature extraction layer based on the feature maps at each level.

[0038] The first-level feature extraction layer, second-level feature extraction layer, third-level feature extraction layer, and fourth-level feature extraction layer mentioned above correspond to respectively Figure 2 Layer1, Layer2, Layer3, and Layer4.

[0039] Specifically, to further improve the segmentation accuracy of small emboli, the multi-level feature extraction layer in the feature extraction module embeds residual blocks based on the Simple Parameter-Free Attention Module (SimAM) into the residual module of Res2Net. SimAM is an attention module that does not require learnable parameters. Its core idea is based on the energy function theory in neuroscience, which evaluates the importance of each spatial location in the feature map by calculating the energy value. Specifically, regions with high energy values ​​(such as emboli) are enhanced, while regions with low energy values ​​(such as the background) are suppressed.

[0040] refer to Figure 3 As shown, Figure 3 This diagram illustrates a comparison between the traditional Res2Net residual module (left) and the residual module with multi-level feature extraction layers in this invention (right). As shown in the left diagram, in the traditional Res2Net residual module, the input features are first processed by a 1×1 convolutional layer for channel transformation. Then, the features are divided into multiple sub-feature groups X1, X2, X3, and X4. Each sub-feature group undergoes progressive feature transformation through cascaded 3×3 convolutional layers, outputting corresponding features Y1, Y2, Y3, and Y4 respectively. Finally, the features are integrated by a 1×1 convolutional layer. As shown in the right diagram, the residual module of this application, based on the multi-sub-feature group structure, introduces a SimAM residual block in the output stage to perform weighted modulation processing on the features before output.

[0041] Furthermore, the SimAM process is described using the first-level feature extraction layer (Layer 1) to perform feature transformation on the first feature map, and calculating spatial location weights based on the energy function of the first feature map. The feature transformation results are then weighted and modulated to obtain the first-level feature map. For each spatial location in the first feature map... The corresponding neuron features are The characteristics of the remaining positions are ,in, It is an index of spatial location. The formula for the energy function is:

[0042] in, , indicating except The mean of the features at other locations; , indicating except The variance at other locations; To remove The total number of neurons outside the body; This is a hyperparameter; to prevent the denominator from being 0, it can be set to 0.0001.

[0043] The modeling principle of the energy function is as follows: assuming there are two classes of neurons (target neuron vs. other neurons) each following a normal distribution, the energy function is derived by maximizing inter-class differences and minimizing intra-class differences. The smaller the value, the greater the difference between the neuron and its surrounding neurons, and therefore the more important it is.

[0044] The attention weights are calculated as follows:

[0045] in, The smaller the value, the higher its importance; take the reciprocal. Make it as important as it is larger, and then normalize it to (0,1) using sigmoid as the weight.

[0046] This mechanism relies solely on the statistical properties of the features themselves for dynamic adjustment, offering the following advantages: computationally efficient, without introducing additional trainable parameters, resulting in extremely low computational overhead; avoidance of overfitting, with the flexible parameter design significantly reducing the risk of overfitting on small-scale medical data; and the ability to automatically focus on discriminative regions, enhancing the feature representation capabilities of small targets.

[0047] Furthermore, the first-level feature extraction layer is provided with a first preset number of residual blocks based on the parameter free attention mechanism, the second-level feature extraction layer is provided with a second preset number of residual blocks based on the parameter free attention mechanism, the third-level feature extraction layer is provided with a third preset number of residual blocks based on the parameter free attention mechanism, and the fourth-level feature extraction layer is provided with a first preset number of residual blocks based on the parameter free attention mechanism; wherein, the third preset number is greater than the second preset number, and the second preset number is greater than the first preset number.

[0048] For example, refer to Figure 2As shown, Layer 1 is set to s=1 and basicBlock=3, where s=1 indicates that layer 1 does not perform downsampling and basicBlock=3 indicates that layer 1 contains three residual blocks with embedded SimAM units; Layer 2 is set to s=2 and basicBlock=4, where s=2 indicates that downsampling will be performed once and basicBlock=4 indicates that layer 2 contains four residual blocks with embedded SimAM units; Layer 3 is set to s=2 and basicBlock=6, where basicBlock=6 indicates that layer 3 contains six residual blocks with embedded SimAM units; Layer 4 is set to s=2 and basicBlock=3, where basicBlock=3 indicates that layer 4 contains three residual blocks with embedded SimAM units.

[0049] For example, in step S14, the use of the context fusion layer in the feature extraction module to perform hierarchical fusion of the second feature maps corresponding to each level of feature extraction layer to obtain the first fused feature map corresponding to each level of the second feature map includes: Step S141: The feature maps corresponding to the first slice image, second slice image, and third slice image in the first-level feature map, second-level feature map, third-level feature map, and fourth-level feature map are spliced ​​along the channel dimension to obtain spliced ​​features; Step S142: Perform global average pooling on the splicing features according to the channels to obtain channel-level global information; Step S143: Learn channel-level global information through two fully connected layers to obtain fusion weights; Step S144: Multiply the fusion weights and spliced ​​features channel by channel, and fuse the three-channel information through a convolutional layer and reduce the dimensionality to obtain the first fusion feature map corresponding to each level of the second feature map.

[0050] The aforementioned context fusion layer, denoted as ASCF, is used to fuse adjacent slice channels. (See reference...) Figure 4 As shown, Figure 4 This is a schematic diagram of the ASCF framework.

[0051] Specifically, taking the use of ASCF to perform layer-by-layer fusion of the second feature map output from Layer 1 to obtain the first fused feature map corresponding to Layer 1 as an example, the process of fusing adjacent slice channels is explained. The feature map of the current layer... Feature maps of adjacent slices at the corresponding scale , The concatenated features are concatenated along the channel dimension. The concatenated features are then passed through a channel attention gate consisting of global average pooling, a fully connected layer, and a sigmoid function to generate adaptive weights. Finally, these weights are used to weight the concatenated features, outputting an enhanced feature map that is then passed to the next layer.

[0052] The calculation process can be divided into the following steps: Step 1: Input the feature maps of three adjacent slices. The previous slice feature map has a shape of (C, H, W). This is the current slice feature map, and its shape is (C,H,W). The subsequent slice feature map has the same shape (C,H,W).

[0053] Step 2: Concatenate the three feature maps along the channel dimension, as shown in the following formula.

[0054]

[0055] Step 3: Perform global average pooling on each channel to obtain channel-level global information, as shown in the formula below.

[0056]

[0057] Step 4: Channel Attention Weight Learning The fusion weights between the channels are learned through two fully connected (Linear) layers, and the dimensionality reduction formula is as follows:

[0058] in, To reduce the proportion.

[0059] The formula for dimensional restoration is as follows:

[0060] After passing through the sigmoid activation function, we obtain weight information ranging from 0 to 1.

[0061]

[0062] Step 5: Weighted fusion and convolution Weight splicing features Multiply by channel, as shown in the formula below.

[0063]

[0064] The three-channel information is fused and dimensionality reduced by convolutional layers, as shown in the formula below.

[0065]

[0066] Step 6: Obtain the current slice features that incorporate information from neighboring slices. .

[0067] Following the above process, the fusion feature maps of adjacent slice channels in Layer 1 are obtained respectively. Layer 2 adjacent slice channel fusion feature map Layer 3 adjacent slice channel fusion feature map Layer 4 adjacent slice channel fusion feature map Furthermore, ASCF is used to perform hierarchical fusion of the first feature maps corresponding to the three slice images to obtain the second fused feature map. .

[0068] For example, in step S16, the mask-based reconstruction module performs scale recovery using the first fused feature map and the second fused feature map to obtain a probability map with the same resolution as the second slice image, including: Step S161: The fused feature map corresponding to the fourth-level feature map is transformed using the dual convolutional attention layer in the mask reconstruction module to obtain the first-level reconstructed features; wherein, the dual convolutional attention layer is embedded with a triple attention unit. The above architecture reference for the dual convolutional attention layer Figure 5 As shown, the left figure represents the traditional DCMA architecture, while the right figure shows the architecture with triplet attention integrated on top of DCMA. This invention specifically optimizes the Bottleneck part of the U-Net network. The DualConv-Attention Module (DCMA) employs a cascaded feature processing strategy: first, local contextual information is extracted through 3×3 convolutions; then, residual connections are introduced to preserve original features and alleviate the gradient vanishing problem; subsequently, 1×1 convolutions are used to achieve cross-channel feature fusion. Finally, a triplet attention mechanism is innovatively integrated to enhance feature representation capabilities through multi-dimensional collaborative enhancement.

[0069] The innovation of Triplet Attention lies in its three-dimensional feature complementarity mechanism: the HC dimension models the long-range dependency between height and channel dimensions, capturing spatial correlations in the vertical direction; the WC dimension learns the structural continuity between width and channel dimensions, enhancing contextual information in the horizontal direction; and the HW dimension dynamically focuses on key regions in the spatial dimension, suppressing background interference. This attention mechanism significantly improves the robustness to detecting small emboli and low-contrast regions through cross-dimensional feature complementarity.

[0070] Specifically, the calculation process for the HC, WC, and HW dimensions in Triplet Attention is as follows. Triplet Attention first goes through the HC branch and the WC branch. The results of the two branches are added together, and then added to the HW branch. The formula is:

[0071] in, The result of the HC branch, The result of the WC branch, This is the result of the HW branch.

[0072] The steps for calculating the branch are as follows: (1) Rotation tensor

[0073] Rotate the original vector 90 degrees around the height axis, changing it from (C,H,W) to (H,C,W). X represents the input feature.

[0074] (2) Z-Pool dimensionality reduction

[0075] in, This indicates that average pooling and max pooling are performed separately along the H dimension, and then the results are concatenated.

[0076] (3) Cross-dimensional interactive convolution

[0077] The interaction between the C and W dimensions is established through a shared 2D convolutional kernel.

[0078] (4) Generate attention weights

[0079] in, It is the sigmoid activation function.

[0080] (5) Apply attention restoration First, expand the attention dimension to match the original features:

[0081] Reapplying attention:

[0082] in, The Hadamard product symbol indicates element-wise multiplication. Finally, rotate back to the original dimension:

[0083] The calculation steps are as follows: (1) Rotation tensor: Rotate the original vector 90 degrees around the height axis, changing it from (C,H,W) to (W,H,C).

[0084]

[0085] (2) Z-Pool dimensionality reduction:

[0086] in, This indicates that average pooling and max pooling are performed separately along the W dimension, and then the results are concatenated.

[0087] (3) Cross-dimensional interactive convolution: establish the interaction between C and H dimensions through a shared 2D convolution kernel.

[0088]

[0089] (4) Generate attention weights:

[0090] in, It is the sigmoid activation function.

[0091] (5) Apply attention restoration First, expand the attention dimension to match the original features:

[0092] Reapplying attention:

[0093] Finally, rotate back to the original dimension:

[0094] The steps for calculating the branch are as follows: (1) Keep it as it is:

[0095] (2) Z-Pool dimensionality reduction:

[0096] This means that average pooling and max pooling are performed separately along dimension C, and then the results are concatenated.

[0097] (3) Spatial interactive convolution: establishes the interaction between the H and W dimensions through a shared 2D convolution kernel.

[0098]

[0099] (4) Generate attention weights:

[0100] in, It is the sigmoid activation function.

[0101] (5) Apply attention restoration: First, expand the attention dimension to match the original features:

[0102] Reapplying attention:

[0103] Step S162: The first dilated convolutional layer is used to perform feature integration and scale transformation on the fused feature map corresponding to the first-level reconstructed feature and the third-level feature map to obtain the second-level reconstructed feature. Step S163: Use the second dilated convolutional layer to perform feature integration and scale transformation on the fused feature map corresponding to the second-level reconstructed features and the second-level feature map to obtain the third-level reconstructed features. Step S164: Use the third dilated convolutional layer to perform feature integration and scale transformation on the fused feature map corresponding to the third-level reconstructed feature and the first-level feature map to obtain the fourth-level reconstructed feature. Step S165: Based on the second fused feature map and the fourth-level reconstructed features, a probability map with the same resolution as the second slice image is obtained.

[0104] Specifically, referring to Figure 2, the first dilated convolutional layer is Dconv, with parameters set to (H / 8, W / 8, 1024). The first dilated convolutional layer is used to fuse the first-level reconstructed features (the output of DCAM) and the corresponding third-level feature map. Feature integration and scaling transformation are performed to obtain the second-level reconstructed features.

[0105] The second dilated convolutional layer mentioned above is Dconv, with its parameters set to (H / 4, W / 4, 512). This second dilated convolutional layer is used to fuse the second-level reconstructed features and the corresponding feature maps of the second-level feature maps. Feature integration and scaling transformation are performed to obtain the third-level reconstructed features.

[0106] The third dilated convolutional layer mentioned above is Dconv, with parameters set to (H / 2, W / 2, 256). This third dilated convolutional layer is used to fuse the third-level reconstructed features and the corresponding first-level feature maps into a fused feature map. Feature integration and scaling transformation are performed to obtain the fourth-level reconstructed features.

[0107] Specifically, the decoder consists of multiple upsampling blocks. Each block expands the feature map space size by a factor of 2 through bilinear interpolation. The expanded features are then skip-connected with the ASCF-enhanced fused feature map from the corresponding encoder layer to fuse the original information. The concatenated features are refined through several convolutional layers to gradually restore spatial details and refine boundaries. The final decoding layer uses a 1×1 convolution and a sigmoid activation function to output a probability map with the same resolution as the input center slice. , where each pixel value represents the probability that it is a thrombus.

[0108] For example, in step S165, obtaining a probability map with the same resolution as the second slice image based on the second fused feature map and the fourth-level reconstructed features includes: Step S501: The second fused feature map and the fourth-level reconstructed feature map are spliced ​​together to obtain the spliced ​​feature map. Step S502: Using the adaptive multi-scale context modeling layer, the weight allocation of the concatenated features is adaptively adjusted to obtain a multi-scale feature representation; Step S503: Based on the fourth dilated convolutional layer, perform feature transformation processing on the multi-scale feature representation to obtain a probability map with the same resolution as the second slice image.

[0109] For details, please refer to Figure 2 As shown, the second fused feature map is processed using a Concat layer. The features are then combined with the fourth-level reconstructed features to obtain the combined features.

[0110] Before the final decoded output, it passes through an adaptive multi-scale context modeling layer, Ada_ASPP, as shown below. Figure 6 As shown, this layer addresses the limitations of the traditional Atrous Spatial Pyramid Pooling (ASPP) module in pulmonary embolism segmentation tasks: an excessively large receptive field (expansion ratio 6 / 12 / 18) can easily lead to the dilution of microembolic features.

[0111] Ada_ASPP adjusts the dilation rate of the dilated convolution to (2,3,4) to better match the actual physical size of the tiny emboli in the CTPA image. On the other hand, it performs dynamic weight learning, enabling the network to autonomously enhance the weight allocation of the small dilation rate branch (high-frequency detail features), effectively improving the segmentation sensitivity of tiny emboli while retaining the multi-scale representation ability for large and medium-sized emboli, ensuring robustness in detecting targets of different sizes.

[0112] The learning process of dynamic weights in the Ada_ASPP module is as follows:

[0113] in, Indicates the first The weight coefficients of each branch, It is the Sigmoid activation function. This represents the learnable weight matrix.

[0114] This mechanism first utilizes Global Average Pooling (GAP) to extract global contextual representations for each branch, and then learns importance weights across scales through a linear transformation layer. This enables the network to autonomously enhance the weight allocation of branches with low inflation rates (high-frequency detail features), effectively improving the segmentation sensitivity for small emboli while retaining the multi-scale representation capability for medium and large emboli, ensuring robustness in detecting targets of different sizes. The entire process is entirely driven by the network autonomously learning the weight allocation strategy, without the need for manually preset prior parameters, demonstrating the algorithm's adaptive integration capability of features at different scales.

[0115] Furthermore, feature transformation processing is performed on the multi-scale feature representation based on the fourth dilated convolutional layer DConv to obtain a probability map with the same resolution as the second slice image. The parameters of the fourth dilated convolutional layer DConv are set to (H, W, 2).

[0116] For example, in step S18, a threshold (0.5) is set on the probability map P to obtain a binary segmentation mask, i.e., the segmentation result. The final segmentation result is then overlaid onto the original CTPA image for visualization, and a structured diagnostic auxiliary report can be generated subsequently.

[0117] For example, the method further includes: Step S21: Obtain the sample image dataset; wherein, the sample image dataset contains multiple raw pulmonary angiography images; Step S22: Preprocess the sample image dataset to construct the model training dataset, and input the model training dataset into the initial pulmonary embolism segmentation model to obtain the class imbalance loss and adaptive region-specific loss of the initial pulmonary embolism segmentation model. Step S23: Weight the class imbalance loss and the adaptive region-specific loss to calculate the composite loss function; Step S24: With minimizing the composite loss function as the optimization objective, train the initial pulmonary embolism segmentation model until the preset termination condition is met, and obtain the trained initial pulmonary embolism segmentation model.

[0118] Specifically, the entire network is trained end-to-end, driven by a composite loss function:

[0119] in, For class imbalance loss, For adaptive region-specific loss terms.

[0120] On the one hand, it can dynamically adjust the proportion of precision and recall in the optimization objectives, effectively alleviating the limitations of traditional loss functions (such as Dice Loss or cross-entropy) in balancing these two aspects. Especially in the segmentation of small emboli in pulmonary emboli, this feature can avoid missing small emboli due to bias towards high precision (low recall) or introducing background noise due to excessive pursuit of recall (low precision), thus achieving a better balance in terms of clinical needs. On the other hand, by calculating TP, FN, and FP by region, the loss function can apply differentiated penalty weights based on the differences in different anatomical regions (such as the central pulmonary artery and subsegmental branches) or emboli sizes (large emboli and small emboli). This region-specific optimization significantly improves the anatomical rationality of the segmentation results.

[0121]

[0122]

[0123]

[0124] In the above formula, A and B are constants used to adjust... and The range is typically set to 0.3 and 0.4, making and The range is from 0.3 to 0.7. If a certain sub-region has a large number of FP, The number of FNs will increase, and the loss function will place more emphasis on FP, leading to improved accuracy in the next training round. Similarly, if a certain sub-region has a large number of FNs, The loss function will increase, emphasizing full normalization (FN), which will improve recall during subsequent training. By dividing the data into regions and calculating the loss function separately for each small region, the tuning can be more precise, and this loss function can promote a certain balance between precision and recall.

[0125] A visual diagram illustrating the calculation process of this loss term is shown below. Figure 7As shown. Left: Global Loss, calculated using a global prediction region (pink prediction) and ground truth (blue ground truth). The bottom label indicates Dice loss: emphasizing overall overlap (intersection over union), but with limited sensitivity to local errors. Middle: Local Loss. Dividing the image into grids / patches, Local Loss is calculated for each local region (multiple green boxes labeled "Local Loss" in the image). The bottom label indicates region-specific loss: aiming to make the model focus more on "local alignment / local boundaries / local difficulties" rather than just global overlap. Right: Adaptive Local Loss. Also calculated based on local regions, but the loss for each local region is Adaptive Local Loss. A zoomed-in illustration in the upper right corner shows: Within a local grid, the following will appear: TP (True Positive): The predicted target is the target, and the actual result is the target. TN (True Negative): Predicted background and actual background FP (False Positive): The predicted result is the target, but the actual result is the background. FN (False Negative): Predicted as background but the actual target is the target. The red dashed box on the right, Loss Adjustment, indicates that the loss weights or penalty intensity are adjusted based on the distribution of TP / TN / FP / FN in the local area (e.g., penalizing FP or FN more), thereby achieving "paying more attention to where the errors are more numerous".

[0126] Specifically, The specific formula is as follows:

[0127] Where y represents the true label of the sample, usually 0 (positive class) or 1 (negative class). p represents the probability that the model predicts the sample belongs to the positive class (y=1), and its value ranges from [0,1]. To focus on the parameters, adjust the intensity of the difficulty weight.

[0128] For example, in step S21, the preprocessing of the sample image dataset to construct the model training dataset includes: Step S211: Convert the pixel values ​​of the original pulmonary angiography image to Huntsfield unit values ​​to obtain a Huntsfield unit image; Step S212: Window the Huntsfield unit image, and use automatic segmentation technology to extract the mask of the lung blood vessels from the windowed image to obtain the lung blood vessel mask image. Step S213: Perform mask constraint processing on the pulmonary vascular mask image and the original pulmonary artery angiography image to obtain a mask constraint image; Step S214: Perform center cropping on the mask constraint image, and divide the mask constraint image into training set, test set and validation set according to a preset ratio.

[0129] Specifically, the data preprocessing process is as follows: 1. Convert image pixel values ​​to Huntsfield units (HU) to better reflect the degree of X-ray absorption by lung tissue; 2. Window the CTPA images converted to HU values ​​and adjust the contrast of the CTPA images to make the lung tissue and embolism areas stand out more in the images, making it easier to observe embolism areas with small differences. 3. Use automatic segmentation technology to extract the mask of the lung vessels from the normalized CTPA image to obtain a CTPA mask image that only contains the lung vessel region. 4. Convert both the CTPA mask image and the original image from DICOM format to BMP format, and then divide the BMP format CTPA images to obtain the training set, validation set and test set. 5. Multiply the mask obtained in step 4 with the original image. The resulting image retains the original pixel values ​​for the lung blood vessel region, while the pixel values ​​for other regions are all 0. 6. Crop the CTPA mask image from 512*512 to 480*480; 7. After preprocessing, the training set is augmented by adjusting contrast, flipping, rotating, and translating to provide the network with higher quality data and thus obtain better segmentation results.

[0130] The beneficial effects of this invention are as follows: (1) Improved detection sensitivity and segmentation accuracy of microemboli: The core mechanism of the designed pulmonary embolism segmentation model lies in the feature preservation and enhancement of micro-targets. Through an innovative adjacent slice image fusion module, an adaptive multi-scale architecture sensitive to small targets, and a dynamically focused loss function, the model is forced to give equal importance to, or even tilt resources towards, microemboli features during the learning process. This aims to improve the recall rate of microemboli to a level that meets the requirements of early clinical screening and significantly reduce the risk of missed detection by existing methods.

[0131] (2) Efficient and accurate modeling of the anisotropic 3D context of CTPA: A lightweight cross-slice feature fusion mechanism is proposed, which enables the 2D network to explicitly and adaptively fuse information from adjacent layers while ensuring computational advantages. This module aims to simulate the diagnostic behavior of doctors "flipping up and down" adjacent slices to confirm lesions, thereby effectively utilizing Z-axis continuity and enhancing the model's ability to identify targets without introducing a heavy 3D computational burden.

[0132] (3) Achieving a highly adaptive match between feature extraction and task characteristics: A collaborative module group was constructed, including: a) a multi-scale context-aware module that matches the physical size distribution of pulmonary emboli to avoid dilution of small targets; b) a joint attention mechanism across spatial and channel dimensions to achieve more refined feature selection and enhancement; c) a personalized decoder optimization design to ensure efficient fusion of deep semantic information and shallow detail information. The common goal of these designs is to enable the features extracted by the network to serve the specific task of "distinguishing emboli of different sizes from the background" to the greatest extent.

[0133] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0134] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0135] Figure 8 A schematic diagram of an electronic device suitable for implementing embodiments of the present invention is shown.

[0136] It should be noted that, Figure 8 The electronic device 1000 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0137] like Figure 8As shown, the electronic device 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage section 1008 into Random Access Memory (RAM) 1003. The RAM 1003 also stores various programs and data required for system operation. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An Input / Output (I / O) interface 1005 is also connected to the bus 1004. Furthermore, the electronic device 1000 also includes an FPGA device and a System-on-a-Chip (SoC) device.

[0138] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0139] In particular, according to embodiments of the present invention, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0140] Specifically, the aforementioned electronic devices can be airborne intelligent electronic devices.

[0141] It should be noted that the storage medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein computer-readable program code is carried. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0143] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0144] It should be noted that, as another aspect, this application also provides a storage medium, which may be included in an electronic device or may exist independently without being assembled into the electronic device. The aforementioned storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to perform the methods described in the following embodiments. For example, the electronic device may perform... Figure 1 The steps of the method shown.

[0145] In one embodiment, this application provides a computer program product including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0146] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0147] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0148] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A pulmonary embolism segmentation method based on multi-scale context awareness, characterized in that, The method includes: The convolutional downscaling module in the pulmonary embolism segmentation model is used to perform preliminary feature extraction on the first slice image, the second slice image, and the third slice image to obtain the corresponding first feature map; wherein, the first slice image and the third slice image are the previous adjacent slice image and the next adjacent slice image of the second slice image, respectively. Based on the multi-level feature extraction layer in the feature extraction module, the first feature map is enhanced step by step according to the energy function to obtain the second feature map corresponding to each level of feature extraction layer; Using the context fusion layer in the feature extraction module, the second feature maps corresponding to each level of the feature extraction layer are fused hierarchically to obtain the first fused feature map corresponding to each level of the second feature map; and the first feature map is fused hierarchically to obtain the second fused feature map. Based on the mask reconstruction module, scale recovery is performed using the first fused feature map and the second fused feature map to obtain a probability map with the same resolution as the second slice image; wherein, the pixel value of the probability map is used to represent the probability that the corresponding pixel belongs to a thrombus; The probability map is divided into binary segments based on the probability threshold to obtain the pulmonary embolism segmentation results.

2. The method according to claim 1, characterized in that, The multi-level feature extraction layer in the feature extraction module enhances the first feature map step by step according to the energy function to obtain the second feature map corresponding to each level of feature extraction layer, including: The first feature map is transformed using the first-level feature extraction layer, and the spatial position weights are calculated based on the energy function of the first feature map. The feature transformation results are then weighted and modulated to obtain the first-level feature map. The second-level feature extraction layer is used to perform feature transformation on the first-level feature map, and the features are weighted and modulated based on the energy function of the first-level feature map to obtain the second-level feature map; The third-level feature extraction layer is used to perform feature transformation on the second-level feature map, and the features are weighted and modulated based on the energy function of the second-level feature map to obtain the third-level feature map; The fourth-level feature extraction layer is used to perform feature transformation on the third-level feature map, and the features are weighted and modulated based on the energy function of the third-level feature map to obtain the fourth-level feature map. The second feature map corresponding to each feature extraction layer is obtained based on the feature maps at each level.

3. The method according to claim 2, characterized in that, The step of using the context fusion layer in the feature extraction module to perform hierarchical fusion of the second feature maps corresponding to each level of feature extraction layer to obtain the first fused feature map corresponding to each level of the second feature map includes: The feature maps corresponding to the first, second, third, and third slice images in the first, second, third, and fourth level feature maps are concatenated along the channel dimensions to obtain the concatenated features; Global average pooling is performed on the splicing features according to the channels to obtain channel-level global information; The fusion weights are obtained by learning channel-level global information through two fully connected layers; The fusion weights and concatenated features are multiplied channel by channel, and the three-channel information is fused and the dimensionality is reduced through a convolutional layer to obtain the first fused feature map corresponding to each level of the second feature map.

4. The method according to claim 2, characterized in that, The mask-based reconstruction module uses the first fused feature map and the second fused feature map to perform scale recovery, obtaining a probability map with the same resolution as the second slice image, including: The dual convolutional attention layer in the mask reconstruction module is used to perform feature transformation on the fused feature map corresponding to the fourth-level feature map to obtain the first-level reconstructed features; wherein, the dual convolutional attention layer is embedded with triple attention units; The first dilated convolutional layer is used to perform feature integration and scale transformation on the fused feature map corresponding to the first-level reconstructed feature map and the third-level feature map to obtain the second-level reconstructed feature map. The second dilated convolutional layer is used to perform feature integration and scale transformation on the fused feature map corresponding to the second-level reconstructed feature and the second-level feature map to obtain the third-level reconstructed feature. The third dilated convolutional layer is used to perform feature integration and scale transformation on the fused feature map corresponding to the third-level reconstructed feature and the first-level feature map to obtain the fourth-level reconstructed feature. Based on the second fusion feature map and the fourth-level reconstruction feature, a probability map with the same resolution as the second slice image is obtained.

5. The method according to claim 4, characterized in that, The process of obtaining a probability map with the same resolution as the second slice image based on the second fused feature map and the fourth-level reconstructed features includes: The second fused feature map and the fourth-level reconstructed feature map are spliced ​​together to obtain the spliced ​​feature map. By utilizing an adaptive multi-scale context modeling layer, the weight allocation of the concatenated features is adaptively adjusted to obtain a multi-scale feature representation; The feature transformation process of the multi-scale feature representation is performed based on the fourth dilated convolutional layer to obtain a probability map with the same resolution as the second slice image.

6. The method according to claim 2, characterized in that, The first-level feature extraction layer has a first preset number of residual blocks based on the parameter free attention mechanism; the second-level feature extraction layer has a second preset number of residual blocks based on the parameter free attention mechanism; the third-level feature extraction layer has a third preset number of residual blocks based on the parameter free attention mechanism; and the fourth-level feature extraction layer has a first preset number of residual blocks based on the parameter free attention mechanism. The third preset number is greater than the second preset number, and the second preset number is greater than the first preset number.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain a sample image dataset; the sample image dataset contains multiple raw pulmonary angiography images; The sample image dataset is preprocessed to construct the model training dataset, and the model training dataset is input into the initial pulmonary embolism segmentation model to obtain the class imbalance loss and adaptive region-specific loss of the initial pulmonary embolism segmentation model. The class imbalance loss and the adaptive region-specific loss are weighted and matched to calculate the composite loss function; The initial pulmonary embolism segmentation model is trained with the goal of minimizing the composite loss function until a preset termination condition is met, thus obtaining the trained initial pulmonary embolism segmentation model.

8. The method according to claim 7, characterized in that, The preprocessing of the sample image dataset to construct the model training dataset includes: The pixel values ​​of the original pulmonary angiography image are converted to Huntsfield unit values ​​to obtain a Huntsfield unit image; The Huntsfield unit image is windowed, and the pulmonary blood vessel mask is extracted from the windowed image using automatic segmentation technology to obtain the pulmonary blood vessel mask image. The pulmonary vascular mask image and the original pulmonary artery angiography image are subjected to mask constraint processing to obtain a mask constraint image; The mask constraint image is center-cropped, and then divided into training, testing, and validation sets according to a preset ratio.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images