A lung structure abnormal disease image segmentation method based on an attention mechanism

Abstract

The embodiment of the application provides a lung structure abnormal disease image segmentation method based on an attention mechanism, which can solve the technical problem of high error rate in judging congenital pulmonary airway malformations caused by poor CT image quality. The method determines the Nth fusion image,..., the first fusion image of the fusion of the plain scan image, the venous image and the arterial image; the first fusion image and the second fusion image are subjected to attention mechanism to obtain the first weight image; based on the first fusion image, the first size image is obtained through upsampling; the first weight image and the first size image are trained to obtain the first stage result; the mth stage result is taken as input to generate the m+1th size image; the mth stage result and the m+2th fusion image are fused through the attention mechanism to generate the m+1th weight image; the m+1th size image is weighted and fused to output the m+1th stage result, so as to obtain the K+1th lesion segmentation image and obtain the lesion boundary segmentation result.

Description

Technical Field

[0001] This application relates to the field of medical information technology, and in particular to an image segmentation method for lung structural abnormalities based on an attention mechanism. Background Technology

[0002] Congenital lung malformations (CLMs) are a collective term for diseases related to abnormal lung structure that develop during fetal development. Congenital pulmonary airway malformation (CPAM) is the main type. CPAM is a congenital hamartoma-like lesion of the lung characterized by excessive proliferation and dilation of terminal bronchioles, often presenting as unilocular or multilocular cysts or honeycomb structures within the lung parenchyma.

[0003] Currently, the clinical system for CPAM relies on a two-stage imaging assessment, both prenatal and postnatal. Prenatal diagnosis is performed in utero using ultrasound screening and fetal MRI. After birth, computed tomography angiography (CTA) of the chest is required to definitively diagnose the condition. This two-stage imaging approach allows physicians to make an imaging assessment based on their experience, ultimately determining whether CPAM has been diagnosed.

[0004] However, the lack of spontaneous respiratory regulation in infants and young children leads to significant temporal randomness in CT scans, and respiratory motion artifacts severely degrade image quality, causing physicians to make incorrect judgments. Therefore, there is an urgent need to develop an intelligent lung CT image analysis solution that can automatically identify and quantify lesion areas, providing physicians with objective decision support and fundamentally breaking through existing diagnostic bottlenecks. Summary of the Invention

[0005] This application provides an image segmentation method for lung structural abnormalities based on an attention mechanism, which can solve the technical problem of high error rate in the judgment of congenital lung airway malformations due to poor CT image quality.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] In a first aspect, embodiments of this application provide an image segmentation method for lung structural abnormalities based on an attention mechanism. This method includes: training and fusing plain scan images, vein images, and arterial images; sequentially determining an Nth fused image, ..., a second fused image, and a first fused image in descending order of size; where N is a positive integer greater than two; and the Nth fused image, plain scan image, vein image, and arterial image all have the same size; applying an attention mechanism to the first fused image and the second fused image to obtain a first weighted image; obtaining a first size image based on the first fused image through upsampling; and combining the first weighted image with the ... The first-stage result is obtained by training a size image. The following steps are iteratively executed until m=N-2: the m-th stage result is used as input and upsampled to generate the (m+1)-th size image; the m-th stage result and the (m+2)-th fused image are fused through an attention mechanism to generate the (m+1)-th weighted image; the (m+1)-th size image is weighted and fused according to the (m+1)-th weighted image to output the (m+1)-th stage result; the value of m is incremented by 1; where m=1 at the beginning of the iteration, the iteration is terminated when m increases to N-2 and the (N-1)-th stage result is output; according to the preset number of lesions K, the K+1-layer lesion segmentation map is obtained through the (m+1)-th stage result to obtain the boundary segmentation result of each lesion.

[0008] Based on the above description of the attention-based image segmentation method for lung structural abnormalities provided in the embodiments of this application, it can be seen that this attention-based image segmentation method for lung structural abnormalities includes constructing a deep learning architecture that integrates multi-scale and multi-modal fusion and iteratively optimizes using an attention mechanism. It exhibits significant advantages in the precise segmentation and surgical planning of congenital pulmonary airway malformations (CPAM). CPAM lesions are typically large in size, compressing and displacing surrounding normal lung tissue and blood vessels (pulmonary arteries and pulmonary veins). Their boundaries are often complex and ambiguous, and accurately defining the extent of the lesion and its three-dimensional spatial relationship with key vascular structures is crucial for developing a safe and effective surgical path (such as the precise direction of the scalpel). Inputting plain, venous, and arterial CT images of the same size, the method first fuses them into a series of fused images (from the Nth fused image to the first fused image) from low resolution (high-level semantics) to high resolution (low-level details) through training. The core processing flow begins with high-level fusion information. Initial features are fused using an attention mechanism and upsampled to obtain the first-stage result. Subsequent iterations involve upsampling, generating attention weights by combining higher-resolution fused images, and weighted fusion correction of the upsampled results, progressively refining the segmentation until the N-1th stage result is output. This restores the high-level semantic feature map obtained from pediatric lung CT images to the original CT image size. Ultimately, this result is used to generate a "K+1 layer lesion segmentation map" containing K core lesion layers and a dedicated boundary layer. Firstly, multimodal fusion effectively integrates the basic anatomical information from plain scan images, the clearly delineated pulmonary veins (with their compression displacement and abnormal drainage patterns) from venous phase images, and the pulmonary arteries (including abnormal feeding arteries and compression deformation) from arterial phase images. These three complementary pieces of information provide the model with a comprehensive three-dimensional view of the complex relationship between the lesion and surrounding key blood vessels. Secondly, the multi-scale processing and iterative optimization mechanism enables the model to grasp the overall compression range of the lesion from a high-level perspective (robustness). During the decoding process, it gradually utilizes higher-resolution fused image details (especially vascular imaging information), and dynamically learns the importance weights of features at different locations, scales, and modalities through an attention mechanism. This precisely guides information fusion, effectively overcoming the problems of boundary blurring and partial volume effect caused by the large lesion volume and compression, and significantly improving the segmentation accuracy of the boundary layer (K+1 layer). Finally, this structured segmentation output, especially the fine boundary layer that integrates arterial and venous spatial information, directly provides surgeons with an intuitive and reliable navigation basis for planning surgical paths. It clearly marks the main range of the lesion to be removed (K layer) and the precise anatomical interface (boundary layer) that the scalpel should strictly follow to avoid compressed important blood vessels (pulmonary arteries and veins) and to preserve as much functional compressed lung tissue as possible. This greatly improves the safety and precision of the surgery and reduces the risk of accidentally damaging blood vessels or removing too much normal tissue.

[0009] In the feasible implementation of the first aspect, the lung structural abnormality disease image segmentation method based on the attention mechanism further includes: inputting the labeled image, plain scan image, vein image and artery image into the training model together; the labeled image is an image with the lesion location already labeled; comparing the boundary segmentation result and the labeled image to obtain the comparison result; if the comparison result does not meet the requirements of the preset threshold, adjusting the parameters of the attention mechanism, and repeatedly executing all steps to obtain the boundary segmentation result until the updated comparison result meets the requirements of the preset threshold.

[0010] In a feasible implementation of the first aspect, the lung structural abnormality disease image segmentation method based on the attention mechanism further includes: sequentially determining the Nth plain scan feature map, ..., the second plain scan feature map, and the first plain scan feature map of the plain scan image in descending order of size; sequentially determining the Nth vein feature map, ..., the second vein feature map, and the first vein feature map of the vein image in descending order of size; sequentially determining the Nth artery feature map, ..., the second artery feature map, and the first artery feature map of the artery image in descending order of size; fusing the first plain scan feature map, the first vein feature map, and the first artery feature map, adjusting the channels, and restoring the size to obtain a first fused image; fusing the second plain scan feature map, the second vein feature map, and the second artery feature map, adjusting the channels, and restoring the size to obtain a second fused image; and fusing the Nth plain scan feature map, the Nth vein feature map, and the Nth artery feature map, adjusting the channels, and restoring the size to obtain the Nth fused image.

[0011] In a feasible implementation of the first aspect, the lung structural abnormality disease image segmentation method based on the attention mechanism further includes: performing convolution and activation operations on the plain scan image to obtain a first plain scan feature map; iteratively executing the following steps until the Nth plain scan feature map, ..., the second plain scan feature map and the first plain scan feature map of the plain scan image are obtained; taking the nth plain scan feature map as input, performing downsampling, convolution and activation operations to obtain the (n+1)th plain scan feature map; incrementing the value of n by 1; wherein, n=1 at the beginning of the iteration.

[0012] In the feasible implementation of the first aspect, the attention mechanism includes sequential addition, activation, convolution, and reactivation operations.

[0013] In the feasible implementation of the first aspect, the lung structural abnormality disease image segmentation method based on the attention mechanism further includes: the activation operation of the attention mechanism is a ReLU activation operation; and the reactivation operation of the attention mechanism is a Sigmoid activation operation.

[0014] In the feasible implementation of the first aspect, the size of the flat scan image is 480×480, the value of N is 5, the size of the first fused image is 480×480, the size of the second fused image is 240×240, the size of the third fused image is 120×120, the size of the fourth fused image is 60×60, and the size of the fifth fused image is 30×30.

[0015] Secondly, embodiments of this application provide an image segmentation system for lung structural abnormalities based on an attention mechanism, comprising: an encoder, an attention module, and a decoder; the encoder is configured to extract elements from a plain scan image, a vein image, and an artery image; the decoder is configured to, through training, fuse the plain scan image, the vein image, and the artery image, sequentially determine the Nth fused image, ..., the second fused image, and the first fused image in descending order of size; the attention module is configured to pass the first fused image and the second fused image through an attention mechanism to obtain a first weighted image; based on the first fused image, upsampling is performed to obtain a first size image; and the first weighted image and the first size image are passed through an attention mechanism. After training, the first-stage result is obtained; the following steps are iteratively executed until m=N-2: taking the m-th stage result as input, upsampling is performed to generate the (m+1)-th size image; through an attention mechanism, the m-th stage result and the (m+2)-th fused image are fused to generate the (m+1)-th weighted image; based on the (m+1)-th weighted image, the (m+1)-th size image is weighted and fused to output the (m+1)-th stage result; the value of m is incremented by 1; where m=1 at the beginning of the iteration, and the iteration is terminated and the (N-1)-th stage result is output when m increases to N-2; the decoder is also configured to obtain a K+1-layer lesion segmentation map based on the preset number of lesions K and the (m+1)-th stage result to obtain the boundary segmentation result of each lesion.

[0016] Thirdly, embodiments of this application provide an image segmentation system for lung structural abnormalities based on an attention mechanism. The image segmentation system for lung structural abnormalities based on an attention mechanism includes: at least one processor; a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method provided in the first aspect.

[0017] Fourthly, embodiments of this application provide a computer-readable medium having computer program instructions stored thereon, which can be executed by a processor to implement the method as provided in the first aspect. Attached Figure Description

[0018] Figure 1 A schematic diagram of an image segmentation system for lung structural abnormalities based on an attention mechanism, provided in an embodiment of this application;

[0019] Figure 2 A schematic diagram of an image segmentation system for lung structural abnormalities based on an attention mechanism, provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating an image segmentation method for lung structural abnormalities based on an attention mechanism, provided in an embodiment of this application;

[0021] Figure 4 A flowchart illustrating an image segmentation method for lung structural abnormalities based on an attention mechanism, provided in an embodiment of this application;

[0022] Figure 5 This is a flowchart illustrating an attention-based image segmentation method for lung structural abnormalities, provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In the description of the embodiments of the present invention, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0024] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of the present invention, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0025] The principles and features of this application are described below. The examples given are only for explaining this application and are not intended to limit the scope of this application.

[0026] This application provides an image segmentation system for lung structural abnormalities based on an attention mechanism, which can execute the image segmentation method for lung structural abnormalities based on an attention mechanism provided in this application. Figure 1This is a schematic diagram of an image segmentation system for lung structural abnormalities based on an attention mechanism, provided in an embodiment of this application.

[0027] like Figure 1 As shown, the lung structural abnormality disease image segmentation system 001 based on attention mechanism includes at least one processor 011 and a memory 012 communicatively connected to the at least one processor; wherein, the memory 012 stores instructions that can be executed by the at least one processor 011, and the instructions are executed by the at least one processor 011 to enable the at least one processor 011 to execute the lung structural abnormality disease image segmentation method based on attention mechanism provided in the embodiments of this application.

[0028] like Figure 1 As shown, in some embodiments, the lung structural abnormality disease image segmentation system based on the attention mechanism employs a model based on the U-Net network model and combined with the Attention mechanism module, including: encoder 21, attention module 22 and decoder 23.

[0029] Encoder 21 consists of convolutional layers and pooling layers. The convolutional layers capture local features from pediatric lung CT images, while the pooling layers reduce the size of the feature maps while preserving important features. It is configured to extract elements from plain scan images, vein images, and artery images.

[0030] Decoder 23, consisting of deconvolution layers and upsampling operations, recovers the high-level semantic feature maps obtained from pediatric lung CT images to the original CT image size. It is configured to determine the Nth fused image, ..., the second fused image, and the first fused image in descending order of size by training to fuse plain scan images, vein images, and artery images.

[0031] Attention module 22 is configured to pass the first fused image and the second fused image through an attention mechanism to obtain a first weight image. Based on the first fused image, a first size image is obtained through upsampling. The first weight image and the first size image are trained to obtain the first stage result. The following steps are iteratively executed until m=N-2: Taking the m-th stage result as input, an (m+1)-th size image is generated through upsampling. The m-th stage result and the (m+2)-th fused image are fused through the attention mechanism to generate the (m+1)-th weight image. Based on the (m+1)-th weight image, the (m+1)-th size image is weighted and fused to output the (m+1)-th stage result. The value of m is incremented by 1. Wherein, m=1 at the beginning of the iteration, and the iteration terminates when m increases to N-2 and the (N-1)-th stage result is output.

[0032] The decoder is also configured to obtain a K+1 layer lesion segmentation map based on the preset number of lesions K and the results of the m+1th stage, so as to obtain the boundary segmentation results of each lesion.

[0033] In some embodiments, a Skip Connection structure is employed, inserting Attention modules into both the encoder and decoder to fuse the feature maps generated by the encoder and decoder. The network adjusts the relationship between the feature maps generated by the encoder and decoder based on the parameters of the Attention module, enabling the network to focus more on meaningful features combined with the high-resolution feature maps when generating segmentation results. Finally, a convolution operation is performed on the feature map generated by the decoder to obtain the final segmentation result.

[0034] Figure 3 This is a schematic flowchart illustrating an attention-based image segmentation method for lung structural abnormalities provided in an embodiment of this application. Figure 3 As shown, in some embodiments, this attention-based lung structural abnormality disease image segmentation method includes the following steps:

[0035] S1, by training and fusing plain scan images, vein images and artery images, the Nth fused image, ..., the second fused image and the first fused image are determined in descending order of size.

[0036] N is a positive integer greater than two.

[0037] The Nth fused image, plain image, vein image, and artery image are all the same size.

[0038] In some embodiments, the plain scan image can be a plain-phase CT image of a child's airway malformation. The venous image can be a venous-phase CT image of a child's airway malformation. The arterial image can be an arterial-phase CT image of a child's airway malformation.

[0039] like Figure 4 and Figure 5 As shown, for example, the size of the flat scan image is 480×480, the value of N is 5, the size of the first fused image is 480×480, the size of the second fused image is 240×240, the size of the third fused image is 120×120, the size of the fourth fused image is 60×60, and the size of the fifth fused image is 30×30.

[0040] In some embodiments, the number of channels in the image changes during training. For example, the image specifications are represented as number of channels × size. A flat scan image is 1×480×480, a first fused image is 64×480×480, a second fused image is 128×240×240, a third fused image is 256×120×120, a fourth fused image is 512×60×60, and a fifth fused image is 1024×30×30.

[0041] In some embodiments, prior to performing step S1, the attention-based lung structural abnormality disease image segmentation method includes the following steps:

[0042] S101, determine the Nth flat scan element map, ..., the second flat scan element map and the first flat scan element map of the flat scan image in descending order of size.

[0043] In some embodiments, when performing step S101, the attention-based lung structural abnormality disease image segmentation method includes the following steps:

[0044] S1011, perform convolution and activation operations on the flat scan image to obtain the first flat scan feature map.

[0045] For example, a first flat scan feature map is obtained by performing a 3×3 convolution operation and a ReLU activation operation.

[0046] S1012, iteratively execute the following steps until the Nth flat scan feature map, ..., the second flat scan feature map and the first flat scan feature map of the flat scan image are obtained.

[0047] Using the nth flat scan feature map as input, the (n+1)th flat scan feature map is obtained through downsampling, convolution, and activation operations.

[0048] Increment the value of n by 1.

[0049] In this iteration, n=1 at the beginning.

[0050] Downsampling can be done through max pooling.

[0051] Convolution operations can be 3×3 convolution operations.

[0052] The activation operation can be a ReLU activation operation.

[0053] For example, if the size of the plain scan image is 1×480×480, the elements of the lung structural abnormality disease image to be segmented are extracted to obtain five plain scan element images corresponding to the plain scan period: the first plain scan element image A1 with a size of 64×480×480, the first plain scan element image A2 with a size of 128×240×240, the first plain scan element image A3 with a size of 256×120×120, the first plain scan element image A4 with a size of 512×60×60, and the first plain scan element image A5 with a size of 1024×30×30.

[0054] S102, determine the Nth vein element map, ..., the second vein element map and the first vein element map of the vein image in descending order of size.

[0055] The processing method is the same as that for flat scan images, so it will not be described in detail here.

[0056] For example, for a vein image with a size of 1×480×480, elements of the lung structural abnormality disease image are extracted to obtain five sizes corresponding to the venous phase: the first vein element image B1 (size 64×480×480), the second vein element image B2 (size 128×240×240), the third vein element image B3 (size 256×120×120), the fourth vein element image B4 (size 512×60×60), and the fifth vein element image B5 (size 1024×30×30).

[0057] S103, in descending order of size, determine the Nth artery element map, ..., the second artery element map and the first artery element map of the artery image in sequence.

[0058] The processing method is the same as that for flat scan images, so it will not be described in detail here.

[0059] For example, for an arterial image with a size of 1×480×480, elements of the lung structural abnormality disease image are extracted to obtain five arterial element maps corresponding to the arterial phase: the first arterial element map C1 (size 64×480×480), the second arterial element map C2 (size 128×240×240), the third arterial element map C3 (size 256×120×120), the fourth arterial element map C4 (size 512×60×60), and the fifth arterial element map C5 (size 1024×30×30).

[0060] S104, the first plain scan feature image, the first vein feature image, and the first artery feature image are fused, the channels are adjusted, and the size is restored to obtain the first fused image.

[0061] For example, A1, B1, and C1 of 64×480×480 are fused, channel adjusted, and size restored to obtain a first fused image of 64×480×480.

[0062] S105, the second plain scan feature image, the second vein feature image, and the second artery feature image are fused, the channels are adjusted, and the size is restored to obtain the second fused image.

[0063] For example, by fusing, adjusting the channels, and restoring the size of A2, B2, and C2 (128×240×240), a second fused image of 128×240×240 is obtained.

[0064] This process is repeated to obtain more fused images.

[0065] S106, fuse the Nth plain scan feature image, the Nth vein feature image, and the Nth artery feature image, adjust the channels, and restore the size to obtain the Nth fused image.

[0066] For example, the smallest feature images are A5, B5 and C5, which are 1024×30×30. The first feature images A5, B5 and C5, which are 1024×30×30, are fused, channel adjusted and size restored to obtain a fifth fused image of 1024×30×30.

[0067] S2, the first fused image and the second fused image are processed through an attention mechanism to obtain the first weighted image.

[0068] In some embodiments, the attention mechanism includes sequentially performed addition, activation, convolution, and reactivation operations. The activation operation of the attention mechanism is a ReLU activation operation. The reactivation operation of the attention mechanism is a Sigmoid activation operation.

[0069] In one implementation, the calculation formula for the attention mechanism includes:

[0070] ;

[0071] in, This represents the final attention score after fusing the feature maps of the same layer with the downsampled feature maps; This represents the feature map downsampled by the encoder; This represents the feature map downsampled by the decoder; A set of parameters representing the attention mechanism. This represents the first fusion operation process.

[0072] In another implementation, the calculation formula for the attention mechanism includes:

[0073]

[0074] in, Represented as the attention weights after fusion via skip connections. Indicates the first The Skip Connection operation process; Represents the first weighted image; This represents the i-th fusion graph; and Represents the parameters of a linear transformation; Represents the linear transformation parameters used to calculate attention weights; It is the Sigmoid activation function.

[0075] By combining the U-Net network model with the Attention mechanism, the advantages of U-Net in image segmentation and the ability of the Attention mechanism to fuse features and focus on important features are fully utilized. This combination improves the segmentation performance of the network model when processing pediatric lung CT images. An Attention module is inserted between the encoder and decoder, allowing the network to adjust the relationship between the feature maps generated by the encoder and decoder based on the parameters of the Attention module. This design allows the network to focus more on meaningful features combined with high-resolution feature maps, thereby improving the network's segmentation accuracy. A Skip Connection structure is used to fuse the feature maps generated by the encoder and decoder. This structure helps the network better transmit information, avoids information loss, and helps the network learn features in the image better, improving segmentation accuracy. The decoder uses deconvolution layers and upsampling operations to restore the high-level semantic feature maps obtained from pediatric lung CT images to the original CT image size. This design preserves important feature information and makes the final segmentation result more accurate.

[0076] S3, based on the first fused image, obtains a first-size image through upsampling.

[0077] S4. The first weighted image and the first size image are trained to obtain the first stage result.

[0078] Iteratively execute the following steps S5 to S8 until m = N-2:

[0079] S5 takes the result of stage m as input, and after upsampling, generates the (m+1)th dimension map.

[0080] S6 uses an attention mechanism to fuse the results of the m-th stage and the (m+2)-th fused image to generate the (m+1)-th weighted image.

[0081] S7. Based on the (m+1)th weighted image, perform weighted fusion on the (m+1)th size image and output the (m+1)th stage result.

[0082] S8 increments the value of m by 1.

[0083] In this iteration, m=1 at the beginning and the iteration terminates when m increases to N-2, and the result of the N-1 stage is output.

[0084] The following provides an example of iteratively executing steps S5 to S8.

[0085] In one instance, using the first-stage result as input, a second-sized image is obtained through upsampling. The first-stage result and the third fused image are then processed through an attention mechanism to obtain a second weighted image. The second weighted image and the second-sized image are trained to obtain the second-stage result. Using the second-stage result as input, a third-sized image is obtained through upsampling. The second-stage result and the fourth fused image are then processed through an attention mechanism to obtain a third weighted image. The third weighted image and the third-sized image are trained to obtain the third-stage result. Using the third-stage result as input, a fourth-sized image is obtained through upsampling. The third-stage result and the fifth fused image are then processed through an attention mechanism to obtain a fourth weighted image. The fourth weighted image and the fourth-sized image are trained to obtain the fourth-stage result.

[0086] S9. Based on the preset number of lesions K, the lesion segmentation map of layer K+1 is obtained through the results of stage m+1, so as to obtain the boundary segmentation result of each lesion.

[0087] Boundary segmentation results can provide information about lesion regions in lung CT images (such as the location, size, and / or bounding box of the lesion region).

[0088] For example, the boundary segmentation result is obtained based on the result of the fourth stage.

[0089] With a preset number of lesions K of 2, a three-layer lesion segmentation map is obtained based on the results of the fourth stage. These are the background region, the first lesion layer, and the second lesion layer.

[0090] In some embodiments, prior to performing step S1, the attention-based lung structural abnormality disease image segmentation method includes the following steps:

[0091] S01: Input the labeled image, plain scan image, vein image, and artery image together into the training model. The labeled image is the image with the lesion location already marked.

[0092] In some embodiments, ROIs (Regions of Interest), which are lesion areas in CT images, are manually labeled as tags on the original images. This results in labeled images.

[0093] After performing steps S01 and S1 to S9, the attention-based lung structural abnormality disease image segmentation method further includes the following steps:

[0094] S10: Compare the boundary segmentation results with the labeled image to obtain the comparison results.

[0095] S11. If the comparison result does not meet the preset threshold requirement, adjust the parameters of the attention mechanism and repeat all steps to obtain the boundary segmentation result until the updated comparison result meets the preset threshold requirement.

[0096] In this way, the training samples are input into the initial image segmentation network for model training and parameter adjustment until the output of the image segmentation network is the same as the label or converges to the label, thus completing the training and obtaining the optimal network parameters, that is, the optimal image segmentation network.

[0097] In some feasible approaches, the segmentation performance of image segmentation networks can be evaluated based on the Dice Similarity Coefficient (DSC). DSC is a function that measures the degree of overlap between the labels and the segmentation results; the closer the value is to 1, the better the segmentation performance of the image segmentation network.

[0098] In one implementation, the mathematical expression for the Dice coefficient is as follows:

[0099]

[0100] in, and These represent the segmentation label and the lesion area in the segmentation result, respectively.

[0101] In other feasible approaches, Mean Intersection over Union (Mean IoU) can be used as a metric to evaluate segmentation performance. Mean IoU comprehensively considers the degree of overlap between the predicted results and the ground truth labels for each class, reflecting the overall accuracy of the image segmentation network in multi-class scenarios.

[0102] In one implementation, the mathematical expression for Mean IoU is as follows:

[0103]

[0104]

[0105] in, This represents the area of ​​the lesion region in the segmentation result; This indicates the area of ​​the lesion region in the segmentation label.

[0106] The technical effects of this application are explained below based on simulation results.

[0107] A segmentation study was conducted on CT images of children with congenital lung and airway malformations. First, a network architecture based on the U-Net network model combined with an Attention mechanism module was adopted. This network includes an encoder, an attention module, and a decoder. The decoder consists of convolutional and pooling layers to capture local features and retain important features. Simultaneously, a SkipConnection structure is used to fuse the feature maps generated by the encoder and decoder, which helps with information transfer and reduces information loss. During the training phase, data from the training set and their corresponding hand-labeled maps were used for network training. The segmentation model was evaluated based on the test set. The predicted segmentation maps of CT images of children with lung malformations obtained by the neural network model were evaluated by comparing them with the hand-labeled maps. The Dice coefficient and Mean IoU were used as evaluation metrics to assess the model's performance. The results showed that the model achieved a Dice coefficient of 0.92 and a Mean IoU of 0.93, indicating that the network achieved high accuracy and performance in the prediction segmentation task. This demonstrates that the proposed method based on the U-Net network model combined with the Attention mechanism is effective in processing CT images of children with congenital lung and airway malformations and has the potential to play an important role in clinical practice. Validated results show that the DSC and Mean IoU of the attention-based image segmentation network are both above 0.9, indicating that the image segmentation network has high segmentation accuracy and is effective in segmenting CT images of children with congenital pulmonary tracheal malformations, and has the potential to play an important role in clinical practice.

[0108] As demonstrated by the above embodiments, this application constructs a deep learning architecture that integrates multiple scales and modalities and utilizes an attention mechanism for iterative optimization, exhibiting significant advantages in the precise segmentation and surgical planning of congenital pulmonary airway malformations (CPAM). CPAM lesions are typically large, compressing and displacing surrounding normal lung tissue and blood vessels (pulmonary arteries and pulmonary veins), and their boundaries are often complex and ambiguous. Accurately defining the lesion's extent and its three-dimensional spatial relationship with key vascular structures is crucial for developing safe and effective surgical pathways (such as the precise direction of the scalpel). The scheme requires input plain, venous, and arterial CT images of the same size. First, through training, these images are fused into a series of fused images ranging from low resolution (high-level semantics) to high resolution (low-level details) (from the Nth fused image to the first fused image). The core processing flow begins with high-level fusion information, fusing initial features through an attention mechanism and upsampling to obtain the first-stage result. Subsequently, iterative upsampling is performed, attention weights are generated by combining higher-resolution fused images, and the upsampling results are weighted and fused for correction, gradually refining the segmentation result until the N-1th stage result is output. Ultimately, this result was used to generate a "K+1 layer lesion segmentation map" containing K lesion core layers and a dedicated boundary layer. First, multimodal fusion effectively integrates the basic anatomical information from plain scan images, the clearly delineated pulmonary veins (with their compression displacement and abnormal drainage patterns) from venous phase images, and the pulmonary arteries (including abnormal feeding arteries and compression deformation) from arterial phase images. These three complementary pieces of information provide the model with a comprehensive three-dimensional view of the complex relationship between the lesion and surrounding key vessels. Second, multi-scale processing and iterative optimization mechanisms enable the model to grasp the overall compression range of the lesion from a high-level perspective (robustness), and gradually utilize higher-resolution fused image details (especially vascular visualization information) during decoding. Through an attention mechanism, it dynamically learns the importance weights of features at different locations, scales, and modalities, accurately guiding information fusion. This effectively overcomes the problems of boundary blurring and partial volume effects caused by the large lesion volume and compression, significantly improving the segmentation accuracy of the boundary layer (K+1 layer). Ultimately, this structured segmentation output, especially the fine boundary layer that integrates spatial information of arteries and veins, provides surgeons with an intuitive and reliable navigation basis for planning surgical paths. It clearly marks the main range of the lesion to be removed (K layer) and the precise anatomical interface (boundary layer) that the scalpel should strictly follow to avoid compressed important blood vessels (pulmonary arteries and veins) and to preserve as much functional compressed lung tissue as possible. This greatly improves the safety and precision of the surgery and reduces the risk of accidentally damaging blood vessels or removing too much normal tissue.

[0109] Based on the same concept, this application also provides an image segmentation system for lung structural abnormalities based on an attention mechanism. The method corresponding to this system can be the same as the attention-based image segmentation method for lung structural abnormalities described in the preceding embodiments, and its problem-solving principle is similar. The attention-based image segmentation system for lung structural abnormalities provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the methods and / or technical solutions of the various embodiments of this application.

[0110] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0111] Specifically, this embodiment may employ any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0112] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0113] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0114] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or page components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0120] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0122] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. An image segmentation method for lung structural abnormalities based on an attention mechanism, characterized in that, include: By training and fusing plain scan images, vein images, and artery images, the Nth fused image, ..., the second fused image, and the first fused image are determined sequentially in descending order of size; N is a positive integer greater than two. The first fused image and the second fused image are processed through an attention mechanism to obtain a first weighted image; wherein the attention mechanism includes sequentially performing an addition operation, an activation operation, a convolution operation, and a reactivation operation; the activation operation of the attention mechanism is a ReLU activation operation; the reactivation operation of the attention mechanism is a Sigmoid activation operation; the calculation formula of the attention mechanism includes: ; Represented as the attention weights after skip connections and fusion. Indicates the first The next jump connection operation process; Represents the first weighted image; This represents the i-th fusion graph; and Represents the parameters of a linear transformation; Represents the linear transformation parameters used to calculate attention weights; It is the Sigmoid activation function; Based on the first fused image, a first-size image is obtained through upsampling; The first weighted image and the first size image are trained to obtain the first stage result; Iteratively execute the following steps until m = N - 2: Using the result of stage m as input, after upsampling, a dimension map of stage m+1 is generated; By using an attention mechanism, the m-th stage result and the (m+2)-th fused image are fused to generate the (m+1)-th weight image; Based on the (m+1)th weighted image, the (m+1)th size image is weighted and fused to output the (m+1)th stage result; Increment the value of m by 1; In this iteration, m=1 at the beginning and the iteration terminates when m increases to N-2, and the result of the N-1th stage is output. Based on the preset number of lesions K, the lesion segmentation map of layer K+1 is obtained through the results of the (m+1)th stage, so as to obtain the boundary segmentation result of each lesion; The attention-based image segmentation method for lung structural abnormalities also includes: The Nth flat scan element image, ..., the second flat scan element image and the first flat scan element image are determined in descending order of size. The Nth vein element image, ..., the second vein element image and the first vein element image are determined sequentially according to the order of size from largest to smallest; The Nth artery feature map, ..., the second artery feature map and the first artery feature map of the artery image are determined in descending order of size; The first plain scan feature image, the first vein feature image, and the first artery feature image are fused, channel adjusted, and size restored to obtain the first fused image. The second plain scan feature image, the second vein feature image, and the second artery feature image are fused, channel adjusted, and size restored to obtain the second fused image; The Nth plain scan feature image, the Nth vein feature image, and the Nth artery feature image are fused, channel adjusted, and size restored to obtain the Nth fused image; The attention-based image segmentation method for lung structural abnormalities also includes: Perform convolution and activation operations on the flat scan image to obtain the first flat scan feature map; The following steps are performed iteratively until the Nth flat scan feature map, ..., the second flat scan feature map and the first flat scan feature map of the flat scan image are obtained; Using the nth flat scan feature map as input, the (n+1)th flat scan feature map is obtained through downsampling, convolution, and activation operations. Increment the value of n by 1; Where n=1 at the beginning of the iteration; The size of the flat scan image is 480×480, N is 5, the size of the first fused image is 480×480, the size of the second fused image is 240×240, the size of the third fused image is 120×120, the size of the fourth fused image is 60×60, and the size of the fifth fused image is 30×30.

2. The lung structural abnormality disease image segmentation method based on attention mechanism according to claim 1, characterized in that, The attention-based image segmentation method for lung structural abnormalities also includes: The labeled image, the plain scan image, the vein image, and the artery image are input together into the training model; the labeled image is an image with the lesion location already marked. By comparing the boundary segmentation result with the labeled image, a comparison result is obtained; If the comparison result does not meet the preset threshold requirement, the parameters of the attention mechanism are adjusted, and all steps to obtain the boundary segmentation result are executed cyclically until the updated comparison result meets the preset threshold requirement.

3. An image segmentation system for lung structural abnormalities based on an attention mechanism, characterized in that, include: Encoder, attention module, and decoder; The encoder is configured to extract elements from plain scan images, vein images, and artery images; The decoder is configured to determine the Nth fused image, ..., the second fused image and the first fused image in descending order of size by training on fused plain images, vein images and artery images; The attention module is configured to pass the first fused image and the second fused image through an attention mechanism to obtain a first weighted image; Based on the first fused image, a first-size image is obtained through upsampling; the first weighted image and the first-size image are trained to obtain the first-stage result; The following steps are iteratively executed until m=N-2: the result of the m-th stage is used as input, and after upsampling, a size map of size m+1 is generated; through the attention mechanism, the result of the m-th stage and the fused image of m+2 are fused to generate the weight image of m+1. Based on the (m+1)th weighted image, the (m+1)th size image is weighted and fused to output the (m+1)th stage result; the value of m is incremented by 1; where m=1 at the beginning of the iteration, and the iteration is terminated when m increases to N-2 and the N-1th stage result is output. The attention mechanism includes sequentially performed addition, activation, convolution, and reactivation operations; the activation operation of the attention mechanism is a ReLU activation operation; the reactivation operation of the attention mechanism is a Sigmoid activation operation; and the calculation formula of the attention mechanism includes: ; Represented as the attention weights after skip connections and fusion. Indicates the first The next jump connection operation process; Represents the first weighted image; This represents the i-th fusion graph; and Represents the parameters of a linear transformation; Represents the linear transformation parameters used to calculate attention weights; It is the Sigmoid activation function; The decoder is further configured to obtain a K+1 layer lesion segmentation map based on the preset number of lesions K and the results of the (m+1)th stage, so as to obtain the boundary segmentation result of each lesion; The attention module is also configured to: The Nth flat scan element image, ..., the second flat scan element image and the first flat scan element image are determined in descending order of size. The Nth vein element image, ..., the second vein element image and the first vein element image are determined sequentially according to the order of size from largest to smallest; The Nth artery feature map, ..., the second artery feature map and the first artery feature map of the artery image are determined in descending order of size; The first plain scan feature image, the first vein feature image, and the first artery feature image are fused, channel adjusted, and size restored to obtain the first fused image. The second plain scan feature image, the second vein feature image, and the second artery feature image are fused, channel adjusted, and size restored to obtain the second fused image; The Nth plain scan feature image, the Nth vein feature image, and the Nth artery feature image are fused, channel adjusted, and size restored to obtain the Nth fused image; The attention module is also configured to: Perform convolution and activation operations on the flat scan image to obtain the first flat scan feature map; The following steps are performed iteratively until the Nth flat scan feature map, ..., the second flat scan feature map and the first flat scan feature map of the flat scan image are obtained; Using the nth flat scan feature map as input, the (n+1)th flat scan feature map is obtained through downsampling, convolution, and activation operations. Increment the value of n by 1; Where n=1 at the beginning of the iteration; The size of the flat scan image is 480×480, N is 5, the size of the first fused image is 480×480, the size of the second fused image is 240×240, the size of the third fused image is 120×120, the size of the fourth fused image is 60×60, and the size of the fifth fused image is 30×30.

4. An image segmentation system for lung structural abnormalities based on an attention mechanism, characterized in that, include: At least one processor; A memory that is communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 2.

5. A computer-readable medium having computer program instructions stored thereon, characterized in that, The computer program instructions can be executed by a processor to implement the method as described in any one of claims 1 to 2.

Images