Edge-enhanced left atrial appendage medical image segmentation method and system, and storage medium

Abstract

The invention discloses an edge-enhanced left atrial appendage medical image segmentation method and system and a storage medium, and the method comprises the steps: constructing a convolutional neural network model of a U-shaped coding structure based on an attention mechanism and a weight loss function, and recording the model as EMCLUNet + +; the medical image segmentation model (EMCLUNet + +) based on edge enhancement adopts an improved UNet + + as a backbone network, and comprises a convolution encoder module, a decoder part, a multi-scale attention module EMSA and an efficient attention mechanism module Clotion. And transmitting a medical image to be segmented into the trained convolutional neural network model to obtain a segmentation result. According to the method, redundant information interference is reduced and the accuracy of boundary information is improved by utilizing a channel and a space attention mechanism. For conditions (such as pericardial effusion or pleural effusion of a patient) with complications, the robustness to complex backgrounds and noise is enhanced by the multi-scale feature fusion and lightweight module, and a high-quality segmentation result is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and image segmentation, and in particular to a left atrial appendage medical image segmentation method and system based on edge enhancement and a storage medium.

[0002] The project on which this invention is based: Shanxi Province Key R&D Plan "Research and Application of Key Technologies for Predicting Risk of Stroke Based on Big Data Analysis". The left atrial appendage CT medical image dataset used is from the Second Hospital of Shanxi Medical University and was annotated by professional physicians. Background Art

[0003] Medical image segmentation can make images of pathological structures clearer and plays an important role in computer-aided diagnosis and intelligent medicine.

[0004] The main difficulties in segmenting the left atrial appendage are: the LAA region presents diverse morphological features, uneven size, shape, and intensity distribution; the boundary between the LAA and surrounding tissues is unclear, and there may be complications, such as fluid accumulation in the patient. These difficulties increase the complexity of medical image segmentation. The results obtained by existing LAA medical image segmentation methods often contain a large amount of relevant redundant information and irrelevant feature information, resulting in slow segmentation progress and low efficiency.

[0005] In the field of LAA segmentation, researchers have proposed a variety of methods to address this challenge. Early work focused on traditional image processing techniques and semi-automatic methods, such as combining the maximum flow algorithm and the random forest algorithm to segment LAA. The maximum flow algorithm is used to find the optimal path to segment the LAA area, while the random forest algorithm is used for feature selection and classification to improve the accuracy of segmentation; another example is to segment LAA by combining the segmentation results of multiple heart regions. This method first segments multiple areas of the heart, and then finally segments the LAA by combining the results of these areas. This method can utilize the relative position relationship between the various areas of the heart to improve the robustness of the segmentation. However, although traditional methods have certain advantages in LAA segmentation, they still face the problem of high computational complexity. With the rapid development of deep learning technology, researchers have gradually transitioned from traditional methods to deep learning methods in the research direction of LAA segmentation. Researchers have made significant progress in LAA segmentation, especially methods based on graph segmentation technology, Unet, and fully convolutional neural network (FCN) have shown significant advantages in LAA segmentation tasks. However, these advances are accompanied by key issues that need to be solved urgently. There are still some limitations when dealing with LAA with unclear boundaries and irregular shapes. These problems are particularly prominent in actual clinical applications, because the boundaries of LAA are usually unclear, irregular in shape, and easily affected by noise and partial volume effects. These factors make it difficult for deep learning models to accurately capture the boundary information of LAA during training and testing, thus affecting the accuracy of segmentation results. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the problems of low segmentation efficiency and accuracy and lack of comprehensiveness of the left atrial appendage medical image segmentation method in the prior art, and to provide a left atrial appendage medical image segmentation method, system and storage medium, which have fast segmentation progress, high efficiency, comprehensiveness and can obtain high-precision segmentation results.

[0007] In a first aspect, in order to solve the above technical problems, the present invention provides a left atrial appendage medical image segmentation method, comprising the following steps:

[0008] S110 collects medical image data of the left atrial appendage lesion area, and divides the medical image data into a training set and a test set according to a ratio set independently;

[0009] S120 original image dataset and label dataset are used for data preprocessing, and test and training datasets are constructed;

[0010] S130 builds a convolutional neural network model with a U-shaped coding structure based on the attention mechanism and weighted loss function, denoted as EMCL_UNet++;

[0011] S140 training the convolutional neural network model based on the training set and the test set;

[0012] S150: passing the medical image to be segmented into the trained convolutional neural network model to obtain a segmentation result;

[0013] The data preprocessing method in step S120 includes steps S210 to S230, specifically, S210 slicing three-dimensional medical images, S220 normalizing two-dimensional images, and S230 scaling two-dimensional images.

[0014] S210: If the medical image is a three-dimensional image, each original medical image data and the corresponding target region manual segmentation result data are two-dimensionally sliced ​​along a cross section.

[0015] S220: In order to accelerate the convergence of neural network training and ensure fast convergence of the network, all two-dimensional images are first normalized, that is, each pixel value of the image is changed from [0, 255] to [0, 1].

[0016] Secondly, the normalized original medical image data and the corresponding target area manual segmentation result data are used to construct the original image training dataset I in a ratio of 80% and 20% respectively. train , original image test dataset I test , Manual segmentation results training data set L train And the manual segmentation results test dataset L test .

[0017] S230: All image data are scaled using the resize() function in the PIL package, and the image size is scaled to 1×256×256.

[0018] In step S130, a medical image segmentation model based on edge enhancement (EMCL_UNet++) is constructed, using improved UNet++ as the backbone network, including a convolutional encoder module, a decoder part, a multi-scale attention module EMSA and an efficient attention mechanism module Cloattention, as follows:

[0019] 310 In the convolution encoder module, five convolution encoding blocks are connected in sequence: the first to the fifth convolution encoding blocks, each of which is composed of two 2D convolution layers, a batch normalization layer and a ReLU activation layer; the specific calculation process includes: first, Conv1 performs a convolution operation on the input two-dimensional image and outputs a feature map, then the output of the convolution layer is processed by the normalization layer to stabilize the training process, and finally the output of the normalization layer is applied to the ReLU activation function to output the high-level abstract features of the feature map;

[0020] 320 In the multi-scale attention module EMSA, it includes a Stem module, four groups of sequentially connected position encoding modules and an efficient Transformer block; the Stem module contains a convolutional layer for preliminary processing of the input image and extracting low-level features; the specific process includes: the Stem module first extracts the low-level information output by the fifth convolutional encoding block; the formula is as follows:

[0021] x=Conv(x input )

[0022] Among them, x input is the input image, Conv represents the convolution operation; the function of the Stem module is to reduce the resolution of the input feature map and expand the channel dimension; the function of the position encoding module (Position embed) is to suppress the position information and enhance the feature extraction ability of the Stem module; the formula is as follows:

[0023] X pos =PosEmbed(x)

[0024] Among them, PosEmbed(·) is the position encoding operation, X pos is the feature map after position encoding; next, the low-level information extracted by the Stem module is sent to the efficient Transformer block after the position encoding module enhances the feature extraction capability, so as to reduce the computational complexity and maintain the diversity of multi-head attention; the formula is as follows:

[0025]

[0026] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O

[0027]

[0028] Among them, FC(·) is a fully connected layer used to generate query Q, key K and value V; IN(·) is a normalization operation. is the dimension of the key, Softmax(·) is used to calculate the attention weight; Concat(·) is the output of concatenating multiple heads, W O is the output projection matrix; LayerNorm(·) represents the layer normalization operation, and FFN(·) is a feed-forward neural network;

[0029] Finally, the output of the multi-scale attention module EMSA is input into the first convolutional encoding block of the convolutional decoder module to maintain the integrity of the feature map; the formula is as follows:

[0030] X output =Conv dec (X ffn ), X overlap =Conv overlap (X pos )

[0031] X final =σ(DWConv(X overlap ))

[0032] Among them, Conv overlap is an overlapped convolution operation, σ is an activation function, and DWConv is a depthwise convolution operation.

[0033] 330 sets up a total of four efficient attention mechanism Cloattention modules, corresponding to the first to fourth convolutional coding blocks. In each efficient attention mechanism Cloattention module, there is a Conv Stem block, four groups of Clo blocks and ConvFFN blocks. The specific calculation process includes: first, the input image is preliminarily processed by the Conv Stem block to extract features: each convolution contains four convolution layers, and the Stride of each convolution layer is 2, 2, 1 and 1 respectively, gradually reducing the spatial size of the feature map and increasing the number of channels; the formula is as follows:

[0034] x stem =Conv4(Conv3(Conv2(Conv1(x input ))))

[0035] Among them, x stem It is the feature map after convolution processing;

[0036] Then, the extracted features are passed through the Clo block for multi-scale feature extraction; and the ConvFFN block is used to enhance the ability to capture local information; the ConvFFN block uses a deep separable convolution DWConv; the Clo block contains a local branch and a global branch; the local branch processes high-frequency information through the AttnConv module, and the formula is as follows:

[0037] Q,K,V=FC(x in )

[0038] V s =DWConv(V)

[0039] context_weights=σ(DWConv(Q)⊙DWConv(K))

[0040] x local =x in +V enhanced

[0041] Among them, DWConv is a deep convolution, which is used for local feature fusion; V s is the local feature after aggregation; context_weights is the context-aware weight, which is used for context-local awareness enhancement; V enhanced is the enhanced local feature, x local is the output feature map of the local branch.

[0042] The global branch uses the standard attention mechanism to process low-frequency global information. The formula is as follows:

[0043] K g =Pool(K),V g =Pool(V)

[0044] x global =Attention(Q,K g ,V g )

[0045] x global =x in +x global

[0046] The first step is to perform downsampling to reduce the spatial size of the feature map, K g and V g is the downsampled key and value; Attention is the standard attention mechanism, which calculates the query Q and the key K g The similarity of and weighted sum value V g ;x global is the output feature map of the global branch;

[0047] Finally, the output feature maps of the local branch and the global branch are concatenated to achieve the fusion of multi-scale features; the formula is as follows:

[0048] x out =Concat(x local ,x global )

[0049] Finally, the final binary classification result is generated through global average pooling and fully connected layers; the formula is as follows:

[0050] x pool =GlobalAvgPool(x out )

[0051] y pred =FC(x pool )

[0052] The structure of the convolutional decoding module 340 includes four convolutional decoding blocks, which correspond to four Cloattention modules of efficient attention mechanism. The first to third convolutional decoding modules are composed of a 2D convolutional layer, a batch normalization layer, a ReLU activation layer and a deconvolutional layer; the fourth convolutional decoding module is composed of a 2D convolutional layer, a batch normalization layer, a ReLU activation layer, a 2D convolutional layer and a Sigmoid layer; taking one of the first to third convolutional decoding modules as an example, the specific calculation process includes: firstly, the input feature map is enlarged by a Conv2D, and then the enlarged feature map is spliced ​​with the feature map of the corresponding stage of the encoder to achieve feature fusion; then, the spliced ​​feature map is passed through a batch normalization layer and a ReLU activation layer to further extract and process features. Finally, the feature map passes through a deconvolution layer, which upsamples the feature map to restore the spatial resolution, thereby generating an output feature map of a larger size. The fourth convolution decoding module omits the deconvolution layer and directly processes the features through a 2D convolution layer, a batch normalization layer, and a ReLU activation layer, and finally obtains the final segmentation result through a 2D convolution layer and a Sigmoid layer.

[0053] In step S140, the original image training data set I in step S120 is used. train And the manual segmentation result training dataset L train The specific steps are as follows: First, I train Input into the network to calculate the result of this round of iteration, and compare the result with L train The corresponding manual segmentation results are compared with the loss function, and the loss value is calculated; the gradient is calculated by the stochastic gradient descent optimizer and the weights in the network are updated by back propagation; the above process is iterated until the error requirements are met to obtain the network training model; finally, the original image test dataset I is used test And the manual segmentation results test dataset L test Verify the model. The loss function in the above steps uses a weighted loss function, calculated as follows

[0054] L s =α·Loss BCE +β·Loss DICE

[0055] Among them, α and β represent the weights of the two loss functions. BCE Represents the balanced binary cross entropy loss function, Loss DICE Represents the Dice loss function. The calculation formula is as follows

[0056]

[0057]

[0058] Here, ε is a small constant to prevent division by zero errors. i is the label image, For the prediction results.

[0059] In a second aspect, in order to solve the above technical problems, the present invention also provides a left atrial appendage medical image segmentation system, comprising:

[0060] A data acquisition module, which collects medical image data of the left atrial appendage lesion area and divides the medical image data into a training set and a test set according to a self-set ratio;

[0061] A model construction module, which constructs a weighted loss function by adopting a multi-loss fusion method, and constructs a convolutional neural network model with a U-type encoding and decoding structure based on the attention mechanism and the weighted loss function;

[0062] A model training module, which trains the convolutional neural network model based on the training set and the test set;

[0063] An image segmentation module, which passes the medical image to be segmented into the trained convolutional neural network model to obtain a segmentation result;

[0064] Among them, the U-type codec structure includes an encoder, a decoder and a jump connection part; the attention mechanism is introduced between the bottom layers of the encoder and the decoder; and the attention mechanism is introduced between the decoder and the jump connection part.

[0065] In a third aspect, in order to solve the above-mentioned technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the left atrial appendage medical image segmentation method are implemented.

[0066] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0067] Compared with the prior art, the present invention provides a left atrial appendage medical image segmentation method, system and storage medium based on attention mechanism and weighted loss function, which solves multiple problems existing in the prior art.

[0068] First, in order to address the problems of high computational complexity and slow segmentation progress in existing methods, the present invention adopts a convolutional neural network model with a U-type codec structure and introduces a multi-scale attention module (EMSA) to capture accurate local feature information at the bottom of the encoder, accelerate the feature extraction process, and significantly improve segmentation efficiency. Secondly, in order to solve the problem that the existing methods use a single loss function to handle special samples poorly, the present invention introduces a weighted loss function to dynamically adjust the weights to adapt to LAA areas with different forms, sizes, shapes and uneven intensity distributions, thereby improving the comprehensiveness and robustness of the model. In addition, in order to address the problem that existing methods are difficult to accurately capture LAA areas with unclear boundaries and irregular shapes, the present invention uses channel and spatial attention mechanisms to reduce redundant information interference and improve the accuracy of boundary information. Finally, for situations accompanied by complications (such as pericardial effusion or pleural effusion in patients), the multi-scale feature fusion and lightweight modules of the present invention enhance the robustness to complex backgrounds and noise, ensuring high-quality segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 :Comparison of algorithm effects; a original image; b original algorithm; c improved algorithm;

[0070] Figure 2 : Overall flow chart;

[0071] Figure 3 : Data preprocessing flow chart;

[0072] Figure 4 : EMCL_UNet++ network structure diagram;

[0073] Figure 5 : Convolutional coding block flow chart;

[0074] Figure 6 : Flowchart of the first to third convolutional decoding blocks;

[0075] Figure 7 : The fourth convolutional decoding block flow chart;

[0076] Figure 8 : EMSA module structure diagram;

[0077] Fig. 9 : Cloattention module structure diagram; DETAILED DESCRIPTION

[0078] The present invention is described in detail below in conjunction with specific embodiments.

[0079] The present invention provides a left atrial appendage medical image segmentation method, comprising the following steps:

[0080] S110 collects medical image data of the left atrial appendage lesion area, and divides the medical image data into a training set and a test set according to a ratio set independently;

[0081] S120 original image dataset and label dataset are used for data preprocessing, and test and training datasets are constructed;

[0082] S130 builds a convolutional neural network model with a U-shaped coding structure based on the attention mechanism and weighted loss function, denoted as EMCL_UNet++;

[0083] S140 training the convolutional neural network model based on the training set and the test set;

[0084] S150: passing the medical image to be segmented into the trained convolutional neural network model to obtain a segmentation result;

[0085] The data preprocessing method in step S120 is as follows: Figure 3 As shown, it includes steps S210-S230. Specifically, S210 is 3D medical image slicing, S220 is 2D image normalization, and S230 is 2D image scaling. Each module is described below:

[0086] S210: If the medical image is a three-dimensional image, each original medical image data and the corresponding target region manual segmentation result data are two-dimensionally sliced ​​along a cross section.

[0087] S220: In order to accelerate the convergence of neural network training and ensure fast convergence of the network, all two-dimensional images are first normalized, that is, each pixel value of the image is changed from [0,255] to [0,1]. The normalization formula is as follows:

[0088]

[0089] where x i represents the i-th pixel value, max(x) and min(x) represent the maximum and minimum values ​​of the pixel respectively.

[0090] Secondly, the normalized original medical image data and the corresponding target area manual segmentation result data are used to construct the original image training dataset I in a ratio of 80% and 20% respectively. train , original image test dataset I test , Manual segmentation results training data set L train And the manual segmentation results test dataset L test .

[0091] S230: All image data are scaled using the resize() function in the PIL package, and the image size is scaled to 1×256×256.

[0092] In step S130, a medical image segmentation model based on edge enhancement (EMCL_UNet++) is constructed, and the network structure is as follows: Figure 4 As shown in the figure, the improved UNet++ is used as the backbone network, which includes a convolutional encoder module, a decoder part, a multi-scale attention module EMSA and an efficient attention mechanism module Cloattention, as follows:

[0093] In the convolutional encoder module 310, five convolutional encoding blocks (first to fifth convolutional encoding blocks) are included, wherein each convolutional encoding block consists of a 2D convolution layer, a batch normalization layer, and a ReLU activation layer, such as Figure 5 The specific calculation process includes: first, Conv2D performs convolution operation on the input two-dimensional image and outputs the feature map, then the output of the convolution layer is processed by the normalization layer to stabilize the training process, and finally the output of the normalization layer is applied to the ReLU activation function to output the high-level abstract features of the feature map.

[0094] 320 In the multi-scale attention module EMSA, it includes the Stem module, four groups of sequentially connected position encoding modules and an efficient Transformer block. The Stem module contains several convolutional layers for preliminary processing of the input image and extracting low-level features. The EMSA module is as follows Figure 8 The specific calculation process includes: first extracting the low-level information output by the fifth convolutional coding block. The formula is as follows:

[0095] x=Conv(x input )

[0096] Among them, x input is the input image, and Conv represents the convolution operation. The function of the Stem module is to reduce the resolution of the input feature map and expand the channel dimension. The function of the position encoding module (Position embed) is to suppress the position information and enhance the feature extraction ability of the Stem module. The formula is as follows:

[0097] X pos =PosEmbed(x)

[0098] Among them, PosEmbed(·) is the position encoding operation, X pos is the feature map after position encoding. Next, the low-level information extracted by the Stem module is sent to the efficient Transformer block after the position encoding module enhances the feature extraction capability, thereby reducing the computational complexity and maintaining the diversity of multi-head attention. The formula is as follows:

[0099] Q,K,V=FC(X pos ),

[0100] MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O

[0101]

[0102] Among them, FC(·) is a fully connected layer used to generate query Q, key K and value V; IN(·) is a normalization operation. is the dimension of the key, Softmax(·) is used to calculate the attention weight; Concat(·) is the output of concatenating multiple heads, W O is the output projection matrix; LayerNorm(·) represents the layer normalization operation, and FFN(·) is a feed-forward neural network.

[0103] Finally, the output of the multi-scale attention module EMSA is input into the first convolutional encoding block of the convolutional decoder module to maintain the integrity of the feature map. The formula is as follows:

[0104]

[0105] X final =σ(DWConv(X overlap ))

[0106] Among them, Conv overlap is an overlapped convolution operation, σ is an activation function, and DWConv is a depthwise convolution operation.

[0107] 330 A total of four efficient attention mechanism Cloattention modules are set, corresponding to the first to fourth convolutional coding blocks, such as Fig. 9 As shown in the figure, each Cloattention module of the efficient attention mechanism includes a Conv Stem block and four stages, each of which contains a Clo block and a ConvFFN block. The specific calculation process includes: first, the input image is preliminarily processed to extract features: each convolution contains four convolution layers, and the Stride of each convolution layer is 2, 2, 1, and 1, respectively, gradually reducing the spatial size of the feature map and increasing the number of channels. The formula is as follows:

[0108] x stem =Conv4(Conv3(Conv2(Conv1(x input ))))

[0109] Among them, x stemIt is a feature map after convolution processing. Then the extracted features are passed through the Clo block to extract multi-scale features; and the ConvFFN block is used to enhance the ability to capture local information. The ConvFFN block uses a deep separable convolution DWConv. The Clo block contains a local branch and a global branch. The local branch processes high-frequency information through the AttnConv module. The formula is as follows:

[0110] Q,K,V=FC(x in )

[0111] V s =DWConv(V)

[0112] context_weights=σ(DWConv(Q)⊙DWConv(K))

[0113] x local =x in +V enhanced

[0114] Among them, DWConv is a deep convolution, which is used for local feature fusion; V s is the local feature after aggregation; context_weights is the context-aware weight, which is used for context-local awareness enhancement; V enhanced is the enhanced local feature, x local is the output feature map of the local branch.

[0115] The global branch uses the standard attention mechanism to process low-frequency global information. The formula is as follows:

[0116] K g =Pool(K),V g =Pool(V)

[0117] x global =Attention(Q,K g ,V g )

[0118] x global =x in +x global

[0119] The first step is to perform downsampling to reduce the spatial size of the feature map, K g and V g is the downsampled key and value; Attention is the standard attention mechanism, which calculates the query Q and the key K g The similarity of and weighted sum value V g ;x global is the output feature map of the global branch.

[0120] Finally, the output feature maps of the local branch and the global branch are concatenated to achieve multi-scale feature fusion. The formula is as follows:

[0121] x out =Concat(x local ,x global )

[0122] Finally, the final binary classification result is generated through global average pooling and full connection layer. The formula is as follows:

[0123] x pool =GlobalAvgPool(x out )

[0124] y pred =FC(x pool )

[0125] The structure of the convolutional decoding module 340 includes four convolutional decoding blocks, which correspond to four efficient attention mechanism Cloattention modules. The first to third convolutional decoding modules consist of a 2D convolution layer, a batch normalization layer, a ReLU activation layer and a deconvolution layer, such as Figure 6 The fourth convolutional decoding module consists of a 2D convolution layer, a batch normalization layer, a ReLU activation layer, a 2D convolution layer, and a Sigmoid layer, as shown in Figure 7 As shown. Taking one of the first to third convolutional decoding modules as an example, the specific calculation process includes: first, the input feature map is enlarged through a Conv2D, and then the enlarged feature map is spliced ​​with the feature map of the corresponding stage of the encoder to achieve feature fusion; then, the spliced ​​feature map passes through a batch normalization layer and a ReLU activation layer to further extract and process features. Finally, the feature map passes through a deconvolution layer, which upsamples the feature map to restore the spatial resolution, thereby producing an output feature map of a larger size. The fourth convolutional decoding module omits the deconvolution layer, and directly processes the features through a 2D convolution layer, a batch normalization layer, and a ReLU activation layer, and finally obtains the final segmentation result through a 2D convolution layer and a Sigmoid layer.

[0126] In step S140, the original image training data set I in step S120 is used. train And the manual segmentation result training dataset L train The specific steps are as follows: First, I train Input into the network to calculate the result of this round of iteration, and compare the result with L trainThe corresponding manual segmentation results are compared with the loss function, and the loss value is calculated; the gradient is calculated by the stochastic gradient descent optimizer and the weights in the network are updated by back propagation; the above process is iterated until the error requirements are met to obtain the network training model; finally, the original image test dataset I is used test And the manual segmentation results test dataset L test Verify the model. The loss function in the above steps uses a weighted loss function, calculated as follows

[0127] L s =α·Loss BCE +β·Loss DICE

[0128] Among them, α and β represent the weights of the two loss functions. BCE Represents the balanced binary cross entropy loss function, Loss DICE Represents the Dice loss function. The calculation formula is as follows

[0129]

[0130]

[0131] Here, ε is a small constant to prevent division by zero errors. i is the label image, For the prediction results.

[0132] In a second aspect, in order to solve the above technical problems, the present invention also provides a left atrial appendage medical image segmentation system, comprising:

[0133] A data acquisition module, which collects medical image data of the left atrial appendage lesion area and divides the medical image data into a training set and a test set according to a self-set ratio;

[0134] A model construction module, which constructs a weighted loss function by adopting a multi-loss fusion method, and constructs a convolutional neural network model with a U-type encoding and decoding structure based on the attention mechanism and the weighted loss function;

[0135] A model training module, which trains the convolutional neural network model based on the training set and the test set;

[0136] An image segmentation module, which passes the medical image to be segmented into the trained convolutional neural network model to obtain a segmentation result;

[0137] Among them, the U-type codec structure includes an encoder, a decoder and a jump connection part; the attention mechanism is introduced between the bottom layers of the encoder and the decoder; and the attention mechanism is introduced between the decoder and the jump connection part.

[0138] In a third aspect, in order to solve the above-mentioned technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the left atrial appendage medical image segmentation method are implemented.

[0139] Example 1: Application to the image segmentation of the pulmonary vein CT left atrial appendage dataset

[0140] The specific steps are as follows:

[0141] The original data in the pulmonary vein CT left atrial appendage dataset and the corresponding target area manual segmentation result data are respectively formed into a CT dataset I and a label dataset L, and the data are further preprocessed. Each CT data and the corresponding target area manual segmentation result data are sliced ​​two-dimensionally along the cross section, and all two-dimensional slices are normalized, that is, each pixel value of the image is changed from [0,255] to [0,1]. The normalization formula is as follows:

[0142]

[0143] where x i represents the i-th pixel value, max(x) and min(x) represent the maximum and minimum values ​​of the pixel respectively.

[0144] The normalized CT data and the corresponding target area manual segmentation result data are used to construct the CT training dataset I in a ratio of 80% and 20% respectively. train , CT test dataset I test , Manual segmentation results training data set L train And the manual segmentation results test dataset L test All image data is scaled using the resize() function in the PIL package, and the image size is scaled to 1×256×256.

[0145] Taking a piece of CT data as an example, we build and train a medical image segmentation model (EMCL_UNet++). The specific steps are as follows:

[0146] First, a convolutional encoding module is constructed. In the convolutional encoder module, there are five convolutional encoding blocks, each of which consists of two 2D convolutional layers, a batch normalization layer, and a ReLU activation layer. The specific calculation process is as follows: the original input image is subjected to five convolutional block calculations and four downsampling operations, and the resulting feature map will contain rich high-level semantic information. Finally, the rich semantic information is passed through the EMSA module to generate the feature map F EMSA .

[0147] Secondly, construct a convolutional decoding module. The structure of the convolutional decoding module includes four convolutional decoding modules. The first to third convolutional decoding modules are composed of two 2D convolutional layers, a batch normalization layer, a ReLU activation layer, a deconvolution layer and a splicing layer, and the fourth convolutional decoding module is composed of two 2D convolutional layers, a batch normalization layer, and a ReLU activation layer. The specific calculation process includes: first, the input feature map is enlarged through a Conv, and then the enlarged feature map is spliced ​​with the feature map of the corresponding stage of the encoder to achieve feature fusion. Then, after two 2D convolutional layers, a batch normalization layer and a ReLU activation layer, the features are further extracted and processed.

[0148] Again, the convolutional decoding block not only recovers spatial information from the feature map extracted by the encoder, but also enhances the feature expression capability through multi-scale feature fusion. The structure of the decoder includes four convolutional decoding blocks, four Cloattention modules and four upsampling modules. The specific calculation process is as follows: First, the feature map generated by the EMSA module passes through the first upsampling block, the first convolutional decoding block, and the fourth convolutional encoding block after passing through the Cloattention block for channel splicing; secondly, after passing through the second upsampling block, the second convolutional decoding block, the third convolutional encoding block and the first convolutional decoding block after passing through the Cloattention block for channel splicing; again, after passing through the third upsampling block, the third convolutional decoding block, the second convolutional encoding block and the second convolutional decoding block after passing through the Cloattention block for channel splicing; finally, after passing through the fourth upsampling block, the fourth convolutional decoding block, the first convolutional encoding block and the third convolutional decoding block after passing through the Cloattention block for channel splicing. The fourth convolutional decoding module omits the deconvolution layer and the concatenation layer, and directly processes the features through two 2D convolution layers, a batch normalization layer, and a ReLU activation layer. Finally, the final segmentation result is obtained after calculation through a two-dimensional convolution layer and a Sigmoid layer.

[0149] Finally, use the CT training dataset I train And the manual segmentation result training dataset L train Train the model. The specific steps are as follows: train Input into the network to calculate the result of this round of iteration, and compare the result with L train The loss function is used to calculate the loss value. The loss function uses L s =α·Loss BCE +β·Loss DICE . Among them, α and β represent the weights of the two loss functions. BCE Represents the balanced binary cross entropy loss function, LossDICE represents the Dice loss function. In this embodiment, the values ​​are α=0.5 and β=0.5 respectively. Next, the gradient is calculated by the stochastic gradient descent optimizer and the weights in the network are updated by back propagation. The above process is iterated until the error requirement is met to obtain the network training model. Finally, the CT test data set I test Input into the model to obtain the final segmentation result, and compare it with the manual segmentation result test data set L test The model is verified. The experimental results of the EMCL_UNet++ network in this embodiment are shown in Table 1.

[0150] Table 1 Experimental results of EMCL_UNet++ on the left atrial appendage dataset

[0151]

[0152] Performance indicators:

[0153] DSC (Dice Similarity Coefficient): used to measure the similarity between two sample sets, with a value range of [0,1]. The closer the value is to 1, the higher the overlap between the two samples. In this context, DSC measures the similarity between the predicted segmented region and the actual manually segmented region.

[0154] IoU (Intersection over Union): Measures the overlap between the predicted segmentation and the true label, with a value between [0,1], and the larger the value, the better. IoU is another widely used criterion for evaluating segmentation accuracy.

[0155] Experimental results analysis:

[0156] In the comparative experiment, the EMCL_UNet++ model showed better segmentation performance than the traditional U-Net, with DSC and IoU on the left atrial appendage dataset reaching 90.7% and 78.4% respectively, which is higher than 87.7% and 73.7% of U-Net. This improvement shows that the new model can more accurately identify and separate the left atrial appendage area, and its enhanced segmentation accuracy is attributed to the introduction of EMSA and Cloattention modules and other mechanisms. These features help capture richer semantic information and spatial relationships, and achieve efficient, comprehensive and high-precision LAA medical image segmentation, providing strong support for clinical diagnosis.

[0157] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A method for edge-enhanced left atrial appendage medical image segmentation, characterized in that: The following steps are involved: S110 collects medical image data of the left atrial appendage lesion area, and divides the medical image data into a training set and a test set according to a ratio set independently; S120 original image dataset and label dataset are used for data preprocessing, and test and training datasets are constructed; S130 builds a convolutional neural network model with a U-shaped coding structure based on the attention mechanism and weighted loss function, denoted as EMCL_UNet++; the edge-enhanced medical image segmentation model EMCL_UNet++ uses the improved UNet++ as the backbone network, including a convolutional encoder module, a decoder part, a multi-scale attention module EMSA, and an efficient attention mechanism module Cloattention; S140 training the convolutional neural network model based on the training set and the test set; S150 passes the medical image to be segmented into the trained convolutional neural network model to obtain a segmentation result.

2. The method according to claim 1, characterized in that The data preprocessing method in step S120 includes steps S210-S230; S210: If the medical image is a three-dimensional image, two-dimensionally slice each original medical image data and the corresponding target area manual segmentation result data along the cross section; S220: In order to accelerate the convergence of neural network training and ensure fast convergence of the network, all two-dimensional images are first normalized, that is, each pixel value of the image is changed from [0, 255] to [0, 1]; Secondly, the normalized original medical image data and the corresponding target area manual segmentation result data are used to construct the original image training dataset I in a ratio of 80% and 20% respectively. train , original image test dataset I test , Manual segmentation results training data set L train And the manual segmentation results test dataset L test ; S230: All image data are scaled using the resize() function in the PIL package, and the image size is scaled to 1×256×256.

3. The method according to claim 1, characterized in that In the convolution encoder module, there are five convolution encoding blocks connected in sequence: the first to the fifth convolution encoding blocks, each of which consists of two 2D convolution layers, a batch normalization layer and a ReLU activation layer; the specific calculation process includes: first, Conv1 performs a convolution operation on the input two-dimensional image and outputs a feature map, then the output of the convolution layer is processed through the normalization layer to stabilize the training process, and finally the output of the normalization layer is applied to the ReLU activation function to output the high-level abstract features of the feature map.

4. The method according to claim 1, characterized in that: The multi-scale attention module EMSA includes a Stem module, four sets of sequentially connected position encoding modules, and an efficient Transformer block; The Stem module contains convolutional layers, which are used to preliminarily process the input image and extract low-level features; The specific process includes: the Stem module first extracts the low-level information output by the fifth convolutional coding block; the formula is as follows: x=Conv(x input ) Among them, x input is the input image, Conv represents the convolution operation; the function of the Stem module is to reduce the resolution of the input feature map and expand the channel dimension; the function of the position encoding module (Position embed) is to suppress the position information and enhance the feature extraction ability of the Stem module; the formula is as follows: X pos =PosEmbed(x) Among them, PosEmbed(·) is the position encoding operation, X pos is the feature map after position encoding; next, the low-level information extracted by the Stem module is sent to the efficient Transformer block after the position encoding module enhances the feature extraction capability, so as to reduce the computational complexity and maintain the diversity of multi-head attention; the formula is as follows: MultiHead(Q,K,V)=Concat(head1,head2,...,head h )W O X ffn =LayerNorm(X multihead +FFN(X multihead )) Among them, FC(·) is a fully connected layer used to generate query Q, key K and value V; IN(·) is a normalization operation. is the dimension of the key, Softmax(·) is used to calculate the attention weight; Concat(·) is the output of concatenating multiple heads, W O is the output projection matrix; LayerNorm(·) represents the layer normalization operation, and FFN(·) is a feed-forward neural network; Finally, the output of the multi-scale attention module EMSA is input into the first convolutional encoding block of the convolutional decoder module to maintain the integrity of the feature map; the formula is as follows: X output =Conv dec (X ffn ),X overlap =Conv overlap (X pos ) X final =σ(DWConv(X overlap )) Among them, Conv overlap is an overlapped convolution operation, σ is an activation function, and DWConv is a depthwise convolution operation.

5. The method according to claim 1, characterized in that A total of four Cloattention modules are set up, corresponding to the first to fourth convolutional coding blocks. In each Cloattention module, there is a Conv Stem block, four groups of Clo blocks and a ConvFFN block. The specific calculation process includes: first, the input image is preliminarily processed through the Conv Stem block to extract features: each convolution contains four convolution layers, and the stride of each convolution layer is 2, 2, 1 and 1 respectively, gradually reducing the spatial size of the feature map and increasing the number of channels; the formula is as follows: x stem =Conv4(Conv3(Conv2(Conv1(x input )))) Among them, x stem It is the feature map after convolution processing; Then, the extracted features are passed through the Clo block for multi-scale feature extraction; and the ConvFFN block is used to enhance the ability to capture local information; the ConvFFN block uses a deep separable convolution DWConv; the Clo block contains a local branch and a global branch; the local branch processes high-frequency information through the AttnConv module, and the formula is as follows: Q,K,V=FC(x in ) V s =DWConv(V) context_weights=σ(DWConv(Q)⊙DWConv(K)) x local =x in +V enhanced Among them, DWConv is a deep convolution, which is used for local feature fusion; V s is the local feature after aggregation; context_weights is the context-aware weight, which is used for context-local awareness enhancement; V enhanced is the enhanced local feature, x local is the output feature map of the local branch; The global branch uses the standard attention mechanism to process low-frequency global information. The formula is as follows: K g =Pool(K),V g =Pool(V) x global =Attention(Q,K g ,V g ) x global =x in +x global The first step is to perform downsampling to reduce the spatial size of the feature map, K g and V g is the downsampled key and value; Attention is the standard attention mechanism, which calculates the query Q and the key K g The similarity of and weighted sum value V g ;x global is the output feature map of the global branch; Finally, the output feature maps of the local branch and the global branch are spliced ​​to achieve the fusion of multi-scale features; the formula is as follows: x out =Concat(x local ,x global ) Finally, the final binary classification result is generated through global average pooling and fully connected layers; the formula is as follows: x pool =GlobalAvgPool(x out ) y pred =FC(x pool )。 6. The method according to claim 1, characterized in that The structure of the convolutional decoding module includes four convolutional decoding blocks, which correspond to four efficient attention mechanism Cloattention modules respectively; the first to third convolutional decoding modules are composed of a 2D convolutional layer, a batch normalization layer, a ReLU activation layer and a deconvolution layer; the fourth convolutional decoding module is composed of a 2D convolutional layer, a batch normalization layer, a ReLU activation layer, a 2D convolutional layer and a Sigmoid layer; taking one of the first to third convolutional decoding modules as an example, the specific calculation process includes: firstly, a Conv2D is used to enlarge the input feature map, The enlarged feature map is then spliced ​​with the feature map of the corresponding stage of the encoder to achieve feature fusion; then, the spliced ​​feature map passes through a batch normalization layer and a ReLU activation layer to further extract and process features; finally, the feature map passes through a deconvolution layer, which upsamples the feature map to restore the spatial resolution, thereby producing an output feature map of a larger size; the fourth convolution decoding module omits the deconvolution layer and directly processes the features through a 2D convolution layer, a batch normalization layer, and a ReLU activation layer, and finally obtains the final segmentation result through a 2D convolution layer and a Sigmoid layer.

7. The method according to claim 1, characterized in that In step S140, the original image training data set I in step S120 is used. train And the manual segmentation result training dataset L train The model is trained; the specific steps are as follows: First, I train Input into the network to calculate the result of this round of iteration, and compare the result with L train The corresponding manual segmentation results are compared with the loss function, and the loss value is calculated; the gradient is calculated by the stochastic gradient descent optimizer and the weights in the network are updated by back propagation; the above process is iterated until the error requirements are met to obtain the network training model; finally, the original image test dataset I is used test And the manual segmentation results test dataset L test Verify the model; the loss function in the above steps uses a weighted loss function, which is calculated as follows: L s =a·Loss BCE +b·Loss DICE Among them, α and β represent the weights of the two loss functions; Loss BCE Represents the balanced binary cross entropy loss function, Loss DICE Represents the Dice loss function; the calculation formula is as follows Where ε is a small constant to prevent division by zero errors; y i is the label image, For the prediction results.

8. A left atrial appendage medical image segmentation system according to any one of claims 1 to 7, characterized in that: include: A data acquisition module, which collects medical image data of the left atrial appendage lesion area and divides the medical image data into a training set and a test set according to a self-set ratio; A model construction module, which constructs a weighted loss function by adopting a multi-loss fusion method, and constructs a convolutional neural network model with a U-type encoding and decoding structure based on the attention mechanism and the weighted loss function; A model training module, which trains the convolutional neural network model based on the training set and the test set; An image segmentation module, which passes the medical image to be segmented into the trained convolutional neural network model to obtain a segmentation result; The U-shaped encoding and decoding structure includes an encoder, a decoder and a jump connection part; the attention mechanism is introduced between the bottom layers of the encoder and the decoder; The attention mechanism is introduced in the decoder and the skip connection part.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a left atrial appendage medical image segmentation method described in any one of claims 1-7 are implemented.

Images