A liver tumor segmentation method fusing three attentions

Abstract

The application discloses a liver tumor segmentation method fusing three kinds of attentions, which effectively fuses spatial attention, self-attention and edge fusion attention, and realizes accurate liver tumor segmentation. The method comprises the following steps: S1, preprocessing abdominal CT data, and dividing the data into a training set, a verification set and a test set in proportion; S2, combining double attention with convolution operation to construct double attention dynamic convolution; S3, fusing edge information and attention mechanism to construct an edge fusion attention module; S4, fusing edge supervision loss and segmentation loss to construct a multi-scale edge segmentation loss; S5, fusing double attention dynamic convolution, the edge fusion attention module and the multi-scale edge segmentation loss to construct a Triple Attention Fusion Network (TAF-Net) model; and S6, using an experimental data set to complete training, verification optimization and performance evaluation of the model. The application can adaptively adjust a convolution kernel and effectively capture edge features, effectively improves segmentation precision, and is suitable for automatic segmentation of liver tumors in CT images.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a liver tumor segmentation method fusing three kinds of attention. BACKGROUND

[0002] In recent years, medical image segmentation has played a crucial role in the diagnosis, treatment planning, and postoperative evaluation of liver tumors, especially in terms of automated and accurate segmentation. However, due to the significant diversity of liver tumors in terms of morphology, size, and fuzzy boundaries, the segmentation task faces great challenges. Especially in abdominal computerized tomography (CT) images, the contrast between tumors and normal liver tissue is low, and the images are easily disturbed by noise, artifacts, and physiological differences among different patients, making the accuracy and robustness of traditional segmentation methods difficult to meet actual needs. With the rapid development of computer technology, deep learning technology provides a new research direction for achieving automated and high-precision liver tumor segmentation.

[0003] Existing segmentation methods based on convolutional neural networks (CNN), such as U-Net, have shown significant effects in liver tumor segmentation tasks. However, these methods still have deficiencies in capturing subtle features and multi-scale information, and enhancing the perception of boundaries. Especially for liver tumors with fuzzy boundaries, accurate identification of their contours is difficult, which may lead to missed detection of small tumors or inaccurate identification of complex morphological tumors. In addition, U-Net and other models also have certain limitations in modeling global information, and the lack of receptive field may limit their segmentation performance for large lesions.

[0004] Dynamic convolution, as a mechanism that can dynamically adjust the weights of convolution kernels according to the input, provides a new idea for enhancing the feature extraction capability of the model. However, existing dynamic convolution methods still have certain limitations in some complex features, making it difficult to fully express the detailed features of tumors. On the other hand, attention mechanisms have been widely used in medical image segmentation in recent years. However, existing attention mechanisms usually focus on significant feature regions, and less on boundary information. For liver tumor segmentation tasks, modeling of boundary information is crucial for accurate identification of tumor regions.

[0005] To solve these problems, it is necessary to explore new technologies and methods to improve the accuracy and robustness of liver tumor segmentation. Therefore, how to utilize the characteristics of dynamic convolution and attention mechanisms and the advantages of U-Net architecture to effectively improve the precision and reliability of liver tumor segmentation has become a key problem to be solved in the field of medical image segmentation. SUMMARY

[0006] To solve the above technical problems, the application provides a liver tumor segmentation method fusing three attentions, which can provide a more accurate, efficient and adaptable liver tumor segmentation solution for users.

[0007] Specifically, the method comprises the following steps:

[0008] S1: preprocessing the abdominal CT data, and dividing it into a training set, a validation set and a test set in proportion;

[0009] S2: combining double attention and convolution operation to construct a double attention dynamic convolution;

[0010] S3: fusing edge information and attention mechanism to construct an edge fusion attention module;

[0011] S4: fusing edge supervision loss and segmentation loss to construct a multi-scale edge segmentation loss;

[0012] S5: fusing the double attention dynamic convolution, the edge fusion attention module and the multi-scale edge segmentation loss to construct a TripleAttentionFusionNetwork (TAF-Net) model;

[0013] S6: using experimental data sets to complete the training, verification optimization and performance evaluation of the model.

[0014] Preferably, S1 comprises the following steps:

[0015] S1.1: preprocessing the abdominal CT data, including data format conversion, cropping, resampling, denoising and normalization operation;

[0016] S1.2: dividing the preprocessed abdominal CT data into a training set, a validation set and a test set in the ratio of 8:1:1.

[0017] Preferably, S2 comprises the following steps:

[0018] S2.1: enhancing features using spatial attention and self-attention mechanism;

[0019] S2.2: fusing convolution operation to construct a double attention dynamic convolution Dual-Attention Dynamic Convolution (DADC) module.

[0020] Preferably, S2.1 comprises the following steps:

[0021] First, the original image is subjected to convolution operation to obtain an input feature map F, a 1x1 convolution is used to reduce the dimension of F, and then a sigmoid activation function is used to normalize the output tensor to obtain a spatial attention weight As (F), finally, the obtained spatial attention weights are multiplied pixel by pixel by the input feature map to obtain the output feature map F. s The calculation process is as follows:

[0022] A s (F)=σ(Conv 1×1 (F)),

[0023]

[0024] In the above formula, F is the input feature map, and A s (F) is the spatial attention weight. F represents the element-wise multiplication operation, σ represents the sigmoid activation function, and F... s This is the output feature map.

[0025] The output feature map F of the spatial attention module s A self-attention mechanism is used to capture the global dependencies and interactions between channels, resulting in the channel attention weights A. c Finally, the channel weights W, enhanced by the spatial attention module and self-attention mechanism, are obtained. c The calculation process is as follows:

[0026] F c =AvgPool(F s ),

[0027] Q = W Q ⊙F c ,

[0028] K = W K ⊙F c ,

[0029] V = W V ⊙F c ,

[0030]

[0031] W c =A c ⊙V,

[0032] In the above formula, F c W represents the channel features after global average pooling (AvgPool). Q W K W V It is a learnable weight matrix, where Q, K, and V represent the query, key, and value in the channel dimension, respectively, and A c The weights represent the channel dimensions, ⊙ represents matrix multiplication, C represents the channel dimension of the input feature map, and W represents the channel dimension. cThe channel weight enhanced by the dual attention mechanism is represented as W

[0033] Preferably, S2.2 comprises the following steps:

[0034] The channel weight is converted into a dynamic convolution weight μ, and a convolution operation is performed, and the constructed dual-attention dynamic convolution (DADC) module can be described as:

[0035] The channel weight enhanced by the dual attention mechanism is converted into a dynamic convolution weight:

[0036] μ = Softmax(W p ⊙ W c ),

[0037] In the above formula, W p is a linear transformation matrix, which realizes the transformation from the channel dimension to the convolution kernel dimension, and ⊙ represents matrix multiplication.

[0038] Then, based on the preset four convolution kernels, the final dynamic convolution kernel k is generated by dynamic weight weighting:

[0039]

[0040] In the above formula, μ i (i = 1, 2, 3, 4) is the dynamic weight of the four convolution kernels, and conv i (i = 1, 2, 3, 4) represents the convolution kernel.

[0041] Finally, the output F s of the spatial attention module is taken as the convolution object, and a dynamic convolution kernel k is used to perform a convolution operation on it to obtain the output F out of the DADC module:

[0042] F out = Conv(k(F s )),

[0043] Preferably, S3 comprises the following steps:

[0044] S3.1: Extracting edge information to generate boundary features, background features, and high-frequency features;

[0045] S3.2: Fusing attention mechanisms to construct an edge fusion attention (EFA) module.

[0046] Preferably, S3.1 comprises the following steps:

[0047] To enhance the perception ability of the network to edge information, the feature expression ability is further optimized by extracting edge information and generating multiple features. For a given input feature F inFirst, a predicted segmentation result P is generated. Then, P is processed to obtain the edge weights A. edge The boundary features F are obtained by multiplying the edge weights and input features. edge Then, the background weight A is obtained by calculating the back attention. bg The background features F are obtained by multiplying the background weights and input features. bg Finally, the high-frequency edge weights A are extracted using the Laplacian pyramid. hf The high-frequency edge weights and input features are multiplied by a dot product to obtain the high-frequency feature F. hf The overall generation process is as follows:

[0048] From input features F in Generate predicted segmentation result P:

[0049] P=σ(Conv 1×1 (F in )),

[0050] Extract boundary features F from the prediction result P. edge :

[0051] P filtered =ConvGauss(P),

[0052] P up =Upsample(Downsample(P filtered )),

[0053]

[0054]

[0055] The background feature F is obtained by calculating the background interest region of the prediction result P. bg :

[0056]

[0057]

[0058] High-frequency feature F is extracted using the Laplacian pyramid. hf :

[0059] A hf =Laplacian pyramid(F in ),

[0060]

[0061] In the above formula, σ(·) represents a Sigmoid function, Laplacian pyramid(·) represents a Laplacian pyramid method, represents an exclusive or operation, P represents a prediction result, ConvGauss(·) represents a Gaussian convolution, Downsample represents down-sampling, and Upsample represents up-sampling, represents an element-wise multiplication operation.

[0062] Preferably, S3.2 comprises the following steps:

[0063] The boundary feature F edge , the background feature F bg , and the high-frequency feature F hf are fused, and an attention mechanism is introduced to highlight the salient information of the target region, to construct an edge fusion attention (EFA) module, which is implemented as follows:

[0064] The three features are fused to obtain a fused feature F fusion :

[0065] F fusion = Conv(Concat(F edge , F bg , F hf )),

[0066] An attention weight A fusion of the fused feature is generated, and an enhanced feature F enhanced is obtained:

[0067] A fusion = σ(Conv(F fusion )),

[0068]

[0069] The enhanced fused feature and the residual input are added to obtain an intermediate feature F temp :

[0070]

[0071] Finally, the feature representation capability is further enhanced through channel attention and spatial attention, to obtain the final output F EFA of the EFA module:

[0072]

[0073] M s (F temp ) = σ(f 7×7 (Concat(AvgPool(Ftemp ), MaxPool(F temp )))),

[0074]

[0075]

[0076] In the above formula, Concat(·) represents concatenating features in the channel dimension, Conv(·) represents a convolution operation, represents an element-wise multiplication operation, represents an element-wise addition operation, σ represents a sigmoid activation function, f 7×7 represents a filter with a kernel size of 7, MLP represents a multi-layer perceptron, M c , M s respectively represent a channel attention layer and a spatial attention layer.

[0077] Preferably, S4 comprises the following steps:

[0078] S4.1: Calculate edge supervision loss;

[0079] S4.2: Calculate segmentation loss;

[0080] S4.3: Build a multi-scale edge segmentation loss.

[0081] Preferably, S4.1 comprises the following steps:

[0082] For feature maps of different scales, edge features are extracted by a Laplacian operator to guide the model to learn the boundary area more accurately. The edge supervision loss L ES is defined as follows:

[0083]

[0084]

[0085] In the above formula, represents the predicted edge feature, e represents the real edge feature, represents the segmentation prediction result, Laplace(·) represents a Laplacian operator operation, and N represents the total number of pixels.

[0086] Preferably, S4.2 comprises the following steps:

[0087] The segmentation loss combines the Dice loss and the cross-entropy loss. The segmentation loss L SEG is defined as follows:

[0088]

[0089]

[0090] L SEG = λ1L DICE + λ2L CE ,

[0091] where y i represents the real segmentation result, represents the predicted segmentation result, N represents the total number of pixels, and λ1 and λ2 are weight hyperparameters.

[0092] Preferably, S4.3 comprises the following steps:

[0093] By using multi-scale supervision, edge supervision loss L ES and segmentation loss L SEG are fused to obtain multi-scale edge segmentation loss L MES , as follows:

[0094]

[0095] where S is set to 4, α s represents the loss weight of the s-th scale, represents the edge supervision loss of the s-th scale, represents the segmentation loss of the s-th scale.

[0096] Preferably, S5 comprises the following steps:

[0097] Taking a U-shaped network as the basic architecture, the DADC module is first replaced with the second, third and fourth layers of convolutional layers in the encoder and all convolutional layers in the decoder of the U-shaped network, then the EFA module is introduced at the bottleneck layer of the U-shaped network, and finally the multi-scale edge segmentation loss function L MES optimizes the overall learning process of the model, thereby constructing the TAF-Net model.

[0098] Preferably, S6 comprises the following steps:

[0099] By using the divided experimental data set, the training and verification optimization of the model are completed, the optimal model weight is saved, and then the performance of the model is evaluated by using the test set, thereby obtaining various indicators of the model.

[0100] The method has the beneficial effects that: firstly, in the data preprocessing stage, through data format conversion, cropping, resampling, denoising and normalization operations on the collected abdominal CT image, the data quality is comprehensively improved, the interference of irrelevant information is effectively reduced, and the data is divided into a training set, a validation set and a test set in proportion, thereby laying a solid foundation for subsequent model training and testing. In the model construction process, the double attention mechanism is innovatively combined with the convolution operation to design a double attention dynamic convolution (DADC), which realizes dynamic weight distribution of feature information. In addition, an edge fusion attention (EFA) module is constructed by fusing edge information and attention mechanism. Further, for feature information at different scales, an edge supervision loss and a segmentation loss are fused to construct a multi-scale edge segmentation loss L MES . Finally, a TAF-Net model is constructed based on a U-shaped network architecture. Through multi-dimensional optimization design, the model exhibits excellent ability in capturing edge features and multi-scale information, significantly improves the accuracy of liver tumor segmentation, and effectively avoids the drag of feature redundancy and invalid features on the performance of the model by adaptively adjusting the convolution kernel weight. The overall design has high efficiency and robustness while accurately identifying the liver tumor region, and provides strong technical support for accurate liver tumor segmentation. BRIEF DESCRIPTION OF DRAWINGS

[0101] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings needed to be used in the prior art and embodiments, and the following drawings are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0102] Figure 1 is a flowchart of a liver tumor segmentation method fusing three kinds of attention according to the present application;

[0103] Figure 2 is a TAF-Net model architecture diagram of a liver tumor segmentation method fusing three kinds of attention according to the present application;

[0104] Figure 3 is a DADC structure diagram of a liver tumor segmentation method fusing three kinds of attention according to the present application;

[0105] Figure 4 is an EFA structure diagram of a liver tumor segmentation method fusing three kinds of attention according to the present application; DETAILED DESCRIPTION

[0106] In order to make the inventive purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be indicated that the following detailed description is exemplary and is intended to provide further description of the present application. Unless otherwise specified, all other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0107] The embodiment of the present application provides a liver tumor segmentation method fusing three attentions, which is used for realizing accurate segmentation of liver tumors, assisting clinical diagnosis and treatment decision, and improving the accuracy and efficiency of medical image processing.

[0108] With reference to Figure 1 The method comprises the following steps:

[0109] S1: preprocessing abdominal CT data, and dividing the abdominal CT data into a training set, a verification set and a test set in proportion;

[0110] S2: combining double attention and convolution operation to construct double attention dynamic convolution;

[0111] S3: fusing edge information and attention mechanism to construct an edge fusion attention module;

[0112] S4: fusing edge supervision loss and segmentation loss to construct a multi-scale edge segmentation loss;

[0113] S5: fusing double attention dynamic convolution, the edge fusion attention module and the multi-scale edge segmentation loss to construct a TripleAttentionFusionNetwork (TAF-Net) model;

[0114] S6: using an experimental data set to complete training, verification optimization and performance evaluation of the model.

[0115] Further, S1 comprises the following steps:

[0116] S1.1: preprocessing abdominal CT data, comprising data format conversion, cropping, resampling, denoising and normalization operation;

[0117] Further, the preprocessing of the abdominal CT data comprises data format conversion, cropping, resampling, denoising and normalization operation, which specifically comprises:

[0118] The collected abdominal CT data, including public data sets and data sets provided by cooperative hospitals, each case image completely covers the entire liver region, first convert the labeled abdominal CT data stored in nii format to npy matrix format, then crop the abdominal CT data to remove irrelevant areas, resample to 512x512 uniform size, then use Gaussian filtering to denoise the abdominal CT data, and normalize the pixel value of each data to a uniform range (0-1). The mask corresponding to the liver and background area is set to 0, and the mask corresponding to the liver tumor area is set to 1.

[0119] S1.2: Divide the preprocessed abdominal CT data into training set, validation set and test set according to 8:1:1.

[0120] Further, the step of dividing the preprocessed abdominal CT data into training set, validation set and test set according to 8:1:1 specifically includes:

[0121] The preprocessed abdominal CT data is divided according to the proportion of 8:1:1, and is respectively used as the training set, the validation set and the test set.

[0122] Further, referring to Figure 3 , S2 includes the following steps:

[0123] S2.1: Enhance features using spatial attention and self-attention mechanisms;

[0124] S2.2: Fuse convolution operations to build a dual-attention dynamic convolution (DADC) module.

[0125] Further, S2.1 includes the following steps:

[0126] First, the original image is convolved to obtain the input feature map F, and 1x1 convolution is used to reduce the dimension of F, and then the sigmoid activation function is used to normalize the output tensor to obtain the spatial attention weight A s (F), finally, the obtained spatial attention weight is multiplied by the input feature map pixel by pixel to obtain the output feature map F s . The calculation process is as follows:

[0127] A s (F)=σ(Conv 1×1 (F)),

[0128]

[0129] In the above formula, F is the input feature map, A s (F) is the spatial attention weight, represents an element-wise multiplication operation, σ represents a sigmoid activation function, F s is the output feature map.

[0130] The output feature map F s of the spatial attention module is input into a self-attention mechanism to capture global dependencies and interaction relationships between channels, and channel attention weights A c are obtained. c The final channel weights W c enhanced by the spatial attention module and the self-attention mechanism are obtained. The calculation process is as follows:

[0131] F s ,

[0132] Q = W Q ⊙F c ,

[0133] K = W K ⊙F c ,

[0134] V = W V ⊙F c ,

[0135]

[0136] W c = A c ⊙V,

[0137] In the above formula, F c is the channel feature after global average pooling (AvgPool), W Q , W K , and W V are learnable weight matrices, Q, K, and V represent the query, key, and value in the channel dimension, A c represents the weight in the channel dimension, ⊙ represents matrix multiplication, C represents the channel dimension of the input feature map, and W c represents the channel weight enhanced by the double-attention mechanism.

[0138] Further, S2.2 includes the following steps:

[0139] The channel weights are converted into dynamic convolution weights μ, and convolution operation is performed. The constructed double-attention dynamic convolution (DADC) module can be described as:

[0140] The channel weights enhanced by the double-attention are converted into dynamic convolution weights:

[0141] μ = Softmax(W p ⊙Wc ),

[0142] In the above formula, W p is a linear transformation matrix, which realizes the transformation from the channel dimension to the convolution kernel dimension, and represents matrix multiplication.

[0143] Then, based on the preset four convolution kernels, the final dynamic convolution kernel k is generated by dynamic weight weighting:

[0144]

[0145] In the above formula, μ i (i=1, 2, 3, 4) is the dynamic weight of the four convolution kernels, and conv i (i=1, 2, 3, 4) represents the convolution kernel.

[0146] Finally, the output F s of the spatial attention module is taken as the convolution object, and a dynamic convolution kernel k is used for convolution operation to obtain the output F out of the DADC module:

[0147] F out =Conv(k(F s )),

[0148] Further, referring to Figure 4 , S3 includes the following steps:

[0149] S3.1: Extracting edge information, generating boundary features, background features, and high-frequency features;

[0150] S3.2: Fusing attention mechanism, constructing edge fusion attention Edge-Fusion Attention (EFA) module.

[0151] Further, S3.1 includes the following steps:

[0152] In order to enhance the perception ability of the network to edge information, the feature expression ability is further optimized by extracting edge information and generating multiple features. For a given input feature F in , first, a prediction segmentation result P is generated. Then, P is processed and edge weight A edge is obtained. The edge weight and the input feature are calculated by point multiplication to obtain the boundary feature F edge ; Then, the background weight A bg is obtained by calculating the reverse attention, and the background weight and the input feature are calculated by point multiplication to obtain the background feature F bg ; Finally, the high-frequency edge weight A hf is extracted by Laplacian pyramid, and the high-frequency edge weight and the input feature are calculated by point multiplication to obtain the high-frequency feature F hf. The overall generation procedure is shown as follows:

[0153] The input feature F in is generated by:

[0154] P = σ(Conv 1×1 (F in )),

[0155] The boundary feature F edge is extracted from the prediction result P:

[0156] P filtered = ConvGauss(P),

[0157] P up = Upsample(Downsample(P filtered )),

[0158]

[0159]

[0160] The background feature F bg is obtained by calculating the background attention region of the prediction result P:

[0161]

[0162]

[0163] The high-frequency feature F hf is extracted by Laplacian pyramid:

[0164] A hf = Laplacian pyramid(F in ),

[0165]

[0166] In the above formula, σ(·) represents the Sigmoid function, Laplacian pyramid(·) represents the Laplacian pyramid method, ^ represents the XOR operation, P represents the prediction result, ConvGauss(·) represents the Gaussian convolution, Downsample represents the down-sampling, Upsample represents the up-sampling, and represents the element-wise multiplication operation.

[0167] Further, S3.2 includes the following steps:

[0168] The boundary feature F edge , the background feature F bg , and the high-frequency feature Fhf The three features are fused, and an attention mechanism is introduced to highlight the significant information of the target area, to construct an edge fusion attention Edge-FusionAttention (EFA) module, which is implemented as follows:

[0169] The three features are fused to obtain the fused feature F fusion :

[0170] F fusion = Conv(Concat(F edge ,F bg ,F hf )),

[0171] The attention weight A fusion of the generated fused feature is obtained, and the enhanced feature F enhanced :

[0172] A fusion = σ(Conv(F fusion )),

[0173]

[0174] The enhanced fused feature is added to the residual input to obtain the intermediate feature F temp :

[0175]

[0176] Finally, the channel attention and spatial attention are used to further enhance the feature representation capability, and the final output F EFA of the EFA module is obtained:

[0177] M c (F temp ) = σ(MLP(AvgPool(F temp )) ⊕ MLP(MaxPool(F temp ))),

[0178] M s (F temp ) = σ(f 7×7 (Concat(AvgPool(F temp ), MaxPool(F temp )))),

[0179]

[0180]

[0181] In the above formula, Concat(·) represents concatenating features in the channel dimension, Conv(·) represents convolution operation, represents an element-wise multiplication operation, represents an element-wise addition operation, σ represents a sigmoid activation function, f 7×7 represents a filter with a kernel size of 7, MLP represents a multi-layer perception, M c , M s respectively represent a channel attention layer and a spatial attention layer.

[0182] Further, S4 comprises the following steps:

[0183] S4.1: Calculate edge supervision loss;

[0184] S4.2: Calculate segmentation loss;

[0185] S4.3: Construct a multi-scale edge segmentation loss.

[0186] Further, S4.1 comprises the following steps:

[0187] For feature maps of different scales, edge features are extracted by a Laplacian operator to guide the model to learn the boundary area more accurately. The edge supervision loss L ES is defined as follows:

[0188]

[0189]

[0190] In the above formula, represents predicted edge features, e represents real edge features, represents segmentation prediction results, Laplace(·) represents a Laplacian operator operation, and N represents the total number of pixels.

[0191] Further, S4.2 comprises the following steps:

[0192] The segmentation loss combines Dice loss and cross-entropy loss. The segmentation loss L SEG is defined as follows:

[0193]

[0194]

[0195] L SEG = λ1L DICE + λ2L CE ,

[0196] where y i represents real segmentation results, represents predicted segmentation results, N represents the total number of pixels, and λ1 and λ2 are weight hyperparameters.

[0197] Further, S4.3 comprises the following steps:

[0198] Using multi-scale supervision, the edge supervision loss L ES and the segmentation loss L SEG are fused to obtain a multi-scale edge segmentation loss L MES as follows:

[0199]

[0200] wherein S is set to 4, and α s represents the loss weight of the s-th scale, represents the edge supervision loss of the s-th scale, represents the segmentation loss of the s-th scale.

[0201] Further, referring to Figure 2 , S5 comprises the following steps:

[0202] Taking a U-shaped network as the basic architecture, the DADC module is first replaced with the second, third and fourth layers of convolutional layers in the encoder and all convolutional layers in the decoder of the U-shaped network, then the EFA module is introduced at the bottleneck layer of the U-shaped network, and finally the multi-scale edge segmentation loss function L MES optimizes the overall learning process of the model, thereby constructing the TAF-Net model.

[0203] Further, S6 comprises the following steps:

[0204] Using the divided experimental data set, the training and verification optimization of the model are completed, the optimal model weight is saved, then the performance of the model is evaluated using the test set, and the indicators of the model are obtained.

[0205] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make other equivalent modifications or replacements without departing from the spirit of the application. These equivalent modifications or replacements are included in the scope defined by the claims.

Claims

1. A liver tumor segmentation method fusing three attentions, characterized in that, The method comprises the following steps: S1: preprocessing the abdominal CT data, and proportionally dividing into a training set, a verification set and a test set; S2: combining double attention and convolution operation to construct double attention dynamic convolution; S3: fusing edge information and attention mechanism to construct edge fusion attention module; S3 comprises the following steps: S3.1: For a given input feature , first generate a prediction segmentation result P ; then process P and get edge weights , the edge weights and the input features are calculated by point multiplication to get the boundary features ; then get the background weight by calculating the reverse attention, the background weight and the input features are calculated by point multiplication to get the background features ; finally, extract the high-frequency edge weight by Laplacian pyramid , the high-frequency edge weight and the input features are calculated by point multiplication to get the high-frequency features ; S3.2: Boundary features , background features and high-frequency features The three features are fused, and the attention mechanism is introduced to highlight the significant information of the target area to construct the edge fusion attention EFA module; S4: fusing edge supervision loss and segmentation loss to construct multi-scale edge segmentation loss; S5: fusing double attention dynamic convolution, edge fusion attention module and multi-scale edge segmentation loss to construct TAF-Net model; S6: using experimental data set to complete model training, verification optimization and performance evaluation.

2. The liver tumor segmentation method of claim 1, wherein, S1 comprises the following steps: S1.1: preprocessing the abdominal CT data, including data format conversion, cropping, resampling, denoising and normalization operation; S1.2: dividing the preprocessed abdominal CT data into a training set, a verification set and a test set according to 8:1:

1. 3.The liver tumor segmentation method of fusing three attentions according to claim 1, characterized in that, S2 comprises the following steps: S2.1: enhancing features by using spatial attention and self-attention mechanism; S2.2: fusing convolution operation to construct double attention dynamic convolution DADC module; S2.1 comprises the following steps: First, the original image is subjected to a convolution operation to obtain an input feature map , and then , the output tensor is normalized using a sigmoid activation function, obtaining spatial attention weights , and finally, the obtained spatial attention weights are multiplied pixel by pixel with the input feature map to obtain the output feature map ; the calculation process is as follows: , , In the above formula, is an input feature map, is a spatial attention weight, denotes an element-wise multiplication operation, denotes a sigmoid activation function, is an output feature map; The output feature map of the spatial attention module , the global dependency and interaction between channels are captured by using a self-attention mechanism to obtain channel attention weights , and finally the channel weights enhanced by the spatial attention module and the self-attention mechanism are obtained ; the calculation process is as follows: , , , , , , In the above formula, is a global average pooling (AvgPool ) the channel feature after is a learnable weight matrix, Q, K, V respectively represent the query, key and value of the channel dimension, represents the weight on the channel dimension, represents matrix multiplication, C represents the channel dimension of the input feature map, represents the channel weight enhanced by the double attention mechanism; S2.2 comprises the following steps: Converting channel weights to dynamic convolution weights and performing a convolution operation, the constructed dual-attention dynamic convolution (DADC) module can be described as: Convert the channel weight enhanced by double attention into dynamic convolution weight: , In the above formula, is a linear transformation matrix that implements the transformation of the channel dimension to the convolution kernel dimension, denotes a matrix multiplication; Then based on the preset four convolution kernels, a final dynamic convolution kernel is generated by dynamic weight weighting : , In the above formula, i are dynamic weights of the 4 convolution kernels, i denote the convolution kernels;​​ Finally, the output of the spatial attention module is added to the output of the DADC module As a convolution object, a dynamic convolution kernel is applied k A convolution operation is performed thereon to obtain the output of the DADC module : 。 4. The liver tumor segmentation method of claim 1, wherein, S3 comprises the following steps: S3.1 comprises the following steps: from the input features generating a predicted segmentation result P : , extracting boundary features from prediction results P :​ , , , , By calculating the prediction result P The background feature is obtained from the background region of interest : , , extracting high frequency features by laplacian pyramid : , , In the above formula denotes a Sigmoid function, denotes a Laplacian pyramid method, denotes an XOR operation, P denotes a prediction result, denotes a Gaussian convolution, denotes a down-sampling, denotes an up-sampling, denotes an element-wise multiplication operation; S3.2 comprises the following steps: The three features are fused to obtain fused features : , Generating attention weights for fused features and obtaining enhanced features : , , The enhanced fusion feature and the residual input are added to obtain an intermediate feature : , Finally, the channel attention and spatial attention are used to further enhance the feature representation ability, and the final output of the EFA module is obtained : , , , , In the above formulae, denotes concatenating features in the channel dimension, denotes a convolution operation, denotes an element-wise multiplication operation, denotes an element-wise addition operation, denotes a sigmoid activation function, denotes max-pooling, denotes average-pooling, denotes a filter with a kernel size of 7, MLP denotes a multi-layer perceptron, , denote a channel attention layer and a spatial attention layer, respectively.

5. The liver tumor segmentation method of claim 1, wherein, S4 comprises the following steps: S4.1: calculating edge supervision loss; S4.2: calculating segmentation loss; S4.3: constructing multi-scale edge segmentation loss; S4.1 comprises the following steps: For different scale feature maps, edge features are extracted by Laplacian operator , guiding the model to learn the boundary region more accurately; edge supervision loss is defined as follows: , , In the above formulae, denotes a predicted edge feature, denotes a real edge feature, denotes a segmentation prediction, denotes a Laplacian operation, denotes the total number of pixels; S4.2 comprises the following steps: The segmentation loss combines the loss and the cross-entropy loss; the segmentation loss is defined as follows: , , , In the above formula, denotes the real segmentation result, denotes the predicted segmentation result, denotes the total number of pixels, and is a weight hyperparameter; S4.3 comprises the following steps: Applying multi-scale supervision, fusing edge supervision loss and segmentation loss , obtaining multi-scale edge segmentation loss as follows: , ( s =1,2,3,4) wherein, S is set to 4, denotes the loss weight for the s scale, denotes the edge supervision loss for the s scale, denotes the segmentation loss for the s scale.

6. The liver tumor segmentation method of claim 1, wherein, S5 comprises the following steps: Taking the U-shaped network as the basic framework, firstly, the DADC module is respectively replaced with the second, third and fourth layers of the encoder convolution layer and all the decoder convolution layers in the U-shaped network, then the EFA module is introduced into the bottleneck layer of the U-shaped network, and finally the multi-scale edge segmentation loss function is combined The overall learning process of the optimization model is optimized, thereby constructing the TAF-Net model.

7. The liver tumor segmentation method of claim 1, wherein, S6 comprises the following steps: Using the divided experimental data set, the model is trained and verified and optimized, the optimal model weight is saved, then the model is evaluated in performance by using the test set, and various indexes of the model are obtained.

Images