A lightweight image segmentation method, device and storage medium

Abstract

The present invention provides an image segmentation method, device and storage medium. The method comprises: obtaining first image data; segmenting the first image data through a hole gated attention model to obtain a segmentation result, wherein the hole gated attention model comprises two parts: an encoding part and a decoding part; wherein the encoding part comprises a common encoding unit and an attention encoding unit; the decoding part comprises an attention decoding unit and a common decoding unit; the common encoding unit, the attention encoding unit, the attention decoding unit and the common decoding unit are connected in sequence. By segmenting the first image data through the hole gated attention model, the amount of parameters and the amount of calculation in image segmentation are greatly reduced, and a better segmentation result is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a lightweight image segmentation method, device and storage medium. Background Art

[0002] Medical image segmentation is the key to medical imaging analysis. A good medical image segmentation model can accurately locate the lesion area in the image, assist doctors in pathological judgment, and better perform subsequent diagnosis and treatment. However, the computing power of medical equipment currently used in clinics is far behind that of laboratory computing equipment. This has led to the fact that the medical image segmentation model that can run well on laboratory computing equipment is limited in its application on actual clinical equipment.

[0003] Recently, the UNet model has become one of the most important models in the field of medical image segmentation. Its basic idea is to reduce the width and height of the image through the encoder, increase the number of channels of the image, and then restore the image through the decoder until the resolution is consistent with the input image, using cropping and splicing as jump connections in the middle. However, the model has a large number of parameters and calculations, which will limit its use on mobile terminals. Therefore, how to lightweight the medical image segmentation model and improve the segmentation performance of the model while reducing the number of model parameters and calculations has become a research topic. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a lightweight image segmentation method, device and storage medium.

[0005] The present invention is achieved through the following technical solutions:

[0006] The embodiment of the present invention provides a lightweight image segmentation method based on hole gated attention, including:

[0007] Acquiring first image data;

[0008] The first image data is segmented by a hole gated attention model to obtain a segmentation result, wherein the hole gated attention model includes two parts: an encoding part and a decoding part; wherein,

[0009] The encoding part includes a common encoding unit and an attention encoding unit;

[0010] The decoding part includes an attention decoding unit and a common decoding unit;

[0011] The ordinary encoding unit, the attention encoding unit, the attention decoding unit and the ordinary decoding unit are connected in sequence.

[0012] Furthermore, before the first image data is segmented by the hole gated attention model to obtain the segmentation result, the method further includes:

[0013] Using the second image data to train the first hole gated attention model, so as to train the two-dimensional convolution and batch normalization in the common encoding unit and the common decoding unit, and respectively train the hole gated attention and batch normalization in the attention encoding unit and the attention decoding unit;

[0014] The trained hole gated attention model is used as the hole gated attention model for training the first image data.

[0015] Furthermore, the common coding unit in the coding part includes:

[0016] The first image data input is denoted as t 0 , perform three two-dimensional convolutions with a kernel size of 3, batch normalization, double maximum pooling, and GELU activation function operations on it in sequence, and record the feature map output of the above three times as t 1 , t 2 , t 3 , then t 3 As the input of the attention encoding unit.

[0017] Furthermore, the attention encoding unit in the encoding part includes:

[0018] t 3 As input, perform two operations of hole gated attention, batch normalization, double maximum pooling and GELU activation function in sequence, and record the output of the above two feature maps as t 4 , t 5 , then t 5 Perform another hole gated attention and GELU activation function operation to get the output result Out of the encoding part 6 .

[0019] Furthermore, the attention decoding unit in the decoding part includes:

[0020] Out 6 and t 3 , t 4 , t 5 As input, to Out 6 Perform hole gated attention, batch normalization, and GELU activation function operations, and then add the results of the operations to t 5 Perform element-by-element addition to obtain the feature map Out 5 ;

[0021] For the feature map Out 5 Perform hole gated attention, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add the results of the operations to t 4 Perform element-by-element addition to obtain the feature map Out 4 ;

[0022] For the feature map Out 4 Perform hole gated attention, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add t 3 Perform element-by-element addition to obtain the feature map Out 3 , as output.

[0023] Furthermore, the common decoding unit in the decoding part includes:

[0024] The feature map Out 3 and t 1 , t 2 As input;

[0025] For the feature map Out 3 Perform two-dimensional convolution with a convolution kernel size of 3, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add t 2 Perform element-by-element addition to obtain the feature map Out 2 ;

[0026] For the feature map Out 2 Perform two-dimensional convolution with a convolution kernel size of 3, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add t 1 Perform element-by-element addition to obtain the feature map Out 1 ;

[0027] For the feature map Out 1 Perform a two-dimensional convolution with a convolution kernel size of 1 and a two-fold linear interpolation upsampling operation to restore the feature map size to the same as the data label and obtain the feature map Out.

[0028] Furthermore, the hole-gated attention in the attention encoding unit and the attention decoding unit includes:

[0029] The hole gated attention includes a split hole convolution unit and a gated attention unit;

[0030] The split hole convolution unit includes: i Split in the channel dimension and divide it into four equal parts to obtain the feature map after splitting Then, through the deep convolution with a convolution kernel size of 3 but different void ratios, the global feature information and local feature information are obtained, and the feature map is obtained. The global feature information includes feature information obtained by deep convolution with dilation rates of 5 and 7, and the local feature information includes feature information obtained by deep convolution with dilation rates of 1 and 2;

[0031] In the channel dimension Perform a concatenation operation to restore the size of the feature map, followed by a two-dimensional convolution operation with a convolution kernel size of 1 to obtain the feature map x′ i , so that global feature information and local feature information can interact;

[0032] Transmitted to the gated attention unit to input feature map x′ through two depth-separable convolutions with a convolution kernel size of 3 i One of them is followed by a Sigmoid activation function to generate an attention map Att, and the other is element-wise multiplied with the generated attention map Att, followed by a depth-wise separable convolution operation with a convolution kernel size of 3 to obtain a feature map Next, input feature map x′ i Perform a two-dimensional convolution operation with a convolution kernel size of 1 to obtain the feature map Finally, the feature map and Perform element-by-element addition to obtain the output feature map x o .

[0033] Furthermore, after obtaining the segmentation result, the method further includes:

[0034] Use the Sigmoid activation function to activate the feature map Out to obtain the final prediction result.

[0035] The embodiment of the present invention further provides a lightweight image segmentation device based on hole gated attention, comprising:

[0036] Acquisition module: used for acquiring first image data;

[0037] A processing module is used to segment the first image data through a hole gated attention model to obtain a segmentation result, wherein the hole gated attention model includes two parts: an encoding part and a decoding part; wherein the encoding part includes a common encoding unit and an attention encoding unit; the decoding part includes an attention decoding unit and a common decoding unit; the common encoding unit, the attention encoding unit, the attention decoding unit and the common decoding unit are connected in sequence.

[0038] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image segmentation method based on hole gated attention as described in any one of the above items is implemented.

[0039] Compared with the prior art, the present invention has the following beneficial technical effects:

[0040] The present invention provides an image segmentation method based on hole gated attention, which obtains first image data; the first image data is segmented and processed through a hole gated attention model to obtain a segmentation result, wherein the hole gated attention model includes two parts: an encoding part and a decoding part; wherein the encoding part includes a common encoding unit and an attention encoding unit; the decoding part includes an attention decoding unit and a common decoding unit; the common encoding unit, the attention encoding unit, the attention decoding unit and the common decoding unit are connected in sequence. The first image data is segmented through the hole gated attention model, which greatly reduces the amount of parameters and calculations in image segmentation and obtains a better segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A block diagram of a lightweight image segmentation method based on hole gated attention provided by an embodiment of the present invention;

[0042] Figure 2 A structural diagram of the DGA-UNet model provided in an embodiment of the present invention;

[0043] Figure 3 A structural diagram of a split hole convolution unit provided in an embodiment of the present invention;

[0044] Figure 4 A structural diagram of a gated attention unit provided in an embodiment of the present invention;

[0045] Figure 5 A structural diagram of a hole gated attention module provided in an embodiment of the present invention;

[0046] Figure 6 A schematic diagram of the results of segmenting skin lesions and melanoma lesion images using different methods provided in an embodiment of the present invention;

[0047] Figure 7 An image segmentation device based on hole gated attention is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below in conjunction with specific embodiments, which are intended to explain the present invention rather than to limit it.

[0049] In order to better understand the present invention, the technical solution of the present invention will be fully and clearly described below in conjunction with the accompanying drawings and specific embodiments. Of course, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0050] The embodiment of the present invention provides a lightweight medical image segmentation method based on hole gated attention, such as Figure 1 As shown, including:

[0051] S1: Acquire first image data;

[0052] In this embodiment, the first image data may also be original image data, or may be obtained by dividing an image data set. For example, before acquiring the first image data, the image data set is divided to obtain first image data and second image data. The first image data may be a test set, and the second image data may be a training set.

[0053] For example, the image dataset may be an ISIC18 dataset, i.e., skin lesion and melanoma lesion image data, hereinafter referred to as ISIC18. In the ISIC18 dataset, the official provides 2594 training set images and 100 test set images. Since the test samples are less than the training set, in this embodiment, all 2694 images are shuffled, and then the dataset is divided at a ratio of 7:3, to obtain 1886 images as the training set of this embodiment, and 808 images as the test set of this embodiment.

[0054] S2: Segment the first image data through a hole gated attention model to obtain a segmentation result.

[0055] The hole gated attention model in this embodiment may include two parts: an encoding part and a decoding part; wherein,

[0056] The encoding part includes a common encoding unit and an attention encoding unit;

[0057] The decoding part includes an attention decoding unit and a common decoding unit;

[0058] The ordinary encoding unit, the attention encoding unit, the attention decoding unit and the ordinary decoding unit are connected in sequence.

[0059] Specifically, different data preprocessing operations are performed on the divided training set and test set;

[0060] On the training set, the hole gated attention model proposed in the present invention, i.e., the DGA-UNet lightweight model, is trained to train the two-dimensional convolution and batch normalization in the ordinary encoding unit and the ordinary decoding unit, and the hole gated attention and batch normalization in the attention encoding unit and the attention decoding unit are trained respectively to obtain training weights;

[0061] The training weights are further used to predict the segmentation results of the test set image data.

[0062] In this embodiment, the original medical image data described in S1 may refer to ISIC2018 skin lesion and melanoma lesion image data, hereinafter referred to as ISIC18.

[0063] In this embodiment, the division of the data set in S2 is performed on the ISIC18 data set; in the ISIC18 data set, the official provides 2594 training set images and 100 test set images; since the test samples are less than the training set, in this embodiment, all 2694 images are shuffled and then the data set is divided at a ratio of 7:3, obtaining 1886 images as the training set of this embodiment and 808 images as the test set of this embodiment.

[0064] In this embodiment, the data preprocessing operation described in S3 is different for the training set and the test set. The preprocessing steps for the training set images are as follows: (1) normalize the input image; (2) randomly flip horizontally with a probability of 50%; (3) randomly flip vertically with a probability of 50%; (4) randomly rotate with a probability of 50%, and the rotation angle range is (0, 360); (5) scale the image size to 256×256.

[0065] For the test set images, no data augmentation operations (2) to (4) are performed, only normalization and scaling are performed.

[0066] In this embodiment, the DGA-UNet lightweight model described in S4 is combined with Figure 2-5 ,The structure of the DGA-UNet lightweight model includes two parts: an encoding part and a decoding part;

[0067] The encoding part includes a common encoding unit and an attention encoding unit; the decoding part includes an attention decoding unit and a common decoding unit;

[0068] The ordinary encoding unit and the attention encoding unit as well as the attention decoding unit and the ordinary decoding unit are connected in sequence to perform medical image segmentation prediction.

[0069] The present invention provides another embodiment, which is Figure 1 On the basis of the embodiment shown, the first image data is segmented by a hole gated attention model, and before obtaining the segmentation result, the method further includes:

[0070] Using the second image data to train the first hole gated attention model, so as to train the two-dimensional convolution and batch normalization in the common encoding unit and the common decoding unit, and respectively train the hole gated attention and batch normalization in the attention encoding unit and the attention decoding unit;

[0071] The trained hole gated attention model is used as the hole gated attention model for testing the first image data.

[0072] Specifically, the common coding unit in the coding part, specifically, records the input medical image as t 0 , perform three two-dimensional convolutions with a kernel size of 3, batch normalization, double maximum pooling, and GELU activation function operations on it in sequence, and record the feature map output of the above three times as t 1 , t 2 , t 3 , then t 3 As the input of the subsequent attention encoding unit; it is expressed by the formula as follows:

[0073] t i+1 =GELU(MaxPool(BN(Conv2d 3*3 (t i ))))

[0074] Among them, Conv2d 3*3 represents a two-dimensional convolution operation with a convolution kernel size of 3, BN represents a batch normalization operation, MaxPool represents a double maximum pooling, GELU is the activation function, and here, i takes 0, 1, and 2.

[0075] Specifically, the attention encoding unit in the encoding part converts t 3 As input, perform two operations of hole gated attention, batch normalization, double maximum pooling and GELU activation function in sequence, and record the output of the above two feature maps as t 4 , t 5 , then t 5 Perform another hole gated attention and GELU activation function operation to get the output of the encoding part, denoted as Out 6 ; It is expressed by the formula as follows:

[0076] t i+1 =GELU(MaxPool(BN(DGA(ti ))))

[0077] Out 6 =GELU(DGA(t 5 ))

[0078] Among them, DGA represents the void gated attention module, BN represents the batch normalization operation, MaxPool represents the double maximum pooling, GELU is the activation function, and here, i is 3, 4.

[0079] The attention decoding unit in the decoding part, specifically, Out 6 and t 3 , t 4 , t 5 As input; first, Out 6 Perform hole gated attention, batch normalization, and GELU activation function operations, and then add t 5 Perform element-by-element addition to obtain the feature map Out 5 ; For feature map Out 5 Perform hole gated attention, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add t 4 Perform element-by-element addition to obtain the feature map Out 4 ; For feature map Out 4 Perform hole gated attention, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add t 3 Perform element-by-element addition to obtain the feature map Out 3 , as the output; expressed by the formula as follows:

[0080]

[0081]

[0082] Among them, DGA represents the hole gated attention module, BN represents the batch normalization operation, Up represents the two-fold linear interpolation upsampling, GELU is the activation function, It represents the element-by-element addition operation, and here .i takes the values ​​of 3 and 4.

[0083] The ordinary decoding unit in the decoding part, specifically, converts the feature map Out 3 and t 1 , t 2 As input; for feature map Out 3 Perform two-dimensional convolution with a convolution kernel size of 3, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add t 2 Perform element-by-element addition to obtain the feature map Out2 ; For feature map Out 2 Perform two-dimensional convolution with a convolution kernel size of 3, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations, and then add t 1 Perform element-by-element addition to obtain the feature map Out 1 ; For feature map Out 1 Perform a two-dimensional convolution with a convolution kernel size of 1 and a two-fold linear interpolation upsampling operation to restore the feature map size to the same as the data label to obtain the feature map Out; finally, use the Sigmoid activation function to activate the feature map Out to obtain the final prediction result; it is expressed by the following formula:

[0084]

[0085] Out = σ(Up(Conv2d 1*1 (Out 1 )))

[0086] Among them, Conv2d 3*3 represents a two-dimensional convolution operation with a convolution kernel size of 3, BN represents a batch normalization operation, Up represents a two-fold linear interpolation upsampling, and GELU is the activation function. Represents element-by-element addition operation, Conv2d 1*1 represents a two-dimensional convolution operation with a convolution kernel size of 1, σ represents the Sigmoid activation function, and here, i takes 1, 2.

[0087] The hole gated attention module in the attention encoding unit and the attention decoding unit is composed of two sub-units, namely the split hole convolution unit and the gated attention unit. The first is the split hole convolution unit, which performs the input feature map x i Split in the channel dimension and divide it into four equal parts to obtain the feature map after splitting Then, through the depth convolution with kernel size of 3 but different void ratios, the global (obtained through the depth convolution with void ratios of 5 and 7) and local (obtained through the depth convolution with void ratios of 1 and 2) feature information are obtained, thus obtaining the feature map Afterwards, in the channel dimension Perform a concatenation operation to restore the size of the feature map, followed by a two-dimensional convolution operation with a convolution kernel size of 1 to obtain the feature map x′ i , this operation enables global information and local information to interact; next, enter the gated attention unit, and perform two depth-separable convolutions with a kernel size of 3 on the input feature map x′ iOne of them is followed by a Sigmoid activation function to generate an attention map Att, and the other is element-wise multiplied with the generated attention map Att, followed by a depth-wise separable convolution operation with a convolution kernel size of 3 to obtain a feature map This operation can suppress the unimportant parts of the feature information and make the model focus on the key information. Then, the input feature map x′ i Perform a two-dimensional convolution operation with a convolution kernel size of 1 to obtain the feature map Finally, the feature map and Perform element-by-element addition to obtain the output feature map x o ; It is expressed by the formula as follows:

[0088]

[0089]

[0090]

[0091] Att=σ(DW 3*3 (x′ i ))

[0092]

[0093]

[0094]

[0095] Among them, Chunk 4 Indicates that the input feature map is divided into four parts in the channel dimension, W d＝i represents a deep convolution operation with a void rate of i and a convolution kernel size of 3. Concat represents concatenating feature maps in the channel dimension. Conv2d 1*1 Denotes a two-dimensional convolution operation with a convolution kernel size of 1, DW 3*3 represents a depth-wise separable convolution operation with a kernel size of 3, σ represents the Sigmoid activation function, and ⊙ represents an element-by-element multiplication operation. Represents an element-wise addition operation.

[0096] By training the DGA-UNet lightweight model proposed in the present invention on the training set in S4, the training weights after the training are obtained.

[0097] During the model training process, the optimizer used was AdamW optimizer, the learning rate adjustment strategy was cosine annealing, the number of training epochs was 300, the batch-size was 8, the initial learning rate was 0.001, and all experiments were performed on a single NVIDIA2080Ti card. The Pytorch architecture was used to build the model, and the Sklearn architecture was used to evaluate the model performance. The model with the best performance on the test set was saved as the final model weight.

[0098] The loss function expression used in the model training process is as follows:

[0099]

[0100]

[0101] L BceDice =λ 1 L Bce +λ 2 L Dice

[0102] Among them, N is the total number of samples, y i is the true label, p i is the model prediction value; |X| and |Y| represent the true mask and the prediction value respectively; λ 1 and λ 2 They represent weighted weights respectively. In this embodiment, both weights are assumed to be 1 by default.

[0103] The trained model is tested. The evaluation indicators used in the test are: F1 score (DSC), accuracy (Acc), sensitivity (Sen), specificity (Spe) and mean intersection over union (mIoU), which are expressed as follows:

[0104]

[0105]

[0106]

[0107]

[0108]

[0109] Among them, TP, TN, FP and FN represent true positive, true negative, false positive and false negative, respectively.

[0110] The test results on the ISIC18 dataset are shown in Table 1.

[0111] Table 1 Segmentation results on the ISIC18 test set

[0112]

[0113] From the results in Table 1, it can be seen that the DGA-UNet lightweight model of the present invention replaces part of the convolution operation with the hole gated attention module proposed in the present invention. Compared with the UNet model, the mIoU index is improved by 1.26%, the DSC index is improved by 0.88%, the Acc index is improved by 0.28%, and the Sen index is improved by 2.39%, so that the segmentation performance of the model is comprehensively improved. At the same time, compared with the UNet model, the DGA-UNet lightweight model of the present invention is reduced by 97 times in the number of parameters and 230 times in the amount of calculation, which greatly reduces the number of parameters and the amount of calculation of the model, making it possible for the model to perform medical image segmentation in actual clinical scenarios.

[0114] In this embodiment, by Figure 6 The visualization of the test samples shows more clearly that the DGA-UNet lightweight model proposed in the present invention can provide more complete and accurate segmentation results.

[0115] The present invention provides an image segmentation method based on hole gated attention. In the present invention, we propose a hole gated attention module and a DGA-UNet lightweight model of the present invention. Compared with the prior art, our method can greatly reduce the number of parameters and the amount of calculation of the model, and can also improve the performance of the model in medical image segmentation tasks.

[0116] Figure 7 The image segmentation device based on hole gated attention provided by the embodiment of the present invention is as follows: Figure 7 As shown, the embodiment of the present invention further provides an image segmentation device based on hole gated attention, the device comprising: an acquisition module 71 and a processing module 72;

[0117] Acquisition module 71: used to acquire first image data;

[0118] The processing module 72 is used to segment the first image data through a hole gated attention model to obtain a segmentation result, wherein the hole gated attention model includes two parts: an encoding part and a decoding part; wherein the encoding part includes a common encoding unit and an attention encoding unit; the decoding part includes an attention decoding unit and a common decoding unit; the common encoding unit, the attention encoding unit, the attention decoding unit and the common decoding unit are connected in sequence.

[0119] The implementation method and technical effect of the image segmentation device based on hole gated attention are similar to the above Figure 1-Figure 6 The implementation method and technical effect of the illustrated embodiment are similar and will not be described in detail here.

[0120] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image segmentation method based on hole gated attention as described above is implemented.

[0121] The above is a detailed description of the implementation method of the present invention, but the present invention is not limited to the above implementation method. For those skilled in the art, without departing from the principles and methods of the present invention, different changes, substitutions, modifications or deformations of these embodiments are still within the scope of protection of the present invention.

Claims

1. A lightweight image segmentation method based on hole gated attention, characterized in that: include: Acquiring first image data; The first image data is segmented by a hole gated attention model to obtain a segmentation result, wherein the hole gated attention model includes two parts: an encoding part and a decoding part; wherein, The encoding part includes a common encoding unit and an attention encoding unit; The decoding part includes an attention decoding unit and a common decoding unit; The common encoding unit, the attention encoding unit, the attention decoding unit and the common decoding unit are connected in sequence; The common coding unit in the coding part includes: The first input image data is recorded as t0, and three two-dimensional convolutions with a convolution kernel size of 3, batch normalization, double maximum pooling and GELU activation function operations are performed on it in sequence, and the feature map outputs of the above three times are recorded as t1, t2, and t3, and then t3 is used as the input of the attention encoding unit; The attention encoding unit in the encoding part includes: Take t3 as input, perform two operations of hole gated attention, batch normalization, double maximum pooling and GELU activation function in sequence, record the above two feature map outputs as t4, t5, then perform hole gated attention and GELU activation function operations on t5 again, and get the output result Out6 of the encoding part; The attention decoding unit in the decoding part includes: Take Out6 and t3, t4, and t5 as input, perform hole gated attention, batch normalization, and GELU activation function operations on Out6, and perform element-by-element addition operation on the result of the operation and t5 to obtain feature map Out5; Perform hole gated attention, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations on the feature map Out5, and perform element-by-element addition operation on the result of the operation and t4 to obtain the feature map Out4; Perform hole gated attention, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations on the feature map Out4, and then perform element-by-element addition operation with t3 to obtain the feature map Out3 as output; The common decoding unit in the decoding part comprises: Take the feature map Out3 and t1, t2 as input; Perform two-dimensional convolution with a convolution kernel size of 3, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations on the feature map Out3, and then perform element-by-element addition operation with t2 to obtain the feature map Out2; Perform two-dimensional convolution with a convolution kernel size of 3, batch normalization, two-fold linear interpolation upsampling, and GELU activation function operations on the feature map Out2, and then perform element-by-element addition operation with t1 to obtain the feature map Out1; Perform a two-dimensional convolution with a convolution kernel size of 1 and a two-fold linear interpolation upsampling operation on the feature map Out1 to restore the size of the feature map to the same as the data label, thereby obtaining the feature map Out; The hole-gated attention in the attention encoding unit and the attention decoding unit includes: The hole gated attention includes a split hole convolution unit and a gated attention unit; The split hole convolution unit includes: i Split in the channel dimension and divide it into four equal parts to obtain the feature map after splitting Then, through the deep convolution with a convolution kernel size of 3 but different void ratios, the global feature information and local feature information are obtained, and the feature map is obtained. The global feature information includes feature information obtained by deep convolution with dilation rates of 5 and 7, and the local feature information includes feature information obtained by deep convolution with dilation rates of 1 and 2; In the channel dimension Perform a concatenation operation to restore the size of the feature map, followed by a two-dimensional convolution operation with a convolution kernel size of 1 to obtain the feature map x′ i , so that global feature information and local feature information can interact; Transmitted to the gated attention unit to input feature map x′ through two depth-separable convolutions with a convolution kernel size of 3 i One of them is followed by a Sigmoid activation function to generate an attention map Att, and the other is element-wise multiplied with the generated attention map Att, followed by a depth-wise separable convolution operation with a convolution kernel size of 3 to obtain a feature map Next, input feature map x′ i Perform a two-dimensional convolution operation with a convolution kernel size of 1 to obtain the feature map Finally, the feature map and Perform element-by-element addition to obtain the output feature map x o .

2. According to claim 1, a lightweight image segmentation method based on hole gated attention is characterized in that: Before the first image data is segmented by the hole gated attention model to obtain the segmentation result, the method further includes: Using the second image data to train the first hole gated attention model, so as to train the two-dimensional convolution and batch normalization in the common encoding unit and the common decoding unit, and respectively train the hole gated attention and batch normalization in the attention encoding unit and the attention decoding unit; The trained hole gated attention model is used as the hole gated attention model for testing the first image data.

3. According to claim 2, a lightweight image segmentation method based on hole gated attention is characterized in that: After obtaining the segmentation result, the method further includes: Use the Sigmoid activation function to activate the feature map Out to obtain the final prediction result.

4. A lightweight image segmentation device based on hole gated attention, characterized in that: The device is used to execute the lightweight image segmentation method based on hole gated attention according to any one of claims 1 to 3, comprising: An acquisition module, used for acquiring first image data; A processing module is used to segment the first image data through a hole gated attention model to obtain a segmentation result, wherein the hole gated attention model includes two parts: an encoding part and a decoding part; wherein the encoding part includes a common encoding unit and an attention encoding unit; the decoding part includes an attention decoding unit and a common decoding unit; the common encoding unit, the attention encoding unit, the attention decoding unit and the common decoding unit are connected in sequence.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the image segmentation method based on hole gated attention as described in any one of claims 1 to 3 is implemented.

Images