A lightweight medical image segmentation method based on dynamic convolution

Abstract

The present invention discloses a lightweight medical image segmentation method based on dynamic convolution. The method obtains a medical image to be segmented and inputs it into a constructed segmentation network model. The segmentation network model includes an encoding module and a decoding module. In the encoding module, the medical image to be segmented passes through three encoding units in sequence to obtain a first encoding feature map, a second encoding feature map, and a third encoding feature map output by each encoding unit. The third encoding feature map is then input into a semantic branch and a spatial branch, respectively, to extract semantic and spatial features. In the decoding module, the decoded feature map passes through four decoding units in sequence and is finally input into a dynamic convolution unit to obtain the segmentation result. The model of the present invention significantly reduces the number of parameters and computational complexity, while being able to well fit the data set and achieve accurate segmentation results.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence computer vision technology, and in particular relates to a lightweight medical image segmentation method based on dynamic convolution. Background Art

[0002] Image segmentation has always been a difficult problem in the field of image processing and analysis, and is also one of the key issues in the field of computer vision. Medical image segmentation is an important research direction in the field of computer vision, and has important auxiliary value in the detection, diagnosis and treatment of diseases. By segmenting medical images, different tissues and organs in the image can be extracted. Accurate medical image segmentation can provide doctors with more references and help them make more accurate judgments.

[0003] As researchers pursue ever-higher segmentation accuracy, they have developed a number of deep learning models with more complex structures and deeper networks. This inevitably increases the number of model parameters and computational complexity, leading to higher demands on experimental equipment. To address this issue, numerous lightweight network models have been proposed. However, these approaches often compromise performance and scale, making them less suitable for medical image segmentation. Summary of the Invention

[0004] The purpose of this application is to propose a lightweight medical image segmentation method based on dynamic convolution to solve the problems in the background technology, reduce the number of model parameters and calculation amount, and obtain accurate segmentation results.

[0005] In order to achieve the above objectives, the technical solutions of this application are as follows:

[0006] A lightweight medical image segmentation method based on dynamic convolution, comprising:

[0007] Acquire a medical image to be segmented and input it into a constructed segmentation network model, wherein the segmentation network model includes an encoding module and a decoding module;

[0008] In the encoding module, the medical image to be segmented passes through three encoding units in sequence to obtain the first encoding feature map, the second encoding feature map and the third encoding feature map output by each encoding unit, and then the third encoding feature map is input into the semantic branch and the spatial branch respectively to extract semantic features and spatial features;

[0009] In the decoding module, the semantic features and spatial features are input into the first decoding unit to obtain the first decoding feature map, the first decoding feature map and the third encoding feature map are input into the second decoding unit to obtain the second decoding feature map, the second decoding feature map and the second encoding feature map are input into the third decoding unit to obtain the third decoding feature map, the third decoding feature map and the first encoding feature map are input into the fourth decoding unit to obtain the fourth decoding feature map, and finally the fourth decoding feature map is input into the dynamic convolution unit to obtain the segmentation result.

[0010] Furthermore, the step of inputting the third encoded feature map into the semantic branch and the spatial branch to extract semantic features and spatial features respectively includes:

[0011] The third encoded feature map is input into the semantic branch and the spatial branch respectively, wherein the semantic branch includes two encoding units, and the spatial branch includes two 1×1 convolutions. The output of the first encoding unit in the semantic branch is downsampled and activated, multiplied by the output of the first convolution in the spatial branch, and then input into the second convolution in the spatial branch. The output of the first convolution in the spatial branch is upsampled and activated, multiplied by the output of the first encoding unit in the semantic branch, and then input into the second encoding unit in the semantic branch.

[0012] The output of the second coding unit in the semantic branch is downsampled and activated, and then multiplied by the output of the second convolution in the spatial branch to obtain the spatial feature. The output of the second convolution in the spatial branch is upsampled and activated, and then multiplied by the output of the second coding unit in the semantic branch to obtain the semantic feature.

[0013] Furthermore, the encoding unit performs the following operations:

[0014] Use depthwise convolution to process the input feature map, then use 1×1 convolution to increase the dimension to 4 times the number of channels, and record it as X1. Then use 1×1 convolution to reduce the dimension and compress the channel to 1. Then use Sigmoid function to get X2, and then multiply X2 and X1 element by element to get X3.

[0015] After connecting X1 and X3, the input is fed into a 1×1 convolution to reduce the number of channels again. Then, the input is added to the input and fed into a dynamic convolution. Finally, the maximum pooling operation is performed to obtain the output of the encoding unit.

[0016] Furthermore, the second encoding unit in the semantic branch performs the following operations:

[0017] Use depthwise convolution to process the input feature map, then use 1×1 convolution to increase the dimension to 4 times the number of channels, and record it as X1. Then use 1×1 convolution to reduce the dimension and compress the channel to 1. Then use Sigmoid function to get X2, and then multiply X2 and X1 element by element to get X3.

[0018] After connecting X1 and X3, the input is fed into a 1×1 convolution to reduce the number of channels again. Then, the sum is added to the input and fed into a dynamic convolution to obtain the output of the encoding unit.

[0019] Furthermore, the decoding unit performs the following operations:

[0020] For the inputs F1 and F2 of the decoding unit, first a 1×1 convolution is performed to reduce the number of channels of F1 to half of the input, and then the 1×1 convolution output feature map is enlarged to twice the size by linear interpolation;

[0021] The result obtained by linear difference is processed using squeeze and excitation operation, and then the result is concatenated with F2 and input into dynamic convolution to obtain the output of the decoding unit.

[0022] This application proposes a lightweight medical image segmentation method based on dynamic convolution, which introduces dynamic convolution. Dynamic convolution has the characteristic of adaptively adjusting weights according to input to generate different convolution kernel parameters. By adding a dynamic convolution at each layer, the model can better adapt to changes in input data, so that the model can expand the model capacity without changing the depth, thereby better fitting the data and improving the model's adaptability, accuracy and robustness; the "inverted bottleneck" structure is used in the encoder to further reduce the number of parameters, and spatial attention is used for weighting, so that the encoder can extract richer features, improve feature extraction capabilities and generalization; the upsampling strategy in the decoder also greatly reduces the number of parameters and computational complexity of the model, and uses squeezing and excitation mechanisms to strengthen features, making the decoder more efficient and stable. The overall design greatly reduces the number of parameters and computational complexity of the model, while being able to fit the data set well and obtain accurate segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flowchart of the lightweight medical image segmentation method based on dynamic convolution in this application.

[0024] Figure 2 Schematic diagram of the segmentation network model for this application.

[0025] Figure 3 This is a schematic diagram of the structure of the encoder of this application.

[0026] Figure 4 This is a structural diagram of the decoding unit of this application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0028] One embodiment of the present application, such as Figure 1 As shown in the figure, a lightweight medical image segmentation method based on dynamic convolution is proposed, including:

[0029] Step S1: Obtain a medical image to be segmented and input it into a constructed segmentation network model, wherein the segmentation network model includes an encoding module and a decoding module.

[0030] This application segments medical images by constructing and training a segmentation network model. The segmentation network model is as follows: Figure 2 As shown, it includes an encoding module and a decoding module, which will be explained in detail in subsequent steps.

[0031] When training the segmentation network model, the selected image dataset is first preprocessed to obtain the training dataset, validation dataset, and test dataset.

[0032] If a 3D dataset is selected, the 3D data is first read and sliced ​​according to the cross-sectional coordinate axis. The labeled slices and the corresponding source data slices are then converted to npy format and saved for subsequent training. If a 2D dataset is selected, the images are directly read and converted to npy format and saved for subsequent training.

[0033] It should be noted that when preparing the training dataset, the images in the npy format in the dataset are also processed, including data enhancement preprocessing such as translation, horizontal flipping, rotation transformation, grayscale normalization, perspective transformation, and adding Gaussian noise. Data enhancement can force the model to learn more robust features, thereby effectively improving the generalization ability of the model.

[0034] Finally, the dataset is divided into a training dataset, a validation dataset, and a test dataset in proportion. Using the above datasets to train the network model is a relatively mature technology in this field and will not be described in detail here.

[0035] When training the segmentation network model, this application uses the binary cross entropy loss function as the main loss for training. The specific formula can be expressed as follows:

[0036] L=L BCE (y,y)=-ylogy+(1-y)log(1-y)

[0037] in and y represent the predicted value and the true value respectively.

[0038] After the segmentation network model is trained, you only need to obtain the medical image to be segmented, input it into the trained segmentation network model, and perform subsequent processing to obtain the segmentation result.

[0039] Step S2: In the encoding module, the medical image to be segmented passes through three encoding units in sequence to obtain the first encoding feature map, the second encoding feature map and the third encoding feature map output by each encoding unit, and then the third encoding feature map is input into the semantic branch and the spatial branch respectively to extract semantic features and spatial features.

[0040] like Figure 2 As shown in Figure 3, the encoding module includes three encoding units connected in sequence, and the output of the last encoding unit is input into the semantic branch and the spatial branch respectively.

[0041] In one embodiment, the encoding unit is as follows Figure 3 As shown, perform the following operations:

[0042] Step S2.1.1. Use depthwise convolution to process the input feature map, then use 1×1 convolution to increase the number of channels to 4 times, and record it as X1. Then use 1×1 convolution to reduce the dimension and compress the channel to 1. Then use the Sigmoid function to obtain X2, and then multiply X2 and X1 element by element to obtain X3.

[0043] Step S2.1.2: Connect X1 and X3 and input them into 1×1 convolution to reduce the number of channels again. Then add them to the input and input them into dynamic convolution. Finally, the output of the encoding unit is obtained through the maximum pooling operation.

[0044] Specifically, the input feature map X is first processed using depthwise convolution. A 1×1 convolution is then used to quadruple the number of channels, denoted as X1. A 1×1 convolution is then used to reduce the dimensionality, compressing the channels to 1. The sigmoid function is then used to obtain X2. X2 and X1 are then element-wise multiplied to update the feature map to obtain X3. This method of updating the feature map is very convenient and efficient because it compresses the number of channels to 1, generating more diverse feature information with almost no parameters. Finally, the Concat operation is used to concatenate the two feature maps, X1 and X3, to obtain a richer feature map, further enhancing the network's expressive power.

[0045] Then, 1×1 convolution is used to reduce the number of channels to the same as X, recorded as X4, and then X4 is added to X and input into the dynamic convolution. After dynamic convolution, the maximum pooling operation is performed to obtain the feature map finally output by the encoding unit.

[0046] It should be noted that depthwise convolution and dynamic convolution are both relatively mature technologies in this field and will not be described in detail here. Figure 3 DW stands for depthwise convolution, and DynamicConv stands for dynamic convolution.

[0047] like Figure 2 As shown, the encoding unit includes an encoder and a maximum pooling operation (Max Pool, represented by P in the figure) to obtain an encoding feature map. The three encoding units finally obtain the first encoding feature map, the second encoding feature map and the third encoding feature map.

[0048] The output of the last encoding unit passes through the semantic branch and spatial branch respectively to extract semantic features and spatial features.

[0049] In a specific embodiment, the semantic branch and the spatial branch perform the following operations:

[0050] Step S2.2.1, input the third encoded feature map into the semantic branch and the spatial branch respectively, the semantic branch includes two encoding units, the spatial branch includes two 1×1 convolutions, the output of the first encoding unit in the semantic branch is down-sampled and activated, and then multiplied with the output of the first convolution in the spatial branch and input into the second convolution in the spatial branch, the output of the first convolution in the spatial branch is up-sampled and activated, and then multiplied with the output of the first encoding unit in the semantic branch and input into the second encoding unit in the semantic branch.

[0051] Among them, the first coding unit is the same as the coding unit described above, and the second coding unit can be the same as the coding unit described above, and may not include the subsequent maximum pooling operation, which will not be repeated here.

[0052] This step is used to extract deeper semantic information containing more details. The spatial branch uses 1×1 convolution to keep the size of the feature map unchanged and only changes the channel, thereby preserving the spatial information of the feature map.

[0053] Step S2.2.2: The output of the second coding unit in the semantic branch is downsampled and activated, and then multiplied by the output of the second convolution in the spatial branch to obtain the spatial feature. The output of the second convolution in the spatial branch is upsampled and activated, and then multiplied by the output of the second coding unit in the semantic branch to obtain the semantic feature.

[0054] This step fuses the semantic branch and the spatial branch. The output of the second coding unit in the semantic branch is first upsampled by bilinear interpolation, and its feature map size is expanded to the same size as the output of the second convolution in the spatial branch. Sigmoid is used to obtain the weight of each pixel, and then element-wise multiplication is used to obtain the feature map of the spatial branch. The output of the second convolution in the spatial branch is first downsampled by bilinear interpolation, and the feature map is scaled to the same size as the output of the second coding unit in the semantic branch. Sigmoid is used to obtain the weight of each pixel, and then element-wise multiplication is used to obtain the feature map of the semantic branch. This allows both branches to obtain the information of the other branch to improve prediction accuracy.

[0055] Here, using linear interpolation for image scaling can further save parameters and computation, making the model lighter. The specific calculation formula is as follows:

[0056] S1'=σ(up(S1))S2

[0057] S2'=σ(down(S2))S1

[0058] Where S1 and S2 represent the output of the semantic branch encoding unit and the output of the spatial branch convolution respectively, S1' and S2' represent the output of the semantic branch and the spatial branch respectively, σ represents the Sigmoid activation function, up and down represent upsampling and downsampling respectively.

[0059] Step S3: In the decoding module, the semantic features and the spatial features are input into the first decoding unit to obtain a first decoding feature map, the first decoding feature map and the third encoding feature map are input into the second decoding unit to obtain a second decoding feature map, the second decoding feature map and the second encoding feature map are input into the third decoding unit to obtain a third decoding feature map, the third decoding feature map and the first encoding feature map are input into the fourth decoding unit to obtain a fourth decoding feature map, and finally the fourth decoding feature map is input into the dynamic convolution unit to obtain the segmentation result.

[0060] This step enters the decoding stage. The decoding module includes four decoding units and a dynamic convolution unit. The feature map of the semantic branch and the feature map of the spatial branch are input into the first decoding unit to obtain the first decoding feature map. The first decoding feature map and the third encoding feature map are input into the second decoding unit to obtain the second decoding feature map. The second decoding feature map and the second encoding feature map are input into the third decoding unit to obtain the third decoding feature map. The third decoding feature map and the first encoding feature map are input into the fourth decoding unit to obtain the fourth decoding feature map. Finally, the fourth decoding feature map is input into the dynamic convolution unit to obtain the segmentation result.

[0061] like Figure 4As shown, for the inputs F1 and F2 of the decoding unit, the number of channels of F1 is first reduced to half of the input through a 1×1 convolution, and then the output feature map is enlarged to twice the size through linear difference, achieving the same effect as the transposed convolution, but compared with the transposed convolution, it can save more parameters and computation. This upsampling method has clear order requirements. Channel compression must be performed before linear interpolation. If linear interpolation is performed before channel compression, the number of parameters and computation used in the operation will be larger due to the larger image. The result obtained by the above method is processed using the squeeze and excitation (SE) operation, and then the result is spliced ​​with F2 and input into the dynamic convolution to obtain the output of the decoding unit. The specific calculation formula is as follows:

[0062] F1'=SE(Inter(C(F1)))

[0063] output=DyConv(Cat(F1',F2))

[0064] Among them, F1 and F2 represent the two inputs of the decoding unit, F1' represents the feature map of the intermediate process, output represents the output, Inter represents the linear interpolation operation, C represents the convolution operation, SE represents the SE operation, and DyConv represents the dynamic convolution operation.

[0065] Finally, the result of the last decoding unit is input into the dynamic convolution and output after final processing.

[0066] It should be noted that this application also uses images in the corresponding data set to train the segmentation network model multiple times. The training method adopts five-fold cross-validation, and the training set and test set are replaced respectively to ensure the comprehensiveness and reliability of the training results. The training of the network will not be described here.

[0067] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A lightweight medical image segmentation method based on dynamic convolution, characterized in that: The lightweight medical image segmentation method based on dynamic convolution includes: Acquire a medical image to be segmented and input it into a constructed segmentation network model, wherein the segmentation network model includes an encoding module and a decoding module; In the encoding module, the medical image to be segmented passes through three encoding units in sequence to obtain the first encoding feature map, the second encoding feature map and the third encoding feature map output by each encoding unit, and then the third encoding feature map is input into the semantic branch and the spatial branch respectively to extract semantic features and spatial features; In the decoding module, the semantic features and spatial features are input into the first decoding unit to obtain a first decoding feature map, the first decoding feature map and the third encoding feature map are input into the second decoding unit to obtain a second decoding feature map, the second decoding feature map and the second encoding feature map are input into the third decoding unit to obtain a third decoding feature map, the third decoding feature map and the first encoding feature map are input into the fourth decoding unit to obtain a fourth decoding feature map, and finally the fourth decoding feature map is input into the dynamic convolution unit to obtain the segmentation result; The step of inputting the third encoded feature map into the semantic branch and the spatial branch to extract semantic features and spatial features includes: The third encoded feature map is input into the semantic branch and the spatial branch respectively, wherein the semantic branch includes two encoding units, and the spatial branch includes two 1×1 convolutions. The output of the first encoding unit in the semantic branch is downsampled and activated, multiplied by the output of the first convolution in the spatial branch, and then input into the second convolution in the spatial branch. The output of the first convolution in the spatial branch is upsampled and activated, multiplied by the output of the first encoding unit in the semantic branch, and then input into the second encoding unit in the semantic branch. The output of the second coding unit in the semantic branch is downsampled and activated, and then multiplied by the output of the second convolution in the spatial branch to obtain the spatial feature. The output of the second convolution in the spatial branch is upsampled and activated, and then multiplied by the output of the second coding unit in the semantic branch to obtain the semantic feature. The encoding unit performs the following operations: The input feature map is processed using depthwise convolution, and then the number of channels is increased to 4 times by 1×1 convolution, which is recorded as X 1 Then use 1×1 convolution to reduce the dimension, compress the channel to 1, and then use the Sigmoid function to process it. X 2 , and then X 2 and X 1 Perform element-wise multiplication to get X 3 ; Will X 1 and X 3 After the connection, the input is input into the 1×1 convolution to reduce the number of channels again, and then added to the input and input into the dynamic convolution. Finally, the output of the encoding unit is obtained through the maximum pooling operation.

2. The lightweight medical image segmentation method based on dynamic convolution according to claim 1, characterized in that: The second encoding unit in the semantic branch performs the following operations: The input feature map is processed using depthwise convolution, and then the number of channels is increased to 4 times by 1×1 convolution, which is recorded as X 1 Then use 1×1 convolution to reduce the dimension, compress the channel to 1, and then use the Sigmoid function to process it. X 2 , and then X 2 and X 1 Perform element-wise multiplication to get X 3 ; Will X 1 and X 3 After the connection, the input is input into the 1×1 convolution to reduce the number of channels again, and then added to the input and input into the dynamic convolution to obtain the output of the encoding unit.

3. The lightweight medical image segmentation method based on dynamic convolution according to claim 1, characterized in that: The decoding unit performs the following operations: Input to the decoding unit F 1 and F 2 , first pass a 1×1 convolution to F 1 The number of channels is reduced to half of the input, and then the 1×1 convolution output feature map is enlarged to twice the size by linear interpolation; The result of the linear difference is processed by squeeze and excitation operation, and then the result is compared with F 2 After splicing, it is input into the dynamic convolution to obtain the output of the decoding unit.

Images