A neural network-based colorectal polyp segmentation method

Abstract

The present application aims to provide a kind of neural network-based colorectal polyp segmentation method, comprising the following steps: A, construct neural network, and neural network structure is as follows: including coding network, filter, decoding network;The coding network includes five residual convolution layers connected in turn;The filter includes five cascaded local feature fusion module CLFF, context edge guide module CEG;B, original image is input into coding network, and is processed in turn through five residual convolution layers, obtains 5 residual convolution processing results, and is input into filter respectively;C, then the result obtained after processing is input into decoding network;D, after fusion of all input results by decoding network, final segmentation result is obtained.The present application can improve the accuracy and robustness of colorectal polyp segmentation network, and provide more reliable auxiliary diagnostic tool for clinicians.

Description

Technical Field

[0001] This invention relates to the field of computer image processing, and more specifically to a method for segmenting colorectal polyps based on neural networks. Background Technology

[0002] Colorectal polyps are a major precursor lesion of colorectal cancer, and accurate segmentation of colorectal polyps is of great significance in clinical diagnosis and treatment. In recent years, with the rapid development of computer vision and deep learning, many advanced technologies have emerged in the field of medical imaging. Among them, convolutional neural network models have been widely used, including fully convolutional networks (FCNs) and U-net. The U-net model achieves multi-level feature fusion by combining low-level semantic information in the encoding network with high-level semantic information in the decoding network, thereby improving feature representation capabilities. Furthermore, the use of Res2Net-50 as the encoder for colorectal polyp segmentation networks has also been widely adopted. Its powerful feature extraction capabilities add new possibilities to the colorectal polyp segmentation task, further improving accuracy and stability. Experiments have demonstrated that these models have achieved significant segmentation results on colorectal polyp datasets, providing strong support for the automated diagnosis and treatment of colorectal polyps.

[0003] While U-shaped segmentation methods for colorectal polyps have achieved some success, most models lack a deep understanding of the morphological characteristics of colorectal polyps. For example, ... Figure 1 As shown, colorectal polyps vary greatly in color and size, have blurred and indistinct boundaries with surrounding tissues, and may contain interfering factors such as folds or blood in the background. Summary of the Invention

[0004] The present invention aims to provide a neural network-based method for colorectal polyp segmentation, which can improve the accuracy and robustness of the colorectal polyp segmentation network and provide clinicians with a more reliable auxiliary diagnostic tool.

[0005] The technical solution of the present invention is as follows:

[0006] The aforementioned neural network-based method for segmenting colorectal polyps includes the following steps:

[0007] A. Construct a neural network. The specific structure of the neural network is as follows:

[0008] This includes encoding networks, filters, and decoding networks;

[0009] The encoding network comprises five sequentially connected residual convolutional layers; the filter comprises five cascaded local feature fusion modules (CLFF) and a context edge guidance module (CEG).

[0010] B. The original image is input into the encoding network and processed through five residual convolutional layers in sequence to obtain five residual convolutional processing results, which are then input into the filters respectively.

[0011] C. In the filter, the five residual convolution results are processed by 1×1 convolution to obtain convolution results F1, F2, F3, F4, and F5.

[0012] The convolution results F1 and F2 are input into the first cascaded local feature fusion module CLFF for processing, and the first CLFF processing result is then input into the decoding network.

[0013] The convolution results F2 and F3 are input into the second cascaded local feature fusion module CLFF for processing, and the second CLFF processing result is then input into the decoding network.

[0014] Convolution results F3 and F4 are input into the third cascaded local feature fusion module CLFF block for processing, resulting in the third CLFF processing result, which is then input into the decoding network.

[0015] The convolution results F4 and F5 are input into the fourth cascaded local feature fusion module CLFF for processing, and the fourth CLFF processing result is then input into the decoding network.

[0016] The convolution results F1, F2, F3, F4, and F5 are respectively input into the context edge guidance module CEG to obtain the CEG processing results, which are then input into the decoding network.

[0017] D. The decoding network fuses the results of all inputs to obtain the final segmentation result.

[0018] The processing procedure in the cascaded local feature fusion module CLFF is as follows:

[0019] The two input results are first added together and then fused to obtain the first fused result, which is then divided into two paths;

[0020] The first fusion result of the first path is processed by the channel attention module and the 1×1 convolutional block in sequence. The resulting convolutional block processing result is divided into four paths. The first convolutional block processing result is processed by the spatial attention module to obtain the spatial attention processing result, which is divided into three paths.

[0021] The results of the first spatial attention processing are added and fused with the results of the second convolutional block processing to obtain the second fused result. The second fused result is then processed by a 3×3 convolutional block and added and fused with the second fused result to obtain the third fused result.

[0022] The result of the second spatial attention processing is processed by a 3×3 convolution block, then by a 3×3 convolution, and the result is added to and fused with the result of the third spatial attention processing to obtain the fourth fused result.

[0023] The result of the third convolutional block is processed by a 7×7 depth-separable convolutional block, then added and fused with the result of the fourth convolutional block. After being processed by a 1×1 convolutional block, the resulting convolutional result, along with the third and fourth fusion results, is processed by the Concat function and a 3×3 convolutional block, respectively. Finally, it is added and fused with the result of the first fusion result of the second path after being processed by a 3×3 convolutional block to obtain the output result.

[0024] The processing procedure in the Context Edge Guidance Module (CEG) is as follows: Convolution results F1, F2, F3, F4, and F5 are processed sequentially by depthwise separable convolution and 1×1 convolution, respectively. The results are then concatenated using the Concat function. The concatenated results are then processed sequentially by 3×3 convolution blocks and 1×1 convolutions, and then split into two paths. One path is processed by the MSM block and input into the decoding network, while the other path is substituted into the loss formula to calculate the polyp boundary results, which are used to assist in the training of the decoding network.

[0025] The aforementioned loss formula can be expressed as:

[0026]

[0027] Among them G i P represents the true boundary label map of polyps. i This represents a prediction map of polyp boundaries.

[0028] The decoding network includes four multi-scale feature optimization modules (MFOs).

[0029] The fourth CLFF result, the convolution result F5, and the CEG processing result are input into the fourth multi-scale feature optimization module MFO to obtain the fourth MFO result;

[0030] The third CLFF result, the fourth MFO result, and the CEG processing result are input into the third multi-scale feature optimization module MFO to obtain the third MFO result.

[0031] The second CLFF result, the third MFO result, and the CEG processing result are input into the second multi-scale feature optimization module MFO to obtain the second MFO result.

[0032] The first CLFF result, the second MFO result, and the CEG processing result are input into the first multi-scale feature optimization module MFO to obtain the first MFO result; the first MFO result is the final segmentation result.

[0033] The processing procedure in the Multi-Scale Feature Optimization (MFO) module is as follows:

[0034] After the two convolution results are added and fused, they are processed by directional convolution to obtain four-directional processing results. After the four-directional processing results are added and fused, they are multiplied and fused with the CEG processing result. This multiplication and fusion result is divided into three paths. The first path multiplication and fusion result is processed by two 7×7 convolution blocks to obtain a 7×7 convolution block processing result. The second path multiplication and fusion result is processed by two 3×3 convolution blocks to obtain a 3×3 convolution block processing result. The 7×7 convolution block processing result and the 3×3 convolution block processing result are concatenated by the Concat function, processed by 1×1 convolution, and the result is added and fused with the third path multiplication and fusion result to obtain the output result.

[0035] This invention designs a cascaded refinement network for polyp segmentation. The network includes an encoder, a transitioner, and a decoder. The encoder extracts rich structural features from the input image, achieving comprehensive capture of colorectal polyp features through multi-level convolutional operations. The transitioner is designed to effectively integrate cross-layer correlations, enabling the network to more comprehensively understand the complex structure of the polyps. The decoder restores the encoded and transitioned feature maps to pixel-level segmentation results, thereby achieving accurate localization and segmentation of colorectal polyps. By introducing a cascaded refinement network, the aim is to improve the accuracy and robustness of colorectal polyp segmentation networks, providing clinicians with a more reliable auxiliary diagnostic tool, and is expected to play an important role in clinical practice.

[0036] The neural network proposed in this invention can effectively extract feature information from colorectal polyps, thereby achieving more accurate segmentation. Unlike general segmentation networks, this invention simultaneously considers the combination of high-level and low-level features, and designs a cascaded local feature fusion module (CLFF), a context edge guidance module (CEG), and a multi-scale feature optimization module (MFO), which helps to improve the accuracy and robustness of segmentation. Experimental results demonstrate that this neural network model exhibits significant effectiveness in the colorectal polyp segmentation task. Attached Figure Description

[0037] Figure 1 Examples of challenges faced by existing colorectal polyp images: (a) complex backgrounds, (b) blurred polyp boundaries, and (c) varying polyp colors and sizes;

[0038] Figure 2 This is a schematic diagram of the neural network structure in Embodiment 1 of the present invention;

[0039] Figure 3 This is a schematic diagram of the cascaded local feature fusion module CLFF in Embodiment 1 of the present invention;

[0040] Figure 4 This is a schematic diagram of the Context Edge Guidance Module (CEG) of Embodiment 1 of the present invention;

[0041] Figure 5 This is a schematic diagram of the structure of the multi-scale feature optimization module (MFO) in Embodiment 1 of the present invention;

[0042] Figure 6 This is a comparison chart of the visualization results of the method of this invention and the Prnet of the literature. Detailed Implementation

[0043] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] Example 1

[0045] A neural network-based method for segmenting colorectal polyps includes the following steps:

[0046] A. Construct a neural network. The specific structure of the neural network is as follows:

[0047] This includes encoding networks, filters, and decoding networks;

[0048] The encoding network comprises five sequentially connected residual convolutional layers; the filter comprises five cascaded local feature fusion modules (CLFF) and a context edge guidance module (CEG).

[0049] B. The original image is input into the encoding network and processed through five residual convolutional layers in sequence to obtain five residual convolutional processing results, which are then input into the filters respectively.

[0050] C. In the filter, the five residual convolution results are processed by 1×1 convolution to obtain convolution results F1, F2, F3, F4, and F5.

[0051] The convolution results F1 and F2 are input into the first cascaded local feature fusion module CLFF for processing, and the first CLFF processing result is then input into the decoding network.

[0052] The convolution results F2 and F3 are input into the second cascaded local feature fusion module CLFF for processing, and the second CLFF processing result is then input into the decoding network.

[0053] Convolution results F3 and F4 are input into the third cascaded local feature fusion module CLFF block for processing, resulting in the third CLFF processing result, which is then input into the decoding network.

[0054] The convolution results F4 and F5 are input into the fourth cascaded local feature fusion module CLFF for processing, and the fourth CLFF processing result is then input into the decoding network.

[0055] The processing procedure in the cascaded local feature fusion module CLFF is as follows:

[0056] The two input results are first added together and then fused to obtain the first fused result, which is then divided into two paths;

[0057] The first fusion result of the first path is processed by the channel attention module and the 1×1 convolutional block in sequence. The resulting convolutional block processing result is divided into four paths. The first convolutional block processing result is processed by the spatial attention module to obtain the spatial attention processing result, which is divided into three paths.

[0058] The results of the first spatial attention processing are added and fused with the results of the second convolutional block processing to obtain the second fused result. The second fused result is then processed by a 3×3 convolutional block and added and fused with the second fused result to obtain the third fused result.

[0059] The result of the second spatial attention processing is processed by a 3×3 convolution block, then by a 3×3 convolution, and the result is added to and fused with the result of the third spatial attention processing to obtain the fourth fused result.

[0060] The result of the third convolutional block is processed by a 7×7 depth-separable convolutional block, then added and fused with the result of the fourth convolutional block. After being processed by a 1×1 convolutional block, the resulting convolutional result is processed by the Concat function and the third and fourth fusion results, respectively, and then added and fused with the result of the first fusion result of the second path after being processed by a 3×3 convolutional block to obtain the output result.

[0061] The convolution results F1, F2, F3, F4, and F5 are respectively input into the context edge guidance module CEG to obtain the CEG processing results, which are then input into the decoding network.

[0062] The processing procedure in the Context Edge Guidance Module (CEG) is as follows: Convolution results F1, F2, F3, F4, and F5 are processed sequentially by depthwise separable convolution and 1×1 convolution, respectively. The results are then concatenated using the Concat function. The concatenated results are then processed sequentially by 3×3 convolution blocks and 1×1 convolutions, and then split into two paths. One path is processed by the MSM block and input into the decoding network, while the other path is substituted into the loss formula to calculate the polyp boundary results, which are used to assist in the training of the decoding network.

[0063] The aforementioned loss formula can be expressed as:

[0064]

[0065] Among them G i P represents the true boundary label map of polyps.i This represents a prediction map of polyp boundaries.

[0066] D. The decoding network fuses the results of all inputs to obtain the final segmentation result.

[0067] The decoding network includes four multi-scale feature optimization modules (MFOs).

[0068] The fourth CLFF result, the convolution result F5, and the CEG processing result are input into the fourth multi-scale feature optimization module MFO to obtain the fourth MFO result;

[0069] The third CLFF result, the fourth MFO result, and the CEG processing result are input into the third multi-scale feature optimization module MFO to obtain the third MFO result.

[0070] The second CLFF result, the third MFO result, and the CEG processing result are input into the second multi-scale feature optimization module MFO to obtain the second MFO result.

[0071] The first CLFF result, the second MFO result, and the CEG processing result are input into the first multi-scale feature optimization module MFO to obtain the first MFO result; the first MFO result is the final segmentation result.

[0072] The processing procedure in the Multi-Scale Feature Optimization (MFO) module is as follows:

[0073] After the two convolution results are added and fused, they are processed by directional convolution to obtain four-directional processing results. After the four-directional processing results are added and fused, they are multiplied and fused with the CEG processing result. This multiplication and fusion result is divided into three paths. The first path multiplication and fusion result is processed by two 7×7 convolution blocks to obtain a 7×7 convolution block processing result. The second path multiplication and fusion result is processed by two 3×3 convolution blocks to obtain a 3×3 convolution block processing result. The 7×7 convolution block processing result and the 3×3 convolution block processing result are concatenated by the Concat function, processed by 1×1 convolution, and the result is added and fused with the third path multiplication and fusion result to obtain the output result.

[0074] Example 2

[0075] To quantitatively evaluate the performance of colorectal polyp segmentation, this invention introduces the method from Reference 1 for comparison.

[0076] Document 1: D.-P.Fan et al., "Pranet: Parallel reverse attention network for polyp segmentation," in International conference on medical image computing and computer-assisted intervention, 2020: Springer, pp.263-273.

[0077] The parameters used in Reference 1 are the same as those in the original text, and they are guaranteed to be the optimal parameters for the model.

[0078] Table 1 summarizes the experimental data from Reference 1 and Example 1 on the Kvasir-SEG dataset. Figure 6 This demonstrates the effectiveness of the experimental results. Based on the experimental results, the detection method in Example 1 is superior to the detection method in Reference 1.

[0079] Table 1 Quantitative Data

[0080]

[0081] Depend on Figure 6 As can be seen, through comparative analysis of visualization results, the method in Example 1 can effectively learn the features of polyps of different sizes, shapes and blurred edges, thereby improving the ability to identify various challenging polyps and generating segmentation results that are closer to the real labels.

Claims

1. A method for segmenting colorectal polyps based on neural networks, characterized in that, Includes the following steps: A. Construct a neural network. The specific structure of the neural network is as follows: This includes encoding networks, filters, and decoding networks; The encoding network comprises five sequentially connected residual convolutional layers; the filter comprises five cascaded local feature fusion modules (CLFF) and a context edge guidance module (CEG). B. The original image is input into the encoding network and processed through five residual convolutional layers in sequence to obtain five residual convolutional processing results, which are then input into the filters respectively. C. In the filter, the five residual convolution results are processed by 1×1 convolution to obtain convolution results F1, F2, F3, F4, and F5. The convolution results F1 and F2 are input into the first cascaded local feature fusion module CLFF for processing, and the first CLFF processing result is then input into the decoding network. The convolution results F2 and F3 are input into the second cascaded local feature fusion module CLFF for processing, and the second CLFF processing result is then input into the decoding network. Convolution results F3 and F4 are input into the third cascaded local feature fusion module CLFF block for processing, resulting in the third CLFF processing result, which is then input into the decoding network. The convolution results F4 and F5 are input into the fourth cascaded local feature fusion module CLFF for processing, and the fourth CLFF processing result is then input into the decoding network. The convolution results F1, F2, F3, F4, and F5 are respectively input into the context edge guidance module CEG to obtain the CEG processing results, which are then input into the decoding network. D. The decoding network fuses the results of all inputs to obtain the final segmentation result; The processing procedure in the cascaded local feature fusion module CLFF is as follows: The two input results are first added together and then fused to obtain the first fused result, which is then divided into two paths; The first fusion result of the first path is processed by the channel attention module and the 1×1 convolutional block in sequence. The resulting convolutional block processing result is divided into four paths. The first convolutional block processing result is processed by the spatial attention module to obtain the spatial attention processing result, which is divided into three paths. The results of the first spatial attention processing are added and fused with the results of the second convolutional block processing to obtain the second fused result. The second fused result is then processed by a 3×3 convolutional block and added and fused with the second fused result to obtain the third fused result. The result of the second spatial attention processing is processed by a 3×3 convolution block, then by a 3×3 convolution, and the result is added to and fused with the result of the third spatial attention processing to obtain the fourth fused result. The result of the third convolutional block is processed by a 7×7 depth-separable convolutional block, then added and fused with the result of the fourth convolutional block. After being processed by a 1×1 convolutional block, the resulting convolutional result is processed by the Concat function and the third and fourth fusion results, respectively, and then added and fused with the result of the first fusion result of the second path after being processed by a 3×3 convolutional block to obtain the output result. The processing procedure in the Context Edge Guidance Module (CEG) is as follows: Convolution results F1, F2, F3, F4, and F5 are processed sequentially by depthwise separable convolution and 1×1 convolution, respectively. The results are then concatenated using the Concat function. The concatenated results are then processed sequentially by 3×3 convolution blocks and 1×1 convolutions, and then split into two paths. One path is processed by the MSM block and input into the decoding network, while the other path is substituted into the loss formula to calculate the polyp boundary results, which are used to assist in the training of the decoding network. The aforementioned loss formula can be expressed as: in This represents a label map showing the true boundaries of polyps. This represents a predicted boundary map of polyps.

2. The method for segmenting colorectal polyps based on neural networks as described in claim 1, characterized in that: The decoding network includes four multi-scale feature optimization modules (MFOs). The fourth CLFF result, the convolution result F5, and the CEG processing result are input into the fourth multi-scale feature optimization module MFO to obtain the fourth MFO result; The third CLFF result, the fourth MFO result, and the CEG processing result are input into the third multi-scale feature optimization module MFO to obtain the third MFO result; The second CLFF result, the third MFO result, and the CEG processing result are input into the second multi-scale feature optimization module MFO to obtain the second MFO result; The first CLFF result, the second MFO result, and the CEG processing result are input into the first multi-scale feature optimization module MFO to obtain the first MFO result; the first MFO result is the final segmentation result.

3. The method for segmenting colorectal polyps based on neural networks as described in claim 2, characterized in that: The processing procedure in the Multi-Scale Feature Optimization (MFO) module is as follows: After the two convolution results are added and fused, they are processed by directional convolution to obtain four-directional processing results. After the four-directional processing results are added and fused, they are multiplied and fused with the CEG processing result. This multiplication and fusion result is divided into three paths. The first path multiplication and fusion result is processed by two 7×7 convolution blocks to obtain a 7×7 convolution block processing result. The second path multiplication and fusion result is processed by two 3×3 convolution blocks to obtain a 3×3 convolution block processing result. The 7×7 convolution block processing result and the 3×3 convolution block processing result are concatenated by the Concat function, processed by 1×1 convolution, and the result is added and fused with the third path multiplication and fusion result to obtain the output result.

Images