Polyp image segmentation method based on double-branch feature progressive fusion network

Abstract

The invention belongs to the technical field of image processing, and discloses a polyp image segmentation method based on a double-branch feature progressive fusion network, and the method comprises the steps: constructing a double-branch network based on feature and boundary information extraction, and carrying out the progressive fusion training; the method mainly comprises a double-branch backbone network based on point convolution visual transformation and a reverse perception information layer, a dislocation fusion module, a dislocation single-layer fusion module, a perception information fusion module and a multistage residual decoding module, a trained optimal model is used for segmenting a polyp image, and a result is evaluated. According to the method, boundary information and hierarchical features are effectively extracted through a double-branch structure, global semantic information is reserved and refined in a progressive fusion mode, the limitation of a traditional algorithm in the polyp boundary feature utilization process is overcome, the boundary difference is adopted in the training process to combine with the loss to optimize the model training process, and the training efficiency is improved. And high-precision recognition and segmentation of the polyp region are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a polyp image segmentation method based on a double-branch feature progressive fusion network. Background Art

[0002] With the rapid development of deep learning methods, the efficiency of feature extraction and utilization in polyp image segmentation has been significantly improved. Architectures based on convolutional neural networks (CNNs) have been widely used in the field of image segmentation, among which the typical network architecture represented by U-Net has greatly promoted the application of CNN in medical image segmentation. Many studies have made improvements on the basis of U-Net to overcome its limitations in practical applications, so that various segmentation models based on U-Net have significantly improved the segmentation performance. In addition, the Transformer-based method realizes the feature extraction capabilities of dynamic weight allocation and global receptive field through the multi-head self-attention layer. On this basis, the Visual Transformer (ViT) proposes a strategy of dividing the image into multiple regions for encoding and classification, which improves the training efficiency with the help of parallel computing and effectively improves the ability to extract global information.

[0003] However, the current deep learning-based segmentation methods mainly focus on the extraction of texture information features in the polyp region, and it is difficult to effectively extract and utilize the boundary information of the polyp region. Due to the lack of deep fusion of global features and boundary information, these methods often show inadequacies in processing polyp segmentation tasks of different sizes and shapes. Therefore, it is necessary to explore new methods to improve the accuracy and reliability of polyp segmentation, so as to achieve a more complete semantic structure expression. Summary of the invention

[0004] The purpose of the present invention is to provide a polyp image segmentation method based on a double-branch feature progressive fusion network to solve the above-mentioned technical problems.

[0005] In order to solve the above technical problems, the specific technical solution of a polyp image segmentation method based on a dual-branch feature progressive fusion network of the present invention is as follows: A polyp image segmentation method based on a dual-branch feature progressive fusion network comprises the following steps: Step 1: Collect colonoscopic polyp images and polyp mask label datasets, and divide the datasets into training sets and test sets; Step 2: Construct a polyp image segmentation model based on a dual-branch feature progressive fusion network; Step 3: Perform data enhancement on the training data obtained in step 1 and input it into the dual-branch feature progressive fusion network training model; Step 4: Use the best model obtained after training to test the colonoscopy images in the test set, obtain the polyp segmentation results after the test, and evaluate the segmentation results of the model.

[0006] Furthermore, the data set of step 1 includes five sub-datasets, of which 1450 images selected from two sub-datasets are used as training sets, 798 images are selected from five data sets as test sets, and finally all five data sets are used to evaluate the segmentation performance of the final model.

[0007] Furthermore, the dual-branch feature progressive fusion network proposed in step 2 is used to perform the following steps: Step 2.1: Use the PVTv2 backbone network as the multi-level feature extraction branch to extract four-stage multi-level features from the original input image T ={ T 1 , T 2 , T 3 , T 4}; Step 2.2: Use a single-line gating mechanism to perform feature weighted extraction on the original input image and perform preliminary feature filtering; Step 2.3: Using the boundary information perception branch, high-fidelity boundary information is extracted from the features obtained in step 2.2 through hierarchical feature enhancement and cascade fusion strategies, and the polyp boundary segmentation mask is constructed to extract boundary feature information; Step 2.4: Use the dislocation fusion module to fuse the adjacent level features in step 2.1 pair by pair to generate a new comprehensive feature representation, and use the dislocation single layer fusion module to fuse the bottom layer features. T 4 Extraction, using the channel and spatial attention fusion mechanism to refine and enhance high-level semantic feature information, thereby obtaining a new four-stage multi-level feature F ={ F 1 , F 2 , F 3 , F 4}; Step 2.5: Further mine effective semantic information from the boundary feature information extracted in step 2.3, and combine the mined effective semantic information with the multi-level feature information through a multi-level cascade perception information fusion module. F Perform progressive global fusion to obtain boundary aggregation perception features; Step 2.6: Cross-level fusion of multi-level features using a multi-level residual decoding module FThe features obtained at each level are fused and reduced in dimension by 1×1 convolution to participate in deep supervision during the training process.

[0008] Furthermore, the single-line gating mechanism proposed in step 2.2 is used to input the original image of the network X Perform preliminary feature extraction and filtering, first generate weight tensors through convolution and normalization operations W as , and then use the weight tensor after channel division , ( i =1, 2) Perform weighted summation on the input features to obtain fusion features G f , the specific expression method is as follows: , in Softmax represents the normalized exponential function, div c Indicates the channel division operation on the tensor. represents element-wise multiplication, Represents addition of elements.

[0009] Furthermore, the boundary information perception branch proposed in step 2.3 includes the following steps: Step 2.3.1: The boundary information perception branch contains three sets of cascaded reverse perception information layers. As a parallel branch with the multi-level feature extraction branch, the boundary information perception branch uses the original image of the input network as the perception target to extract boundary feature information. Each set of reverse perception information layers first extracts the feature E j ( j =1,2,3) is obtained by 3×3 convolution, and then activation and inversion operations are performed to obtain the inverted features. E rj , and E rj and Multiply elements to get the original boundary features E cj , the specific expression method is as follows: , In the formula express Sigmoid function, E is an all-one matrix; Step 2.3.2: Together with E rj and E cj The two modules are input into the residual inverse enhancement module, and the features are fused by combining residual connection and convolution operation. Ecj , and E rj , and finally obtain enhanced boundary-aware features with strong semantic information E xj , the specific expression method is as follows: , Where || represents the concatenation operation along the channel dimension, Conv 1 represents a 1×1 convolution operation, Conv 3 Represents a 3×3 convolution operation.

[0010] Furthermore, the dislocation fusion module proposed in step 2.4 is based on adjacent level features. T i and T i+1 ( i =1,2,3,4) as input, after convolution at two scales, the feature information is enriched by splicing operation, and the channel attention operation and spatial attention operation are used to obtain features with good information expression ability. M 2 ,Will M 2 Self-splicing is performed in the channel dimension to construct a new feature representation, which doubles the number of channels of the output tensor, expands the representation capacity of the feature space while maintaining the integrity of the original feature information, and finally extracts local features through 3×3 convolution to obtain features with strong spatial context expression capabilities. F i , the specific expression method is as follows: , In the formula U represents the upsampling operation, Conv 5 represents a 5×5 convolution, s represents spatial attention, ca Represents channel attention.

[0011] Furthermore, the misaligned single-layer enhancement module proposed in step 2.4 is composed of channel attention and spatial attention. i = 4, the staggered single-layer enhancement module uses a dual-branch splicing method of parallel attention operation and cascade attention operation to perform hierarchical features. T 4 The semantic features are enhanced, and finally the enhanced feature information after splicing is further extracted through 3×3 convolution to obtain the feature F 4 , the specific expression method is as follows:

[0012] Furthermore, the perception information fusion module proposed in step 2.5 is based on the hierarchical features F i and boundary-aware features E xj As input, the hierarchical features are globally fused with the boundary perception features through a multi-level cascade method. The perception information fusion module first uses the channel attention operation to reduce the redundancy and noise in the boundary perception features, and adopts a dual-branch parallel structure to splice the two features in different orders to obtain the feature and , thus forming two feature combination branches, using 3×3 and 5×5 convolution operations of two different scales in the two branches respectively, and using the global average pooling layer and residual connection to further enhance the fusion features to obtain and , and finally the elements are added to obtain the perceptual features The specific expression method is as follows: , In the formula GAP Represents a global average pooling layer.

[0013] Furthermore, the multi-level residual decoding module proposed in step 2.6 includes the following steps: Step 2.6.1: The multi-level residual decoding module combines the output of the hierarchical perception information fusion module in a cascade manner. , boundary features E x3 The output features of the previous multi-level residual decoding module are used as input, and the features between different levels and the boundary information features are globally fused to finally obtain the fused features. D k ( k =1,2,3,4); Step 2.6.2: In the multi-level residual decoding module, first and D k+1 , ( k =2,3) to perform element-by-element addition operation to obtain a new fused feature, which is then concatenated with the two original features to form a joint feature representation containing three features. , thereby enhancing the expressiveness of features. The specific expression method is as follows: , Step 2.6.3: Enhanced features The input is sent to two cascaded residual convolution modules, and a two-level residual connection is used to improve the semantic expression ability of the features. The specific expression method is as follows: , when k = 1, the output feature of the multi-level residual decoding module above the top-level perceptual information fusion module D 2 And the output features of the boundary information perception branch E x3 As input; when k =4, the bottom layer perception information fusion module uses the output features of the bottom layer perception information fusion module F P32 and F P42 as input.

[0014] Furthermore, in step 3, the model training process is optimized using boundary difference joint loss, which specifically includes the following steps: Step 3.1: Mask the real G and the prediction mask P As input, the neighborhood features of the real mask are extracted by customizing the edge convolution kernel to generate a tensor containing boundary information. T E , the specific expression method is as follows: , in Pad 01 Indicates that a layer of edge filling is performed with 0. Conv e Represents a custom edge convolution kernel, with: , Step 3.2: Calculate the ground-truth mask G and the boundary information tensor T E The relationship between them determines the parameters ɑ Used to adjust loss calculation, the specific expression method is as follows: , in N Tnz and N Gnz They are the boundary information tensors T E and the true mask G The number of non-zero elements in , using a very small constant to prevent errors caused by the denominator being zero; Step 3.3: Calculate the intersection and sum of squares of the true mask and the predicted mask, and obtain the difference measure between the predicted value and the true value as the numerator, using ɑ As a variable parameter, the real mask G and the boundary information tensor T E The weighted value is used as the denominator to calculate the overall loss. The specific expression is as follows: , in sum ele Represents the sum of the internal elements of a tensor.

[0015] The polyp image segmentation method based on a dual-branch feature progressive fusion network of the present invention has the following advantages: (1) The present invention adopts a dual-branch structure, which can capture multi-level boundary information features while obtaining hierarchical information. The features captured by the dual-branch structure realize the deep fusion of global semantic information, strengthen the relevance of contextual information, and thus effectively improve the performance of polyp segmentation.

[0016] (2) The present invention proposes the reverse perception information layer as the core of the boundary information perception branch, combines the two parts of feature inversion and residual enhancement, uses three groups of reverse perception information layers in cascade to extract boundary features, and combines each group of boundary features with hierarchical features, which significantly improves the semantic expression ability of boundary features.

[0017] (3) The present invention proposes a misaligned fusion module and a misaligned single-layer enhancement module, which utilize a multi-branch attention mechanism to perform pairwise fusion and filtering of adjacent level features and perform separate enhancement on the highest level features, thereby effectively retaining key information while optimizing the segmentation process.

[0018] (4) The present invention proposes a perception information fusion module, which further filters multi-level boundary information and adopts a multi-level cascade method to achieve progressive fusion of features, thereby generating multi-level perception features that fully integrate boundary information.

[0019] (5) The present invention proposes a multi-level residual decoding module, which uses a cascaded residual structure to fully globally fuse multi-level perceptual features to obtain accurate and complete segmentation results.

[0020] (6) The present invention uses boundary difference joint loss to supervise the training process, which effectively improves the training effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic flow chart of a polyp image segmentation method based on a double-branch feature progressive fusion network of the present invention; Figure 2The overall network structure diagram of the present invention; Figure 3 It is a structural diagram of a single-line gating mechanism; Figure 4 It is a diagram of the structure of the reverse perception information layer; Figure 5 It is the structural diagram of the dislocation fusion module; Figure 6 It is a staggered single-layer fusion module structure; Figure 7 It is the structural diagram of the perception information fusion module; Figure 8 It is a structural diagram of the multi-level residual decoding module; Fig. 9 This is an experimental visualization comparison chart. DETAILED DESCRIPTION

[0022] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a polyp image segmentation method based on a double-branch feature progressive fusion network of the present invention in conjunction with the accompanying drawings.

[0023] like Figure 1 As shown, the present invention provides a polyp image segmentation method based on a dual-branch feature progressive fusion network, comprising the following steps: Step 1: Collect colonoscopic polyp images and polyp mask label datasets, and divide the datasets into training sets and test sets; The dataset includes five sub-datasets, of which 1450 images selected from two sub-datasets are used as training sets, 798 images are selected from five datasets as test sets, and finally all five datasets are used to evaluate the segmentation performance of the final model.

[0024] Step 2: Construct a polyp image segmentation model based on a dual-branch feature progressive fusion network; like Figure 2 As shown, in step 2, the proposed dual-branch feature progressive fusion network includes the following steps: Step 2.1: Use the PVTv2 backbone network as the multi-level feature extraction branch to extract four-stage multi-level features from the original input image T ={ T 1 , T 2 , T 3 , T 4}.

[0025] Step 2.2: Use a single-line gating mechanism to perform weighted feature extraction on the original input image and perform preliminary feature filtering.

[0026] like Figure 3 As shown, the single-line gating mechanism proposed in step 2.2 is applied to the original image of the input network X Perform preliminary feature extraction and filtering, first generate weight tensors through convolution and normalization operations W as , and then use the weight tensor after channel division ( i =1, 2) Perform weighted summation on the input features to obtain fusion features G f , the specific expression method is as follows:

[0027] in Softmax represents the normalized exponential function, div c Indicates the channel division operation on the tensor. represents element-wise multiplication, Represents addition of elements.

[0028] Step 2.3: Using the boundary information perception branch, high-fidelity boundary information is extracted from the features obtained in step 2.2 through hierarchical feature enhancement and cascade fusion strategies, and the polyp boundary segmentation mask is constructed to extract boundary feature information.

[0029] like Figure 4 As shown, the boundary information perception branch proposed in step 2.3 includes the following steps: Step 2.3.1: The boundary information perception branch contains three sets of cascaded reverse perception information layers. As a parallel branch to the multi-level feature extraction branch, the boundary information perception branch uses the original image of the input network as the perception target to extract boundary feature information. Each set of reverse perception information layers first performs feature extraction. E j ( j =1,2,3) and perform 3×3 convolution to obtain , then perform activation and inversion operations to obtain the inversion feature E rj , and E rj and Multiply elements to get the original boundary features E cj , the specific expression method is as follows:

[0030] In the formula express Sigmoid function, E is an all-one matrix.

[0031] Step 2.3.2: Together with E rj and E cj The two modules are input into the residual inverse enhancement module, and the features are fused by combining residual connection and convolution operation. E cj , and E rj , and finally obtain enhanced boundary-aware features with strong semantic information E xj , the specific expression method is as follows:

[0032] Where || represents the concatenation operation along the channel dimension, Conv 1 represents a 1×1 convolution operation, Conv 3 Represents a 3×3 convolution operation.

[0033] Step 2.4: Use the dislocation fusion module to fuse the adjacent level features in step 2.1 pair by pair to generate a new comprehensive feature representation, and use the dislocation single layer fusion module to fuse the bottom layer features. T 4 Extraction, using the channel and spatial attention fusion mechanism to refine and enhance high-level semantic feature information, thereby obtaining a new four-stage multi-level feature F ={ F 1 , F 2 , F 3 , F 4}.

[0034] like Figure 5 As shown in the figure, the dislocation fusion module proposed in step 2.4 is based on the adjacent level features T i and T i+1 ( i =1,2,3,4) as input, after convolution at two scales, the feature information is enriched by splicing operation, and the channel attention operation and spatial attention operation are used to obtain features with good information expression ability. M 2 ,Will M 2Self-splicing is performed in the channel dimension to construct a new feature representation, which doubles the number of channels of the output tensor, expands the representation capacity of the feature space while maintaining the integrity of the original feature information, and finally extracts local features through 3×3 convolution to obtain features with strong spatial context expression capabilities. F i , the specific expression method is as follows:

[0035] In the formula U represents the upsampling operation, Conv 5 represents a 5×5 convolution, s represents spatial attention, ca Represents channel attention.

[0036] like Figure 6 As shown in , the misaligned single-layer enhancement module proposed in step 2.4 is mainly composed of channel attention and spatial attention. i = 4, the staggered single-layer enhancement module uses a dual-branch splicing method of parallel attention operation and cascade attention operation to perform hierarchical features. T 4 The semantic features are enhanced, and finally the enhanced feature information after splicing is further extracted through 3×3 convolution to obtain the feature F 4 , the specific expression method is as follows:

[0037] Step 2.5: Further mine effective semantic information from the boundary feature information extracted in step 2.3, and combine the mined effective semantic information with the multi-level feature information through a multi-level cascade perception information fusion module. F Perform progressive global fusion to obtain boundary aggregation-aware features.

[0038] like Figure 7 As shown, in step 2.5, the proposed perceptual information fusion module uses hierarchical features F i and boundary-aware features E xj As input, the hierarchical features are globally fused with the boundary perception features through a multi-level cascade. The perception information fusion module first uses the channel attention operation to reduce the redundancy and noise in the boundary perception features, and adopts a dual-branch parallel structure to splice the two features in different orders to obtain the feature and , thus forming two feature combination branches. In the two branches, two convolution operations of different scales, 3×3 and 5×5, are used respectively, and the global average pooling layer and residual connection are used to further enhance the fusion features to obtain and , and finally the elements are added to obtain the perceptual features The specific expression method is as follows:

[0039] In the formula GAP Represents a global average pooling layer.

[0040] Step 2.6: Cross-level fusion of multi-level features using a multi-level residual decoding module F The features obtained at each level are fused and reduced in dimension by 1×1 convolution to participate in deep supervision during the training process.

[0041] like Figure 8 As shown, in step 2.6, the proposed multi-level residual decoding module includes the following steps: Step 2.6.1: The multi-level residual decoding module combines the output of the hierarchical perception information fusion module in a cascade manner. , boundary features E x3 The output features of the previous multi-level residual decoding module are used as input, and the features between different levels and the boundary information features are globally fused to finally obtain the fused features. D k ( k =1,2,3,4).

[0042] Step 2.6.2: In the multi-level residual decoding module, first and D k+1 , ( k =2,3) to perform element-by-element addition operation to obtain a new fused feature, which is then concatenated with the two original features to form a joint feature representation containing three features. , thereby enhancing the expressiveness of features. The specific expression method is as follows:

[0043] Step 2.6.3: Enhanced features The input is sent to two cascaded residual convolution modules, and a two-level residual connection is used to improve the semantic expression ability of the features. The specific expression method is as follows:

[0044] When k=1, the output features of the multi-level residual decoding module above the top-level perceptual information fusion module are D 2 And the output features of the boundary information perception branch E x3 As input; when k=4, the bottom layer perception information fusion module uses the output features of the bottom layer perception information fusion module F P32 and F P42 as input.

[0045] Step 3: Perform data enhancement on the training data obtained in step 1 and input it into the dual-branch feature progressive fusion network training model; In step 3, the boundary difference joint loss is used to optimize the model training process, which specifically includes the following steps: Step 3.1: Mask the real G and the prediction mask P As input, the neighborhood features of the real mask are extracted by customizing the edge convolution kernel to generate a tensor containing boundary information. T E , the specific expression method is as follows:

[0046] in Pad 01 Indicates that a layer of edge filling is performed with 0. Conv e Represents a custom edge convolution kernel, with:

[0047] Step 3.2: Calculate the ground-truth mask G and the boundary information tensor T E The relationship between them determines the parameters ɑ Used to adjust loss calculation, the specific expression method is as follows:

[0048] in N Tnz and N Gnz They are the boundary information tensors T E and the true mask G The number of non-zero elements in , using a very small constant to prevent errors caused by the denominator being zero.

[0049] Step 3.3: Calculate the intersection and sum of squares of the true mask and the predicted mask, and obtain the difference measure between the predicted value and the true value as the numerator, using ɑ As a variable parameter, the real mask G and the boundary information tensor T E The weighted value is used as the denominator to calculate the overall loss. The specific expression is as follows:

[0050] in sum ele Represents the sum of the internal elements of a tensor.

[0051] Step 4: Use the best model obtained after training to test the colonoscopy images in the test set, obtain the polyp segmentation results after the test, and evaluate the segmentation results of the model.

[0052] In step 4, the polyp images of the test set are input into the trained polyp image segmentation model, and the segmentation mask is output for evaluation.

[0053] The network proposed in this method is trained and tested using five classic polyp segmentation datasets: CVC-300, Kvasir, ColonDB, ETIS, and ClinicDB, and the test results are compared with the mainstream polyp segmentation algorithms in the prior art to illustrate the effectiveness of this method. The proposed network model is implemented using PyTorch and trained on an NVIDIA GeForce RTX 2080 SUPER GPU with 8GB of memory. The input of all images is uniformly set to 352×352, and the Adam optimization algorithm is used to optimize the overall parameters, with the learning rate set to 0.25e-4. The test results on the CVC-ClinicDB and Kvasir datasets are shown in Table 1: Table 1 Test results of polyp segmentation of the present invention and other 12 methods on CVC-ClinicDB and Kvasir datasets

[0054] The experimental results on the CVC-ClinicDB and CVC-ColonDB datasets are shown in Table 2: Table 2 Test results of polyp segmentation of the present invention and other 12 methods on CVC-300 and CVC-ColonDB datasets

[0055] The experimental results on the ETIS dataset are shown in Table 3: Table 3 Test results of the present invention and other 12 methods on polyp segmentation of ETIS dataset

[0056] The bold font indicates the best value in each row, and the underline indicates the second best result.

[0057] Among the five evaluation indicators, the first two are commonly used evaluation indicators in semantic segmentation tasks, and the closer the value of the indicator is to 1, the better the segmentation result is.

[0058] Fig. 9 This is a visualization comparison diagram of the experimental results of the method in the present invention and other methods. From the results, it can be concluded that the polyp segmentation method proposed in the present invention can obtain a segmentation result with more accurate boundaries and more complete semantic structures.

[0059] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A polyp image segmentation method based on a dual-branch feature progressive fusion network, characterized in that: The steps include: Step 1: Collect colonoscopic polyp images and polyp mask label datasets, and divide the dataset into training set and test set; Step 2: Construct a polyp image segmentation model based on a dual-branch feature progressive fusion network; Step 3: Perform data enhancement on the training data obtained in step 1 and input it into the dual-branch feature progressive fusion network training model; Step 4: Use the best model obtained after training to test the colonoscopy images in the test set, obtain the polyp segmentation results after the test, and evaluate the segmentation results of the model.

2. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 1 is characterized in that: The data set of step 1 includes five sub-datasets, among which 1450 images selected from two sub-datasets are used as training sets, 798 images are selected from five data sets as test sets, and finally all five data sets are used to evaluate the segmentation performance of the final model.

3. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 1 is characterized in that: The dual-branch feature progressive fusion network proposed in step 2 is used to perform the following steps: Step 2.1: Use the PVTv2 backbone network as the multi-level feature extraction branch to extract four-stage multi-level features from the original input image T ={ T 1 , T 2 , T 3 , T 4 }; Step 2.2: Use a single-line gating mechanism to perform feature weighted extraction on the original input image and perform preliminary feature filtering; Step 2.3: Using the boundary information perception branch, high-fidelity boundary information is extracted from the features obtained in step 2.2 through hierarchical feature enhancement and cascade fusion strategies, and the polyp boundary segmentation mask is constructed to extract boundary feature information; Step 2.4: Use the dislocation fusion module to fuse the adjacent level features in step 2.1 pair by pair to generate a new comprehensive feature representation, and use the dislocation single layer fusion module to fuse the bottom layer features. T 4 Extraction, using the channel and spatial attention fusion mechanism to refine and enhance high-level semantic feature information, thereby obtaining a new four-stage multi-level feature F ={ F 1 , F 2 , F 3 , F 4 }; Step 2.5: Further mine effective semantic information from the boundary feature information extracted in step 2.3, and combine the mined effective semantic information with the multi-level feature information through a multi-level cascade perception information fusion module. F Perform progressive global fusion to obtain boundary aggregation perception features; Step 2.6: Cross-level fusion of multi-level features using a multi-level residual decoding module F The features obtained at each level are fused and reduced in dimension by 1×1 convolution to participate in deep supervision during the training process.

4. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 3 is characterized in that: The single-line gating mechanism proposed in step 2.2 is used to input the original image of the network X Perform preliminary feature extraction and filtering, first generate weight tensors through convolution and normalization operations W as , and then use the weight tensor after channel division , ( i =1, 2) Perform weighted summation on the input features to obtain fusion features G f , the specific expression method is as follows: ， in Softmax represents the normalized exponential function, div c Indicates the channel division operation on the tensor. represents element-wise multiplication, Represents addition of elements.

5. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 3 is characterized in that: The boundary information perception branch proposed in step 2.3 includes the following steps: Step 2.3.1: The boundary information perception branch contains three sets of cascaded reverse perception information layers. As a parallel branch with the multi-level feature extraction branch, the boundary information perception branch uses the original image of the input network as the perception target to extract boundary feature information. Each set of reverse perception information layers first extracts the feature E j ( j =1,2,3) is obtained by 3×3 convolution, and then activation and inversion operations are performed to obtain the inverted features. E rj , and E rj and Multiply elements to get the original boundary features E cj , the specific expression method is as follows: ， In the formula express Sigmoid function, E is an all-one matrix; Step 2.3.2: Together with E rj and E cj The two modules are input into the residual inverse enhancement module, and the features are fused by combining residual connection and convolution operation. E cj , and E rj , and finally obtain enhanced boundary-aware features with strong semantic information E xj , the specific expression method is as follows: ， Where || represents the concatenation operation along the channel dimension, Conv 1 represents a 1×1 convolution operation, Conv 3 Represents a 3×3 convolution operation.

6. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 3 is characterized in that: The dislocation fusion module proposed in step 2.4 is based on the adjacent level features. T i and T i+1 ( i =1,2,3,4) as input, after convolution at two scales, the feature information is enriched by splicing operation, and the channel attention operation and spatial attention operation are used to obtain features with good information expression ability. M 2 ,Will M 2 Self-splicing is performed in the channel dimension to construct a new feature representation, which doubles the number of channels of the output tensor, expands the representation capacity of the feature space while maintaining the integrity of the original feature information, and finally extracts local features through 3×3 convolution to obtain features with strong spatial context expression capabilities. F i , the specific expression method is as follows: ， In the formula U represents the upsampling operation, Conv 5 represents a 5×5 convolution, s represents spatial attention, ca Represents channel attention.

7. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 3 is characterized in that: The misaligned single-layer enhancement module proposed in step 2.4 is composed of channel attention and spatial attention. i = 4, the staggered single-layer enhancement module uses a dual-branch splicing method of parallel attention operation and cascade attention operation to perform hierarchical features. T 4 The semantic features are enhanced, and finally the enhanced feature information after splicing is further extracted through 3×3 convolution to obtain the feature F 4 , the specific expression method is as follows: ， ， 。 8. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 3 is characterized in that: The perception information fusion module proposed in step 2.5 is based on the hierarchical features F i and boundary-aware features E xj As input, the hierarchical features are globally fused with the boundary perception features through a multi-level cascade method. The perception information fusion module first uses the channel attention operation to reduce the redundancy and noise in the boundary perception features, and adopts a dual-branch parallel structure to splice the two features in different orders to obtain the feature and , thus forming two feature combination branches, using 3×3 and 5×5 convolution operations of two different scales in the two branches respectively, and using the global average pooling layer and residual connection to further enhance the fusion features to obtain and , and finally the elements are added to obtain the perceptual features The specific expression method is as follows: ， In the formula GAP Represents a global average pooling layer.

9. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 3 is characterized in that: The multi-level residual decoding module proposed in step 2.6 comprises the following steps: Step 2.6.1: The multi-level residual decoding module combines the output of the hierarchical perception information fusion module in a cascade manner. , boundary features E x3 The output features of the previous multi-level residual decoding module are used as input, and the features between different levels and the boundary information features are globally fused to finally obtain the fused features. D k ( k =1,2,3,4); Step 2.6.2: In the multi-level residual decoding module, first and D k+1 , ( k =2,3) to perform element-by-element addition operation to obtain a new fused feature, which is then concatenated with the two original features to form a joint feature representation containing three features. , thereby enhancing the expressiveness of features. The specific expression method is as follows: ， Step 2.6.3: Enhanced features The input is sent to two cascaded residual convolution modules, and a two-level residual connection is used to improve the semantic expression ability of the features. The specific expression method is as follows: ， when k = 1, the output feature of the multi-level residual decoding module above the top-level perceptual information fusion module D 2 And the output features of the boundary information perception branch E x3 As input; when k =4, the bottom layer perception information fusion module uses the output features of the bottom layer perception information fusion module F P32 and F P42 as input.

10. The polyp image segmentation method based on a dual-branch feature progressive fusion network according to claim 1, characterized in that: In step 3, the model training process is optimized using the boundary difference joint loss, which specifically includes the following steps: Step 3.1: Mask the real G and the prediction mask P As input, the neighborhood features of the real mask are extracted by customizing the edge convolution kernel to generate a tensor containing boundary information. T E , the specific expression method is as follows: ， in Pad 01 Indicates that a layer of edge filling is performed with 0. Conv e Represents a custom edge convolution kernel, with: ， Step 3.2: Calculate the ground-truth mask G and the boundary information tensor T E The relationship between them determines the parameters ɑ Used to adjust loss calculation, the specific expression method is as follows: ， in N Tnz and N Gnz They are the boundary information tensors T E and the true mask G The number of non-zero elements in , using a very small constant to prevent errors caused by the denominator being zero; Step 3.3: Calculate the intersection and sum of squares of the true mask and the predicted mask, and obtain the difference measure between the predicted value and the true value as the numerator, using ɑ As a variable parameter, the real mask G and the boundary information tensor T E The weighted value is used as the denominator to calculate the overall loss. The specific expression is as follows: ， in sum ele Represents the sum of the internal elements of a tensor.

Images