A deep polyp segmentation method and system based on multi-view fusion

Abstract

This invention relates to the field of medical image segmentation technology, and more specifically, to a method and system for deep polyp segmentation based on multi-view fusion. The invention discloses a method for deep polyp segmentation based on multi-view fusion. On one hand, it processes the original colonoscopy image through multi-angle switching to form multi-view input images, exploring complementary contextual feature associations from these images to output a good segmentation result. On the other hand, the invention extracts multi-scale features for each view and introduces a cross-gating strategy for staged decoding, thereby recursively processing all multi-scale features. The invention combines the results of each view with the generated weights to comprehensively and collaboratively perform segmentation prediction. Simulation comparisons show that this method achieves robust polyp segmentation and good performance on standard datasets, with improved accuracy metrics.

Description

Technical Field

[0001] This invention relates to the field of medical image segmentation technology, and more specifically, to a method and system for deep polyp segmentation based on multi-view fusion. Background Technology

[0002] Polyp segmentation, as a computer vision task, can automatically segment polyp areas in images or videos, reducing the false negative rate of polyps during colonoscopy.

[0003] Currently, an increasing number of researchers are using convolutional neural network-based models for polyp segmentation, such as U-Net, Swin-Unet, MSCAF-Net, and UCTransNet. However, existing models mainly focus on controlling upper-level features or combining features. After analysis, the inventors found that the accuracy of polyp segmentation guided by this approach has reached a bottleneck. Summary of the Invention

[0004] Therefore, it is necessary to address the bottleneck in polyp segmentation accuracy caused by existing models focusing on upper-level features or combined features. This leads to a method and system for deep polyp segmentation based on multi-view fusion.

[0005] This invention is achieved using the following technical solution:

[0006] In a first aspect, the present invention discloses a deep polyp segmentation method based on multi-view fusion, comprising the following steps:

[0007] Build and train the encoder-decoder network model;

[0008] Acquire colonoscopy images to be processed;

[0009] The colonoscopy images are transformed from multiple perspectives to obtain V input images I. 1 ~I V ;

[0010] Using a trained encoder-decoder network model to process input images I from V viewpoints 1 ~I V The polyp segmentation image P is obtained by segmentation processing. final .

[0011] The encoding / decoding network model includes: encoder, decoder, and feature aggregation.

[0012] The encoder unit is used to process the input image I from the v-th viewpoint. v Feature extraction was performed, resulting in feature maps at four different scales. Among them, the four feature maps of different scales are sorted from largest to smallest as follows:

[0013] The decoder unit includes a feature fusion unit and a prediction head. The feature fusion unit fuses four feature maps of different scales in ascending order of scale to obtain intermediate feature maps. in, and Same scale, and Same scale, F1 v and Same scale. The predicted head is used based on... The prediction yielded the auxiliary prediction map P3. v According to F2 v The prediction yielded the auxiliary prediction map P2. v According to F1 v The prediction yields the auxiliary prediction map P1. v .

[0014] The segmentation prediction unit includes a view unification unit and a feature aggregation unit. The view unification unit is used to integrate the F1 score... 1 ~F1 V P1 1 ~P1 V Adjust the viewing angle to unify it into the same viewpoint; among them, F1 1 ~F1 V The corresponding auxiliary prediction map is obtained. P1 1 ~P1 V The corresponding auxiliary prediction map is obtained. Feature aggregation section is used for based on Calculate the weight matrix W, and then apply the weight matrix W to... Aggregation yields polyp segmentation map P final .

[0015] This deep polyp segmentation method based on multi-view fusion implements the method or process according to embodiments of this disclosure.

[0016] Secondly, the present invention discloses a deep polyp segmentation system based on multi-view fusion, which uses the deep polyp segmentation method based on multi-view fusion of the first aspect.

[0017] The deep polyp segmentation system based on multi-view fusion includes: an encoding / decoding network module, a colonoscopy image input module, a multi-view processing module, and a polyp segmentation module.

[0018] The encoding / decoding network module is used to build and train the encoding / decoding network model. The colonoscopy image input module is used to acquire the colonoscopy images to be processed. The multi-view processing module is used to transform the colonoscopy images from multiple perspectives to obtain V input images I. 1 ~IV The polyp segmentation module is used to process V viewpoints of input image I using a trained encoder-decoder network model. 1 ~I V The polyp segmentation image P is obtained by segmentation processing. final .

[0019] Thirdly, the present invention discloses a readable storage medium. This readable storage medium stores computer program instructions that, when read and executed by a processor, perform the steps of the deep polyp segmentation method based on multi-view fusion disclosed in the first aspect.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] This invention employs a research approach different from existing polyp segmentation methods: On one hand, based on the human tendency to recognize multiple views, it utilizes different perspectives of the same object for better identification. This involves switching the original colonoscopy image from multiple angles to create multi-view input images, exploring complementary contextual feature associations from these images to output a better segmentation result. On the other hand, this invention extracts multi-scale features for each view and introduces a cross-gating strategy for staged decoding, recursively processing all multi-scale features to achieve robust view-specific feature fusion and mask prediction. Furthermore, this invention comprehensively and collaboratively performs segmentation prediction by combining the results of each view with the generated weights. It can adaptively fuse multi-view prediction maps for the final prediction through soft attention learning weights between different views. Simulation results demonstrate that this method achieves robust polyp segmentation and good performance on standard datasets, with improved accuracy metrics. Attached Figure Description

[0022] Figure 1 This is a flowchart of the deep polyp segmentation method based on multi-view fusion in Embodiment 1 of the present invention;

[0023] Figure 2 for Figure 1 The overall structure of the Chinese encoding / decoding network model;

[0024] Figure 3 for Figure 2 Data flow diagram of the encoder and decoder sections;

[0025] Figure 4 for Figure 3 Data flow diagram of the first-level cross-gating fusion unit;

[0026] Figure 5 The intermediate feature map F3 corresponds to the three different instance images in Embodiment 1 of the present invention. v ~F1v ;

[0027] Figure 6 This refers to the auxiliary prediction map P1 corresponding to the 0° and 180° images of two different instance images in Embodiment 1 of the present invention. v ;

[0028] Figure 7 for Figure 2 Data flow diagram of the feature aggregation section;

[0029] Figure 8 This is a diagram showing the results of a comparative experiment between the method of Example 1 and existing methods in Example 2 of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] It should be noted that when a component is said to be "installed on" another component, it can be directly on the other component or it may be in a component that is centered on it. When a component is said to be "set on" another component, it can be directly set on the other component or it may also be in a component that is centered on it. When a component is said to be "fixed to" another component, it can be directly fixed to the other component or it may also be in a component that is centered on it.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0033] Example 1

[0034] Please see Figure 1 , Figure 1 This is a flowchart of the deep polyp segmentation method based on multi-view fusion in this invention.

[0035] In summary, the deep polyp segmentation method based on multi-view fusion includes the following steps:

[0036] Step 1: Build and train the encoder-decoder network model;

[0037] This step aims to obtain the most accurate encoder-decoder network model, which requires training. Specific training methods are detailed in Example 2 and will not be elaborated upon here.

[0038] Step 2: Obtain the colonoscopy images to be processed.

[0039] It should be noted that there is no strict order between steps one and two; they can be swapped or performed simultaneously.

[0040] Step 3: Perform multi-view transformations on the colonoscopy images to obtain V input images I. 1 ~I V .

[0041] For a given original colonoscopy image, horizontal, vertical, and diagonal transformations can be performed to obtain at least three images with different viewing angles. Therefore, including the original colonoscopy image, there are at least four input images with different viewing angles. 1 ~I 4 Of course, other perspective changes can be made to obtain a greater number of input images. Therefore, V≥2.

[0042] Step four: Use the trained encoder-decoder network model to process the input images I from V viewpoints. 1 ~I V The polyp segmentation image P is obtained by segmentation processing. final .

[0043] It should be noted that while a larger V value theoretically leads to higher prediction and segmentation accuracy, it also increases the computational load of the entire model, potentially resulting in a situation where there is no significant improvement in accuracy but computational overload. Experiments have shown that a V value of 3 can achieve very good prediction accuracy.

[0044] See Figure 2 The encoding / decoding network model includes: encoder, decoder, and feature aggregation.

[0045] The following is a description of each module:

[0046] (1) The encoder unit is used to process the input image I at the vth viewpoint. v Feature extraction was performed, resulting in feature maps at four different scales. v∈[1,V].

[0047] The four feature maps at different scales are sorted from largest to smallest as follows:

[0048] See Figure 3In this embodiment 1, the encoder section uses a PVT-v2 network, which can process the input image to obtain feature maps of four different scales, namely... (i.e., X1) v ), (Right now ), (i.e., X3) v ), (Right now Where H is the length of the input image, W is the width of the input image, and C1, C2, C3, and C4 are the number of channels of the four feature maps, respectively.

[0049] (2) The decoder section includes the feature fusion section and the prediction head.

[0050] The feature fusion unit is used to fuse four feature maps of different scales in ascending order of scale to obtain the intermediate feature map F3. v ~F1 v .

[0051] Specifically, the feature fusion unit includes a three-level cross-gate fusion unit. The first-level cross-gate fusion unit is used to... and Merge into F3 v The second-level cross-gating fusion unit is used to integrate F3. v and Merge into F2 v The third-level cross-gating fusion unit is used to integrate F2. v With X1 v merged into F1 v .

[0052] It is important to note that F3 v With X3 v Same scale, F2 v With X2 v Same scale, F1 v With X1 v Same scale.

[0053] In other words, it uses the feature map at the smallest scale as a basis and recursively merges the feature maps at the next higher level until it processes the feature map at the largest scale. In each merging process, a first-level cross-gated fusion unit is used to selectively combine these features to generate a more refined intermediate feature map. Then, another level of cross-gated fusion unit is used to merge this intermediate feature map with the feature map at the next scale. The final output feature is the intermediate feature map after passing through three levels of cross-gated fusion units.

[0054] For F3 v F2 v F1v The operation of the corresponding level cross-gating fusion unit is similar. See below. Figure 4 With F3 v For example, let's explain in detail:

[0055] right Perform upsampling to generate and Intermediate feature maps of the same scale Right now UP(.) indicates upsampling. Perform 1×1 convolution to obtain intermediate feature maps. Right now C1(.) represents a 1×1 convolution. First, calculate... Perform 1×1 convolution, then apply the Sigmoid activation function to obtain the attention map. To highlight Key information; namely σ(.) represents the Sigmoid activation function. and Perform residual connections to generate intermediate feature maps. Right now ⊙ represents the Hadamard.

[0056] right Perform 1×1 convolution to obtain intermediate feature maps. Right now First to Perform 1×1 convolution, then apply the Sigmoid activation function to obtain the attention map. To highlight Key information; namely σ(.) represents the Sigmoid activation function. and Perform residual connections to generate intermediate feature maps. Right now ⊙ represents the Hadamard.

[0057] Then The concatenation is followed by sequential processing using 1×1 convolution, ReLU activation function, and 1×1 convolution to obtain F3. v .Right now [.||.] indicates splicing according to the channel dimension.

[0058] The above process can be expressed by the following formula:

[0059]

[0060] F2 v F1 v With F3v Similarly, the specific process will not be repeated here.

[0061] F2 v The calculation formula is:

[0062]

[0063] F1 v The calculation formula is:

[0064]

[0065] Through the above operations, we progressively obtain increasingly stronger intermediate feature maps, thus continuously improving the quality of the theory. (See also...) Figure 5 The example demonstrates how three different input images are processed to obtain intermediate feature maps F3. v ~F1 v It can be seen that the intermediate feature map F3 v F2 v F1 v The difference from the theoretical segmentation map gradually decreases, verifying the function of the feature fusion unit in this model.

[0066] For predicting the head, it is used based on F3 v The prediction yielded the auxiliary prediction map P3. v According to F2 v The prediction yielded the auxiliary prediction map P2. v According to F1 v The prediction yields the auxiliary prediction map P1. v .

[0067] The prediction head is shared. That is, the prediction head is connected to the three-level cross-gated fusion section. Specifically, the prediction head contains a 1×1 convolutional layer. In this way, the output of the three-level cross-gated fusion section is also input into the prediction head, generating an auxiliary prediction map for supervision.

[0068] In summary, the decoder can combine features from adjacent scales, and each level of cross-gated fusion unit obtains segmentation results at different scales through a shared prediction head, and outputs intermediate feature maps and their corresponding auxiliary prediction maps.

[0069] Additionally, see Figure 6 The diagram shows auxiliary prediction maps P1 corresponding to the 0° and 180° views of two different instance images. v The 0° image represents the original image with no change in perspective, while the 180° image represents the image obtained by changing the perspective of the original image by 180°. Figure 6 It can be seen that the F1 obtained from the 0° diagram v F2 obtained from the 180° diagramv The two have different key features, which also verifies the rationality of using different views as input graphs in this method.

[0070] (3) The segmentation prediction unit includes the view unification unit and the feature aggregation unit.

[0071] For the perspective unification part, the perspective unification part is used to unify F1 1 ~F1 V P1 1 ~P1 V Adjust the viewing angle to unify it into the same viewpoint; among them, F1 1 ~F1 V Corresponding to P1 1 ~P1 V Corresponding to

[0072] Because the input images are from different perspectives I 1 ~I V Therefore, the F1 score is obtained. 1 ~F1 V P1 1 ~P1 V The perspectives are also different. These images need to be standardized to the same perspective (generally using the perspective of the original colonoscopy image) to facilitate subsequent operations.

[0073] For the feature aggregation section, the feature aggregation section is used based on Calculate the weight matrix W, and then apply the weight matrix W to... Aggregation yields polyp segmentation map P final .

[0074] This is because different viewpoints contribute differently to the final predicted segmentation, therefore, it is necessary to fuse the intermediate feature maps from all viewpoints. Auxiliary prediction map It should achieve adaptive fusion, rather than simple average pooling or max pooling.

[0075] See Figure 7 The feature aggregation unit is equipped with a soft two-layer FCN network to generate view-specific weights for predicting aggregations: the input of the first layer is... The feature map is obtained by concatenating the channels; the output of the first layer is further input into the second 1×1 V filter convolutional layer, and finally the weight matrix W is obtained, which can be expressed by the following formula:

[0076]

[0077] In the formula, ρ(.) represents the softmax activation function; C1(.) represents the 1×1 convolution calculation; C3(.) represents the 3×3 convolution calculation;

[0078] Then W is split into V weighted graphs along the channel dimension, which are used for the final polyp segmentation graph P. final The generation of polyps. (P) final It can be expressed by the following formula:

[0079]

[0080] In the formula, W v This represents the v-th auxiliary prediction graph. The weight.

[0081] Through the above operations, colonoscopy images can be segmented and predicted to obtain a highly accurate polyp segmentation image P. final .

[0082] Simultaneously, this embodiment 1 also discloses a deep polyp segmentation system based on multi-view fusion, which uses the above-mentioned deep polyp segmentation method based on multi-view fusion.

[0083] The deep polyp segmentation system based on multi-view fusion includes: an encoding / decoding network module, a colonoscopy image input module, a multi-view processing module, and a polyp segmentation module.

[0084] The encoding / decoding network module is used to build and train the encoding / decoding network model. The colonoscopy image input module is used to acquire the colonoscopy images to be processed. The multi-view processing module is used to transform the colonoscopy images from multiple perspectives to obtain V input images I. 1 ~I V The polyp segmentation module is used to process V viewpoints of input image I using a trained encoder-decoder network model. 1 ~I V The polyp segmentation image P is obtained by segmentation processing. final .

[0085] The polyp segmentation module includes an encoder submodule, a decoder submodule, and a segmentation prediction submodule. The encoder submodule is used to process the input image I from the v-th viewpoint. v Feature extraction was performed, resulting in feature maps at four different scales. The decoder submodule includes a feature fusion unit and a prediction head unit. The feature fusion unit fuses four feature maps of different scales in ascending order of scale to obtain the intermediate feature map F3. v ~F1 v The prediction head unit is used to predict based on F3. v The prediction yielded the auxiliary prediction map P3. v According to F2 vThe prediction yielded the auxiliary prediction map P2. v According to F1 v The prediction yields the auxiliary prediction map P1. v The segmentation prediction submodule includes a viewpoint unification unit and a feature aggregation unit. The viewpoint unification unit is used to integrate the F1 score... 1 ~F1 V P1 1 ~P1 V Adjust the perspective to unify the intermediate feature maps to the same viewpoint. Auxiliary prediction map Feature aggregation section is used for based on Calculate the weight matrix W, and then apply the weight matrix W to... Aggregation yields polyp segmentation map P final .

[0086] Using the above-mentioned deep polyp segmentation method / system based on multi-view fusion, accurate feature analysis can be performed on colonoscopy images, and accurate polyp segmentation maps can be obtained quickly, providing correct references for clinical practice, improving clinical efficiency, and demonstrating strong practicality.

[0087] Example 2

[0088] For untrained encoder-decoder network models, they need to be trained to obtain trained encoder-decoder network models, which can then be applied to deep polyp segmentation methods based on multi-view fusion.

[0089] This embodiment 2 discloses the training method of the encoding / decoding network model in embodiment 1.

[0090] During the training phase, the working principle of each module of the encoder-decoder network model remains unchanged, but full supervision is introduced, and a loss function is calculated to update the network parameters in reverse.

[0091] Specifically, the training methods for encoder-decoder network models include:

[0092] S1.1 Obtain the polyp image sample dataset. The polyp image sample dataset contains the true labels of the samples.

[0093] The polyp image sample dataset can be any one, several, or a combination of five standard datasets (Kvasir-SEG, ClinicDB, ColonDB, ETIS, Endoscene).

[0094] The images in the polyp image sample dataset were resized to 352×352 and normalized using mean {0.485, 0.456, 0.406} and variance {0.229, 0.224, 0.225}; each image was then flipped along the horizontal and diagonal directions to form a three-view input.

[0095] S1.2 divides the polyp image sample dataset into a training set and a test set.

[0096] S1.3, use the training set to train the codec network model, and use the test set to verify the performance of the trained codec network model.

[0097] In this second embodiment, the Adam optimizer is used, and the learning rate is set to 5e. -5 .

[0098] The three views of the training set are input into the encoding / decoding network model. During training, the three auxiliary prediction maps P1 are used. v ~P3 v Perform supervision and calculate the loss function L. total Network parameters are updated via backpropagation.

[0099] Wherein, the loss function L total It consists of two parts: one part is the polyp segmentation map P obtained through training. final Compared to the loss L(P) of the true labels on the training set samples final The other part is the auxiliary prediction map P1 obtained through training. v ~P3 v Compared to the loss L(P) of the true labels on the training set samples i v The loss function Ltotal can be expressed by the formula:

[0100]

[0101] The polyp images from the test set are input into the encoder-decoder network model after each round of training, and then the polyp segmentation images P obtained from the test are processed. final The model's performance—accuracy—is validated by comparing it with the true labels of the samples in the test set and calculating metrics.

[0102] Among them, six indicators are generally used: mDSC, mIoU, S α mE ξ MAE. Among them, the higher the first five indicators, the better, and the lower the last indicator, the better.

[0103] S1.4: Obtain the best-performing codec network model from S1.3 as the trained codec network model.

[0104] After multiple iterations, the loss converges, meaning the model converges. Generally, testing on the test set at this point yields optimal performance. However, if performance is poor, it indicates ineffective model training, and the model parameters should be modified and retrained according to steps S1.3–S1.4.

[0105] In this second embodiment, the performance of the model from the first embodiment is compared with that of existing models (U-Net, PraNet, SANet, Polyp-PVT, TGANet, SSFormer, CFA-Net, MSCAF-Net):

[0106] The above nine models were trained and tested using Kvasir-SEG, ClinicDB, ColonDB, ETIS, and Endoscene respectively. The results are shown in the comparison chart. Figure 8 ,Depend on Figure 8 It can be seen that the model in Example 1 is closest to the theoretical segmentation graph (GT); the comparison results of the indicators are shown in Tables 1 to 5.

[0107] Table 1. Comparison of metrics for the nine models on the Kvasir-SEG dataset.

[0108]

[0109] Table 2. Comparison of metrics for the nine models on the ClinicDB dataset.

[0110]

[0111]

[0112] Table 3. Comparison of metrics for the nine models on the ColonDB dataset.

[0113]

[0114] Table 4. Comparison of metrics for the nine models on the ETIS dataset.

[0115]

[0116] Table 5. Comparison of metrics for the nine models on the Endoscene dataset.

[0117]

[0118]

[0119] As shown in Tables 1-5, the model in Example 1 achieves the best performance metrics on the Kvasir-SEG and ClinicDB datasets, surpassing the other eight models. While the model in Example 1 performs slightly worse on the ColonDB, ETIS, and Endoscene datasets compared to the Kvasir-SEG and ClinicDB datasets, it still outperforms the other eight models on most metrics. In summary, the model of this invention achieves robust polyp segmentation and good performance, improves accuracy metrics to a certain extent, and demonstrates good generalization ability.

[0120] Example 3

[0121] This embodiment 3 discloses a readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, they perform the steps of the deep polyp segmentation method based on multi-view fusion disclosed in embodiment 1, or / and the steps of the training method of the encoding and decoding network model disclosed in embodiment 2.

[0122] When applying the methods in Examples 1 and 2, they can be applied in software form, such as by designing a program that can run independently on a computer-readable storage medium. The computer-readable storage medium can be a USB flash drive, designed as a USB security token, and the program can be designed to start the entire method through an external trigger.

[0123] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A deep polyp segmentation method based on multi-view fusion, comprising the following steps: constructing and training a coding-decoding network model; obtaining a colonoscopy image to be processed; characterized in that The colonoscopy images are changed in multi-view to obtain input images of V I 1 I V ​​​ The trained coding-decoding network model is used to process input images of the view angle V I 1 I V Segmentation processing is performed to obtain a polyp segmentation map P final ​​​ wherein the coding-decoding network model comprises: An encoder unit is configured to perform feature extraction on the input image of the first view to obtain four feature maps of different scales v I v v V ​​​​​​​​​​ The decoder part comprises a feature fusion part and a prediction head; the feature fusion part is used for fusing feature maps of four different scales in order of scale from small to large to obtain intermediate feature maps ~ ; wherein, The same scale as The same scale as The same scale as The same scale as The same scale as ; the prediction head is used for obtaining auxiliary prediction maps according to , according to , according to ; the feature fusion part comprises a three-stage cross-gate fusion part; wherein, the first-stage cross-gate fusion part is used for fusing into ; the second-stage cross-gate fusion part is used for fusing into ; the third-stage cross-gate fusion part is used for fusing into ;​​​​​​ and The segmentation prediction unit comprises a view angle unification unit and a feature aggregation unit; the view angle unification unit is configured to unify the intermediate feature maps into intermediate feature maps of the same view angle by performing view angle adjustment W W P final .​​​​​​​​​​​​​​​​​​​​​​ 2.The deep polyp segmentation method based on multi-view fusion according to claim 1, characterized in that, V ≥2。 3.The deep polyp segmentation method based on multi-view fusion according to claim 1, characterized in that, the encoder part is a PVT-v2 network. 4.The deep polyp segmentation method based on multi-view fusion according to claim 1, characterized in that, The calculation formula is: ； The calculation formula is: ； The calculation formula is: ； In the formulae, , , , , , represent intermediate features; C1(.) denotes a 1 1 convolution calculation; ReLU(.) denotes a ReLU activation function; denotes concatenation along the channel dimension; UP(.) denotes up-sampling; σ (.) denotes a Sigmoid activation function; and denotes a Hadamard product.

5. The multi-view fusion based deep polyp segmentation method of claim 4, wherein, the prediction head and the 3-level cross-gated fusion part are connected respectively; the prediction head contains a 1x1 convolution layer. 6.The deep polyp segmentation method based on multi-view fusion according to claim 1, characterized in that, W The calculation formula is: ； In the formula, ρ (.) denotes a softmax activation function; C1(.) denotes a 1 x 1 convolution calculation; C3(.) denotes a 3 x 3 convolution calculation; P final The calculation formula is: ； In the formula, W v denotes the weight of the v auxiliary prediction picture . 7.The deep polyp segmentation method based on multi-view fusion according to claim 1, characterized in that, the training method of the coding-decoding network model comprises: S1.1, obtaining a polyp image sample dataset; S1.2, dividing the polyp image sample dataset into a training set and a test set; S1.3, training the coding-decoding network model using the training set and verifying the performance of the trained coding-decoding network model using the test set; S1.4, obtaining the coding-decoding network model with the best performance in S1.3 as the trained coding-decoding network model.

8. A multi-view fusion based deep polyp segmentation system, characterized in that, It uses the deep polyp segmentation method based on multi-view fusion as claimed in any one of claims 1-7; the deep polyp segmentation system based on multi-view fusion comprises: a coding-decoding network module for constructing and training a coding-decoding network model; a colonoscopy image input module for obtaining a colonoscopy image to be processed; a multi-view processing module, configured to perform multi-view conversion on the colonoscopy image to obtain an input image of V I 1 I V ​​​ and a polyp segmentation module configured to segment the input image of the view using the trained encoder-decoder network model to obtain a polyp segmentation map V I 1 I V P final ​​​​ 9. A readable storage medium, characterized by, the computer program instructions stored in the readable storage medium are read and run by a processor, and perform the steps of the deep polyp segmentation method based on multi-view fusion as claimed in any one of claims 1-7.

Images