Bidirectional enhanced network system for colon polyp segmentation and colon polyp segmentation method

Abstract

The application belongs to the field of computer vision, and discloses a bidirectional enhancement network system for colon polyp segmentation and a colon polyp segmentation method. The system comprises: a backbone network module for extracting low-level features including first and second low-level features and high-level features including first and second high-level features; an edge highlighting module for generating first and second guided low-level features by using the two high-level features to bidirectionally guide the low-level features through a double guide submodule; a first detail-context extraction submodule for extracting the inner boundary details and the inter-feature context information of the two guided low-level features respectively to generate first and second re-extracted features; a self-adaptive gating mechanism submodule for adaptively fusing the two features to output first and second boundary features; a salient edge generation module for fusing the two boundary features with the two high-level features respectively through a second detail-context extraction submodule to generate first and second enhanced high-level features; and a decoding segmentation module for fusing the boundary features and the enhanced high-level features to decode a polyp segmentation map.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a bidirectional enhancement network system for colon polyp segmentation and a colon polyp segmentation method. BACKGROUND

[0002] Colorectal cancer (CRC) is one of the malignant tumors with high incidence and mortality worldwide. Clinical practice shows that early detection and resection of precancerous lesions (i.e. colon polyps) through colonoscopy is the most effective strategy to reduce the incidence and mortality of colorectal cancer. However, during colonoscopy, due to the various shapes and sizes of polyps, and often affected by the complex intestinal environment, the automatic polyp segmentation task faces great challenges.

[0003] Among them, the fuzzy polyp boundary is the key technical bottleneck leading to the decline in segmentation accuracy. This problem is usually caused by the following three situations:

[0004] (1) Low contrast: the color and texture of polyp tissue are highly similar to the surrounding normal intestinal mucosa tissue, resulting in low visual differentiation.

[0005] (2) Poor imaging quality: unstable lighting conditions or rapid movement of the lens during endoscopy can cause local overexposure, underexposure or motion blur of the image, making the boundary between the polyp and the normal tissue unclear.

[0006] (3) Complex background environment: complex background structures such as folds, blood vessels and residues in the intestine may be confused with the polyp edge, causing serious interference to the accurate identification of the boundary.

[0007] To address the above challenges, existing technologies mainly adopt deep learning-based methods, especially architectures based on convolutional neural networks (CNN) or Transformers, such as U-Net, SSformer, etc. In addition, some methods are specifically designed to address the boundary blur problem, which can be roughly divided into three categories: the first category is to capture the boundary by designing specific modules, such as the reverse attention module of PraNet and the bilateral boundary extraction module of BCNet; the second category is to strengthen the supervision of boundary pixels by designing boundary-aware loss functions, such as the bdm loss of BDGNet; the third category is to set up a special branch network to extract boundary features, such as the gating mechanism branch of CFANet.

[0008] Although the above methods have achieved certain results in improving the clarity of the segmentation boundary, they generally have the problem of insufficient utilization of polyp edge information. Specifically, these methods usually extract boundary features from images in one direction or make simple boundary corrections at the decoding stage, lacking a mechanism for effective interaction and mutual enhancement between boundary information and deep and shallow features in the network. Therefore, when facing the above three typical boundary blur situations, the performance of existing methods is still limited, and it is difficult to achieve accurate segmentation in all scenarios. SUMMARY

[0009] Based on this, the present application provides a bidirectional enhancement network method, device and medium for colon polyp segmentation, aiming to solve the problem of polyp boundary blur caused by low contrast, poor imaging quality and complex background environment.

[0010] In a first aspect, the present application provides a bidirectional enhancement network system for colon polyp segmentation, comprising:

[0011] A backbone network module for encoding the acquired colonoscopy image data to extract low-level features including first and second low-level features, and high-level features including first and second high-level features;

[0012] An edge highlighting module, comprising:

[0013] A dual guidance sub-module for dual guidance of the low-level features using the first and second high-level features, respectively, to obtain first and second guided low-level features;

[0014] A first detail-context extraction sub-module for capturing intra-feature boundary details and inter-feature context information from the first and second guided low-level features, respectively, to generate first and second re-extracted guided low-level features;

[0015] An adaptive gating mechanism sub-module for applying an adaptive gating mechanism to the first and second re-extracted guided low-level features to obtain first and second boundary features;

[0016] A salient edge generation module, comprising a second detail-context extraction sub-module for fusing the first and second boundary features with the first high-level feature to obtain a first enhanced high-level feature, and fusing the first and second boundary features with the second high-level feature to obtain a second enhanced high-level feature;

[0017] a decoding segmentation module configured to generate a polyp segmentation map based on the first boundary feature, the second boundary feature, the first enhanced high-level feature, and the second enhanced high-level feature through a decoding operation.

[0018] As an optional implementation form of the first aspect of the present application, in the dual-guiding sub-module, the first guiding low-level feature and the second guiding low-level feature are obtained by: performing upsampling processing on the first high-level feature, then performing element-wise multiplication with the first low-level feature, and then performing element-wise addition of the multiplication result and the first low-level feature to obtain the first guiding low-level feature; and performing upsampling processing on the second high-level feature, then performing element-wise multiplication with the second low-level feature, and then performing element-wise addition of the multiplication result and the second low-level feature to obtain the second guiding low-level feature.

[0019] As an optional implementation form of the first aspect of the present application, in the first detail-context extraction sub-module, the first re-extracted guiding low-level feature is generated in the same way as the second re-extracted guiding low-level feature, wherein the first re-extracted guiding low-level feature is generated by: inputting the first guiding low-level feature into a parallel structure including an upper branch and a lower branch for processing: the upper branch extracts intra-feature boundary details of the guiding low-level feature through stacked convolution operations; the lower branch extracts inter-feature context information of the guiding low-level feature through stacked self-attention mechanisms; the intra-feature boundary details output by the upper branch and the inter-feature context information output by the lower branch are merged to obtain fused features; the reverse attention operation is applied to the fused features to generate a reverse mask; and based on the reverse mask and the fused features, the intra-feature boundary details and the inter-feature context information are enhanced to extract boundary regions in the foreground view and the background view to obtain the first re-extracted guiding low-level feature.

[0020] As an optional implementation form of the first aspect of the present application, in the adaptive gating mechanism sub-module, the first boundary feature and the second boundary feature are obtained by: after merging the first re-extracted guiding low-level feature and the second re-extracted guiding low-level feature, calculating adaptive weights for the first re-extracted guiding low-level feature and the second re-extracted guiding low-level feature through a global average pooling operation, a fully connected layer, and a Softmax function; and performing weighted summation on the first re-extracted guiding low-level feature and the second re-extracted guiding low-level feature according to the adaptive weights to obtain the first boundary feature and the second boundary feature.

[0021] As an optional implementation of the first aspect of the application, in the significant edge generation module, the first enhanced high-level feature and the second enhanced high-level feature are obtained in the same way, and the way of obtaining the first enhanced high-level feature comprises: performing down-sampling processing on the first boundary feature and the second boundary feature respectively, merging the down-sampled first boundary feature and the second boundary feature with the first high-level feature in the channel dimension to obtain the first fused high-level feature after the first boundary detail enhancement; inputting the first fused high-level feature into the second detail-context extraction submodule for feature extraction to obtain the first fused high-level feature after the second boundary detail enhancement; integrating the first fused high-level feature after the second boundary detail enhancement into the first high-level feature through the convolution block attention submodule to obtain the first enhanced high-level feature.

[0022] As an optional implementation of the first aspect of the application, in the decoding segmentation module: based on the first boundary feature, the second boundary feature, the first enhanced high-level feature and the second enhanced high-level feature, features are gradually up-sampled and fused to generate four polyp segmentation maps of different resolutions.

[0023] As an optional implementation of the first aspect of the application, it further includes a result output module, which trains the bidirectional enhancement network system in a deep supervision manner, and finally obtains a polyp segmentation result by cascading and aggregating the four polyp segmentation maps.

[0024] In a second aspect, the embodiments of the application provide a colon polyp segmentation method, which comprises: acquiring colonoscopy image data to be processed, inputting the colonoscopy image data to be processed into the bidirectional enhancement network system as described in the first aspect, and obtaining a polyp segmentation map corresponding to the colonoscopy image data to be processed.

[0025] In a third aspect, the embodiments of the application provide an electronic device, which comprises a processor, a memory, and a program or instructions stored on the memory and executable on the processor, and the program or instructions are executed by the processor to implement the steps of the method as described in the first aspect.

[0026] In a fourth aspect, the embodiments of the application provide a readable storage medium, and the readable storage medium stores a program or instructions, and the program or instructions are executed by the processor to implement the steps of the method as described in the first aspect.

[0027] Compared with the prior art, the application has the following remarkable beneficial effects:

[0028] (1) An innovative bidirectional enhancement network system (BE-Net) is proposed, which realizes the effective interaction and mutual enhancement of low-order boundary details and high-order semantic information through the collaborative work of edge prominence module (EPM) and salient edge generation module (DEGM). This bidirectional information flow mechanism can make more full use of the edge clues of the image, and effectively overcome the defects of the existing methods that do not make full use of the edge information.

[0029] (2) The edge prominence module (EPM) designed can densely mine and refine accurate boundary features from low-order features by using the double guidance of high-order features and combining with the internal detail-context extraction module (DCEM), especially for low-contrast and poorly lit blurred boundary scenes, the extraction effect is remarkable.

[0030] (3) The salient edge generation module (DEGM) designed can feed back the accurate boundary features captured by EPM to the high-order feature map, significantly enhancing the polyp edge discrimination ability of the deep network, reducing the boundary information loss problem caused by deepening the network, and thus improving the segmentation accuracy of complex background polyps. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The overall architecture diagram of the bidirectional enhancement network system (BE-Net) proposed in the application is shown in the figure;

[0032] Figure 2 The detailed structure diagram of the edge prominence module (EPM) in the application is shown in the figure;

[0033] Figure 3 The detailed structure diagram of the detail-context extraction module (DCEM) in the application is shown in the figure;

[0034] Figure 4 The detailed structure diagram of the salient edge generation module (DEGM) in the application is shown in the figure;

[0035] Figure 5 The visualization comparison diagram of the segmentation results of different methods in three blurred boundary scenes is shown in the figure;

[0036] Figure 6 The comparison diagram of the segmentation results of the bidirectional enhancement network system (BE-Net) and its modules is shown in the figure. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0038] The terms "first", "second", and the like in the description and in the claims of this application are used for distinguishing between similar objects and are not necessarily used to describe a particular sequential or chronological order. It is to be understood that the use of the term data herein is not limited to nor is it to be construed as a reference to any particular structure or manner by which the data is presented, and that the embodiments of this application can be implemented in software and / or hardware components without departing from the scope of the present application. Furthermore, the term "and / or" means at least one of the connected objects, and the character " / " generally means a "or" relationship between the front and rear associated objects. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified.

[0039] Embodiment 1

[0040] Please refer to Figure 1 , the architecture diagram of a bidirectional enhancement network system (BE-Net) for colon polyp segmentation provided by an embodiment of the present application. The system architecture mainly includes:

[0041] The backbone network module 100 is used for encoding processing of the acquired colonoscopy image data, so as to extract low-level features including first low-level features and second low-level features, and high-level features including first high-level features and second high-level features;

[0042] The edge highlighting module 200 includes:

[0043] The double-guiding sub-module 210 is used for double-guiding the low-level features by using the first high-level features and the second high-level features respectively, so as to obtain first guided low-level features and second guided low-level features;

[0044] The first detail-context extraction sub-module 220 is used for capturing intra-feature boundary details and inter-feature context information of the first guided low-level features and the second guided low-level features respectively, so as to generate first re-extracted guided low-level features and second re-extracted guided low-level features;

[0045] The adaptive gating mechanism sub-module 230 is used for adopting an adaptive gating mechanism to the first re-extracted guided low-level features and the second re-extracted guided low-level features, so as to obtain first boundary features and second boundary features;

[0046] The salient edge generation module 300 includes a second detail-context extraction sub-module 310, which is used for fusing the first boundary features, the second boundary features, and the first high-level features to obtain first enhanced high-level features; and fusing the first boundary features, the second boundary features, and the second high-level features to obtain second enhanced high-level features;

[0047] The decoding and segmentation module 400 is used to generate a polyp segmentation map through decoding operations based on the first boundary feature, the second boundary feature, the first enhanced high-level feature, and the second enhanced high-level feature.

[0048] In this embodiment, TransNext is used as the backbone network module 100 for extracting initial features. For example... Figure 1 As shown, specifically, a colonoscopy image with a resolution adjusted to 352×352 pixels (where H represents the image height and W represents the image width) is input into BE-Net. The backbone network module 100 uses the TransNext model pre-trained on ImageNet, responsible for hierarchical feature extraction from the input image. Then, through a 1×1 convolution operation and four feature maps of different scales, the first low-level features are obtained. Second-lowest feature First advanced feature Second advanced features These features are then used as inputs to the Edge Prominence Module 200 (EPM) and the Significant Edge Generation Module 300 (DEGM).

[0049] In this embodiment, for the edge protrusion module 200 (EPM), accurate edge extraction is one of the core factors for achieving accurate segmentation results. Its main idea is to capture the accurate location of the polyp and then perform boundary extraction.

[0050] In the dual bootstrap submodule 210 (EPM), a high-level feature (i.e., the first high-level feature) is used. Second advanced features ) for low-level features The method employs a dual-guide approach, rather than the single-guide approach commonly used in existing methods, to locate polyps. This indicates the first low-level feature. This indicates the second lowest level feature.

[0051] Specifically, such as Figure 2 As shown, for the first high-level feature After upsampling, it is compared with low-level features. Perform element-wise multiplication, and then combine the result with the low-level feature. By performing element-wise addition, we obtain the first guiding low-level feature. ; for the second advanced feature After upsampling, it is compared with the low-level features. Perform element-wise multiplication, then add the result of the multiplication to the aforementioned low-level feature element-wise to obtain the second guiding low-level feature. :

[0052]

[0053] wherein, denotes element-wise multiplication, denotes up-sampling operation, denotes element-wise addition.

[0054] In the first detail-context extraction sub-module 220 (DCEM), the feature extraction of the first guiding low-level feature is the same as that of the second guiding low-level feature. As shown in Figure 3 to make full use of the boundary information of and , they are respectively input into the first detail-context extraction module 220 (DCEM), which is responsible for densely capturing the intra-feature boundary details and the inter-feature context information.

[0055] Specifically, the guiding feature (divided into and ) is respectively input into a parallel structure including an upper branch and a lower branch for processing: the upper branch is responsible for extracting the intra-feature boundary details of the guiding low-level feature by three stacked convolutional and activation functions; the lower branch extracts the inter-feature context information of the guiding low-level feature by stacked self-attention mechanisms; thus, and can be calculated as:

[0056]

[0057] wherein denotes a convolution operation using a convolution kernel followed by a activation function, and denotes a self-attention operation.

[0058] Further, the intra-feature boundary details output by the upper branch are merged with the inter-feature context information output by the lower branch to obtain the fusion feature , which can serve as a guide map to highlight the polyp edge; in order to highlight the boundary regions with low prediction confidence, a reverse attention operation is applied to the fusion feature to generate a reverse mask; based on the reverse mask and the fusion feature , the intra-feature boundary details and the inter-feature context information are enhanced to extract the boundary regions in the foreground view and the background view, obtaining the extracted guiding low-level feature The first re-extracted guided low-level feature and the second re-extracted guided low-level feature

[0059]

[0060] wherein, represents a reverse attention operation, and , represents boundary details enhanced by a guide map, represents context information enhanced by a guide map.

[0061] Therefore, as known from the above, when the guided feature input is the first guided low-level feature , the first detail-context extraction sub-module outputs the first re-extracted guided low-level feature ; when the guided feature input is the second guided low-level feature , the first detail-context extraction sub-module outputs the second re-extracted guided low-level feature .

[0062] In the adaptive gating mechanism sub-module 230, a gating mechanism is used to adaptively select features from the first extracted guided feature and the second extracted guided feature , and capture a large number of significant boundary features.

[0063] Specifically, after the first re-extracted guided low-level feature and the second re-extracted guided low-level feature are combined, global average pooling operations, fully connected layers and Softmax functions are used to calculate adaptive weights and for the first re-extracted guided low-level feature and the second re-extracted guided low-level feature ;

[0064] According to the adaptive weights and , the first re-extracted guided low-level feature and the second re-extracted guided low-level feature are weighted and summed to obtain boundary features (the first boundary feature and the second boundary feature ):

[0065] = * * (i = 1,2)​

[0066] In the embodiment, as the neural network is deepened, the boundary details are lost, but the semantic information becomes rich. In order to enhance the discriminative ability of high-level features to boundary details, a significant edge generation module 300 (DEGM) is developed in the BE-Net.

[0067] Specifically, as shown in Figure 4 , the first boundary feature and the second boundary feature are respectively down-sampled, the down-sampled first boundary feature and the second boundary feature are merged with the high-level feature in the channel dimension to obtain the first boundary detail enhanced fusion high-level feature .

[0068] The fusion high-level feature is input into the second detail-context extraction submodule 310 (DCEM) for feature extraction to obtain the second boundary detail enhanced fusion high-level feature .

[0069] The second boundary detail enhanced fusion high-level feature is integrated into the corresponding high-level feature by the convolution block attention submodule 320 (CBAM) to obtain the enhanced high-level feature , which is divided into the first enhanced high-level feature and the second enhanced high-level feature .

[0070] Embodiment 2

[0071] The colon polyp segmentation method provided by the embodiment of the application adopts the bidirectional enhancement network system (BE-Net) as described in Embodiment 1, and the method comprises the following steps:

[0072] Step S1: Obtain the colonoscopy image data to be processed and input it into the backbone network module.

[0073] Step S2: The colonoscopy image data is encoded and processed by the backbone network module to extract multi-level features, including at least low-level features and high-level features. The low-level features retain rich spatial and boundary details, and the high-level features contain stronger semantic information.

[0074] Step S3: An edge highlighting module (EPM) is used to process the low-level features to generate boundary features. This step is the first direction of bidirectional enhancement (bottom-up enhancement). The key is to use high-level features , ) rich semantic information, double guidance to low-level features (F , ) to concentrate and densely mine the boundary details inside the low-order feature map and the cross-feature context information, generating high-quality boundary features.

[0075] Step S4: using the significant edge generation module (DEGM), the boundary features generated in step S3 are integrated into the high-level features (F , ) to enhance the edge discrimination ability of the high-level features, obtaining enhanced high-level features. This step is the second direction of bidirectional enhancement (top-down enhancement). By supplementing accurate boundary information to high-order semantic features, the lost boundary details in the encoding process of the deep network are compensated.

[0076] Step S5: based on the enhanced high-level features, and aggregating other high-level feature information extracted by the backbone network module, the final boundary clear polyp segmentation result is generated through the decoder or cascading aggregation operation.

[0077] In order to verify the effectiveness and generalization of the method proposed in the embodiment, the present application adopts two-stage experimental design: first, 1450 training images (900 from Kvasir-SEG dataset and 550 from ClinicDB dataset) are used for model training, and the remaining images are used as visible test set to evaluate the basic learning ability. To further verify the performance of the model, three datasets, ColonDB, ETIS and CVC-300, are introduced to form an invisible test set, focusing on the generalization performance of the model on the invisible dataset. Such experimental design not only guarantees the learning ability of the model in the known field, but also effectively tests its generalization ability.

[0078] a. Comparison with the most advanced method

[0079] (1) Quantitative results of polyp segmentation: The values of the six metrics for all methods are listed in Table 1, Table 2 and Table 3, where the best results are shown in bold and '-' means the result is not available. The learning capability test results of all methods on Kvasir-SEG and CVC-ClinicDB are presented in Table 1. From all six evaluation metrics, BE-Net performs best on CVC-ClinicDB which contains most colonoscopy images, which are taken under dim lighting conditions, making it difficult to find the polyp boundary. Moreover, BE-Net outperforms the second best method BRNet by 2.5% and 1.9% on mDice and mloU respectively. In addition, among the six evaluation metrics on Kvasir-SEG, the proposed BE-Net outperforms other methods on five metrics and ranks second on one metric (i.e. mloU). Therefore, BE-Net has stronger learning capability than other comparative methods and can better segment the fuzzy polyps due to insufficient lighting.

[0080] Table 1 Values of 6 evaluation metrics for all models on visible datasets

[0081]

[0082] To evaluate the generalization capability of all methods, this experiment uses three unseen datasets, i.e. CVC-ColonDB, ETIS-LaribPolypDB and CVC-300, whose images are not used in the training process. The results are shown in Table 2 and Table 3. It can be noted that BE-Net performs best on two important segmentation metrics (i.e. mDice and mloU) on all three datasets. Moreover, BE-Net ranks either first or second on the other four metrics. Therefore, it can be concluded that the generalization capability of BE-Net is superior to other comparative methods.

[0083] Table 2 Values of 6 evaluation metrics for all models on unseen datasets CVC-ColonDB and ETIS-LaribPolypDB

[0084]

[0085] Table 3 Values of 6 evaluation metrics for all models on unseen dataset CVC-300

[0086]

[0087] (2) Qualitative results of polyp segmentation: Figure 5 Visual comparisons of segmentation results under three fuzzy ring edge scenarios are shown. From Figure 5As can be seen, the BE-Net proposed in this invention significantly outperforms other comparative methods in terms of segmentation accuracy. For example, BE-Net performs particularly accurately in the segmentation of low-contrast polyps, while other methods show significant shortcomings (see [link to relevant documentation]). Figure 5 (First, fifth, and sixth rows). For colonoscopy images in poor lighting conditions, some contrast methods either completely miss polyps or severely over-segment them. In contrast, BE-Net's segmentation results are basically consistent with the true values ​​(see...). Figure 5 (Second and third rows). More importantly, in image processing with cluttered backgrounds, BE-Net outperforms other methods (see...). Figure 5 (Fourth line). Overall, BE-Net demonstrates significant advantages in both quantitative and qualitative aspects.

[0088] (3) Experimental Details: The proposed BE-Net was implemented using the Py-Torch framework and trained on an NVIDIA RTX 3090 GPU with 24GB of memory. Furthermore, the AdamW optimizer was used in the experiments, and the image resolution was adjusted to 352×352. For the backbone network, it was pre-trained on ImageNet. The training epochs of BE-Net were 30, the batch size was 4, the learning rate was 1e-4, and the learning rate decay rate was 0.1.

[0089] b. Ablation test

[0090] To verify the effectiveness of the proposed EPM and DEGM, a series of ablation experiments were conducted under the same settings described in the experimental details above. Furthermore, this invention used TranNext as the backbone network, and then progressively added EPM and DEGM to the backbone network. Quantitative and qualitative results are shown in Table 4 and... Figure 6 As shown.

[0091] (1) Effectiveness of EPM: The purpose of EPM is to capture more detailed features and small polyps by utilizing low-level features. As shown in Table 4, in the CVC-300 dataset (with unclear boundaries and flat polyps as the main feature), adding EPM to the backBone dataset improved mean and mean iou by 1.3% and 1.6%, respectively.

[0092] (2) Effectiveness of DEGM: The purpose of DEGM is to enhance high-level features using low-level features. Here, this invention uses low-level features that have not been processed by EPM. As can be seen from the first and third rows of the table, meanice and meaniou are improved by 3.2% and 3.8% respectively on the CVC-ColonDB dataset with complex backgrounds, and there are also improvements on other datasets.

[0093] (3) Effectiveness of EPM+DEGM: From the first and fourth rows of the table, it can be seen that when the two modules are used together, the meandice and meaniou are improved by 4.9% and 4.9% respectively on the CVC-ClinicDB dataset, 2.8% and 3.2% respectively on the CVC-300 dataset, and 4.2% and 4.6% respectively on the CVC-ColonDB dataset.

[0094] Table 4 Ablation experiments

[0095]

[0096] In addition to these quantitative results, it can also be seen from Figure 4 that the segmentation results of the backbone network gradually approach the true value under the modification of EPM and DEGM. Therefore, the effectiveness of EPM and DEGM is verified quantitatively and qualitatively.

[0097] The bidirectional enhancement network system for colon polyp segmentation provided by the embodiment of the application can implement each process of the method embodiment of the colon polyp segmentation method, and thus will not be described here again to avoid repetition. Figure 1

[0098] Optionally, the embodiment of the application further provides an electronic device, which includes a processor, a memory, a program or instructions stored on the memory and executable on the processor, the program or instructions being executed by the processor to implement each process of the above-mentioned colon polyp segmentation method embodiment and achieve the same technical effects, and thus will not be described here again to avoid repetition.

[0099] The embodiment of the application further provides a readable storage medium, which stores a program or instructions, the program or instructions being executed by a processor to implement each process of the above-mentioned colon polyp segmentation method embodiment and achieve the same technical effects, and thus will not be described here again to avoid repetition.

[0100] The processor is the processor in the electronic device in the above-mentioned embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0101] ​It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it is to be understood that the method and apparatus of the present application can be carried out by more than one process, method, article, or apparatus either simultaneously, concurrently, or with intervening action that are carried out at the same time, either in a simultaneous fashion or in a fashion that is staggered in time. For example, the described methods can be carried out in a different order than described, and / or various steps can be combined or omitted, and / or additional steps can be added, without departing from the scope of the present application. Also, features described with respect to certain examples can be combined in other examples.

[0102] From the above description of the embodiments, it is apparent that the above-described method of the embodiments can be realized by means of software and general-purpose hardware platforms, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such an understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a number of instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the methods described in the various embodiments of the present application.

[0103] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, rather than limiting, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the protection scope of the claims, and all of them belong to the protection scope of the present application.

Claims

1. A bidirectional enhanced network system for colon polyp segmentation, characterized in that, The bidirectional enhanced network system comprises: a backbone network module, configured to encode the acquired colonoscopy image data to extract low-level features comprising first low-level features and second low-level features, and high-level features comprising first high-level features and second high-level features; an edge highlighting module, comprising: a dual-guiding sub-module, configured to perform dual-guiding on the low-level features by using the first high-level features and the second high-level features respectively, to obtain first guided low-level features and second guided low-level features; a first detail-context extraction sub-module, configured to capture intra-feature boundary details and inter-feature context information of the first guided low-level features and the second guided low-level features respectively, to generate first re-extracted guided low-level features and second re-extracted guided low-level features; in the first detail-context extraction sub-module, the first re-extracted guided low-level features are generated in the same way as the second re-extracted guided low-level features, wherein the generation of the first re-extracted guided low-level features comprises: inputting the first guided low-level features into a parallel structure comprising an upper branch and a lower branch respectively for processing: the upper branch extracts intra-feature boundary details of guided low-level features through stacked convolution operations; the lower branch extracts inter-feature context information of guided low-level features through stacked self-attention mechanisms; the intra-feature boundary details output by the upper branch and the inter-feature context information output by the lower branch are merged to obtain fused features; reverse attention operations are applied to the fused features to generate reverse masks; based on the reverse masks and the fused features, the intra-feature boundary details and the inter-feature context information are enhanced to extract boundary regions in foreground views and background views, to obtain the first re-extracted guided low-level features; an adaptive gating mechanism sub-module, configured to adopt an adaptive gating mechanism on the first re-extracted guided low-level features and the second re-extracted guided low-level features, to obtain first boundary features and second boundary features; the obtaining of the first boundary features and the second boundary features comprises: after merging the first re-extracted guided low-level features and the second re-extracted guided low-level features, global average pooling operations, fully connected layers and Softmax functions are used to calculate adaptive weights for the first re-extracted guided low-level features and the second re-extracted guided low-level features; the first re-extracted guided low-level features and the second re-extracted guided low-level features are weighted and summed according to the adaptive weights, to obtain the first boundary features and the second boundary features; a salient edge generation module, comprising a second detail-context extraction sub-module, configured to fuse the first boundary features, the second boundary features, the first high-level features and the second high-level features to obtain first enhanced high-level features; and fuse the first boundary features, the second boundary features, the first high-level features and the second high-level features to obtain second enhanced high-level features; a decoding segmentation module, configured to generate a polyp segmentation map through decoding operations based on the first boundary features, the second boundary features, the first enhanced high-level features and the second enhanced high-level features. 2.The bidirectional enhanced network system for colon polyp segmentation of claim 1, wherein, In the dual guidance sub-module, the first guided low-level feature and the second guided low-level feature are obtained by: performing element-wise multiplication on the first high-level feature after upsampling processing, and then performing element-wise addition on the multiplication result and the first low-level feature to obtain the first guided low-level feature; performing element-wise multiplication on the second high-level feature after upsampling processing, and then performing element-wise addition on the multiplication result and the second low-level feature to obtain the second guided low-level feature. 3.The bidirectional enhanced network system for colon polyp segmentation of claim 1, wherein, In the significant edge generation module, the first enhanced high-level feature and the second enhanced high-level feature are obtained in the same way, and the way of obtaining the first enhanced high-level feature includes: performing downsampling processing on the first boundary feature and the second boundary feature respectively, merging the downsampling first boundary feature and the second boundary feature with the first high-level feature in the channel dimension to obtain the first fusion high-level feature after the first boundary detail enhancement; inputting the first fusion high-level feature into the second detail-context extraction sub-module for feature extraction to obtain the first fusion high-level feature after the second boundary detail enhancement; integrating the first fusion high-level feature after the second boundary detail enhancement into the first high-level feature through the convolution block attention sub-module to obtain the first enhanced high-level feature.

4. The bidirectional enhanced network system for colon polyp segmentation of any one of claims 1-3, wherein, In the decoding segmentation module: based on the first boundary feature, the second boundary feature, the first enhanced high-level feature and the second enhanced high-level feature, gradually upsample and fuse features to generate four polyp segmentation maps of different resolutions.

5. The bidirectional enhanced network system for colon polyp segmentation of claim 4, wherein, It also includes a result output module that uses a deep supervision method to train the bidirectional enhancement network system, and through cascading aggregation of the four polyp segmentation maps, finally obtains a polyp segmentation result.

6. A colon polyp segmentation method characterized by, The method using the bidirectional enhancement network system of any one of claims 1-5 includes: obtaining the colonoscopy image data to be processed, inputting the colonoscopy image data to be processed into the bidirectional enhancement network system, and obtaining the polyp segmentation map corresponding to the colonoscopy image data to be processed.

7. An electronic device, comprising: It includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor, and when the program or instruction is executed by the processor, the steps of the colon polyp segmentation method of claim 6 are implemented.

8. A readable storage medium, characterized by, The program or instruction is stored on the readable storage medium, and when the program or instruction is executed by the processor, the steps of the colon polyp segmentation method of claim 6 are implemented.

Images