Method and system for segmenting non-significant colon polyp in medical image
By using a multi-branch feature fusion network and feature fusion strategy, combined with global branching, residual branching and boundary refinement modules, the problems of insufficient local information and loss of global features in colon polyp image segmentation are solved, and more efficient segmentation of non-saliency colon polyps is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUNAN INST OF INFORMATION TECH
- Filing Date
- 2025-11-24
- Publication Date
- 2026-04-10
AI Technical Summary
Existing technologies suffer from insufficient local information capture, neglect of global information, and feature loss in colon polyp image segmentation, resulting in low segmentation accuracy and efficiency, especially in the segmentation of non-salient colon polyps.
A multi-branch feature fusion network is adopted, which combines a global branch, a residual branch and a main branch. Feature extraction is performed through a Transformer-to-CNNs module and a residual connection mechanism. By combining a feature fusion strategy and a residual refinement module, the complementarity of local features and global features and the refinement of boundaries are achieved.
It improves the feature representation ability of non-significant colonic polyps, enhances the fine-grained representation of segmentation results, improves the accuracy and efficiency of segmentation, and alleviates the problem of blurred boundaries.
Smart Images
Figure CN121837620A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a method and system for segmenting non-significant colonic polyps in medical images. Background Technology
[0002] Colorectal cancer includes two types: colon cancer and rectal cancer. With a mortality rate of 9.2%, it is the second most common cancer after lung cancer. In recent years, the incidence and mortality rates of colorectal cancer have been rising annually, seriously threatening the lives and health of the people and creating a severe social burden.
[0003] Reducing the incidence of colorectal cancer has become a significant challenge in today's society. The main cause of colorectal cancer is colonic polyps, especially highly proliferative adenomas. With advancements in medical imaging technology, colonoscopy is widely recognized as the best means of preventing colorectal cancer. Colonoscopy screening detects and removes polyps in the intestines, preventing them from developing into cancer and thus reducing the incidence of colorectal cancer. Many studies have shown that early colonoscopy can reduce the incidence of colorectal cancer by 30%. However, colonoscopy images contain a large amount of information and require a high level of expertise from physicians, leading to frequent missed diagnoses. Studies indicate that the missed diagnose rate of polyps during colonoscopy ranges from 20% to 47%, posing a serious threat to patient safety. With the continuous advancement of artificial intelligence and deep learning technologies, the use of computer-aided diagnostic techniques has become a current research focus.
[0004] The heterogeneity of colon polyp images in terms of color, size, shape, location, and texture makes segmentation a very challenging task. Early work relied on manually extracting features such as texture, color, and boundaries of polyps, and then combining these features to model and train a classifier to distinguish polyps from surrounding tissue. However, manually designed features rely on sensitive regions in the visual system to extract features that can be distinguished from the intestinal wall, and their accuracy largely depends on the experience and expertise of the scientists. With the increasing importance of medical images, automated processing of medical images has become a focus of much research in order to improve the efficiency and accuracy of clinical disease diagnosis.
[0005] Most deep learning-based methods employ encoder-decoder architectures based on convolutional networks or Transformers. Convolutional neural networks have been widely used in image processing tasks due to their excellent performance. However, due to the limited field of view of convolution, they can only capture local information from images, leading to the neglect of global information and spatial details, and their learning ability in features such as texture, shape, and size is relatively limited. In recent years, Transformer networks have received widespread attention in the field of image vision due to their powerful global semantic extraction capabilities and spatial adaptive information aggregation capabilities. However, the Transformer series has many parameters, high computational cost, requires powerful hardware, and has insufficient ability to capture local features, especially on small datasets such as medical images. At the same time, the encoder inevitably loses features during the process of downsampling the input into a high-dimensional semantic vector.
[0006] Therefore, how to better combine convolutional networks and Transformers, and how to compensate for features lost during downsampling, remains a research challenge. Using computer technology to assist doctors in diagnosing colon polyps can effectively reduce the pressure on medical personnel, minimize the impact of individual variability, and ultimately improve the accuracy and efficiency of doctors' diagnoses. Summary of the Invention
[0007] To address the aforementioned technical problems, this invention provides a method and system for segmenting non-significant colonic polyps in medical images.
[0008] The technical solution adopted in this invention is:
[0009] Firstly, a method for segmenting non-significant colonic polyps in medical images is provided, including:
[0010] Perform endoscopic image acquisition to obtain images of colon polyps;
[0011] Multi-branch feature fusion network is used to extract multi-level features from colon polyp images and output multi-scale features.
[0012] A feature fusion strategy is adopted to add local features in multi-scale features and then concatenate them with global features to obtain fused features;
[0013] The fused features are then refined by boundary thinning to obtain the refined features.
[0014] The refined features are decoded to output the segmentation result image.
[0015] Furthermore, the multi-branch feature fusion network consists of a global branch, a residual branch, and a main branch.
[0016] Furthermore, the global branch uses the Transformer-to-CNNs module.
[0017] The Transformer-to-CNNs module uses the global context modeling mechanism of Transformer to extract features from colon polyp images, obtain initial global features, and then convert the initial global features into global features that can be processed by a convolutional neural network (CNN).
[0018] Furthermore, the residual branch uses a residual connection mechanism to extract features from the colon polyp image, obtaining local features of the residual branch.
[0019] Furthermore, the main branch uses a CNN structure to extract basic semantic features from colon polyp images, thus obtaining local features of the main branch.
[0020] Furthermore, a feature fusion strategy is employed to add local features from the multi-scale features and concatenate them with the global features to obtain fused features, including:
[0021] A feature fusion strategy is used to perform an addition operation on the residual branch local feature F2 and the main branch local feature F3 in the multi-scale features to obtain (F2+F3).
[0022] The global feature F1 is concatenated with (F2+F3) to obtain the fused feature F. fused =Convcat(F1,(F2+F3)).
[0023] Furthermore, the fused features are refined by boundary refinement to obtain refined features, including:
[0024] By refining the RCF module through residuals, the fused feature F is refined layer by layer during the decoding stage. fused Enhancement is performed to obtain the refined feature F4 = RCF(Convcat(F1, (F2+F3))).
[0025] Secondly, a segmentation system for non-significant colonic polyps in medical images is provided, including:
[0026] The image acquisition unit is used to acquire endoscopic images, specifically images of colon polyps.
[0027] The feature extraction unit is used to extract multi-level features from colon polyp images through a multi-branch feature fusion network and output multi-scale features.
[0028] The feature fusion unit is used to add local features from multi-scale features using a feature fusion strategy and then concatenate them with global features to obtain fused features.
[0029] The residual refinement unit is used to refine the boundaries of the fused features to obtain refined features.
[0030] The segmentation output unit is used to decode the refined features and output the segmentation result image.
[0031] The beneficial effects achieved by this invention are as follows:
[0032] Endoscopic image acquisition was performed to obtain images of colon polyps; multi-branch feature fusion network was used to extract multi-level features from the colon polyp images, outputting multi-scale features; a feature fusion strategy was used to add local features from the multi-scale features and concatenate them with global features to obtain fused features; the fused features were then refined to obtain refined features; the refined features were decoded to output the segmentation result image;
[0033] By leveraging the synergistic effect of the global branch, residual branch, and main branch of the multi-branch feature fusion network, the complementarity between local and global features is achieved, effectively enhancing the feature representation ability of non-significant colonic polyps.
[0034] The feature fusion strategy combines the addition of local features with the concatenation of global features, which ensures the consistency of local features while maintaining the independence of global features, thereby improving the expressiveness and fusion effect of multi-scale features.
[0035] In the decoding stage, an RCF module is introduced. By using a multi-level residual structure to enhance boundary features through layer-by-layer refinement, the segmentation shortcomings of traditional methods in cases of blurred boundaries are effectively alleviated, and the fine-grained performance of the segmentation results is improved. Attached Figure Description
[0036] Figure 1 This is a flowchart of the method for segmenting non-significant colonic polyps in medical images according to the present invention;
[0037] Figure 2 This is a structural diagram of the segmentation system for non-significant colonic polyps in medical images according to the present invention. Detailed Implementation
[0038] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.
[0039] like Figure 1 As shown, this embodiment of the invention provides a method for segmenting non-significant colonic polyps in medical images, including:
[0040] 101. Perform endoscopic image acquisition to obtain images of colon polyps;
[0041] In this embodiment, a novel deep learning model, the pre-built Unified Multi-branch Fusion Network (UMF-Net), is used for the accurate segmentation of non-significant colonic polyps in medical images. The core idea is to combine the powerful local feature extraction capabilities of Convolutional Neural Networks (CNNs) with the advantages of Transformer-like structures in global context modeling, achieving comprehensive capture of polyp features through a carefully designed multi-branch parallel architecture. UMF-Net employs a classic encoder-decoder structure.
[0042] Endoscopic images are acquired using an endoscope to obtain images of colonic polyps, which are then input into UMF-Net.
[0043] It should be noted that various data augmentation techniques were used to improve the robustness of UMF-Net, such as rotation, mirroring, mosaic, RandAffine, and Mixup. Colon polyp images required preprocessing before being input into UMF-Net, including:
[0044] The training samples are uniformly scaled to a fixed resolution, augmented using data augmentation techniques (e.g., random cropping, rotation, flipping, and lighting adjustment), and normalized to ensure the consistency of feature distribution, thereby improving the model's generalization ability.
[0045] 102. Multi-branch feature fusion network is used to extract multi-level features from colon polyp images and output multi-scale features;
[0046] The multi-branch feature fusion network consists of a global branch, a residual branch, and a main branch;
[0047] Global Branch: An innovative Transformer-to-CNNs (TC) module is introduced for a global context modeling mechanism based on Transformer to extract features from colon polyp images, obtain initial global features, and convert the initial global features into global features that can be processed by a convolutional neural network (CNN); this is crucial for identifying polyps that have low contrast with surrounding tissue and whose features are not obvious.
[0048] Residual branch: The residual connection mechanism is used to extract features from the colon polyp image to obtain local features of the residual branch; through simple convolutional downsampling and residual connection, it aims to preserve the low-level details and boundary information of the image to the greatest extent and make up for the fine features that may be lost in the deep feature extraction process.
[0049] Main branch: As the core backbone of the network, it adopts a standard CNN structure and is responsible for extracting the basic semantic features of colon polyp images, obtaining the local features of the main branch, and providing a robust feature foundation for the segmentation task.
[0050] To address the issues of high computational cost and numerous parameters in traditional Transformer models, the TC module efficiently simulates their global modeling capabilities. Its core principle is the use of a deformable convolutional network (DCN) instead of a self-attention mechanism. Standard 2D convolutions output feature maps... Location The calculation on can be expressed as:
[0051] ;
[0052] in, It is the input feature map. These are the convolution kernel weights. These are predefined sampling points; DCN introduces a learnable offset. This allows it to adaptively adjust the sampling position:
[0053] ;
[0054] The above mechanisms enable the convolution kernel to dynamically adapt to the shape and scale of the target, thereby achieving adaptive spatial information aggregation and long-distance dependency modeling with low computational cost;
[0055] The TC module further incorporates an Enhanced Feature Block (EFB), composed of three parallel convolutional branches, whose output... It can be represented as:
[0056] ;
[0057] in, For input features, The diagonal convolution branch is used to capture spatial relationships in the vertical and horizontal directions. Dilated convolution branches with different dilation rates are used to simulate large-size convolution kernels, thereby enhancing the ability to perceive global information at different scales.
[0058] 103. A feature fusion strategy is used to add local features from multi-scale features and then concatenate them with global features to obtain fused features.
[0059] In this embodiment, the feature fusion strategy needs to fuse multi-scale features. For the features extracted from the residual branch local features F2 and the main branch local features F3, which focus on local details, the feature fusion is performed by adding elements one by one to obtain (F2+F3). This can enhance the consistency of features and strengthen common local patterns.
[0060] For the global context features extracted by global feature F1, channel concatenation is used to merge them with the fused local features (F2+F3). The concatenation operation can maintain the independence and integrity of global information and avoid dilution during the addition process; the fused feature F is obtained. fused =Convcat(F1,(F2+F3)).
[0061] 104. The fused features are refined by boundary thinning to obtain the refined features;
[0062] In the decoding stage, to address the potential boundary ambiguity issue caused by multi-scale feature fusion, an RCF module is introduced, consisting of a coarse-grained module and a fine-grained module.
[0063] The main function of the coarse-grained module is to enhance feature diversity while reducing the number of channels; it contains three branches: the first branch is mainly used to capture the spatial relationship of the image in the vertical and horizontal directions; the second branch uses two 3×3 convolutions to extract detailed features; and the third branch uses a 1×1 convolution to retain important information while reducing the number of channels.
[0064] The fine-grained module further optimizes the channel-compressed features, enabling the network to extract high-level features more effectively. The two branches employ dilated convolutions, simulating 7×7 and 15×15 convolution kernels, to extract key features while reducing reliance on low-level details.
[0065] F fusion feature F fused as input features The coarse-grained module first processes the data through three parallel branches to enhance feature diversity and compress channels:
[0066] ;
[0067] in, , , These represent diagonal convolution, standard convolution, and... Convolutional branches;
[0068] Subsequently, the fine-grained module receives the compressed features. Furthermore, it utilizes two parallel dilated convolution branches to further optimize the features, focusing them on high-level semantic information:
[0069] ;
[0070] in , Dilated convolutions representing different dilation rates;
[0071] Finally, the module's output is obtained through a residual connection. Significantly improves the boundary clarity of the segmentation results:
[0072] ;
[0073] Ultimately As a refined feature, F4 = RCF(Convcat(F1, (F2+F3))).
[0074] 105. Decode the refined features and output the segmentation result image.
[0075] The beneficial effects achieved by the embodiments of the present invention are as follows:
[0076] Endoscopic image acquisition was performed to obtain images of colon polyps; multi-branch feature fusion network was used to extract multi-level features from the colon polyp images, outputting multi-scale features; a feature fusion strategy was used to add local features from the multi-scale features and concatenate them with global features to obtain fused features; the fused features were then refined to obtain refined features; the refined features were decoded to output the segmentation result image;
[0077] By leveraging the synergistic effect of the global branch, residual branch, and main branch of the multi-branch feature fusion network, the complementarity between local and global features is achieved, effectively enhancing the feature representation ability of non-significant colonic polyps.
[0078] The feature fusion strategy combines the addition of local features with the concatenation of global features, which ensures the consistency of local features while maintaining the independence of global features, thereby improving the expressiveness and fusion effect of multi-scale features.
[0079] In the decoding stage, an RCF module is introduced. By using a multi-level residual structure to enhance boundary features through layer-by-layer refinement, the segmentation shortcomings of traditional methods in cases of blurred boundaries are effectively alleviated, and the fine-grained performance of the segmentation results is improved.
[0080] above Figure 1 The illustrated embodiments describe a method for segmenting non-significant colonic polyps in medical images. The following embodiments illustrate a segmentation system for non-significant colonic polyps in medical images.
[0081] like Figure 2 As shown, this embodiment of the invention provides a segmentation system for non-significant colonic polyps in medical images, comprising:
[0082] Image acquisition unit 201 is used to acquire endoscopic images and obtain images of colon polyps;
[0083] Feature extraction unit 202 is used to extract multi-level features from colon polyp images through a multi-branch feature fusion network and output multi-scale features;
[0084] The feature fusion unit 203 is used to add local features in multi-scale features using a feature fusion strategy and concatenate them with global features to obtain fused features;
[0085] The residual refinement unit 204 is used to refine the boundaries of the fused features to obtain refined features.
[0086] The segmentation output unit 205 is used to decode the refined features and output the segmentation result image.
[0087] The beneficial effects achieved by the embodiments of the present invention are as follows:
[0088] Image acquisition unit 201 acquires endoscopic images to obtain images of colon polyps; feature extraction unit 202 performs multi-level feature extraction on the colon polyp images through a multi-branch feature fusion network, outputting multi-scale features; feature fusion unit 203 uses a feature fusion strategy to add local features in the multi-scale features and concatenates them with global features to obtain fused features; residual refinement unit 204 performs boundary refinement processing on the fused features to obtain refined features; segmentation output unit 205 decodes the refined features and outputs the segmentation result image.
[0089] By leveraging the synergistic effect of the global branch, residual branch, and main branch of the multi-branch feature fusion network, the complementarity between local and global features is achieved, effectively enhancing the feature representation ability of non-significant colonic polyps.
[0090] The feature fusion strategy combines the addition of local features with the concatenation of global features, which ensures the consistency of local features while maintaining the independence of global features, thereby improving the expressiveness and fusion effect of multi-scale features.
[0091] In the decoding stage, an RCF module is introduced. By using a multi-level residual structure to enhance boundary features through layer-by-layer refinement, the segmentation shortcomings of traditional methods in cases of blurred boundaries are effectively alleviated, and the fine-grained performance of the segmentation results is improved.
[0092] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0093] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0094] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0095] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0096] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.
Claims
1. A method for segmenting non-significant colonic polyps in medical images, characterized in that, include: Perform endoscopic image acquisition to obtain images of colon polyps; Multi-branch feature fusion network is used to extract multi-level features from the colon polyp image and output multi-scale features. A feature fusion strategy is used to add the local features in the multi-scale features and then concatenate them with the global features to obtain the fused features; The fused features are then subjected to boundary refinement processing to obtain refined features; The refined features are decoded to output a segmentation result image.
2. The method for segmenting non-significant colonic polyps in medical images according to claim 1, characterized in that, The multi-branch feature fusion network consists of a global branch, a residual branch, and a main branch.
3. The method for segmenting non-significant colonic polyps in medical images according to claim 2, characterized in that, The global branch uses the Transformer-to-CNNs module. The Transformer-to-CNNs module extracts features from the colon polyp image based on the global context modeling mechanism of Transformer, obtains initial global features, and converts the initial global features into global features that can be processed by a convolutional neural network (CNN).
4. The method for segmenting non-significant colonic polyps in medical images according to claim 2, characterized in that, The residual branch uses a residual connection mechanism to extract features from the colon polyp image, obtaining local features of the residual branch.
5. The method for segmenting non-significant colonic polyps in medical images according to claim 2, characterized in that, The main branch is a CNN structure, which extracts basic semantic features from the colon polyp image to obtain the local features of the main branch.
6. The method for segmenting non-significant colonic polyps in medical images according to any one of claims 2-5, characterized in that, The step of employing a feature fusion strategy to add local features from the multi-scale features and concatenate them with global features to obtain fused features includes: A feature fusion strategy is used to perform an addition operation on the residual branch local feature F2 and the main branch local feature F3 in the multi-scale features to obtain (F2+F3). The global feature F1 is concatenated with (F2+F3) to obtain the fused feature F. fused =Convcat(F1,(F2+F3)).
7. The method for segmenting non-significant colonic polyps in medical images according to claim 6, characterized in that, The process of refining the boundaries of the fused features to obtain refined features includes: By refining the RCF module through residuals, the fused feature F is refined layer by layer during the decoding stage. fused Enhancement is performed to obtain the refined feature F4 = RCF(Convcat(F1, (F2+F3))).
8. A segmentation system for non-significant colonic polyps in medical images, characterized in that, include: The image acquisition unit is used to acquire endoscopic images, specifically images of colon polyps. The feature extraction unit is used to perform multi-level feature extraction on the colon polyp image through a multi-branch feature fusion network and output multi-scale features. The feature fusion unit is used to add the local features in the multi-scale features using a feature fusion strategy and concatenate them with the global features to obtain the fused features; The residual refinement unit is used to refine the boundaries of the fused features to obtain refined features. The segmentation output unit is used to decode the refined features and output a segmentation result image.