Endoscope image polyp segmentation method and device, computer device and storage medium

Abstract

The application relates to an endoscope image polyp segmentation method and device, computer equipment and a storage medium. A trained neural network model comprises a feature extraction layer, an adaptive perception module, a comprehensive feature extraction module and a segmentation head. The method comprises the following steps: acquiring an endoscope image to be processed; inputting the endoscope image to be processed into the trained neural network model, performing feature extraction on the endoscope image by using the feature extraction layer, and obtaining a plurality of feature maps of different scales; inputting the plurality of feature maps of different scales into the adaptive perception module, performing flexible feature processing on the plurality of feature maps of different scales by using a deformable convolution structure in the adaptive perception module, and obtaining an optimized feature map corresponding to each feature map; and inputting the plurality of optimized feature maps after fusion processing into the segmentation head, and obtaining a polyp segmentation image output by the segmentation head. The application can improve the accuracy of endoscope image polyp segmentation.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, computer device and storage medium for polyp segmentation of endoscopic images. Background Technology

[0002] Endoscopic technology enables high-resolution real-time imaging of the gastrointestinal tract, facilitating the precise identification and localization of polyps. Accurately defining the morphological characteristics of polyps (especially subtle boundary variations) is crucial for the early diagnosis of precancerous lesions and the development of targeted treatment plans.

[0003] Currently, the field of medical image segmentation is dominated by two main architectures: Convolutional Neural Networks (CNNs) and Transformers. While CNN-based methods (such as the U-Net architecture) can effectively capture local image features, their fixed receptive field limits accurate segmentation of structures that rely on long-range dependencies. Transformer-based methods (such as UNETR), through their self-attention mechanism, can effectively utilize global information from the entire image, but they still have limitations when dealing with polyp morphologies that vary greatly.

[0004] Existing segmentation frameworks use rectangular convolution kernels of fixed size to perform regular grid sampling on input images. This rigid computational paradigm has the following drawbacks: (1) It cannot adapt to the diversity and deformability of polyp morphology, resulting in inaccurate segmentation of polyps with irregular shapes and blurred boundaries; (2) The deep representation layer is limited by a fixed observation range, causing a mismatch between the regular receptive field and the actual polyp scale or deformation; (3) Simply expanding the observation range without solving the need for adaptive scale determination and flexible receptive field makes it difficult to accurately segment polyps of different shapes and sizes. Summary of the Invention

[0005] This application provides a method, apparatus, computer device, and storage medium for polyp segmentation of endoscopic images, aiming to solve the technical problem that existing segmentation frameworks use fixed-size rectangular convolution kernels to perform regular grid sampling on input images, resulting in relatively low accuracy of the segmented polyp results.

[0006] In a first aspect, embodiments of this application provide a method for polyp segmentation in endoscopic images. The trained neural network model includes a feature extraction layer, an adaptive perception module, a comprehensive feature extraction module, and a segmentation head. The method includes:

[0007] Acquire the endoscopic image to be processed;

[0008] The endoscope image to be processed is input into a trained neural network model, and the feature extraction layer is used to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0009] Multiple feature maps of different scales are input into the adaptive perception module, and the deformable convolutional structure in the adaptive perception module is used to perform flexible feature processing on the multiple feature maps of different scales respectively to obtain the optimized feature map corresponding to each feature map.

[0010] After fusing multiple optimized feature maps, the data is input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0011] In some possible implementations, the adaptive perception module includes a feature segmentation unit, a first processing submodule, and a second processing submodule. The step of inputting multiple feature maps of different scales into the adaptive perception module, and using the deformable convolutional structure in the adaptive perception module to perform flexible feature processing on the multiple feature maps of different scales respectively, to obtain an optimized feature map corresponding to each feature map output by the adaptive perception module, includes:

[0012] The feature map is input into the feature segmentation unit, and the feature segmentation unit is used to segment the feature map into a first sub-feature map and a second sub-feature map.

[0013] The first processing submodule is used to perform local processing on the first sub-feature map to obtain a local detail feature map.

[0014] The second processing submodule is used to perform global processing on the second sub-feature map to obtain a global structural feature map;

[0015] Based on the local detail feature map and the global structure feature map, an optimized feature map corresponding to the feature map is obtained.

[0016] In some possible implementations, the first processing submodule includes multiple adaptive sensing units, and the step of using the first processing submodule to perform local processing on the first sub-feature map to obtain a local detail feature map includes:

[0017] The first sub-feature map is subjected to local feature extraction to obtain multiple local feature maps;

[0018] Each adaptive sensing unit performs local flexible processing on the corresponding local feature map to obtain a local flexible feature map corresponding to each local feature map.

[0019] The local flexible feature maps are averaged to obtain the local detail feature map output by the first processing submodule.

[0020] In some possible implementations, the second processing submodule includes a first depthwise convolutional layer, a 3x3 convolutional layer, and a second depthwise convolutional layer; the step of using the second processing submodule to perform global processing on the second sub-feature map to obtain a global structure feature map includes:

[0021] The second sub-feature map is input into the first depth convolutional layer for depth convolution processing to obtain the depth convolutional image output by the first depth convolutional layer.

[0022] The depth convolutional image is input into the 3*3 convolutional layer for convolution and normalization processing to obtain a normalized depth convolutional image;

[0023] The normalized depthwise convolutional image is input into the second depthwise convolutional layer for secondary depthwise convolution processing to obtain the global structure feature map output by the second depthwise convolutional layer.

[0024] In some possible implementations, obtaining the optimized feature map corresponding to the feature map based on the local detail feature map and the global structure feature map includes:

[0025] The local detail feature map and the global structure feature map are fused at the channel level to obtain a fused feature map;

[0026] The fused feature map is convolved to obtain the optimized feature map corresponding to the feature map output by the adaptive perception module.

[0027] In some possible implementations, the multiple optimized feature maps include a first optimized feature map, a second optimized feature map, a third optimized feature map, and a fourth optimized feature map;

[0028] The process of fusing multiple optimized feature maps and inputting them into the segmentation head to obtain the polyp segmentation image output by the segmentation head includes:

[0029] The first optimized feature map and the second optimized feature map are input into the comprehensive feature extraction module for fusion processing to obtain the first fused optimized feature map;

[0030] The first optimized feature map and the second optimized feature map are input into the comprehensive feature extraction module for fusion processing to obtain the second fused optimized feature map;

[0031] After reconstructing the first fused optimized feature map and the second fused optimized feature map respectively, a first reconstructed feature map corresponding to the first fused optimized feature map and a second reconstructed feature map corresponding to the second fused optimized feature map are obtained.

[0032] The first reconstructed feature map and the second reconstructed feature map are input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0033] In some possible implementations, after using the deformable convolutional structure in the adaptive perception module to perform flexible feature processing on multiple feature maps of different scales to obtain an optimized feature map corresponding to each feature map, the method further includes:

[0034] Each optimized feature map is upsampled to obtain the target optimized feature map corresponding to each optimized feature map.

[0035] The process of fusing multiple optimized feature maps and inputting them into the segmentation head to obtain the polyp segmentation image output by the segmentation head includes:

[0036] After fusing multiple target optimized feature maps, the results are input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0037] Secondly, embodiments of this application also provide an endoscopic image polyp segmentation apparatus, which includes a unit for performing the above-described method.

[0038] Thirdly, embodiments of this application also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0039] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0040] This application provides a method, apparatus, computer device, and storage medium for polyp segmentation in endoscopic images. The method includes: acquiring an endoscopic image to be processed; inputting the endoscopic image to be processed into a trained neural network model, and using a feature extraction layer to extract features from the endoscopic image to obtain multiple feature maps of different scales; inputting the multiple feature maps of different scales into an adaptive perception module, and using a deformable convolutional structure in the adaptive perception module to perform flexible feature processing on the multiple feature maps of different scales respectively, obtaining an optimized feature map corresponding to each feature map; fusing the multiple optimized feature maps and then inputting them into a segmentation head to obtain a polyp segmentation image output by the segmentation head.

[0041] This application embodiment extracts features from medical endoscope images at multiple scales, dynamically adjusting the observation range according to the polyp morphology. This overcomes the limitations of traditional fixed-window methods. Secondly, the flexible representation of features is achieved by utilizing the deformable convolutional structure in the adaptive perception module, which can effectively handle features with irregular boundaries and enable the convolutional kernel to adaptively match the polyp boundary. Furthermore, the fusion of multiple optimized feature maps through the comprehensive feature extraction module enables multi-level feature interaction and fine boundary reconstruction, improving the accuracy of polyp segmentation. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0045] Figure 1 A schematic flowchart of the first embodiment of an endoscopic image polyp segmentation method provided in this application;

[0046] Figure 2 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0049] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0050] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0051] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0052] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0053] Endoscopic technology enables high-resolution real-time imaging of the gastrointestinal tract, facilitating the precise identification and localization of polyps. Accurately defining the morphological characteristics of polyps (especially subtle boundary variations) is crucial for the early diagnosis of precancerous lesions and the development of targeted treatment plans.

[0054] Currently, the field of medical image segmentation is dominated by two main architectures: Convolutional Neural Networks (CNNs) and Transformers. While CNN-based methods (such as the U-Net architecture) can effectively capture local image features, their fixed receptive field limits accurate segmentation of structures that rely on long-range dependencies. Transformer-based methods (such as UNETR), through their self-attention mechanism, can effectively utilize global information from the entire image, but they still have limitations when dealing with polyp morphologies that vary greatly.

[0055] Existing segmentation frameworks use rectangular convolution kernels of fixed size to perform regular grid sampling on input images. This rigid computational paradigm has the following drawbacks: (1) It cannot adapt to the diversity and deformability of polyp morphology, resulting in inaccurate segmentation of polyps with irregular shapes and blurred boundaries; (2) The deep representation layer is limited by a fixed observation range, causing a mismatch between the regular receptive field and the actual polyp scale or deformation; (3) Simply expanding the observation range without solving the need for adaptive scale determination and flexible receptive field makes it difficult to accurately segment polyps of different shapes and sizes.

[0056] To address the technical problem that existing segmentation frameworks use fixed-size rectangular convolution kernels to perform regular grid sampling on input images, resulting in low accuracy of polyp segmentation results, this application provides an endoscopic image polyp segmentation method that can improve the accuracy of polyp segmentation.

[0057] See Figure 1 This application provides a method for polyp segmentation in endoscopic images. The trained neural network model includes a feature extraction layer, an adaptive perception module, a comprehensive feature extraction module, and a segmentation head. The method includes:

[0058] Step 110: Obtain the endoscope image to be processed.

[0059] Step 120: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0060] Step 130: Input multiple feature maps of different scales into the adaptive perception module, and use the deformable convolutional structure in the adaptive perception module to perform flexible feature processing on the multiple feature maps of different scales respectively, so as to obtain the optimized feature map corresponding to each feature map.

[0061] Step 140: After fusing multiple optimized feature maps, input them into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0062] This embodiment extracts features at multiple scales from medical endoscope images, dynamically adjusting the observation range based on polyp morphology. This overcomes the limitations of traditional methods that use fixed windows. Furthermore, the flexible feature representation achieved by utilizing the deformable convolutional structure in the adaptive perception module effectively handles features with irregular boundaries, enabling the convolutional kernel to adaptively match polyp boundaries. Finally, the fusion of multiple optimized feature maps through the comprehensive feature extraction module enables multi-level feature interaction and fine boundary reconstruction, improving the accuracy of polyp segmentation.

[0063] The trained neural network model can distinguish polyp regions from normal tissue at the pixel level and output corresponding segmentation masks. This segmentation process requires no manual intervention; once the input image meets the processing conditions of the trained neural network model, the system can automatically complete the segmentation task. The trained neural network model typically adopts an encoder-decoder architecture, extracting multi-scale features through convolutional neural networks to ultimately generate binarized segmentation results. Once deployed in a medical diagnostic system, the algorithm flow and parameter settings remain fixed. During clinical use, the model structure cannot be temporarily modified or its inference process interrupted, thus exhibiting deterministic execution rigidity.

[0064] Because endoscopic image segmentation technology must achieve extremely high precision standards to ensure that the segmentation boundaries perfectly match the actual lesions, and the accuracy of the segmentation results directly determines the reliability of clinical diagnosis, this technology, in practical deployment, must ensure that it can stably output segmentation results with sub-pixel precision to avoid false positives or false negatives caused by missegmentation.

[0065] In this embodiment, when an endoscope image segmentation request is received, the system obtains the target endoscope image to be processed. The request may include the original image data to be processed, or it may include the storage path of the target endoscope image. That is, the target endoscope image may be uploaded by the user when submitting the processing request, or it may be automatically retrieved by the system from a specified storage location based on the request.

[0066] After acquiring the medical endoscopic image to be processed, a multi-scale feature representation is first generated through a feature extraction layer. This representation consists of local features at different scales and cross-scale contextual features to comprehensively characterize the multi-level information of polyps in the endoscopic image.

[0067] In some possible implementations, the adaptive perception module includes a feature segmentation unit, a first processing submodule, and a second processing submodule. Referring to a second embodiment of an endoscopic image polyp segmentation method provided in this application, the second embodiment includes the following steps:

[0068] Step 210: Obtain the endoscope image to be processed.

[0069] Step 220: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0070] Step 230: Input the feature map into the feature segmentation unit, and use the feature segmentation unit to segment the feature map into a first sub-feature map and a second sub-feature map.

[0071] Step 240: Use the first processing submodule to perform local processing on the first sub-feature map to obtain a local detail feature map.

[0072] Step 250: Use the second processing submodule to perform global processing on the second sub-feature map to obtain a global structure feature map.

[0073] Step 260: Based on the local detail feature map and the global structure feature map, obtain the optimized feature map corresponding to the feature map.

[0074] Step 270: After fusing multiple optimized feature maps, input them into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0075] In some possible implementations, the first processing submodule includes multiple adaptive sensing units. Referring to a third embodiment of an endoscopic image polyp segmentation method provided in this application, the third embodiment includes the following steps:

[0076] Step 310: Obtain the endoscope image to be processed.

[0077] Step 320: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0078] Step 330: Input the feature map into the feature segmentation unit, and use the feature segmentation unit to segment the feature map into a first sub-feature map and a second sub-feature map.

[0079] Step 340: Extract local features from the first sub-feature map to obtain multiple local feature maps.

[0080] Step 350: Use each of the adaptive sensing units to perform local flexible processing on the corresponding local feature map to obtain the local flexible feature map corresponding to each of the local feature maps.

[0081] Step 360: Average the multiple local flexible feature maps to obtain the local detail feature map output by the first processing submodule.

[0082] Step 370: Use the second processing submodule to perform global processing on the second sub-feature map to obtain a global structure feature map.

[0083] Step 380: Based on the local detail feature map and the global structure feature map, obtain the optimized feature map corresponding to the feature map.

[0084] Step 390: After fusing multiple optimized feature maps, input them into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0085] In some possible implementations, the second processing submodule includes a first depthwise convolutional layer, a 3*3 convolutional layer, and a second depthwise convolutional layer. Referring to a fourth embodiment of an endoscopic image polyp segmentation method provided in this application, the fourth embodiment includes the following steps:

[0086] Step 410: Obtain the endoscope image to be processed.

[0087] Step 420: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0088] Step 430: Input the feature map into the feature segmentation unit, and use the feature segmentation unit to segment the feature map into a first sub-feature map and a second sub-feature map.

[0089] Step 440: Use the first processing submodule to perform local processing on the first sub-feature map to obtain a local detail feature map.

[0090] Step 450: Input the second sub-feature map into the first depth convolutional layer for depth convolution processing to obtain the depth convolutional image output by the first depth convolutional layer.

[0091] Step 460: Input the depth convolution image into the 3*3 convolutional layer for convolution and normalization processing to obtain a normalized depth convolution image.

[0092] Step 470: Input the normalized depth convolution image into the second depth convolution layer for secondary depth convolution processing to obtain the global structure feature map output by the second depth convolution layer.

[0093] Step 480: Based on the local detail feature map and the global structure feature map, obtain the optimized feature map corresponding to the feature map.

[0094] Step 490: After fusing multiple optimized feature maps, input them into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0095] Referring to the fifth embodiment of the endoscopic image polyp segmentation method provided in this application, the fifth embodiment includes the following steps:

[0096] Step 510: Obtain the endoscope image to be processed.

[0097] Step 520: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0098] Step 530: Input the feature map into the feature segmentation unit, and use the feature segmentation unit to segment the feature map into a first sub-feature map and a second sub-feature map.

[0099] Step 540: Use the first processing submodule to perform local processing on the first sub-feature map to obtain a local detail feature map.

[0100] Step 550: Use the second processing submodule to perform global processing on the second sub-feature map to obtain a global structure feature map.

[0101] Step 560: Perform channel-level fusion processing on the local detail feature map and the global structure feature map to obtain a fused feature map.

[0102] Step 570: Perform convolution processing on the fused feature map to obtain the optimized feature map corresponding to the feature map output by the adaptive perception module.

[0103] Step 580: After fusing multiple optimized feature maps, input them into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0104] In some possible implementations, the multiple optimized feature maps include a first optimized feature map, a second optimized feature map, a third optimized feature map, and a fourth optimized feature map. Referring to a sixth embodiment of an endoscopic image polyp segmentation method provided in this application, the sixth embodiment includes the following steps:

[0105] Step 610: Obtain the endoscope image to be processed.

[0106] Step 620: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0107] Step 630: Input multiple feature maps of different scales into the adaptive perception module, and use the deformable convolutional structure in the adaptive perception module to perform flexible feature processing on the multiple feature maps of different scales respectively, so as to obtain the optimized feature map corresponding to each feature map.

[0108] Step 640: Input the first optimized feature map and the second optimized feature map into the integrated feature extraction module for fusion processing to obtain the first fused optimized feature map.

[0109] Step 650: Input the first optimized feature map and the second optimized feature map into the integrated feature extraction module for fusion processing to obtain the second fused optimized feature map.

[0110] Step 660: After reconstructing the first fusion optimized feature map and the second fusion optimized feature map respectively, a first reconstructed feature map corresponding to the first fusion optimized feature map and a second reconstructed feature map corresponding to the second fusion optimized feature map are obtained.

[0111] Step 670: Input the first reconstructed feature map and the second reconstructed feature map into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0112] In some possible implementations, after using the deformable convolutional structure in the adaptive perception module to perform flexible feature processing on multiple feature maps of different scales to obtain an optimized feature map corresponding to each feature map, an upsampling operation can be performed on each optimized feature map to obtain a target optimized feature map corresponding to each optimized feature map. Specifically, refer to the seventh embodiment of an endoscopic image polyp segmentation method provided in this application. The seventh embodiment includes the following steps:

[0113] Step 710: Obtain the endoscope image to be processed.

[0114] Step 720: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0115] Step 730: Input multiple feature maps of different scales into the adaptive perception module, and use the deformable convolutional structure in the adaptive perception module to perform flexible feature processing on the multiple feature maps of different scales respectively, so as to obtain the optimized feature map corresponding to each feature map.

[0116] Step 740: Perform an upsampling operation on each of the optimized feature maps to obtain the target optimized feature map corresponding to each optimized feature map.

[0117] Step 750: After fusing multiple target optimized feature maps, input them into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0118] Based on the above embodiments, the endoscopic image polyp segmentation method provided in this application mainly includes the following steps:

[0119] S1: Obtain the endoscope image to be processed.

[0120] S2: Input the endoscope image to be processed into the trained neural network model, and use the feature extraction layer to extract features from the endoscope image to obtain a first-scale feature map, a second-scale feature map, a third-scale feature map, and a fourth-scale feature map.

[0121] The resolutions of the first-scale feature map, the second-scale feature map, the third-scale feature map, and the fourth-scale feature map can decrease sequentially. For example, the size of the first-scale feature map is (H / 4, W / 4, C), the size of the second-scale feature map is (H / 8, W / 8, 2C), the size of the third-scale feature map is (H / 16, W / 16, 4C), and the size of the fourth-scale feature map is (H / 32, W / 32, 8C).

[0122] The first, second, third, and fourth scale feature maps correspond to feature extraction from shallow to deep layers, with resolutions ranging from high to low. The highest resolution shallow features primarily capture fine details such as the polyp's edge contours, microstructure, and local textures; the second highest resolution mid-shallow features focus on characterizing the polyp's local morphological features; the medium resolution mid-deep features are used to model the polyp's overall shape and regional characteristics; and the lowest resolution deep features focus on learning the semantic relationships between the polyp and surrounding tissues, as well as global contextual information.

[0123] S3: Input the first-scale feature map, the second-scale feature map, the third-scale feature map, and the fourth-scale feature map into the adaptive perception module respectively, and use the adaptive perception module to perform flexible feature processing to obtain the first optimized feature map corresponding to the first-scale feature map, the second optimized feature map corresponding to the second-scale feature map, the third optimized feature map corresponding to the third-scale feature map, and the fourth optimized feature map corresponding to the fourth-scale feature map.

[0124] The dimensions of the first optimized feature map are (H / 4, W / 4, C), the dimensions of the second optimized feature map are (H / 4, W / 4, C), the dimensions of the third optimized feature map are (H / 8, W / 8, C), and the dimensions of the fourth optimized feature map are (H / 8, W / 8, C).

[0125] The adaptive perception module includes a feature segmentation unit, a first processing submodule, and a second processing submodule. Each optimized feature map can be obtained using methods S31-S34.

[0126] S31: Input the optimized feature map into the feature segmentation unit, and use the feature segmentation unit to segment the feature map into a first sub-feature map and a second sub-feature map.

[0127] Assuming the size of the optimized feature map of the input feature segmentation unit is (H, W, n*C), then the size of the first sub-feature map and the second sub-feature map are both (H, W, n*C / 2).

[0128] S32: The first processing submodule is used to perform local processing on the first sub-feature map to obtain a local detail feature map.

[0129] S32 may include the following steps:

[0130] S321: Extract local features from the first sub-feature map to obtain multiple local feature maps.

[0131] S322: Each of the adaptive sensing units performs local flexible processing on the corresponding local feature map to obtain a local flexible feature map corresponding to each local feature map.

[0132] In this way, the parallel adaptive sensing units achieve adaptive offset of the sampling position through deformable convolution operation, so that the shape of the convolution kernel can fit the irregular boundary of the polyp.

[0133] Specifically, the sampling positions that the network generates in spatial variations can be predicted by offset, so that the shape of the convolution kernel can dynamically adapt to the geometric deformation of the polyp boundary.

[0134] S322 mainly includes the following steps:

[0135] S3221: Input the local feature map into the dynamic convolutional layer for convolution processing to obtain the dynamic convolutional local feature map.

[0136] S3222: Input the dynamic convolutional local feature map into the deep convolutional layer for deep convolution processing and normalization processing to obtain the first normalized convolutional local feature map.

[0137] S3223: Input the dynamic convolutional local feature map into a 3*3 convolutional layer for convolution and normalization processing to obtain the second normalized convolutional local feature map.

[0138] S3224: The local feature map, the first normalized convolutional local feature map, and the second normalized convolutional local feature map are fused to obtain the local flexible feature map corresponding to the local feature map.

[0139] The dimensions of the local feature map, the dynamic convolutional local feature map, the first normalized convolutional local feature map, the second normalized convolutional local feature map, and the local flexible feature map are all (H, W, C / 2).

[0140] S323: The multiple local flexible feature maps are averaged to obtain the local detail feature map output by the first processing submodule.

[0141] The number of adaptive sensing units depends on n. For example, if n is 6, then the number of adaptive sensing units is 6, and the number of local feature maps obtained is also 6.

[0142] The dimensions of the local feature map and the local flexible feature map are both (H, W, C / 2).

[0143] S33: The second processing submodule is used to perform global processing on the second sub-feature map to obtain a global structural feature map.

[0144] The second processing submodule, feature retention stream, can establish long-distance dependencies through a spatial-channel dual attention mechanism to accurately capture the global distribution pattern of polyps.

[0145] S33 may include the following steps:

[0146] S331: Input the second sub-feature map into the first depth convolutional layer for depth convolution processing to obtain the depth convolutional image output by the first depth convolutional layer.

[0147] S332: Input the depth convolution image into the 3*3 convolutional layer for convolution and normalization processing to obtain a normalized depth convolution image.

[0148] S333: Input the normalized depth convolution image into the second depth convolution layer for secondary depth convolution processing to obtain the global structure feature map output by the second depth convolution layer.

[0149] In step S33, the image dimensions are all (H, W, C / 2).

[0150] S34: Based on the local detail feature map and the global structure feature map, obtain the optimized feature map corresponding to the feature map.

[0151] Among them, a dynamic feature fusion gating mechanism can be adopted to automatically adjust the fusion weights of local details and global features based on the semantic context of the current feature map.

[0152] Channel attention mechanisms can enhance the characteristic responses of polyp margins and key lesion regions while suppressing interfering signals from irrelevant mucosal background.

[0153] S34 may include the following steps:

[0154] S341: Perform channel-level fusion processing on the local detail feature map and the global structural feature map to obtain a fused feature map.

[0155] The size of the local detail feature map is (H, W, C / 2), the size of the global structure feature map is (H, W, C / 2), and the size of the fused feature map is (H, W, C).

[0156] S342: Perform convolution processing on the fused feature map to obtain the optimized feature map corresponding to the feature map output by the adaptive perception module.

[0157] S4: After fusing multiple optimized feature maps, input them into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0158] S4 includes the following steps:

[0159] S41: Input the first optimized feature map and the second optimized feature map into the comprehensive feature extraction module for fusion processing to obtain the first fused optimized feature map.

[0160] Please refer to the following instructions for details.

[0161] S42: Input the first optimized feature map and the second optimized feature map into the comprehensive feature extraction module for fusion processing to obtain the second fused optimized feature map.

[0162] The size of the first fusion optimized feature map is (H / 4, W / 4, 2C), and the size of the second fusion optimized feature map is (H / 8, W / 8, 2C).

[0163] S43: After reconstructing the first fusion optimized feature map and the second fusion optimized feature map respectively, a first reconstructed feature map corresponding to the first fusion optimized feature map and a second reconstructed feature map corresponding to the second fusion optimized feature map are obtained.

[0164] Among these methods, a progressive upsampling approach can be used to gradually reconstruct the microstructure of the polyp boundary, enabling the polyp segmentation structure to accurately reflect the subtle changes at the interface between the polyp and normal tissue while maintaining the correctness of the topological structure.

[0165] The first and second reconstructed feature maps have the same dimensions as the input endoscope image to be processed, both being (H, W, C), where H is the height, W is the width, and C is the number of channels.

[0166] S44: Input the first reconstructed feature map and the second reconstructed feature map into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0167] The aforementioned adaptive perception module extracts local detail feature maps and global structural feature maps of the endoscopic image through parallel adaptive perception units and feature preservation streams (i.e., the second processing submodule). The adaptive perception unit employs a deformable convolutional structure to dynamically adjust the receptive field size and shape of the unit based on the actual morphological characteristics of the polyps in the medical endoscopic image. This ensures that the extracted local detail features better match the location and shape of the polyps in the medical endoscopic image to be processed. The feature preservation stream, on the other hand, captures global contextual features through a self-attention mechanism. Finally, a channel attention mechanism adaptively fuses the local detail features extracted by the adaptive perception unit with the global structural features acquired by the feature preservation stream to generate a multi-scale optimized feature map containing multi-level semantic information.

[0168] The deformable convolutional structure achieves accurate representation primarily through the following techniques: First, an offset prediction network analyzes the local geometry of the input features to generate a dynamic sampling grid that matches the polyp morphology. Second, a multi-scale feature compensation mechanism uses parallel deformable convolutional branches with different dilation rates to ensure the capture of a large-scale context while preserving local details. Finally, an adaptive feature calibration module highlights the importance of lesion-related feature channels through grouped channel attention weights. This flexible representation method is particularly suitable for handling flat polyps with blurred boundaries and lobulated polyps with complex morphologies.

[0169] The aforementioned integrated feature extraction module includes a bidirectional feature interaction path and a cascaded feature refinement mechanism. The bidirectional feature interaction path includes a bottom-up detail transfer path and a top-down semantic guidance path, enabling comprehensive interaction from high-resolution shallow features to semantically rich deep features, achieving cross-resolution feature fusion. The cascaded feature refinement mechanism includes spatial dimension region localization refinement and channel dimension feature representation optimization, as well as progressive upsampling and multi-level feature fusion, gradually reconstructing the microstructure of polyp boundaries, and finally outputting sub-pixel level precision segmentation results.

[0170] The endoscopic imaging device used in this application is mainly used to acquire real-time images or video data of the patient's digestive tract and transmit the raw data to a processor or graphics processing accelerator for processing via a communication bus. The device may include an optical lens, an image sensor, a light source module, and a signal conversion module, and supports modes such as white light imaging, narrow-band imaging (NBI), or fluorescence imaging.

[0171] Corresponding to the above-described endoscopic image polyp segmentation method, this application also provides an endoscopic image polyp segmentation apparatus. This endoscopic image polyp segmentation apparatus includes a unit for performing the above-described endoscopic image polyp segmentation method, and can be configured in a desktop computer, tablet computer, laptop computer, or other terminal.

[0172] like Figure 2 As shown in the figure, this application provides a computer device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114.

[0173] Memory 113 is used to store computer programs;

[0174] In one embodiment of this application, the processor 111, when executing a program stored in the memory 113, implements the endoscopic image polyp segmentation method provided in any of the foregoing method embodiments, including:

[0175] Acquire the endoscopic image to be processed;

[0176] The endoscope image to be processed is input into a trained neural network model, and the feature extraction layer is used to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0177] Multiple feature maps of different scales are input into the adaptive perception module, and the deformable convolutional structure in the adaptive perception module is used to perform flexible feature processing on the multiple feature maps of different scales respectively to obtain the optimized feature map corresponding to each feature map.

[0178] After fusing multiple optimized feature maps, the data is input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0179] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0180] Therefore, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the endoscopic image polyp segmentation method provided in any of the foregoing method embodiments.

[0181] Acquire the endoscopic image to be processed;

[0182] The endoscope image to be processed is input into a trained neural network model, and the feature extraction layer is used to extract features from the endoscope image to obtain multiple feature maps of different scales.

[0183] Multiple feature maps of different scales are input into the adaptive perception module, and the deformable convolutional structure in the adaptive perception module is used to perform flexible feature processing on the multiple feature maps of different scales respectively to obtain the optimized feature map corresponding to each feature map.

[0184] After fusing multiple optimized feature maps, the data is input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

[0185] The storage medium is a physical, non-transitory storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0186] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0187] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0188] The steps in the methods of this application embodiment can be adjusted, merged, or deleted according to actual needs. The units in the apparatus of this application embodiment can be merged, divided, or deleted according to actual needs. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0189] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0190] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0191] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Since these modifications and variations fall within the scope of the claims and their equivalents, this application also intends to include these modifications and variations.

[0192] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for polyp segmentation in endoscopic images, characterized in that, The trained neural network model includes a feature extraction layer, an adaptive perception module, a comprehensive feature extraction module, and a segmentation head. The adaptive perception module includes a feature segmentation unit, a first processing submodule, and a second processing submodule. The first processing submodule includes multiple adaptive perception units, and the second processing submodule includes a first deep convolutional layer and a 3D convolutional layer. The method includes: 3 convolutional layers and a second depthwise convolutional layer. Acquire the endoscopic image to be processed; The endoscope image to be processed is input into a trained neural network model, and the feature extraction layer is used to extract features from the endoscope image to obtain multiple feature maps of different scales. Inputting multiple feature maps of different scales into the adaptive perception module specifically includes: The feature map is input into the feature segmentation unit, and the feature segmentation unit is used to segment the feature map into a first sub-feature map and a second sub-feature map. The first processing submodule is used to perform local processing on the first sub-feature map to obtain a local detail feature map, including: extracting local features from the first sub-feature map to obtain multiple local feature maps; using each of the adaptive perception units to perform local flexible processing on the corresponding local feature map to obtain a local flexible feature map corresponding to each local feature map; and averaging the multiple local flexible feature maps to obtain the local detail feature map. The second processing submodule performs global processing on the second sub-feature map to obtain a global structure feature map, including: inputting the second sub-feature map into the first depthwise convolutional layer for depthwise convolution processing to obtain a depthwise convolutional image output by the first depthwise convolutional layer; inputting the depthwise convolutional image into the 3 The third convolutional layer performs convolution and normalization processing to obtain a normalized depth convolutional image; and the normalized depth convolutional image is input into the second depth convolutional layer for secondary depth convolution processing to obtain the global structure feature map. Based on the local detail feature map and the global structure feature map, an optimized feature map corresponding to each feature map is obtained; The multiple optimized feature maps are fused using the integrated feature extraction module and then input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

2. The method according to claim 1, characterized in that, The process of obtaining the optimized feature map corresponding to the feature map based on the local detail feature map and the global structure feature map includes: The local detail feature map and the global structure feature map are fused at the channel level to obtain a fused feature map; The fused feature map is convolved to obtain the optimized feature map corresponding to the feature map output by the adaptive perception module.

3. The method according to claim 1, characterized in that, The multiple optimized feature maps include a first optimized feature map, a second optimized feature map, a third optimized feature map, and a fourth optimized feature map; After fusing multiple optimized feature maps, the data is input into the segmentation head to obtain a polyp segmentation image output by the segmentation head, including: The first optimized feature map and the second optimized feature map are input into the comprehensive feature extraction module for fusion processing to obtain the first fused optimized feature map; The first optimized feature map and the second optimized feature map are input into the comprehensive feature extraction module for fusion processing to obtain the second fused optimized feature map; After reconstructing the first fused optimized feature map and the second fused optimized feature map respectively, a first reconstructed feature map corresponding to the first fused optimized feature map and a second reconstructed feature map corresponding to the second fused optimized feature map are obtained. The first reconstructed feature map and the second reconstructed feature map are input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

4. The method according to claim 1, characterized in that, After using the deformable convolutional structure in the adaptive perception module to perform flexible feature processing on multiple feature maps of different scales to obtain an optimized feature map corresponding to each feature map, the method further includes: Each optimized feature map is upsampled to obtain the target optimized feature map corresponding to each optimized feature map. The process of fusing multiple optimized feature maps and inputting them into the segmentation head to obtain the polyp segmentation image output by the segmentation head includes: After fusing multiple target optimized feature maps, the results are input into the segmentation head to obtain the polyp segmentation image output by the segmentation head.

5. An endoscopic image polyp segmentation device, characterized in that, The endoscopic image polyp segmentation device includes a unit for performing the method as described in any one of claims 1-4.

6. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-4.

Images