A polyp image segmentation method and system based on detail restoration network

Abstract

The present invention relates to the field of image segmentation, and proposes a polyp image segmentation method and system based on a detail restoration network. A detail restoration network is designed, including a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module. The detail enhancement module retains rich detail information in low-level features, enhances detail features, and further improves the accuracy of segmentation. The feature seamless integration module supplements high-level semantic information, avoids feature dislocation caused by semantic gaps, and further improves the accuracy of segmentation. The selective feature aggregation module obtains detail feature weights and high-level semantic feature weights to dynamically adjust the emphasis on detail features and high-level semantic features. The present invention greatly improves the accuracy of polyp image segmentation.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation, and in particular to a polyp image segmentation method and system based on a detail restoration network. Background Art

[0002] With the rapid development of computer vision technology, image segmentation is increasingly widely used in image processing in various scenarios. Especially in polyp images, manual marking of polyps is very time-consuming, so image segmentation has gradually become an important part of polyp image processing.

[0003] In the prior art, convolutional neural networks are usually used to segment polyp images. However, polyps themselves are of different sizes and shapes, especially smaller polyp targets, which are difficult to capture due to the loss of detail information in the neural network. In addition, the contrast between the polyp target and its surrounding tissue is low, resulting in blurred polyp boundaries and difficult to accurately segment the polyp edges. Existing convolutional neural networks often cannot cope with such complex polyp image segmentation situations, resulting in poor segmentation accuracy.

[0004] Therefore, how to design a polyp image segmentation method to avoid the influence of polyp morphology and boundary fuzziness is an urgent problem to be solved. Summary of the invention

[0005] Based on this, the purpose of the present invention is to provide a polyp image segmentation method and system based on a detail restoration network. By designing a detail restoration network, including a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module, the detail enhancement module retains the rich detail information in the low-level features, enhances the detail features, and further improves the accuracy of segmentation. The feature seamless integration module supplements the high-level semantic information, avoids the feature dislocation caused by the semantic gap, and further improves the accuracy of segmentation. The selective feature aggregation module obtains the detail feature weights and the high-level semantic feature weights to dynamically adjust the emphasis on the detail features and the high-level semantic features. The present invention greatly improves the accuracy of polyp image segmentation.

[0006] The present invention proposes a polyp image segmentation method based on a detail restoration network, comprising:

[0007] Acquire a target polyp image and preprocess it, and input the preprocessed polyp image into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module;

[0008] The basic feature extraction module extracts features from the polyp image according to the feature level to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, medium-level basic features and high-level basic features, and then generates a guide map according to the basic features;

[0009] The detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide mapping to obtain detail enhancement features, wherein the detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement;

[0010] The feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement feature to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the order of feature levels to obtain seamless integration features;

[0011] After the selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature, it obtains the detail feature weight and the high-level semantic feature weight, and then weighted-enhances the detail enhancement feature and the seamless integration feature according to the detail feature weight and the high-level semantic feature weight, respectively, to obtain the final segmentation result.

[0012] In summary, according to the above-mentioned polyp image segmentation method based on the detail restoration network, a detail restoration network is designed, including a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module. The detail enhancement module retains the rich detail information in the low-level features, enhances the detail features, and further improves the accuracy of segmentation. The feature seamless integration module supplements the high-level semantic information, avoids the feature dislocation caused by the semantic gap, and further improves the accuracy of segmentation. The selective feature aggregation module obtains the detail feature weights and the high-level semantic feature weights to dynamically adjust the emphasis on the detail features and the high-level semantic features. The present invention greatly improves the accuracy of polyp image segmentation. Specifically, a target polyp image is acquired and preprocessed, and the preprocessed polyp image is input into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module. The basic feature extraction module performs feature extraction on the polyp image according to the feature level to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, mid-level basic features, and high-level basic features, and then a guide map is generated according to the basic features. The detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide map to obtain detail enhancement features. The detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement, and retains The rich detail information in the low-level features is obtained. The feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement features to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the sequence of feature levels to obtain seamless integration features and supplement high-level semantic information to avoid feature dislocation caused by semantic gaps. The selective feature aggregation module aggregates the detail enhancement features and the seamless integration features to obtain detail feature weights and high-level semantic feature weights, and then weighted enhances the detail enhancement features and the seamless integration features according to the detail feature weights and the high-level semantic feature weights to obtain the final segmentation result. The present invention greatly improves the accuracy of polyp image segmentation.

[0013] Furthermore, the basic feature extraction module extracts features from the polyp image according to the feature level to obtain basic features at multiple feature levels, and then generates a guide mapping according to the basic features, specifically including:

[0014] The basic feature extraction module includes a parallel connection block and a plurality of feature encoders, wherein the feature encoders are linearly connected, each of which corresponds to a single feature level. After obtaining the polyp image, the feature encoders obtain basic features according to the order of feature levels, wherein the basic features include low-level basic features, medium-level basic features and high-level basic features;

[0015] The parallel connection block performs coarse segmentation according to the mid-level basic features and the high-level basic features to obtain a guide map;

[0016] All of the basic features and the guide map are input into a detail enhancement module.

[0017] Further, the detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide mapping to obtain detail enhancement features, and the detail enhancement includes the steps of foreground weighted detail enhancement and background weighted detail enhancement, specifically including:

[0018] After obtaining the basic features and guided mapping, the mid-level basic features and low-level basic features are cleaned according to the high-level basic features to obtain key features. The key features corresponding to the mid-level basic features and low-level basic features are fused after passing through the receptive field block to obtain receptive field enhancement features.

[0019] The algorithm for obtaining the receptive field enhancement feature is as follows:

[0020]

[0021] in, represents the receptive field enhancement feature, express Convolutional blocks, represents the feature fusion operation, Indicates the key features corresponding to the low-level basic features, Indicates the key features corresponding to the mid-level basic features;

[0022] Performing foreground detail weighting and background detail weighting on the guide map to obtain a foreground detail guide map and a background detail guide map;

[0023] According to the foreground detail guide map and the background detail guide map, respectively, the receptive field enhancement feature is subjected to foreground weighted detail enhancement and background weighted detail enhancement to obtain a detail enhancement feature;

[0024] The algorithm for obtaining detail enhancement features is as follows:

[0025] ,

[0026] ,

[0027] ,

[0028] in, represents the foreground weighted detail enhancement feature, represents the background weighted detail enhancement feature, Indicates the expansion operation. represents the activation function, represents the upsampling operation, represents the boot map, represents an all-1 matrix, Represents detail enhancement features.

[0029] Further, the feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement feature to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the sequence of feature levels to obtain the step of seamless integration features, specifically including:

[0030] The feature seamless integration module includes an object revision submodule and a group extension fusion submodule;

[0031] After obtaining the basic features and detail enhancement features, the object revision submodule performs detail fusion processing on the basic features of different levels according to the detail enhancement features to obtain detail fusion features. Each level of the basic features has a unique corresponding detail fusion feature, and then performs attention enhancement processing on the detail fusion features according to the channel attention mechanism to obtain boundary attention enhancement features of multiple different feature levels;

[0032] The group expansion fusion submodule performs cascade fusion processing on boundary attention enhancement features of different feature levels, and splits two boundary attention enhancement features of adjacent feature levels into feature groups along the channel dimension according to the feature level order from high to low to obtain the same number of feature groups, and then seamlessly integrates them according to the inter-group crossover mechanism to obtain seamless integrated features.

[0033] Furthermore, the object revision submodule specifically includes:

[0034] The specific algorithm of the object revision submodule is as follows:

[0035]

[0036] ,

[0037] in, represents the detail fusion feature, represents the boundary attention enhancement feature, Represents the basic features, express Convolutional blocks, represents the global average pooling operation, express Convolutional blocks, Represents detail enhancement features.

[0038] Furthermore, the group expansion fusion submodule specifically includes:

[0039] The specific algorithm of the group expansion fusion submodule is as follows:

[0040]

[0041] ,

[0042] in, Indicates seamless integration features, express Convolutional blocks, express Dilated convolutional blocks, Indicates seamless integrated operation, represents the fusion features between feature groups, Represents a feature group.

[0043] Furthermore, the selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature, obtains the detail feature weight and the high-level semantic feature weight, and then weighted-enhances the detail enhancement feature and the seamless integration feature according to the detail feature weight and the high-level semantic feature weight, respectively, to obtain the final segmentation result, specifically including:

[0044] After obtaining the detail enhancement feature and the seamless integration feature aggregation, the selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature to obtain the weight feature;

[0045] The detail feature weight and the high-level semantic feature weight are obtained respectively according to the weight feature. The algorithm for obtaining the detail feature weight and the high-level semantic feature weight is as follows:

[0046]

[0047] in, represents the detail feature weight, represents the weight of high-level semantic features, represents the activation function, FC represents the fully connected layer, represents the global average pooling operation, represents the weight feature;

[0048] The detail enhancement feature and the seamless integration feature are weightedly enhanced according to the detail feature weight and the high-level semantic feature weight to obtain a final segmentation result;

[0049] The algorithm for obtaining the final segmentation result is as follows:

[0050]

[0051] in, represents the final segmentation result, represents the detail enhancement feature, Indicates seamless integration characteristics.

[0052] The present invention proposes a polyp image segmentation system based on a detail restoration network, comprising:

[0053] A preprocessing module, used for acquiring and preprocessing a target polyp image, and inputting the preprocessed polyp image into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module;

[0054] A basic feature extraction module, used for performing feature extraction on the polyp image according to feature levels to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, medium-level basic features and high-level basic features, and then generating a guide map according to the basic features;

[0055] A detail enhancement module, used to perform receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then perform detail enhancement according to the guide mapping to obtain detail enhancement features, wherein the detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement;

[0056] A feature seamless integration module, used for performing boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement feature to obtain boundary attention enhancement features of multiple feature levels, and then cascading and fusing the boundary attention enhancement features according to the sequence of feature levels to obtain seamless integration features;

[0057] The selective feature aggregation module is used to aggregate the detail enhancement features and the seamless integration features to obtain the detail feature weights and the high-level semantic feature weights, and then weighted enhance the detail enhancement features and the seamless integration features according to the detail feature weights and the high-level semantic feature weights to obtain the final segmentation result.

[0058] The present invention also provides a storage medium, wherein the storage medium stores one or more programs, and when the programs are executed by a processor, the polyp image segmentation method based on the detail restoration network as described above is implemented.

[0059] The present invention also provides a computer device, the computer device comprising a memory and a processor, wherein:

[0060] The memory is used to store computer programs;

[0061] When the processor is used to execute the computer program stored in the memory, it implements the polyp image segmentation method based on the detail restoration network as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a polyp image segmentation method based on a detail restoration network proposed in the first embodiment of the present invention;

[0063] Figure 2 This is a flow chart of a polyp image segmentation method based on a detail restoration network proposed in a second embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of the structure of a polyp image segmentation system based on a detail restoration network proposed in the third embodiment of the present invention.

[0065] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0066] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0067] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0069] See also Figure 1 , which is a flow chart of a polyp image segmentation method based on a detail restoration network proposed in a first embodiment of the present invention, the polyp image segmentation method based on a detail restoration network comprises steps S01 to S05, wherein:

[0070] Step S01: Acquire a target polyp image and perform preprocessing, and input the preprocessed polyp image into a detail restoration network;

[0071] Step S02: the basic feature extraction module extracts features from the polyp image according to the feature level to obtain basic features at multiple feature levels, and then generates a guide map according to the basic features;

[0072] It should be noted that the basic feature extraction module in this embodiment includes a parallel connection block and a plurality of feature encoders, the feature encoders are linearly connected, each of the feature encoders corresponds to a single feature level, and after obtaining the polyp image, the feature encoders obtain basic features according to the order of feature levels, and the basic features include low-level basic features, medium-level basic features and high-level basic features;

[0073] The parallel connection block performs coarse segmentation according to the mid-level basic features and the high-level basic features to obtain a guide map;

[0074] Inputting all of the basic features and the guide map into a detail enhancement module;

[0075] The low-level basic features in this embodiment refer to the basic features obtained by the feature encoder corresponding to the lowest feature level, and the high-level basic features refer to the basic features obtained by the feature encoder of the highest feature level. Except for the feature encoders corresponding to the highest feature level and the lowest feature level, the basic features obtained by the remaining feature encoders are mid-level basic features.

[0076] Step S03: the detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guided mapping to obtain detail enhancement features;

[0077] It should be noted that, in this embodiment, after the basic features and the guide mapping are obtained, the mid-level basic features and the low-level basic features are cleaned up according to the high-level basic features to obtain key features, and the key features corresponding to the mid-level basic features and the low-level basic features are fused after passing through the receptive field block to obtain the receptive field enhancement feature;

[0078] The algorithm for obtaining the receptive field enhancement feature is as follows:

[0079]

[0080] in, represents the receptive field enhancement feature, express Convolutional blocks, represents the feature fusion operation, Indicates the key features corresponding to the low-level basic features, Indicates the key features corresponding to the mid-level basic features;

[0081] Performing foreground detail weighting and background detail weighting on the guide map to obtain a foreground detail guide map and a background detail guide map;

[0082] According to the foreground detail guide map and the background detail guide map, respectively, the receptive field enhancement feature is subjected to foreground weighted detail enhancement and background weighted detail enhancement to obtain a detail enhancement feature;

[0083] The algorithm for obtaining detail enhancement features is as follows:

[0084] ,

[0085] ,

[0086] ,

[0087] in, represents the foreground weighted detail enhancement feature, represents the background weighted detail enhancement feature, Indicates the expansion operation. represents the activation function, represents the upsampling operation, represents the boot map, represents an all-1 matrix, Represents detail enhancement features.

[0088] Step S04: the feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement features to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the order of feature levels to obtain seamless integration features;

[0089] It should be noted that the feature seamless integration module in this embodiment includes an object revision submodule and a group expansion fusion submodule;

[0090] After obtaining the basic features and detail enhancement features, the object revision submodule performs detail fusion processing on the basic features of different levels according to the detail enhancement features to obtain detail fusion features. Each level of the basic features has a unique corresponding detail fusion feature, and then performs attention enhancement processing on the detail fusion features according to the channel attention mechanism to obtain boundary attention enhancement features of multiple different feature levels;

[0091] The group expansion fusion submodule performs cascade fusion processing on boundary attention enhancement features of different feature levels, splits two boundary attention enhancement features of adjacent feature levels into feature groups along the channel dimension according to the feature level order from high to low, so as to obtain the same number of feature groups, and then seamlessly integrates them according to the inter-group crossover mechanism to obtain seamless integrated features;

[0092] The specific algorithm of the object revision submodule is as follows:

[0093]

[0094] ,

[0095] in, represents the detail fusion feature, represents the boundary attention enhancement feature, Represents the basic features, express Convolutional blocks, represents the global average pooling operation, express Convolutional blocks, Represents detail enhancement features;

[0096] The specific algorithm of the group expansion fusion submodule is as follows:

[0097]

[0098] ,

[0099] in, Indicates seamless integration features, express Convolutional blocks, express Dilated convolutional blocks, Indicates seamless integrated operation, represents the fusion features between feature groups, Represents a feature group.

[0100] Step S05: After the selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature, it obtains the detail feature weight and the high-level semantic feature weight, and then weighted-enhances the detail enhancement feature and the seamless integration feature according to the detail feature weight and the high-level semantic feature weight, respectively, to obtain the final segmentation result;

[0101] It should be noted that, in this embodiment, after the detail enhancement feature and the seamless integration feature are aggregated, the selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature to obtain a weight feature;

[0102] The detail feature weight and the high-level semantic feature weight are obtained respectively according to the weight feature. The algorithm for obtaining the detail feature weight and the high-level semantic feature weight is as follows:

[0103]

[0104] in, represents the detail feature weight, represents the weight of high-level semantic features, represents the activation function, FC represents the fully connected layer, represents the global average pooling operation, represents the weight feature;

[0105] The detail enhancement feature and the seamless integration feature are weightedly enhanced according to the detail feature weight and the high-level semantic feature weight to obtain a final segmentation result;

[0106] The algorithm for obtaining the final segmentation result is as follows:

[0107]

[0108] in, represents the final segmentation result, represents the detail enhancement feature, Indicates seamless integration characteristics.

[0109] In summary, according to the above-mentioned polyp image segmentation method based on the detail restoration network, a detail restoration network is designed, including a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module. The detail enhancement module retains the rich detail information in the low-level features, enhances the detail features, and further improves the accuracy of segmentation. The feature seamless integration module supplements the high-level semantic information, avoids the feature dislocation caused by the semantic gap, and further improves the accuracy of segmentation. The selective feature aggregation module obtains the detail feature weights and the high-level semantic feature weights to dynamically adjust the emphasis on the detail features and the high-level semantic features. The present invention greatly improves the accuracy of polyp image segmentation. Specifically, a target polyp image is acquired and preprocessed, and the preprocessed polyp image is input into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module. The basic feature extraction module performs feature extraction on the polyp image according to the feature level to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, mid-level basic features, and high-level basic features, and then a guide map is generated according to the basic features. The detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide map to obtain detail enhancement features. The detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement, and retains The rich detail information in the low-level features is obtained. The feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement features to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the sequence of feature levels to obtain seamless integration features and supplement high-level semantic information to avoid feature dislocation caused by semantic gaps. The selective feature aggregation module aggregates the detail enhancement features and the seamless integration features to obtain detail feature weights and high-level semantic feature weights, and then weighted enhances the detail enhancement features and the seamless integration features according to the detail feature weights and the high-level semantic feature weights to obtain the final segmentation result. The present invention greatly improves the accuracy of polyp image segmentation.

[0110] See also Figure 2 , which is a flow chart of a polyp image segmentation method based on a detail restoration network proposed in a second embodiment of the present invention, the polyp image segmentation method based on a detail restoration network comprises steps S11 to S16, wherein:

[0111] Step S11: acquiring a target polyp image and performing preprocessing, and inputting the preprocessed polyp image into a detail restoration network;

[0112] Step S12: After the polyp image is acquired, the feature encoder acquires basic features according to the order of feature levels, the basic features including low-level basic features, medium-level basic features and high-level basic features, and performs rough segmentation according to the medium-level basic features and the high-level basic features to acquire a guide map;

[0113] Step S13: After obtaining the basic features and the guide map, the mid-level basic features and the low-level basic features are cleaned up according to the high-level basic features to obtain key features, the key features corresponding to the mid-level basic features and the low-level basic features are fused after passing through the receptive field block to obtain the receptive field enhancement feature, the guide map is weighted foreground details and background details to obtain the foreground detail guide map and the background detail guide map, and the receptive field enhancement feature is enhanced by foreground weighted details and background weighted details according to the foreground detail guide map and the background detail guide map to obtain the detail enhancement feature;

[0114] Step S14: after obtaining the basic features and the detail enhancement features, the basic features of different levels are subjected to detail fusion processing according to the detail enhancement features to obtain detail fusion features, and then the detail fusion features are subjected to attention enhancement processing according to the channel attention mechanism to obtain boundary attention enhancement features of multiple different feature levels;

[0115] Step S15: performing cascade fusion processing on the boundary attention enhancement features of different feature levels, splitting the two boundary attention enhancement features of adjacent feature levels into feature groups along the channel dimension according to the feature level order from high to low, so as to obtain the same number of feature groups, and then seamlessly integrating them according to the inter-group crossover mechanism to obtain seamless integrated features;

[0116] Step S16: After the detail enhancement feature and the seamless integration feature are obtained, the detail enhancement feature and the seamless integration feature are aggregated to obtain a weight feature, and the detail feature weight and the high-level semantic feature weight are respectively obtained according to the weight feature, and the detail enhancement feature and the seamless integration feature are weightedly enhanced according to the detail feature weight and the high-level semantic feature weight to obtain a final segmentation result;

[0117] It should be noted that the polyp image segmentation method based on the detail restoration network of the present invention is compared with the existing advanced model in an experiment, and the experimental results are shown in Tables 1 and 2 below:

[0118] Table 1

[0119]

[0120] Table 2

[0121]

[0122] Among them, data set 1 and data set 2 are the invisible data sets ColonDB and ETIS respectively, and the evaluation indicators 1, 2, 3, 4, 5, and 6 are mDice, mIoU, wFm, Sm, maxEm, and mae respectively. Based on the data in the above table, it can be seen that the polyp image segmentation effect of the present invention is significantly improved compared with the existing image segmentation model, and the accuracy of polyp image segmentation is greatly improved.

[0123] In summary, according to the above-mentioned polyp image segmentation method based on the detail restoration network, a detail restoration network is designed, including a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module. The detail enhancement module retains the rich detail information in the low-level features, enhances the detail features, and further improves the accuracy of segmentation. The feature seamless integration module supplements the high-level semantic information, avoids the feature dislocation caused by the semantic gap, and further improves the accuracy of segmentation. The selective feature aggregation module obtains the detail feature weights and the high-level semantic feature weights to dynamically adjust the emphasis on the detail features and the high-level semantic features. The present invention greatly improves the accuracy of polyp image segmentation. Specifically, a target polyp image is acquired and preprocessed, and the preprocessed polyp image is input into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module. The basic feature extraction module performs feature extraction on the polyp image according to the feature level to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, mid-level basic features, and high-level basic features, and then a guide map is generated according to the basic features. The detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide map to obtain detail enhancement features. The detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement, and retains The rich detail information in the low-level features is obtained. The feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement features to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the sequence of feature levels to obtain seamless integration features and supplement high-level semantic information to avoid feature dislocation caused by semantic gaps. The selective feature aggregation module aggregates the detail enhancement features and the seamless integration features to obtain detail feature weights and high-level semantic feature weights, and then weighted enhances the detail enhancement features and the seamless integration features according to the detail feature weights and the high-level semantic feature weights to obtain the final segmentation result. The present invention greatly improves the accuracy of polyp image segmentation.

[0124] See also Figure 3, which is a schematic diagram of the structure of a polyp image segmentation system based on a detail restoration network proposed in the third embodiment of the present invention, and the system includes:

[0125] A preprocessing module 10 is used to obtain a target polyp image and perform preprocessing, and input the preprocessed polyp image into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module;

[0126] A basic feature extraction module 20 is used to extract features from the polyp image according to feature levels to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, medium-level basic features and high-level basic features, and then generate a guide map according to the basic features;

[0127] A detail enhancement module 30, configured to perform receptive field enhancement according to the basic features to obtain a receptive field enhancement feature, and then perform detail enhancement according to the guide mapping to obtain a detail enhancement feature, wherein the detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement;

[0128] A feature seamless integration module 40 is used to perform boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement feature to obtain boundary attention enhancement features of multiple feature levels, and then cascade fuse the boundary attention enhancement features according to the order of feature levels to obtain seamless integration features;

[0129] The selective feature aggregation module 50 is used to aggregate the detail enhancement features and the seamless integration features, obtain the detail feature weights and the high-level semantic feature weights, and then weighted enhance the detail enhancement features and the seamless integration features according to the detail feature weights and the high-level semantic feature weights to obtain the final segmentation result.

[0130] The present invention also proposes a computer storage medium on which one or more programs are stored, and when the program is executed by a processor, the above-mentioned polyp image segmentation method based on the detail restoration network is implemented.

[0131] The present invention also proposes a computer device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned polyp image segmentation method based on the detail restoration network.

[0132] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain storage, communication, propagation or transmission of a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0133] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0134] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0135] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0136] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A polyp image segmentation method based on detail restoration network, characterized in that: include: Acquire a target polyp image and preprocess it, and input the preprocessed polyp image into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module; The basic feature extraction module extracts features from the polyp image according to the feature level to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, medium-level basic features and high-level basic features, and then generates a guide map according to the basic features; The detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide mapping to obtain detail enhancement features, wherein the detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement; The detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide mapping to obtain detail enhancement features, wherein the detail enhancement includes the steps of foreground weighted detail enhancement and background weighted detail enhancement, specifically including: After obtaining the basic features and guided mapping, the mid-level basic features and low-level basic features are cleaned according to the high-level basic features to obtain key features. The key features corresponding to the mid-level basic features and low-level basic features are fused after passing through the receptive field block to obtain receptive field enhancement features. The algorithm for obtaining the receptive field enhancement feature is as follows: in, represents the receptive field enhancement feature, express Convolutional blocks, represents the feature fusion operation, Indicates the key features corresponding to the low-level basic features, Indicates the key features corresponding to the mid-level basic features; Performing foreground detail weighting and background detail weighting on the guide map to obtain a foreground detail guide map and a background detail guide map; According to the foreground detail guide map and the background detail guide map, respectively, the receptive field enhancement feature is subjected to foreground weighted detail enhancement and background weighted detail enhancement to obtain a detail enhancement feature; The algorithm for obtaining detail enhancement features is as follows: ， ， ， in, represents the foreground weighted detail enhancement feature, represents the background weighted detail enhancement feature, Indicates the expansion operation. represents the activation function, represents the upsampling operation, represents the boot map, represents an all-1 matrix, Represents detail enhancement features; The feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement feature to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the order of feature levels to obtain seamless integration features; After the selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature, it obtains the detail feature weight and the high-level semantic feature weight, and then weighted-enhances the detail enhancement feature and the seamless integration feature according to the detail feature weight and the high-level semantic feature weight, respectively, to obtain the final segmentation result.

2. The polyp image segmentation method based on detail restoration network according to claim 1 is characterized in that: The basic feature extraction module extracts features from the polyp image according to the feature level to obtain basic features at multiple feature levels, and then generates a guide mapping according to the basic features, specifically comprising: The basic feature extraction module includes a parallel connection block and a plurality of feature encoders, wherein the feature encoders are linearly connected, each of which corresponds to a single feature level. After obtaining the polyp image, the feature encoders obtain basic features according to the order of feature levels, wherein the basic features include low-level basic features, medium-level basic features and high-level basic features; The parallel connection block performs coarse segmentation according to the mid-level basic features and the high-level basic features to obtain a guide map; All of the basic features and the guide map are input into a detail enhancement module.

3. The polyp image segmentation method based on detail restoration network according to claim 1, characterized in that: The feature seamless integration module performs boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement feature to obtain boundary attention enhancement features of multiple feature levels, and then cascades and fuses the boundary attention enhancement features according to the order of feature levels to obtain the step of seamless integration features, specifically including: The feature seamless integration module includes an object revision submodule and a group extension fusion submodule; After obtaining the basic features and detail enhancement features, the object revision submodule performs detail fusion processing on the basic features of different levels according to the detail enhancement features to obtain detail fusion features. Each level of the basic features has a unique corresponding detail fusion feature, and then performs attention enhancement processing on the detail fusion features according to the channel attention mechanism to obtain boundary attention enhancement features of multiple different feature levels; The group expansion fusion submodule performs cascade fusion processing on boundary attention enhancement features of different feature levels, and splits two boundary attention enhancement features of adjacent feature levels into feature groups along the channel dimension according to the feature level order from high to low to obtain the same number of feature groups, and then seamlessly integrates them according to the inter-group crossover mechanism to obtain seamless integrated features.

4. The polyp image segmentation method based on detail restoration network according to claim 3 is characterized in that: The object revision submodule specifically includes: The specific algorithm of the object revision submodule is as follows: ， in, represents the detail fusion feature, represents the boundary attention enhancement feature, Represents the basic features, express Convolutional blocks, represents the global average pooling operation, express Convolutional blocks, Represents detail enhancement features.

5. The polyp image segmentation method based on detail restoration network according to claim 3 is characterized in that: The group expansion fusion submodule specifically includes: The specific algorithm of the group expansion fusion submodule is as follows: ， in, Indicates seamless integration features, express Convolutional blocks, express Dilated convolutional blocks, Indicates seamless integrated operation, represents the fusion features between feature groups, Represents a feature group.

6. The polyp image segmentation method based on detail restoration network according to claim 1, characterized in that: The selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature, obtains the detail feature weight and the high-level semantic feature weight, and then weighted-enhances the detail enhancement feature and the seamless integration feature according to the detail feature weight and the high-level semantic feature weight, respectively, to obtain the final segmentation result, specifically comprising: After obtaining the detail enhancement feature and the seamless integration feature aggregation, the selective feature aggregation module aggregates the detail enhancement feature and the seamless integration feature to obtain the weight feature; The detail feature weight and the high-level semantic feature weight are obtained respectively according to the weight feature. The algorithm for obtaining the detail feature weight and the high-level semantic feature weight is as follows: in, represents the detail feature weight, represents the weight of high-level semantic features, represents the activation function, FC represents the fully connected layer, represents the global average pooling operation, represents the weight feature; The detail enhancement feature and the seamless integration feature are weightedly enhanced according to the detail feature weight and the high-level semantic feature weight to obtain a final segmentation result; The algorithm for obtaining the final segmentation result is as follows: in, represents the final segmentation result, represents the detail enhancement feature, Indicates seamless integration characteristics.

7. A polyp image segmentation system based on detail restoration network, characterized in that: include: A preprocessing module, used for acquiring and preprocessing a target polyp image, and inputting the preprocessed polyp image into a detail restoration network, wherein the detail restoration network includes a basic feature extraction module, a detail enhancement module, a feature seamless integration module, and a selective feature aggregation module; A basic feature extraction module, used for performing feature extraction on the polyp image according to feature levels to obtain basic features at multiple feature levels, wherein the basic features include low-level basic features, medium-level basic features and high-level basic features, and then generating a guide map according to the basic features; A detail enhancement module, used to perform receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then perform detail enhancement according to the guide mapping to obtain detail enhancement features, wherein the detail enhancement includes foreground weighted detail enhancement and background weighted detail enhancement; The detail enhancement module performs receptive field enhancement according to the basic features to obtain receptive field enhancement features, and then performs detail enhancement according to the guide mapping to obtain detail enhancement features, wherein the detail enhancement includes the steps of foreground weighted detail enhancement and background weighted detail enhancement, specifically including: After obtaining the basic features and guided mapping, the mid-level basic features and low-level basic features are cleaned according to the high-level basic features to obtain key features. The key features corresponding to the mid-level basic features and low-level basic features are fused after passing through the receptive field block to obtain receptive field enhancement features. The algorithm for obtaining the receptive field enhancement feature is as follows: in, represents the receptive field enhancement feature, express Convolutional blocks, represents the feature fusion operation, Indicates the key features corresponding to the low-level basic features, Indicates the key features corresponding to the mid-level basic features; Performing foreground detail weighting and background detail weighting on the guide map to obtain a foreground detail guide map and a background detail guide map; According to the foreground detail guide map and the background detail guide map, respectively, the receptive field enhancement feature is subjected to foreground weighted detail enhancement and background weighted detail enhancement to obtain a detail enhancement feature; The algorithm for obtaining detail enhancement features is as follows: ， ， ， in, represents the foreground weighted detail enhancement feature, represents the background weighted detail enhancement feature, Indicates the expansion operation. represents the activation function, represents the upsampling operation, represents the boot map, represents an all-1 matrix, Represents detail enhancement features; A feature seamless integration module, used for performing boundary attention enhancement processing on the basic features of multiple feature levels according to the detail enhancement feature to obtain boundary attention enhancement features of multiple feature levels, and then cascading and fusing the boundary attention enhancement features according to the sequence of feature levels to obtain seamless integration features; The selective feature aggregation module is used to aggregate the detail enhancement features and the seamless integration features to obtain the detail feature weights and the high-level semantic feature weights, and then weighted enhance the detail enhancement features and the seamless integration features according to the detail feature weights and the high-level semantic feature weights to obtain the final segmentation result.

8. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by the processor, implement the polyp image segmentation method based on the detail restoration network as described in any one of claims 1 to 6.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the polyp image segmentation method based on detail restoration network described in any one of claims 1-6.

Images