Digestive endoscopy image deblurring enhancement method and system

Abstract

The invention relates to the technical field of medical image processing, in particular to a digestive endoscopy image deblurring enhancement method and system.The method comprises the steps that firstly, an input digestive endoscopy original image is processed through a blurred region classification network, and a pixel-level blurred classification map capable of distinguishing an adhesion blurred region and a motion blurred region is generated; then, parallel processing is carried out according to the classification graph: for an adhesion fuzzy region, physical model restoration and color correction are carried out by estimating a transmissivity graph and an ambient light value; for a motion blur region, a self-adaptive non-blind deconvolution kernel is constructed to perform deconvolution sharpness. And finally, inputting the two processing results and the original clear area into a multi-scale feature fusion network together, carrying out adaptive feature weighted fusion and image reconstruction, and outputting a globally clear and detail-enhanced final image. According to the method, accurate identification and targeted enhancement of composite blurring are realized, and the visual quality and diagnosis availability of the digestive endoscopy image are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and system for deblurring and enhancing digestive endoscopy images. Background Technology

[0002] Digestive endoscopy is a core tool for diagnosing gastrointestinal diseases, and its imaging quality directly affects the detection rate and diagnostic accuracy of early lesions. However, in practice, the images acquired often suffer from quality degradation due to various interferences, resulting in complex blurring. These blurrings mainly stem from two distinct types of degradation: first, localized adhesion blurring, caused by contaminants such as blood, mucus, or food residue coming into contact with the endoscope lens, manifesting as localized obstruction, uneven light transmission, and color distortion; second, motion blur, caused by physiological peristalsis of the digestive tract wall, patient respiration, or the movement of the endoscope itself, manifesting as image structure blurring and detail mixing. These two types of blurring often intertwine in the same image, forming a complex degradation scenario that severely interferes with doctors' observation of subtle mucosal morphology and capillary patterns.

[0003] Currently, existing technologies for image deblurring suffer from the following limitations: General deblurring algorithms (such as blind deconvolution) typically assume that the image is affected by a globally uniform blur kernel, making it difficult to effectively handle non-uniform situations where localized adhesion blur and global motion blur coexist, and easily producing ringing artifacts in edge regions; while end-to-end methods based on deep learning can learn certain mapping relationships, their "black box" nature may cause crucial diagnostic features to be smoothed or distorted during enhancement, and the model's generalization ability heavily depends on the training data. More critically, existing methods generally lack the ability to identify the causes of blurring, failing to distinguish between "adhesion blur" and "motion blur" at the pixel level. This leads to the use of a single strategy to process all degraded areas, failing to address the root causes and differentiated restoration of the optical absorption characteristics of adhesion blur and the information mixing characteristics of motion blur. Therefore, when processing complex blurred digestive endoscopy images commonly seen in clinical practice, existing technologies often suffer from incomplete enhancement, resulting in limited enhancement effects and potentially introducing new distortions. Summary of the Invention

[0004] This invention provides a method and system for deblurring and enhancing digestive endoscopy images.

[0005] A method for deblurring and enhancing digestive endoscopy images includes the following steps:

[0006] S1: Input the original digestive endoscopy image to be processed, process it through a fuzzy region classification network, and output a fuzzy classification map including an attached fuzzy region mask and a motion fuzzy region mask;

[0007] S2: Perform parallel processing based on the fuzzy classification map, including restoration of the attached fuzzy region and sharpening of the motion fuzzy region, wherein;

[0008] The restoration of the adhesion blurry area is as follows: for the area marked by the adhesion blurry area mask, the transmittance map and ambient light value of the area are estimated, a preliminary restored image is calculated, and the preliminary restored image is color corrected to obtain a color-corrected adhesion blurry area image.

[0009] The motion-blurred region sharpening is achieved by: constructing an adaptive non-blind deconvolution kernel for the region marked by the motion-blurred region mask, and using the deconvolution kernel to perform deconvolution operation to obtain a sharpened image after motion blur removal;

[0010] S3: The color-corrected attached blurred area image, the motion-blurred sharpened image, and the sharp area in the original image are fused and reconstructed through a multi-scale feature fusion network to output the final enhanced image.

[0011] Optionally, S1 includes:

[0012] S11: Receive the raw image of the digestive endoscopy and input it into a pre-trained fuzzy region classification network;

[0013] S12: The fuzzy region classification network performs multi-scale feature extraction and fusion on the original digestive endoscopy image to generate a comprehensive feature map containing deep semantic information;

[0014] S13: Based on the comprehensive feature map, the fuzzy region classification network calculates the probability of belonging to each pixel through a classifier, and outputs a first probability map of each pixel belonging to the attached fuzzy region, a second probability map of each pixel belonging to the motion fuzzy region, and a third probability map of each pixel belonging to the clear region.

[0015] S14: Competitive decision-making is performed on the first probability map, the second probability map, and the third probability map to assign each pixel to the category with the highest probability, thereby generating a fuzzy classification map that simultaneously contains an attached fuzzy region mask and a motion fuzzy region mask.

[0016] Optionally, the fuzzy region classification network is an encoder-decoder structure. The encoder performs multi-level downsampling on the original digestive endoscopy image to extract multi-scale primary feature maps. The decoder upsamples the multi-scale primary feature maps and performs skip connections and fusion with the features of the corresponding level of the encoder to finally generate the comprehensive feature map.

[0017] Optionally, in the skip connection between the encoder and the decoder, a dual attention unit of channel and space is introduced. This unit processes the feature map from the encoder to generate an attention weight map, and uses the attention weight map to weight the features before fusing them with the features of the decoder, so as to enhance the feature extraction capability of the boundary of the blurred region.

[0018] Optionally, S2 includes:

[0019] S21: Based on the attached blur region mask contained in the blur classification map, extract the local image region corresponding to the attached blur region mask from the original digestive endoscopy image as the attached blur region image block to be restored.

[0020] S22: Perform attached blur region restoration on the attached blur region image block to be restored. Specifically, estimate the transmittance map and ambient light value of the attached blur region image block to be restored based on the dark channel prior model. Use the transmittance map and the ambient light value to obtain the preliminary restored image through atmospheric scattering inverse operation. Then, perform color transfer on the preliminary restored image based on the color distribution of the clear area in the original digestive endoscopy image to generate the color-corrected attached blur region image.

[0021] S23: Based on the motion blur region mask contained in the blur classification map, extract the local image region corresponding to the motion blur region mask from the original digestive endoscopy image as the motion blur region image block to be clarified;

[0022] S24: Perform motion blur region sharpening on the motion-blurred region image block to be sharpened. Specifically, analyze the gradient and texture features of the motion-blurred region image block to be sharpened and its adjacent sharp regions in the original digestive endoscopy image. Using the features of the adjacent sharp regions as prior knowledge, derive an adaptive non-blind deconvolution kernel specifically for the current motion-blurred region image block to be sharpened using the maximum a posteriori probability estimation method. Then, use the adaptive non-blind deconvolution kernel to perform deconvolution operation on the motion-blurred region image block to be sharpened to obtain the sharpened image after de-blurring.

[0023] Optionally, estimating the transmittance map and the ambient light value based on the dark channel prior model specifically involves: calculating the dark channel map of the image block of the attached blurred region to be restored; selecting the original RGB values ​​in the image block of the attached blurred region to be restored from the dark channel map corresponding to the pixels with the highest brightness of the first preset percentage; and using the average of these RGB values ​​as the ambient light value; subsequently, using the ambient light value and a preset atmospheric scattering coefficient, calculating the transmittance map using a formula.

[0024] Optionally, the step of performing color migration on the preliminary restored image based on the color distribution of the clear area in the original digestive endoscopy image specifically involves: calculating the mean and standard deviation of the L channel, a channel, and b channel of the clear area pixels in the original digestive endoscopy image in the LAB color space, and performing a linear transformation on the corresponding channels of the preliminary restored image so that the mean and standard deviation of each channel match the statistics of the clear area, thereby completing the color migration and obtaining the color-corrected attached blurred area image.

[0025] Optionally, S3 includes:

[0026] S31: The color-corrected attached blurry area image, the de-motion-blurred sharpened image, and the sharpened area in the original image extracted from the original digestive endoscopy image according to the blur classification map are used as three parallel input sources and respectively input to the three independent coding branches of the multi-scale feature fusion network.

[0027] S32: The multi-scale feature fusion network extracts features from each input source through convolution and downsampling layers in its coding branch, generating a multi-scale feature map set A corresponding to the color-corrected attached blurred area image, a multi-scale feature map set B corresponding to the de-motion blurred sharpened image, and a multi-scale feature map set C corresponding to the sharp area in the original image.

[0028] S33: In the fusion unit of the multi-scale feature fusion network, for each scale level, the feature maps of the corresponding scale in the multi-scale feature map set A, the multi-scale feature map set B and the multi-scale feature map set C are concatenated by channels, and the features are weighted and fused by the adaptive weights learned by the convolutional layer to generate the fused feature map of that scale level.

[0029] S34: The decoder of the multi-scale feature fusion network upsamples and aggregates the pyramid features composed of the fused feature maps at all scale levels, and reconstructs the final enhanced image with rich spatial details and global semantic consistency through convolutional layers.

[0030] Optionally, the three independent coding branches of the multi-scale feature fusion network have the same network structure. Each coding branch contains four sequentially connected downsampling stages. Each downsampling stage consists of two convolutional layers and one max pooling layer, used to generate the multi-scale feature map set A, the multi-scale feature map set B, and the multi-scale feature map set C, respectively.

[0031] A system for deblurring and enhancing digestive endoscopy images, used to implement the aforementioned method for deblurring and enhancing digestive endoscopy images, includes the following modules:

[0032] Fuzzy Region Classification Module: This module receives the raw input images from the digestive endoscopy system, processes them through a fuzzy region classification network, and outputs a fuzzy classification map containing both attached and motion-fuzzy region masks.

[0033] Targeted deblurring module: used for parallel processing based on the fuzzy classification map, including: an attached fuzzy region restoration unit, used for estimating the transmittance map and ambient light value of the region marked by the attached fuzzy region mask, calculating a preliminary restored image, performing color correction on the preliminary restored image, and outputting a color-corrected attached fuzzy region image; and a motion fuzzy region sharpening unit, used for constructing an adaptive non-blind deconvolution kernel for the region marked by the motion fuzzy region mask, and using the deconvolution kernel to perform deconvolution operation, outputting a sharpened image after motion fuzz removal;

[0034] Multi-scale feature fusion module: This module is used to fuse and reconstruct the features of the color-corrected attached blurred area image, the cleared image after motion blur removal, and the clear areas in the original image through a multi-scale feature fusion network, and output the final enhanced image.

[0035] The beneficial effects of this invention are:

[0036] 1. This invention constructs a pixel-level blur type classification network to automatically and accurately identify and distinguish complex blurs in digestive endoscopy images, generating a blur classification map that includes masks of attached blur regions and masks of motion blur regions. This provides accurate spatial positioning and type discrimination criteria for subsequent processing, thereby solving the fundamental problem of insufficient processing targeting caused by treating blur as a single whole in existing technologies.

[0037] 2. This invention employs a dual-path parallel processing strategy based on physical model restoration and signal reconstruction restoration. Guided by the precise fuzzy classification map, it performs transmittance estimation and color correction on the attached fuzzy region, and adaptive non-blind deconvolution kernel construction and deconvolution operation on the motion fuzzy region. This achieves root-cause differentiated processing for different degradation mechanisms, effectively restoring tissue details and true colors while significantly suppressing the generation of artifacts such as ringing and noise amplification.

[0038] 3. This invention, by designing a multi-scale feature fusion network based on adaptive weights, deeply fuses and reconstructs the processed color-corrected attached blurred area image, the motion-blurred sharpened image, and the original sharp area at the feature level. This achieves natural coherence and visual unity in texture, contrast, and color of the global image, generating a high-quality enhanced image with clear details, realistic colors, and seamless coordination. This greatly enhances the clinical diagnostic value of the image and the trust of doctors in its interpretation. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the system flow according to an embodiment of the present invention. Detailed Implementation

[0042] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0043] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0044] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0045] like Figure 1 As shown, a method for deblurring and enhancing digestive endoscopy images includes the following steps:

[0046] S1: Input the raw image of the digestive endoscopy to be processed, process it through a fuzzy region classification network, and output a fuzzy classification map including attached fuzzy region masks and motion fuzzy region masks. The steps are as follows:

[0047] S11: Receive the raw image from the digestive endoscopy and input it into a pre-trained blurred region classification network, specifically:

[0048] The raw images of the digestive endoscope are acquired by the digestive endoscope imaging equipment. The raw images of the digestive endoscope are stored in the form of a two-dimensional pixel matrix, where each pixel contains the pixel intensity value of three color channels.

[0049] The original images of the digestive endoscopy are processed to unify their size according to the preset input resolution parameters. The original images are scaled using bilinear interpolation to adjust the width and height of the images to the fixed input size required by the fuzzy region classification network.

[0050] After unifying the dimensions, the color channel order of the original digestive endoscopy images is converted to the channel arrangement order used by the network input, and the image data is organized into a three-dimensional data tensor containing image height, image width and number of channels;

[0051] The pixel values ​​in the three-dimensional data tensor are normalized by subtracting the preset mean of the corresponding channel from the channel value of each pixel and dividing it by the preset standard deviation of the corresponding channel, so that the processed pixel value distribution falls into a stable value range, thereby eliminating the influence of brightness and contrast differences between different digestive endoscopy images on the network inference process.

[0052] The three-dimensional data tensor after the above processing is used as the network input and fed into the pre-trained fuzzy region classification network. During the training phase, the fuzzy region classification network is trained by supervised learning using digestive endoscopy image data labeled with attached fuzzy regions, motion fuzzy regions and clear regions. During the execution phase, the network parameters remain fixed.

[0053] S12: The fuzzy region classification network performs multi-scale feature extraction and fusion on the original digestive endoscopy images to generate a comprehensive feature map containing deep semantic information, specifically:

[0054] The three-dimensional data tensor is first input into the encoder part of the fuzzy region classification network. The encoder consists of multiple levels of convolutional layers and downsampling layers, which extract feature representations at different scales by progressively reducing the spatial resolution of the feature map.

[0055] During each downsampling process, the encoder extracts local texture and structural features through convolution operations, and gradually forms semantic representations of fuzzy morphology and regional distribution at deeper levels, thereby generating multi-scale primary feature maps.

[0056] The multi-scale primary feature maps output by the encoder are sequentially input to the decoder. The decoder recovers the spatial resolution of the feature maps through progressive upsampling operations and establishes skip connections with the primary feature maps of the corresponding levels in the encoder during each upsampling process.

[0057] In skip connections, a dual attention unit of channel and space is introduced. The attention unit first performs channel attention calculation on the feature map from the encoder. Specifically, the feature map is globally aggregated in the spatial dimension to obtain the statistical description value of each channel, and the channel weight corresponding to each channel is calculated based on the statistical description value.

[0058] Subsequently, the attention unit performs spatial attention calculation on the feature map, specifically: the feature map is aggregated in the channel dimension to generate a two-dimensional spatial response map, and the spatial weights corresponding to each spatial location are calculated based on the two-dimensional spatial response map.

[0059] The channel weights and spatial weights are applied together to the encoder feature map to enhance the important channels and key spatial locations in the encoder feature map, resulting in a weighted encoder feature map.

[0060] The weighted encoder feature map is fused with the feature map of the current level of the decoder to complete the reconstruction of multi-scale features step by step, and finally outputs a comprehensive feature map containing deep semantic information.

[0061] S13: Based on the comprehensive feature map, the fuzzy region classification network calculates the probability of each pixel belonging to a specific region through a classifier, and outputs a first probability map for each pixel belonging to an attached fuzzy region, a second probability map for belonging to a motion-fuzzy region, and a third probability map for belonging to a sharp region. Specifically:

[0062] The combined feature maps are simultaneously input into a classifier structure consisting of three parallel 1×1 convolutional layers;

[0063] Each 1×1 convolutional layer performs a channel-weighted summation operation on the comprehensive feature map at each pixel location, mapping the feature vector at that pixel location to a single numerical output.

[0064] The first 1×1 convolutional layer outputs a first probability map with the same size as the input image. Each pixel value in the first probability map represents the probability that the pixel belongs to the attached blur region.

[0065] The second 1×1 convolutional layer outputs a second probability map with the same size as the input image. Each pixel value in the second probability map represents the probability that the pixel belongs to the motion blur region.

[0066] The third 1×1 convolutional layer outputs a third probability map with the same size as the input image. Each pixel value in the third probability map represents the probability that the pixel belongs to the sharp region.

[0067] The first, second, and third probability maps correspond one-to-one at the pixel level, providing basic data for subsequent competitive decision-making.

[0068] S14: Competitive decision-making is performed on the first probability map, the second probability map, and the third probability map, assigning each pixel to the category with the highest probability, thereby generating a fuzzy classification map that simultaneously contains both the attached blur region mask and the motion blur region mask, specifically:

[0069] At each pixel location, the probability value of the corresponding pixel is read simultaneously from the first probability map, the second probability map, and the third probability map;

[0070] Perform a pixel-by-pixel comparison on the three probability values ​​to determine the category corresponding to the probability value with the largest value;

[0071] When the maximum probability value corresponds to the first probability map, the pixel is marked as a valid pixel in the attached blur region mask.

[0072] When the maximum probability value corresponds to the second probability map, the pixel is marked as a valid pixel in the motion blur region mask.

[0073] When the maximum probability value corresponds to the third probability map, the pixel is marked as a valid pixel in the clear region mask;

[0074] By performing the above competitive decision-making process on all pixels, a three-channel binary blur classification map is generated, where the first channel is an attached blur region mask, the second channel is a motion blur region mask, and the third channel is a sharp region mask.

[0075] The fuzzy classification map serves as the basis for region division in subsequent steps of restoring attached fuzzy regions and clarifying motion fuzzy regions.

[0076] S2: Parallel processing is performed based on the fuzzy classification map, including restoration of attached fuzzy regions and sharpening of motion fuzzy regions, wherein;

[0077] The restoration of the adherent blurred region involves: estimating the transmittance map and ambient light value of the masked area of ​​the adherent blurred region, calculating a preliminary restored image, and then performing color correction on the preliminary restored image to obtain a color-corrected image of the adherent blurred region. The steps are as follows:

[0078] S21: Based on the attached blur region mask contained in the blur classification map, extract the local image region corresponding to the attached blur region mask from the original digestive endoscopy image, as the attached blur region image block to be restored, specifically:

[0079] Read the fuzzy classification image output by S1 and obtain the attached fuzzy region mask from it;

[0080] Based on the valid pixel positions marked in the adhesive blur region mask, a pixel-by-pixel indexing operation is performed in the original digestive endoscopy image to determine the set of all pixels belonging to the adhesive blur region. The set of pixels belonging to the adhesive blur region is defined as follows:

[0081] ;

[0082] in, This indicates the location of the mask for the adherent blur region at the pixel position. The value at that location, This indicates that the pixel is identified as an area of ​​persistent blur. This represents the set of all pixels in the attached blur region.

[0083] Based on the spatial connectivity of the pixel set, a connectivity analysis is performed on the attached blur region to determine the minimum bounding rectangle region of one or more attached blur regions.

[0084] Using the smallest bounding rectangle region as the cropping window, the corresponding local image region is cropped from the original digestive endoscopy image to form an image block of the attached blurred region to be restored.

[0085] The image patch of the attached blurred region to be restored is used as the input data for subsequent attached blurred region restoration processing.

[0086] S22: Perform attached blur region restoration on the image patch to be restored. Specifically, estimate the transmittance map and ambient light value of the attached blur region image patch to be restored based on the dark channel prior model. Use the transmittance map and ambient light value to obtain a preliminary restored image through atmospheric scattering inverse operation. Then, perform color transfer on the preliminary restored image based on the color distribution of the clear area in the original digestive endoscopy image to generate a color-corrected attached blur region image. Specifically:

[0087] First, a dark channel prior calculation is performed on the image patch of the attached blurred region to be restored. Specifically, within a preset local window, the minimum pixel value in the RGB three color channels is calculated for each pixel position, and this minimum value is used as the pixel value at the corresponding position in the dark channel image, thereby generating the dark channel image of the attached blurred region image patch to be restored. The dark channel image is defined as follows:

[0088] ;

[0089] in, This indicates the image patch representing the attached blurred area to be restored in the channel. pixel values ​​on Indicates A preset local window centered on the user. Indicates the dark channel image at pixel position The value at that location.

[0090] In the dark channel image, sort the pixels according to their brightness values, select the pixels with the highest brightness as the top preset percentage, and obtain the original RGB pixel values ​​of these pixels in the image block of the attached blurred area to be restored.

[0091] Perform a channel-by-channel averaging operation on the original RGB pixel values, and use the resulting average RGB vector as the ambient light value to characterize the global illumination component of the current attached blur region. The ambient light value is defined as:

[0092] ;

[0093] in, This represents the set of pixels with the highest brightness among the preset percentages in the dark channel image. This indicates the location of the image patch representing the attached blurred area to be restored. RGB pixel vector at that location, This indicates the ambient light level.

[0094] After obtaining the ambient light value, and combining it with the preset atmospheric scattering coefficient, the image patch of the attached blurred area to be restored is calculated pixel by pixel according to the atmospheric scattering model to obtain the corresponding transmittance map. The transmittance map represents the proportion of light that is not scattered by the attached object at each pixel position. The transmittance map is calculated as follows:

[0095] ;

[0096] in, This indicates the transmittance map at the pixel location. The value at that location, Indicates the ambient light value in the channel The amount on, This represents the preset atmospheric scattering coefficient.

[0097] Based on the transmittance map and ambient light values, inverse atmospheric scattering is performed on the image patch of the attached blurred region to be restored, recovering the attenuated true image information pixel by pixel, thereby generating a preliminary restored image. The preliminary restored image is calculated as follows:

[0098] ;

[0099] in, This indicates the pixel location of the initially restored image. Pixel value at that location, This represents the minimum transmittance threshold used to prevent numerical instability.

[0100] After the initial restoration is completed, the color distribution of the clear area in the original digestive endoscopy image is used as a reference benchmark. Color transfer processing is then performed on the initial restored image. Specifically, the clear area images in the initial restored image and the original digestive endoscopy image are converted to the LAB color space. The mean and standard deviation of the clear area pixels in the L channel, a channel and b channel are calculated respectively.

[0101] A linear transformation is performed on the corresponding channels of the initially restored image in the LAB color space to ensure that the channel mean and standard deviation are consistent with the statistics of the sharp region, thereby completing the color distribution correction. The linear transformation of each LAB channel is defined as follows:

[0102] ;

[0103] in, This represents the pixel value of the initially restored image in a specific LAB channel. These represent the mean and standard deviation of the preliminary restored image in this channel, respectively. These represent the mean and standard deviation of the clear region in that channel, respectively. This represents the channel pixel value after color correction.

[0104] The image after color transfer processing is converted back to the original color space to obtain the image of the attached blurred area after color correction.

[0105] The process of sharpening motion-blurred regions involves: constructing an adaptive non-blind deconvolution kernel for the region marked by the motion-blurred mask, and then performing deconvolution operations using this kernel to obtain the sharpened image after removing the motion blur. The steps are as follows:

[0106] S23: Based on the motion-blurred region mask contained in the fuzzy classification map, extract the local image region corresponding to the motion-blurred region mask from the original digestive endoscopy image, and use it as the motion-blurred region image block to be clarified. Specifically:

[0107] Read the motion blur region mask from the blur classification map and determine the positions of all pixels marked as motion blur regions;

[0108] Based on the spatial distribution of pixel locations in the original digestive endoscopy image, connected component analysis is performed on the motion-blurred region to identify one or more continuous motion-blurred regions.

[0109] For each motion-blurred region, calculate its corresponding minimum bounding rectangle region, and use this minimum bounding rectangle region as the clipping window;

[0110] Using a cropping window, corresponding local image regions are extracted from the original digestive endoscopy images to form motion-blurred region image blocks to be clarified.

[0111] The motion-blurred region image patch to be sharpened is used as input data for subsequent motion-blurred region sharpening processing.

[0112] S24: Perform motion blur region sharpening on the image patch to be sharpened. Specifically, analyze the gradient and texture features of the motion blur region to be sharpened and its adjacent sharp regions in the original digestive endoscopy image. Using the features of the adjacent sharp regions as prior knowledge, derive an adaptive non-blind deconvolution kernel specifically for the current motion blur region to be sharpened using the maximum a posteriori probability estimation method. Then, use this adaptive non-blind deconvolution kernel to perform deconvolution operation on the motion blur region to be sharpened to obtain the sharpened image after deblurring. Specifically:

[0113] First, the adjacent clear regions of the motion-blurred image patch to be clarified are identified in the original digestive endoscopy image, and the adjacent clear regions are used as the prior feature extraction regions.

[0114] For the sharpened motion-blurred image patch and the adjacent sharp region, a set of preset multi-directional, multi-scale Gabor filter banks are used for convolution operations to obtain the filter response values ​​of each pixel position at different directions and scales. The Gabor convolution operation is expressed as:

[0115] ;

[0116] in, Indicates the first The direction, the first Gabor filters of various scales, This represents the corresponding filter response.

[0117] The filtered response values ​​are combined at each pixel location to form a feature vector containing gradient and texture information, which is used to characterize the local structural characteristics at that location.

[0118] After obtaining the feature vectors, a probabilistic model is established with the motion-blurred region image patch to be sharpened as the observation value and the adaptive non-blind deconvolution kernel and the potentially sharpened image as the variables to be determined.

[0119] In the probabilistic model, the gradient and texture features of adjacent sharp regions are used as natural prior constraints for the potential sharp image. By constraining the structural characteristics of the potential sharp image, the solution process of deconvolution kernel is guided.

[0120] Based on a probabilistic model, the maximum a posteriori probability estimation method is adopted. An iterative optimization algorithm continuously updates the estimates of the adaptive non-blind deconvolution kernel and the potentially sharpened image until the posterior probability reaches the convergence condition. This yields an adaptive non-blind deconvolution kernel specifically for the motion-blurred image patch to be sharpened. The maximum a posteriori probability estimation method is expressed as:

[0121] ;

[0122] in, This represents the adaptive non-blind deconvolution kernel obtained by the maximum a posteriori probability estimation method. This adaptive non-blind deconvolution kernel is used to characterize the motion blur degradation process corresponding to the current motion blur region image patch to be sharpened. This represents the potential sharp image obtained simultaneously through the maximum a posteriori probability estimation method. This potential sharp image is the intermediate sharp image estimate obtained by deconvolution inference of the motion-blurred region image patch to be sharpened under adaptive non-blind deconvolution kernel constraints. This represents an image patch representing the motion-blurred region to be sharpened. This indicates adaptive non-blind deconvolution kernel. Indicates a potentially clear image. Constrained by gradient and texture features of adjacent clear regions.

[0123] After obtaining the adaptive non-blind deconvolution kernel, deconvolution is performed on the motion-blurred region image patch to be sharpened using this kernel. The deconvolution operation employs the Richardson–Lucy iterative deconvolution algorithm, with predetermined iteration numbers and gain parameters set during the iteration process to suppress noise amplification. The Richardson–Lucy iterative deconvolution algorithm is expressed as follows:

[0124] ;

[0125] in, Indicates the first A potentially clearer picture for the next iteration This represents the flipped form of the deconvolution kernel. This is the index for the number of iterations.

[0126] By performing deconvolution, the detailed information caused by motion blur in the motion-blurred region of the image block to be sharpened is recovered, and finally the sharpened image after removing motion blur is obtained.

[0127] S3: The color-corrected image with attached blur, the sharpened image after motion blur removal, and the sharp regions in the original image are fused and reconstructed using a multi-scale feature fusion network to output the final enhanced image. The steps are as follows:

[0128] S31: The color-corrected image of the attached blurred region, the sharpened image after motion blur removal, and the sharp region extracted from the original image of the digestive endoscopy based on the blur classification map are used as three parallel input sources, which are respectively input into the three independent coding branches of the multi-scale feature fusion network, as follows:

[0129] Read the color-corrected attached blur region image and the motion-blurred sharpened image output by S2, and obtain the sharp region mask from the blur classification map output by S1.

[0130] Based on the clear region mask, a pixel-by-pixel indexing operation is performed on the original image of the digestive endoscopy to extract the clear region in the original image containing only clear region pixels, and set the non-clear region pixels to invalid values, thereby forming the third input image;

[0131] Size uniformity processing is performed on the color-corrected image of the attached blurred area, the sharpened image after motion blur removal, and the sharp area in the original image to ensure that the three input images maintain the same spatial resolution and channel structure.

[0132] The three input images are fed into three independent coding branches in the multi-scale feature fusion network. Each coding branch receives data from only one corresponding input source and is independent of each other in terms of network structure, without sharing the feature calculation process.

[0133] S32: The multi-scale feature fusion network extracts features from each input source through convolutional and downsampling layers in its encoding branch, generating a multi-scale feature map set A corresponding to the color-corrected image with attached blur, a multi-scale feature map set B corresponding to the sharpened image after motion blur removal, and a multi-scale feature map set C corresponding to the sharp regions in the original image. Specifically:

[0134] Each coding branch of the multi-scale feature fusion network adopts the same network structure configuration, and each coding branch contains four downsampling stages in sequence;

[0135] In each downsampling stage, the input feature map is first convolved by two consecutive convolutional layers to extract local texture and structural features at that scale.

[0136] Subsequently, the convolutional feature map is spatially downsampled using a max pooling layer to reduce the spatial resolution of the feature map and expand the receptive field, thereby forming a higher-level semantic feature representation.

[0137] In the four downsampling stages, four feature maps with different spatial resolutions are obtained sequentially, and the feature maps output from each downsampling stage in the same coding branch are combined in scale order to form a multi-scale feature map set corresponding to the input source.

[0138] Specifically, a multi-scale feature map set A is generated for the color-corrected attached blurred area image, a multi-scale feature map set B is generated for the sharpened image after motion blur removal, and a multi-scale feature map set C is generated for the sharp area in the original image.

[0139] S33: In the fusion unit of the multi-scale feature fusion network, for each scale level, the feature maps of the corresponding scale in the multi-scale feature map set A, multi-scale feature map set B, and multi-scale feature map set C are concatenated by channels, and then the features are weighted and fused using adaptive weights learned by the convolutional layer to generate the fused feature map at that scale level, specifically:

[0140] At each scale level, feature maps with consistent spatial resolution are selected from multi-scale feature map set A, multi-scale feature map set B, and multi-scale feature map set C, respectively.

[0141] The three selected feature maps are spliced ​​together along the channel dimension to form a spliced ​​feature map containing feature information from the three channels.

[0142] The stitched feature map is input into the sub-network in the fusion unit. The sub-network consists of two consecutive 1×1 convolutional layers, which are used to perform feature mapping on the channel dimension without changing the spatial resolution.

[0143] The subnetwork outputs three sets of attention weight coefficients with the same number of channels as the input feature maps. The three sets of attention weight coefficients correspond to the contribution of multi-scale feature map set A, multi-scale feature map set B, and multi-scale feature map set C at the current scale level, respectively.

[0144] The feature maps from multi-scale feature map set A, multi-scale feature map set B, and multi-scale feature map set C are multiplied with their corresponding attention weight coefficients channel by channel along the channel dimension. The weighted three sets of feature maps are then summed element by element to generate the fused feature map at the current scale level.

[0145] Repeat the above feature concatenation, weight learning, and weighted fusion process for all scale levels to obtain a set of fused feature maps arranged according to scale levels.

[0146] S34: The decoder of the multi-scale feature fusion network upsamples and aggregates the pyramid features composed of fused feature maps from all scale levels, and reconstructs the final enhanced image through convolutional layers, specifically:

[0147] The multi-scale fused feature maps output from step S33 are input into the decoder in order from low resolution to high resolution.

[0148] The decoder consists of four sequentially connected upsampling stages. In each upsampling stage, an upsampling operation is first performed on the feature map from the previous stage to restore a higher spatial resolution.

[0149] After upsampling is completed, the upsampled feature map is fused with the feature map of the corresponding scale from step S33 by skip connection and channel concatenation, thereby introducing detailed feature information at that scale.

[0150] The spliced ​​feature maps are sequentially passed through two convolutional layers for feature integration and reconstruction, so that the fused features are fully expressed in the spatial and channel domains.

[0151] After four upsampling stages of progressive feature aggregation, the resulting high-resolution feature map is input into the last convolutional layer in the decoder, which outputs the final enhanced image with the same size as the input image.

[0152] like Figure 2 As shown, a system for deblurring and enhancing digestive endoscopy images, used to implement the aforementioned method for deblurring and enhancing digestive endoscopy images, includes the following modules:

[0153] Fuzzy Region Classification Module: This module receives the raw input images from the digestive endoscopy system, processes them through a fuzzy region classification network, and outputs a fuzzy classification map containing both attached and motion-fuzzy region masks.

[0154] Targeted deblurring module: This module performs parallel processing based on the fuzzy classification map. It includes: an attached fuzzy region restoration unit, which estimates the transmittance map and ambient light value of the mask-marked area of ​​the attached fuzzy region, calculates a preliminary restored image, performs color correction on the preliminary restored image, and outputs a color-corrected attached fuzzy region image; and a motion fuzzy region sharpening unit, which constructs an adaptive non-blind deconvolution kernel for the mask-marked area of ​​the motion fuzzy region, performs deconvolution operation using the deconvolution kernel, and outputs a sharpened image after motion fuzz removal.

[0155] Multi-scale feature fusion module: This module is used to fuse and reconstruct the features of the color-corrected attached blurred area image, the cleared image after motion blur removal, and the clear areas in the original image through a multi-scale feature fusion network, and output the final enhanced image.

[0156] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0157] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for deblurring and enhancing digestive endoscopy images, characterized in that, Includes the following steps: S1: Input the original digestive endoscopy image to be processed, process it through a fuzzy region classification network, and output a fuzzy classification map including an attached fuzzy region mask and a motion fuzzy region mask; S2: Perform parallel processing based on the fuzzy classification map, including restoration of the attached fuzzy region and sharpening of the motion fuzzy region, wherein; The restoration of the adhesion blurry area is as follows: for the area marked by the adhesion blurry area mask, the transmittance map and ambient light value of the area are estimated, a preliminary restored image is calculated, and the preliminary restored image is color corrected to obtain a color-corrected adhesion blurry area image. The motion-blurred region sharpening is achieved by: constructing an adaptive non-blind deconvolution kernel for the region marked by the motion-blurred region mask, and using the deconvolution kernel to perform deconvolution operation to obtain a sharpened image after motion blur removal; S3: The color-corrected attached blurred area image, the motion-blurred sharpened image, and the sharp area in the original image are fused and reconstructed through a multi-scale feature fusion network to output the final enhanced image.

2. The method for deblurring and enhancing digestive endoscopy images according to claim 1, characterized in that, S1 includes: S11: Receive the raw image of the digestive endoscopy and input it into a pre-trained fuzzy region classification network; S12: The fuzzy region classification network performs multi-scale feature extraction and fusion on the original digestive endoscopy image to generate a comprehensive feature map containing deep semantic information; S13: Based on the comprehensive feature map, the fuzzy region classification network calculates the probability of belonging to each pixel through a classifier, and outputs a first probability map of each pixel belonging to the attached fuzzy region, a second probability map of each pixel belonging to the motion fuzzy region, and a third probability map of each pixel belonging to the clear region. S14: Competitive decision-making is performed on the first probability map, the second probability map, and the third probability map to assign each pixel to the category with the highest probability, thereby generating a fuzzy classification map that simultaneously contains an attached fuzzy region mask and a motion fuzzy region mask.

3. The method for deblurring and enhancing digestive endoscopy images according to claim 2, characterized in that, The fuzzy region classification network is an encoder-decoder structure. The encoder performs multi-level downsampling on the original digestive endoscopy image to extract multi-scale primary feature maps. The decoder upsamples the multi-scale primary feature maps and performs skip connections and fusion with the features of the corresponding level of the encoder to finally generate the comprehensive feature map.

4. The method for deblurring and enhancing digestive endoscopy images according to claim 2, characterized in that, In the skip connection between the encoder and decoder, a dual attention unit of channel and space is introduced. This unit processes the feature map from the encoder to generate an attention weight map, and then uses the attention weight map to weight the features before fusing them with the features of the decoder to enhance the feature extraction capability of the boundary of the blurred region.

5. The method for deblurring and enhancing digestive endoscopy images according to claim 2, characterized in that, S2 includes: S21: Based on the attached blur region mask contained in the blur classification map, extract the local image region corresponding to the attached blur region mask from the original digestive endoscopy image as the attached blur region image block to be restored. S22: Perform attached blur region restoration on the attached blur region image block to be restored. Specifically, estimate the transmittance map and ambient light value of the attached blur region image block to be restored based on the dark channel prior model. Use the transmittance map and the ambient light value to obtain the preliminary restored image through atmospheric scattering inverse operation. Then, perform color transfer on the preliminary restored image based on the color distribution of the clear area in the original digestive endoscopy image to generate the color-corrected attached blur region image. S23: Based on the motion blur region mask contained in the blur classification map, extract the local image region corresponding to the motion blur region mask from the original digestive endoscopy image as the motion blur region image block to be clarified; S24: Perform motion blur region sharpening on the motion-blurred region image block to be sharpened. Specifically, analyze the gradient and texture features of the motion-blurred region image block to be sharpened and its adjacent sharp regions in the original digestive endoscopy image. Using the features of the adjacent sharp regions as prior knowledge, derive an adaptive non-blind deconvolution kernel specifically for the current motion-blurred region image block to be sharpened using the maximum a posteriori probability estimation method. Then, use the adaptive non-blind deconvolution kernel to perform deconvolution operation on the motion-blurred region image block to be sharpened to obtain the sharpened image after de-blurring.

6. The method for deblurring and enhancing digestive endoscopy images according to claim 5, characterized in that, The method of estimating the transmittance map and the ambient light value based on the dark channel prior model is as follows: calculate the dark channel map of the image block of the attached blurry region to be restored, select the original RGB values ​​in the image block of the attached blurry region to be restored corresponding to the pixel with the highest brightness of the first preset percentage from the dark channel map, and take the average value of these RGB values ​​as the ambient light value. Subsequently, the transmittance map is calculated using the ambient light value and a preset atmospheric scattering coefficient through a formula.

7. The method for deblurring and enhancing digestive endoscopy images according to claim 5, characterized in that, The step of color migration of the preliminary restored image based on the color distribution of the clear area in the original digestive endoscopy image is as follows: In the LAB color space, the mean and standard deviation of the L channel, a channel and b channel of the clear area pixels in the original digestive endoscopy image are calculated respectively, and the corresponding channels of the preliminary restored image are linearly transformed so that the mean and standard deviation of each channel match the statistics of the clear area, thereby completing the color migration and obtaining the color-corrected attached blurred area image.

8. The method for deblurring and enhancing digestive endoscopy images according to claim 5, characterized in that, S3 includes: S31: The color-corrected attached blurry area image, the de-motion-blurred sharpened image, and the sharpened area in the original image extracted from the original digestive endoscopy image according to the blur classification map are used as three parallel input sources and respectively input to the three independent coding branches of the multi-scale feature fusion network. S32: The multi-scale feature fusion network extracts features from each input source through convolution and downsampling layers in its coding branch, generating a multi-scale feature map set A corresponding to the color-corrected attached blurred area image, a multi-scale feature map set B corresponding to the de-motion blurred sharpened image, and a multi-scale feature map set C corresponding to the sharp area in the original image. S33: In the fusion unit of the multi-scale feature fusion network, for each scale level, the feature maps of the corresponding scale in the multi-scale feature map set A, the multi-scale feature map set B and the multi-scale feature map set C are concatenated by channels, and the features are weighted and fused by the adaptive weights learned by the convolutional layer to generate the fused feature map of that scale level. S34: The decoder of the multi-scale feature fusion network upsamples and aggregates the pyramid features composed of the fused feature maps at all scale levels, and reconstructs the final enhanced image through convolutional layers.

9. The method for deblurring and enhancing digestive endoscopy images according to claim 8, characterized in that, The three independent coding branches of the multi-scale feature fusion network have the same network structure. Each coding branch contains four sequentially connected downsampling stages. Each downsampling stage consists of two convolutional layers and one max pooling layer, which are used to generate the multi-scale feature map set A, the multi-scale feature map set B, and the multi-scale feature map set C, respectively.

10. A system for deblurring and enhancing digestive endoscopy images, used to implement the method for deblurring and enhancing digestive endoscopy images as described in any one of claims 1-9, characterized in that, Includes the following modules: Fuzzy Region Classification Module: This module receives the raw input images from the digestive endoscopy system, processes them through a fuzzy region classification network, and outputs a fuzzy classification map containing both attached and motion-fuzzy region masks. Targeted deblurring module: used for parallel processing based on the fuzzy classification map, including: an attached fuzzy region restoration unit, used for estimating the transmittance map and ambient light value of the region marked by the attached fuzzy region mask, calculating a preliminary restored image, performing color correction on the preliminary restored image, and outputting a color-corrected attached fuzzy region image; and a motion fuzzy region sharpening unit, used for constructing an adaptive non-blind deconvolution kernel for the region marked by the motion fuzzy region mask, and using the deconvolution kernel to perform deconvolution operation, outputting a sharpened image after motion fuzz removal; Multi-scale feature fusion module: This module is used to fuse and reconstruct the features of the color-corrected attached blurred area image, the cleared image after motion blur removal, and the clear areas in the original image through a multi-scale feature fusion network, and output the final enhanced image.

Images