Diabetic retinopathy fundus image quality improvement method based on edge enhancement

Abstract

The invention discloses a diabetic retinopathy fundus image quality improvement method based on edge enhancement. The method comprises the steps that an original image of diabetic retinopathy fundus is collected and preprocessed to obtain a region-of-interest image; performing multi-scale Gaussian difference operation on the image of the region of interest, obtaining an attention weight map in combination with pathological feature template weighted fusion, and repairing edge fractures through direction controllable sharpening and morphological closed operation to obtain a complete edge enhancement map; carrying out noise suppression on the complete edge enhancement image by adopting an improved anisotropic diffusion algorithm, and superposing the complete edge enhancement image to obtain a detail enhancement image; performing modal processing on the original image to obtain a fused image or an intermediate processing result; lightweight optimization and gray value reverse normalization are carried out on the fused image or the intermediate processing result, and a high-quality fundus image is output; the problems of incomplete edge enhancement coverage, low accuracy, poor multi-modal image fusion information collaboration, prominent contradiction between noise suppression and detail reservation and the like are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a diabetic retinopathy fundus image quality improvement method based on edge enhancement. BACKGROUND

[0002] The clinical diagnosis of diabetic retinopathy (DR) is highly dependent on the clear presentation of key structures such as retinal blood vessels and microangioma in fundus images. However, the existing fundus image quality improvement techniques have two core defects, which seriously affect the accuracy of diagnosis:

[0003] First, the edge enhancement precision is insufficient; the existing technology either uses a single scale edge detection (such as the traditional Canny algorithm), which cannot fully cover the multi-scale structures required for DR diagnosis - the edge of the fine blood vessels with a diameter < 10 pixels is not fully captured, and the boundary of microangioma and hard exudation is blurred; or uses a generalized enhancement strategy (such as global histogram equalization), which does not dynamically adjust the enhancement strength in combination with the anatomical and pathological features of DR fundus, resulting in compression of the blood vessel edge gray gradient, and the subsequent diagnostic system has a high rate of missed diagnosis of subtle lesions;

[0004] Second, the multi-modal image fusion collaboration is poor; the structure clarity of the color fundus image (CFP) and the lesion sensitivity of the fluorescence angiography image (FFA) need to be combined in clinical diagnosis, but the existing fusion algorithms are mostly pixel-level simple superposition (such as weighted average method), without establishing a feature-level dynamic weight mechanism; the blood vessel structure information of CFP and the lesion fluorescence information of FFA cannot be deeply collaborated, and the fused image neither fully retains the fine blood vessel profile of CFP nor effectively highlights the lesion features of FFA, making it difficult to meet the needs of precise staging diagnosis of DR. SUMMARY

[0005] In view of this, the present application proposes a diabetic retinopathy fundus image quality improvement method based on edge enhancement, which can obtain high-quality fundus images that meet medical standards and assist in the diagnosis of diabetic retinopathy.

[0006] The technical solution of the present application is as follows: The diabetic retinopathy fundus image quality improvement method based on edge enhancement comprises the following steps: Step S1, after collecting the original image of the diabetic retinopathy fundus, pre-processing the original image and obtaining the region of interest image; Step S2, performing multi-scale Gaussian difference operation on the region of interest image to obtain a multi-band edge response map, combining a pathological feature template to obtain an attention weight map, and repairing edge breaks through directional controllable sharpening and morphological closing operation to obtain a complete edge enhanced image; Step S3, noise suppression is performed on the complete edge enhanced image by using an improved anisotropic diffusion algorithm, and the denoised image after noise suppression is superimposed on the complete edge enhanced image to obtain a detail enhanced image; Step S4, if the original image is a multi-modal image, feature extraction and fusion are performed on the detail enhanced image to obtain a fused image, and if the original image is a single modal image, the detail enhanced image is taken as an intermediate processing result; Step S5, the fused image or the intermediate processing result is subjected to lightweight optimization and gray value inverse normalization to output a high-quality fundus image.

[0007] Preferably, the step S1 of preprocessing the original image comprises: color space conversion: converting the RGB image of the original image to Lab color space, extracting the luminance channel image, temporarily storing the a and b channel images, and performing resolution adjustment and gray value normalization processing; retinal region segmentation: processing the luminance channel image by using a U-Net segmentation model to output a retinal region mask, and multiplying the original image to obtain an initial region of interest image; illumination unevenness correction: dividing the initial region of interest image into a plurality of sub-blocks, calculating the local entropy value of each sub-block, adjusting the contrast limit parameter according to the local entropy value, and processing the sub-block overlap area by weighted average method to obtain an illumination corrected region of interest image.

[0008] Preferably, the expression of the local entropy value is: ; wherein is the probability of the gray value i in the sub-block, if H>0.8, the contrast limit parameter is set to 1.8; if 0.5≤H≤0.8, the parameter is set to 2.0; if H<0.5, the parameter is set to 2.2.

[0009] Preferably, the specific steps of the step S2 comprise: filtering the region of interest image by using a plurality of groups of different parameter Gaussian kernels, and calculating the difference value of adjacent filtering results to obtain a multi-band edge response map; selecting a plurality of labeled images containing various typical lesions as a training set, training a MobileNet architecture lightweight CNN to form a pathological feature template library; calculating the similarity of the multi-band edge response map and the pathological feature templates in the pathological feature template library by using cosine similarity, selecting the maximum similarity of each frequency band edge response map as the weight coefficient, weighted summing the frequency band edge response maps, and obtaining an attention weight map; The attention weight map is sharpened by a direction controllable sharpening filter to obtain a sharpened edge map, the sharpened edge map is dilated by a morphological closing operation, and then is eroded by a same circular kernel to repair edge breakage, so as to obtain a complete edge enhanced map.

[0010] Preferably, the similarity calculation formula is: ; Wherein is the similarity of , is the edge response map of the nth frequency band, is the mth pathological feature template; The weight coefficient calculation formula is: , wherein is the weight coefficient of the nth frequency band edge response map ; The generation formula of the attention weight map is: .

[0011] Preferably, the specific steps of the step S3 include: Whether the pixel is in the edge region is judged according to the complete edge enhanced map, and different diffusion coefficients are used for calculation in the edge region and the non-edge region; The pixel value is updated according to the diffusion iteration formula in the diffusion process, and four-direction diffusion coefficients are introduced; The denoised image after noise suppression is superimposed with the complete edge enhanced map according to a pixel-level superposition formula, to obtain a detail enhanced image.

[0012] Preferably, the judgment formula of the edge region is: when , it is determined that the pixel is in the edge region, otherwise it is in the non-edge region, wherein E is the complete edge enhanced map; The diffusion coefficient of the edge region is , and the diffusion coefficient of the non-edge region is , wherein is the gradient of the pixel ; The diffusion iteration formula is: , wherein is a step size, is the updated pixel gray value of the image at position (x, y) after the nth+1 iteration, is the pixel gray value of the image at position (x, y) at the nth iteration, , , , are respectively the diffusion intensity of the pixel adjusted to the upward, downward, right and left directions; The expression of the pixel-level superposition formula is as follows: , where is the denoised image.

[0013] Preferably, when the original image is a multimodal image in step S4, the specific steps for feature extraction and fusion of the detail-enhanced image to obtain a fused image are as follows: Input the detail-enhanced images of the color fundus photograph (CFP) and the fluorescein angiography (FFA) into a pre-trained feature extraction model respectively, and the feature extraction model outputs the vascular structure feature FC and the lesion feature FF; Calculate the mean gradient magnitude GC of the vascular structure feature FC and the mean gray value GF of the lesion feature FF respectively, and determine the weights α and β based on the comparison result of the mean gradient magnitude GC and the mean gray value GF; Perform weighted calculations on the vascular structure feature FC and the lesion feature FF based on the weights α and β respectively. The calculation formula is , ; Combine and by splicing in the channel dimension to obtain a fused feature map FM, and upsample the fused feature map FM to obtain a preliminary fused image ; Calculate the gray variances of the preliminary fused image , the detail-enhanced image of the color fundus photograph CFP and the detail-enhanced image of the fluorescein angiography FFA respectively, and obtain the fused image according to the comparison result of the gray variances.

[0014] Preferably, the calculation formula of the mean gradient magnitude GC is: ; where is the number of pixels in the vascular region; The calculation formula of the mean gray value GF is: ; where is the number of pixels in the lesion region; The steps for determining the weights α and β are: if GC ≥ GF, α = 0.6, β = 0.4; if GC ≤ GF, α = 0.4, β = 0.6; The specific steps for obtaining the fused image IM according to the comparison result of the gray variances are: If Var < min(VarC, VarF), then (x, y) = (ID C(x, y) + ID F(x, y)) / 2, otherwise (x, y) = (x, y) + (x, y) * (x, y) * (x, y) (x, y), wherein Var , VarC, VarF are the gray variance of the preliminary fusion image , the color fundus image CFP and the fluorescence angiography image FFA, ID C(x, y) and ID F(x, y) are the detail enhancement images of the color fundus image CFP and the fluorescence angiography image FFA.

[0015] Preferably, the specific step of the step S5 is: L1 regularization is used for model pruning, and instruction set optimization is performed; The fusion image is subjected to gray value inverse normalization according to the formula , wherein is the fusion image; The luminance channel of the output image is merged with the temporarily stored a and b channel images to form a Lab image, which is converted into an RGB image, written into metadata and saved as a high-quality fundus image, and the high-quality fundus image is output.

[0016] Compared with the prior art, the beneficial effects of the present application are: ①Multi-scale Gaussian difference operation is used to obtain multi-band edge response maps, combined with attention weighted fusion guided by pathological feature templates, and then direction controllable sharpening and morphological closing operation is used to repair edge breakage, which can comprehensively cover the multi-scale structures required for DR diagnosis, such as fine blood vessels, microangioma and hard exudation; compared with the existing single scale edge detection technology, it can effectively capture fine blood vessel edges with a diameter of less than 10 pixels, significantly improve the gray gradient difference of blood vessel edges, and improve the clarity of lesion boundary delineation, thereby greatly reducing the missed diagnosis rate of subtle lesions by the subsequent diagnosis system, and providing strong support for early and accurate diagnosis of DR;

[0017] ②Lightweight CNN is used to extract the structural features and lesion features of the two types of images, combined with feature attention mechanism to dynamically allocate weights, and then two-level fusion strategy of feature-level fusion-pixel-level optimization is used to realize the deep cooperation of CFP structural clarity and FFA lesion sensitivity; compared with the existing pixel-level simple superposition fusion algorithm, the fused image not only retains the fine blood vessel profile of CFP, but also effectively highlights the lesion fluorescence features of FFA, so that doctors or AI assisted diagnosis systems can more accurately judge the degree of DR lesions based on the fused image, and significantly improve the accuracy and reliability of DR staging diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only the preferred embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0019] Figure 1 The flow chart of the method for improving the quality of diabetic retinopathy fundus images based on edge enhancement. DETAILED DESCRIPTION

[0020] In order to better understand the technical content of the present application, a specific embodiment is provided below, and the present application is further described in combination with the drawings.

[0021] Referring to Figure 1 The method for improving the quality of diabetic retinopathy fundus images based on edge enhancement provided by the present application includes the following steps: Step S1, after collecting the original image of the diabetic retinopathy fundus, the original image is preprocessed, and an image of a region of interest is obtained; Step S2, a multi-scale Gaussian difference operation is performed on the image of the region of interest to obtain a multi-band edge response map, an attention weight map is obtained by combining a pathological feature template for weighted fusion, edge breakage is repaired through directional controllable sharpening and morphological closing operation, and a complete edge enhancement map is obtained; Step S3, an improved anisotropic diffusion algorithm is used to suppress noise of the complete edge enhancement map, and the denoising image after noise suppression is superimposed with the complete edge enhancement map to obtain a detail enhancement image; Step S4, if the original image is a multi-modal image, feature extraction and fusion are performed on the detail enhancement image to obtain a fusion image, and if the original image is a single modal image, the detail enhancement image is taken as an intermediate processing result; Step S5, the fusion image or the intermediate processing result is subjected to lightweight optimization and gray value inverse normalization, and a high-quality fundus image is output.

[0022] The original image of the diabetic retinopathy fundus can be collected by a specific image collection device, and then the original image is preprocessed, including color space conversion, retinal region segmentation, and uneven illumination correction, etc. After preprocessing, the region of interest image can be obtained, and then the multi-scale Gaussian difference operation can be used to obtain the multi-frequency edge response graph, including the low-frequency edge response graph, the medium-frequency edge response graph and the high-frequency edge response graph. After the multi-frequency edge response graph is combined with the attention weighted fusion guided by the pathological feature template, the attention weight graph can be obtained. Then, the edge breakage can be repaired by the direction controllable sharpening and the morphological closing operation to obtain the complete edge enhancement graph. Then, the improved anisotropic diffusion algorithm is used for noise suppression of the complete edge enhancement graph, and the denoising image is superimposed with the complete edge enhancement graph to obtain the detail enhancement image, which can strengthen the key structural details and can fully cover the multi-scale structures required for DR diagnosis, such as fine blood vessels, microangioma, and hard exudation. Compared with the existing single-scale edge detection technology, the diameter of the fine blood vessel edge is less than 10 pixels, the gray gradient difference of the blood vessel edge is significantly improved, the lesion boundary is clearly depicted, the subsequent diagnostic system has a significantly reduced rate of missed diagnosis of subtle lesions, and strong support is provided for early and accurate diagnosis of DR.

[0023] In addition, for multi-modal images, a feature extraction and fusion method is provided. According to the detail enhancement images of different modal images, structural features and lesion features can be extracted, and then the features are dynamically allocated weights combined with the feature attention mechanism, and then a two-level fusion strategy of feature-level fusion-pixel-level optimization is used to realize the deep cooperation of CFP structure clarity and FFA lesion sensitivity. Compared with the existing pixel-level simple superposition fusion algorithm, the fusion image not only retains the fine blood vessel profile of CFP, but also effectively highlights the lesion fluorescence features of FFA. The subsequent doctor or AI assisted diagnosis system can more accurately judge the DR lesion degree based on the fusion image, significantly improving the accuracy and reliability of DR staging diagnosis. Finally, after the fusion image or intermediate processing result is optimized and the gray value is de-normalized, a high-quality fundus image meeting the medical standards can be output, improving the DR lesion recognition clarity and diagnosis reliability, and taking into account the practicality and real-time performance.

[0024] Preferably, the step S1 of preprocessing the original image comprises: Color space conversion: converting the RGB image of the original image to Lab color space, extracting the luminance channel image, temporarily storing the a and b channel images, and performing resolution adjustment and gray value normalization processing; Retinal region segmentation: using a U-Net segmentation model to process the luminance channel image, outputting a retinal region mask, and multiplying the original image to obtain an initial region of interest image; Lightness correction: the initial region of interest image is divided into several sub-blocks, and the local entropy value of each sub-block is calculated. The contrast limit parameter is adjusted according to the local entropy value, and the region of interest image after lightness correction is obtained by weighted average method after processing the sub-block overlap region. The expression of local entropy value is: ; Wherein is the probability of the gray value i in the sub-block. If H>0.8, the contrast limit parameter is set to 1.8; if 0.5≤H≤0.8, the parameter is set to 2.0; if H<0.5, the parameter is set to 2.2.

[0025] After collecting the original image, the original image needs to be preprocessed, including color space conversion and format standardization. After converting the original image in RGB format to Lab color space by OpenCV, the lightness channel (L channel) image is extracted, and the a and b channels are discarded for temporary storage for subsequent image restoration. Then the L channel image resolution is uniformly adjusted to 2048x2048 pixels by using the resize function, and the gray value is normalized to the [0, 1] interval by using the normalize function. The formula is: , wherein is the original gray value, =0, =255; Then the standardized L channel image is subjected to retinal region segmentation processing. This step applies to the U-Net segmentation model. The model training process is as follows: 5000 labeled images of Kaggle DR Dataset are used as the training set, batchsize=8, the number of iterations=100, and the initial learning rate=1e-4. The encoder of the model contains 4 layers of 3x3 convolution (step 2, padding=same), and after each convolution, BatchNorm2d batch normalization and ReLU activation are performed. The decoder contains 4 layers of 3x3 deconvolution (step 2, padding=same), and after each deconvolution, the corresponding layer feature map of the encoder is connected through the concatenate function to realize the jump connection. After the standardized L channel image is input into the U-Net segmentation model, the retinal region mask can be directly output by the U-Net segmentation model, and the region of interest image can be obtained by multiplying the original image. The interference area such as eyelid and eyelash is excluded, and the segmentation accuracy verification is 98.2%.

[0026] Then the improved CLAHE algorithm is used to realize the createCLAHE function of OpenCV. The region of interest image is divided into 8x8 sub-blocks (each sub-block is 256x256 pixels), and then the local entropy value of each sub-block is calculated , if , the contrast limit parameter is set to 1.8; if , the parameter is set to 2.0; if , the parameter is set to 2.2, and after performing the CLAHE operation on each sub-block, the sub-block overlapping area (overlapping rate 50%) is processed by the weighted average method to obtain the illumination-corrected image of the region of interest.

[0027] Preferably, the specific steps of the step S2 include: The region of interest image is filtered by multiple groups of different parameter Gaussian kernels, and the multi-band edge response map is obtained by calculating the difference between adjacent filtering results; Selecting a labeled image containing multiple typical lesions as a training set, using a MobileNet architecture lightweight CNN for training to form a pathological feature template library; The cosine similarity is used to calculate the similarity between the multi-band edge response map and the pathological feature template in the pathological feature template library, the maximum similarity is selected as the weight coefficient for each frequency band edge response map, the weighted sum of the frequency band edge response maps is obtained, and an attention weight map is obtained, wherein the similarity calculation formula is: ; Wherein is the similarity of , is the nth frequency band edge response map, is the mth pathological feature template; The calculation formula of the weight coefficient is: , wherein is the weight coefficient of the nth frequency band edge response map ; The generation formula of the attention weight map is: ; The attention weight map is sharpened by a direction-controllable sharpening filter to obtain a sharpened edge map, the sharpened edge map is dilated by a morphological closing operation, and then eroded by a same circular kernel to repair edge breaks, thereby obtaining a complete edge enhancement map.

[0028] Multi-scale Gaussian difference operation is realized by the GaussianBlur function of OpenCV, and three groups of Gaussian kernel parameters are set , , to filter the illumination-corrected region of interest image, respectively, to obtain the filtered , , , and then calculate the difference between adjacent filtering results: wherein (medium frequency edge response, corresponding to the lesion boundary) (high frequency edge response, corresponding to the fine blood vessels) (low-frequency edge response, corresponding to coarse blood vessels), and three frequency band edge response maps are obtained.

[0029] The pathological feature templates are stored in a pathological feature template library, and the generation of the pathological feature template library comprises the following steps: 100 labeled images containing microhemangioma, hemorrhagic plaque and hard exudation in the data set are selected as a training set, MobileNetV2 lightweight CNN (input size 224x224, output layer for 3 types of lesion classification) is used for training, and after the training is completed, the features of the 12th layer (deep separable convolution layer) of the network are extracted as pathological feature templates to form a template library containing 3 types of lesions is a microhemangioma template, is a hemorrhagic plaque template, and is a hard exudation template. Then, the cosine similarity is used to calculate the similarity between each frequency band edge response map and each pathological feature template, that is, three similarity values can be obtained for the low-frequency edge response map, the medium-frequency edge response map and the high-frequency edge response map. For each frequency band edge response map, the maximum similarity is selected as the weight coefficient, and then the weight coefficient is used as the weight value of the three frequency band edge response maps for weighted summation to obtain an attention weight map.

[0030] Then, the direction controllable sharpening and morphological closing operation can be performed on the attention weight map, wherein the direction controllable sharpening is calculated by the Sobel operator (range 0°-360°), a 3x3 filter kernel is constructed, when is horizontal (0°), the kernel is when is vertical (90°), the kernel is Based on this, the sharpening operation is performed on the attention weight map , and a sharpened edge map is obtained.

[0031] Then, the morphological closing operation is performed by using the morphologyEx function of OpenCV, and a circular kernel function (the radius is adjusted according to the edge gap: radius=3 when the gap<5 pixels, and radius=5 when the gap≥5 pixels) is set. First, inflation (iteration number 1) and then corrosion (iteration number 1) are performed to repair edge breakage and obtain a complete edge enhancement map .

[0032] Preferably, the specific steps of the step S3 comprise: Whether the pixel is in the edge region is judged according to the complete edge enhancement map, and the judgment formula of the edge region is: when , it is determined that the pixel ​is an edge region, otherwise is a non-edge region, wherein E is a complete edge enhancement map, different diffusion coefficients are used for calculating the edge region and the non-edge region, the diffusion coefficient of the edge region is , and the diffusion coefficient of the non-edge region is , wherein is a gradient of a pixel ; In the diffusion process, the pixel value is updated according to a diffusion iteration formula, and a four-direction diffusion coefficient is introduced, the diffusion iteration formula is: , wherein is a step size, is an updated pixel gray value of the image at position (x, y) after the n+1th iteration, is a pixel gray value of the image at position (x, y) at the nth iteration, , , , respectively adjust the diffusion intensity of the pixel in the upward, downward, right and left directions, the diffusion iteration formula expresses the difference between the pixel point (x, y) and its four-direction neighborhood pixels, and is multiplied by the diffusion weight in the corresponding direction, so that the image is gradually smoothed, and the image edge features are maintained while denoising.

[0033] The denoised image after noise suppression is superimposed with the complete edge enhancement map according to a pixel-level superposition formula, to obtain a detail-enhanced image, and the expression of the pixel-level superposition formula is: , wherein is a denoised image.

[0034] After obtaining the complete edge enhancement map, noise suppression and detail enhancement can be performed, wherein the noise suppression is performed by anisotropic diffusion denoising, first, an edge region is judged, then different diffusion coefficients are set for the edge region and the non-edge region, and the pixel value is updated after diffusion iteration to obtain a denoised image.

[0035] Then, the denoised image needs to be detail-enhanced, and the denoised image is abutted with the complete edge enhancement map according to a pixel-level superposition formula to obtain a detail-enhanced image, if the original image is a single modality image, the processed detail-enhanced image can be output as an intermediate processing result, and if the original image is a multi-modality image, the multi-modality image collaborative fusion process in step S4 needs to be entered.

[0036] Preferably, in step S4, if the original image is a multi-modality image, the detail-enhanced image is feature-extracted and fused to obtain a fused image, and the specific steps are as follows: The detail enhancement image of the color fundus photograph CFP and the fluorescence fundus angiography image FFA is input into a pre-trained feature extraction model respectively, and the blood vessel structure feature FC and the lesion feature FF are output by the feature extraction model; The gradient amplitude mean value GC of the blood vessel structure feature FC and the gray mean value GF of the lesion feature FF are calculated respectively, and the contrast result based on the gradient amplitude mean value GC and the gray mean value GF is used to determine the weights α and β; The calculation formula of the gradient amplitude mean value GC is: ; Wherein is the number of pixels in the blood vessel region, which is labeled by a pre-trained blood vessel segmentation model; The calculation formula of the gray mean value GF is: ; Wherein is the number of pixels in the lesion region, which is obtained by labeling; If GC≥GF, α=0.6, β=0.4; if GC≤GF, α=0.4, β=0.6; The blood vessel structure feature FC and the lesion feature FF are weighted and calculated based on the weights α and β, and the calculation formula is , ; The is spliced with the in the channel dimension to obtain the fusion feature map FM, and the fusion feature map FM is up-sampled to obtain the preliminary fusion image ; The gray variances of the preliminary fusion image , the color fundus photograph CFP and the detail enhancement image of the fluorescence fundus angiography image FFA are calculated respectively, and the fusion image is obtained according to the contrast result of the gray variances, and the specific steps are as follows: If Var <min(VarC, VarF), then (x, y)=(ID C(x, y)+ID F(x, y)) / 2, otherwise (x, y)= (x, y), wherein Var , VarC, VarF are the gray variances of the preliminary fusion image , the color fundus photograph CFP and the fluorescence fundus angiography image FFA, and ID C(x, y) and ID F(x, y) are the detail enhancement images of the color fundus photograph CFP and the fluorescence fundus angiography image FFA.

[0037] The feature extraction network is implemented by a pre-trained MobileNetV3 model. After inputting the detail-enhanced image ID-C of the color fundus image CFP and the detail-enhanced image ID-F of the fluorescence angiography image FFA, respectively, the 16th layer features of the model are extracted as FC (256x256x128, focusing on blood vessel structure) and FF (256x256x128, focusing on lesions). In the feature attention mechanism, the gradient amplitude mean GC of FC and the gray mean GF of FF are calculated, and then GC and GF are compared. If GC≥GF, then α=0.6 and β=0.4; if GC≤GF, then α=0.4 and β=0.6. Based on the weights α and β, FC and FF can be weighted, respectively.

[0038] Then two-level fusion is performed to obtain and The channel dimension is spliced to obtain a 256x256x256 feature map, which is input into a 1x1 convolution layer (output channel 128, ReLU activation) to obtain a fusion feature map FM. The FM is upsampled by a MobileNetV3 decoder (4 layers of deconvolution, output 2048x2048) to obtain a preliminary fusion image The gray variance of and ID-C, ID-F is calculated, and based on the comparison result, the final fusion image is obtained.

[0039] Preferably, the specific steps of step S5 are: L1 regularization is used for model pruning, and instruction set optimization is performed; The fusion image is subjected to gray value inverse normalization according to the formula , wherein is the fusion image; The luminance channel of the output image is merged with the temporarily stored a and b channel images to form a Lab image, which is converted into an RGB image and then written into metadata for saving as a high-quality fundus image, and the high-quality fundus image is output.

[0040] The model pruning uses L1 regularization to calculate the contribution of the MobileNetV3 convolution kernel , sets the threshold T=0.1, removes the convolution kernel of , reduces the model parameters, and in the instruction set optimization, the loop operations in the Gaussian difference and anisotropic diffusion are optimized using the ARM NEON instruction set (through the numpy-neon library of Python), such as replacing the single-pixel loop in the pixel traversal link with 8-pixel vector operation, which improves the operation efficiency by 3.1 times.

[0041] Finally, the image format is converted and output, including gray value de-normalization, color space recovery and format output.

[0042] The basic conditions implemented by the above embodiments include: Hardware equipment: The image acquisition device uses a portable fundus camera, which supports the acquisition of color fundus images (CFP) and fluorescence angiography images (FFA) with a resolution of 2048x2048 pixels; the image processing device uses a basic medical terminal equipped with an ARM Cortex-A53 architecture processor, with 2GB of memory and an Android 10 operating system.

[0043] Software environment: Image processing is based on Python 3.8, deep learning framework uses TensorFlow Lite (adapted to low-power devices), traditional algorithms (such as Gaussian difference, anisotropic diffusion) are based on OpenCV 4.5.5, and DICOM format conversion uses the pydicom library.

[0044] Experimental data: Selecting a hospital endocrinology department diagnosed DR patient fundus image data set, containing 500 groups of images (each group containing 1 CFP image and 1 FFA image), image annotation completed by 2 deputy chief physicians, covering microhemangioma, hemorrhagic plaque, hard exudation and other typical lesions, DR stage includes stages I-IV.

[0045] Through the adaptation of hardware and lightweight software framework (TensorFlow Lite) for primary medical scenarios, based on the continuation of multi-scale Gaussian difference edge enhancement, improved anisotropic diffusion denoising, multi-modal two-level fusion and other core technologies, the standardized process realizes the whole link optimization of DR fundus image from preprocessing to DICOM output, solves the problems of insufficient edge enhancement accuracy and poor multi-modal fusion collaboration, at the same time, through model pruning and NEON instruction set optimization, the single processing time is controlled within 128ms, taking into account noise suppression and detail retention, fully adapting to the performance of primary medical institutions and clinical diagnosis needs, providing an efficient and practical image quality improvement scheme for DR accurate diagnosis.

[0046] In summary, the edge enhancement-based diabetic retinopathy fundus image quality improvement method of the present application, in the edge enhancement link, with the help of multi-scale Gaussian difference operation, pathological feature template guided attention weighted fusion, combined with direction controllable sharpening and morphological closing operation, can comprehensively cover the multi-scale structures such as fine blood vessels and various lesions required for diabetic retinopathy diagnosis, effectively improve the definition of fine blood vessel edges and lesion boundaries, significantly reduce the missed diagnosis risk of subtle lesions in subsequent diagnosis, and provide support for early accurate identification of lesions; In terms of multi-modal image fusion, relying on a lightweight convolutional neural network to extract the structural features of the color fundus image (CFP) and the lesion features of the fluorescence angiography image (FFA), combined with a dynamic weight distribution mechanism and a two-level fusion strategy of "feature-level fusion-pixel-level optimization", the advantages of the two types of images are deeply coordinated, so that the fused image can not only fully retain the fine blood vessel profile of CFP, but also effectively highlight the lesion features of FFA, helping doctors or AI-assisted diagnosis systems to more accurately judge the lesion degree and improve the reliability of staging diagnosis.

[0047] At the same time, through the improved anisotropic diffusion algorithm, the image noise is effectively suppressed while the fine structure details are maximally preserved, avoiding the imbalance between noise suppression and detail preservation; and through model pruning and instruction set optimization adapted to ARM architecture, the algorithm running efficiency is greatly improved, meeting the real-time processing needs of primary medical terminals, and finally outputting high-quality fundus images conforming to DICOM medical standards, fully solving the deficiencies of existing technologies in edge enhancement accuracy, multi-modal fusion synergy, noise-detail balance and low-power device adaptability, and providing an efficient and practical technical solution for accurate diagnosis of diabetic retinopathy.

[0048] The above only describes the preferred embodiments of the present application and should not be used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for improving fundus image quality in diabetic retinopathy based on edge enhancement, characterized in that, Includes the following steps: Step S1: After acquiring the original fundus images of diabetic retinopathy, preprocess the original images and obtain the region of interest image; Step S2: Perform multi-scale Gaussian difference operation on the region of interest image to obtain a multi-band edge response map, combine it with pathological feature template weighted fusion to obtain an attention weight map, and repair edge breaks through directional controllable sharpening and morphological closure operation to obtain a complete edge enhancement map; Step S3: The improved anisotropic diffusion algorithm is used to suppress noise in the complete edge enhancement image. The denoised image after noise suppression is superimposed with the complete edge enhancement image to obtain the detail enhancement image. Step S4: If the original image is a multimodal image, perform feature extraction and fusion on the detail enhancement image to obtain a fused image. If the original image is a single-modal image, use the detail enhancement image as an intermediate processing result. Step S5: Perform lightweight optimization and grayscale inverse normalization on the fused image or intermediate processing result to output a high-quality fundus image.

2. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 1, characterized in that, Step S1, which involves preprocessing the original image, includes: Color space conversion: Convert the original image from RGB to Lab color space, extract the lightness channel image, temporarily store the a and b channel images, and perform resolution adjustment and grayscale value normalization. Retinal region segmentation: The U-Net segmentation model is used to process the brightness channel image, output the retinal region mask, and multiply it with the original image to obtain the initial region of interest image; Illumination unevenness correction: The initial region of interest image is divided into several sub-blocks, and the local entropy value of each sub-block is calculated. The contrast limit parameter is adjusted according to the magnitude of the local entropy value, and the overlapping areas of the sub-blocks are processed by weighted averaging to obtain the illumination-corrected region of interest image.

3. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 2, characterized in that, The expression for the local entropy value is: ； in Let H be the probability of grayscale value i within a sub-block. If H > 0.8, set the contrast limit parameter to 1.8; if 0.5 ≤ H ≤ 0.8, set the parameter to 2.0; if H < 0.5, set the parameter to 2.

2.

4. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 1, characterized in that, The specific steps of step S2 include: The region of interest image is filtered by multiple Gaussian kernels with different parameters, and the difference between adjacent filtering results is calculated to obtain a multi-band edge response map. Annotated images containing various typical lesions were selected as the training set, and a lightweight CNN with the MobileNet architecture was used for training to form a pathological feature template library. Cosine similarity is used to calculate the similarity between the multi-band edge response map and the pathological feature templates in the pathological feature template library. The maximum similarity of each frequency band edge response map is selected as the weight coefficient. The edge response maps of each frequency band are weighted and summed to obtain the attention weight map. The attention weight map is sharpened by a direction-controlled sharpening filter to obtain a sharpened edge map. The sharpened edge map is then expanded by a morphological closing operation, and the edge breaks are repaired by erosion with a circular kernel to obtain a complete edge enhancement map.

5. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 4, characterized in that, The similarity calculation formula is as follows: ； in for similarity, This is the edge response map of the nth frequency band. This is the template for the m-th pathological feature; The formula for calculating the weighting coefficient is: ,in The edge response map of the nth frequency band Weighting coefficients; The formula for generating the attention weight map is: .

6. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 1, characterized in that, The specific steps of step S3 include: The system determines whether a pixel is in an edge region based on the complete edge enhancement map, and calculates different diffusion coefficients for edge regions and non-edge regions. During the diffusion process, pixel values ​​are updated according to the diffusion iteration formula, and a four-directional diffusion coefficient is introduced. The denoised image after noise suppression is superimposed on the complete edge enhancement image according to the pixel-level superposition formula to obtain the detail enhancement image.

7. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 6, characterized in that, The formula for determining the edge region is: when At that time, determine the pixel If E represents an edge region, it is considered an edge region; otherwise, it is considered a non-edge region. E is a complete edge enhancement map. The diffusion coefficient of the edge region is The diffusion coefficient of the non-edge region is ,in For pixels The gradient; The diffusion iteration formula is: ,in Step size, This represents the updated pixel grayscale value at position (x, y) in the image after the (n+1)th iteration. Let (x, y) be the pixel grayscale value at position (x, y) in the nth iteration. , , , These adjust the diffusion intensity of the pixel in the upward, downward, rightward, and leftward directions, respectively. The expression for the pixel-level overlay formula is: ,in This is a denoised image.

8. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 1, characterized in that, In step S4, if the original image is a multimodal image, the specific steps for extracting and fusing features from the detail-enhanced image to obtain the fused image are as follows: The enhanced detail images of color fundus image (CFP) and fluorescence angiography image (FFA) are respectively input into a pre-trained feature extraction model, and the feature extraction model outputs vascular structure feature (FC) and lesion feature (FF). The mean gradient amplitude GC of vascular structural feature FC and the mean gray value GF of lesion feature FF are calculated respectively. The weights α and β are determined based on the comparison results of the mean gradient amplitude GC and the mean gray value GF. The vascular structural feature FC and lesion feature FF are calculated based on weights α and β, respectively. The calculation formula is as follows: , ; Will and The fusion feature map (FM) is obtained by concatenating the features along the channel dimension. The FM is then upsampled to obtain a preliminary fused image. ; Calculate the initial fused images separately The grayscale variance of detail-enhanced images from color fundus images (CFP) and fluorescence angiography (FFA) is used to obtain a fused image based on the comparison results of the grayscale variance. .

9. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 8, characterized in that, The formula for calculating the gradient magnitude mean GC is as follows: ； in This represents the number of pixels in the blood vessel region. The formula for calculating the grayscale mean GF is: ； in This represents the number of pixels in the lesion area. The steps for determining the weights α and β are as follows: if GC≥GF, α=0.6, β=0.4; if GC≤GF, α=0.4, β=0.

6. The fused image is obtained based on the comparison results of grayscale variance. The specific steps are as follows: If Var < min(VarC, VarF), then (x, y) = (ID C(x, y) + ID F(x, y)) / 2, otherwise (x, y) = (x, y), where Var , VarC, and VarF are the gray variances of the preliminary fusion image , the color fundus image CFP, and the fluorescence angiography image FFA, respectively. ID C(x, y) and ID F(x, y) are the detail-enhanced images of the color fundus image CFP and the fluorescence angiography image FFA.

10. The method for improving fundus image quality in diabetic retinopathy based on edge enhancement according to claim 2, characterized in that, The specific steps of step S5 are as follows: L1 regularization is used for model pruning, and instruction set optimization is performed. The fused images are processed according to the formula. Perform inverse normalization of grayscale values, where To merge images; The brightness channel of the output image is merged with the temporarily stored a and b channel images into a Lab image, which is then converted into an RGB image. Metadata is written to save it as a high-quality fundus image, and a high-quality fundus image is output.

Images