Structure preserving enhancement method for unpaired fundus images based on feature decoupling

Abstract

This invention relates to a method for enhancing the structure preservation of unpaired fundus images based on feature decoupling, the method comprising: S1: acquiring unpaired original high-quality and low-quality fundus images: I H and I L After passing through N layers of encoders, semantic features of each layer are extracted; S2: The semantic features of the Nth layer are decoupled from the input enhancement module QDM, and the output semantic features are S3: The features are passed through N layers of decoders respectively, and skip connections are used between the decoders and encoders to ensure structural consistency, and the output is I. L_E and I H_D Step S4: Using the circular consistency model, I H_D and I L_E Steps S1 to S3 are executed again to exchange quality, ultimately resulting in a high-quality reconstructed fundus image I'. H and low-quality fundus images I' L The method provided by this invention decouples and enhances the structural and quality information of fundus images while ensuring structural consistency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of deep learning, and in particular to a feature decoupling-based structure preservation enhancement method and system for unpaired fundus images. BACKGROUND

[0002] With the progress of medical level and the intelligentization of medical equipment, images are increasingly becoming an important basis for clinical diagnosis. In the diagnosis of ophthalmic diseases, the fundus retinal image is extremely important for judgment. Good image quality is a necessary prerequisite for doctors and computer-aided diagnosis systems to make correct diagnoses. Compared with other medical images, fundus image acquisition is more complex and requires more stringent requirements, and there are more factors that affect the quality of the image. Various physiological and pathological reasons such as small pupils of the subject, vitreous opacity, and nystagmus can cause low brightness and blurring of the fundus image, and other low-quality conditions. Patients with cataracts, uveitis, keratitis, and retinal detachment will have foggy fundus images. Therefore, the research on fundus image enhancement has great practical significance.

[0003] There are two kinds of fundus image enhancement methods: (1) traditional image enhancement methods; (2) deep learning methods. Traditional image enhancement methods such as histogram equalization, contrast adaptive histogram equalization, and Gamma correction mostly focus on adjusting the gray distribution histogram and global pixel adjustment. These enhancement methods often have problems such as large noise and overexposure when used for fundus images, and are not suitable for complex, non-uniform, and poor-quality image enhancement. Most deep learning-based enhancement algorithms rely on paired data, which is difficult to obtain in medical images. In particular, for fundus images, the human eye will rotate non-autonomously even in a very short time between two shootings, so it is difficult to obtain paired data. Recently, with the proposal of cyclegan, a large number of studies have begun to enhance unpaired images, attracting the attention of relevant researchers.

[0004] However, there is a serious problem with unpaired enhancement algorithms: the enhanced images always have structure loss and changes, which is unacceptable in medical images, as it can lead to misdiagnosis by doctors or computer-aided diagnosis systems. Therefore, in the process of medical image enhancement, how to ensure the consistency of the structure before and after enhancement has become a problem to be solved. SUMMARY

[0005] To solve the above technical problems, the present application provides a feature decoupling-based structure preservation enhancement method and system for unpaired fundus images.

[0006] The technical solution of the present application is: a feature decoupling-based structure preservation enhancement method for unpaired fundus images, comprising:

[0007] Step S1: Acquire unpaired raw high-quality fundus images I H And the original low-quality fundus image I L After passing through an N-layer encoder, semantic features of each layer of fundus image are extracted. Where L and H represent low-quality fundus images and high-quality fundus images, respectively, and N is the number of layers of the semantic features; the semantic features include: intrinsic structural features and non-intrinsic quality features;

[0008] Step S2: Extract the semantic features from the highest layer, i.e., the Nth layer. and Input the decoupling enhancement module QDM, perform decoupling enhancement and quality exchange, and output semantic features. and in, This involves decoupling and enhancing the semantic features of the low-quality fundus image, as well as improving the semantic features after quality enhancement. This involves decoupling and enhancing the semantic features of the high-quality fundus image and refining the semantic features after quality degradation.

[0009] Step S3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] and The image is processed through N layers of decoders. Simultaneously, an instance-normalized skip connection (I-norm-skip) module is used between the encoders at layers 1 to N-1 and their corresponding decoders to fully utilize the structural information in the low-level semantic features. Instance normalization is used to remove non-intrinsic quality features from the low-level semantic features while retaining their intrinsic structural features. The normalized semantic features of each encoder layer are then concatenated with the corresponding features of the corresponding size in the decoder along the channel dimension to finally obtain a quality-enhanced image I with the original low-quality fundus image structure. L_E and degraded images with original high-quality fundus structure I H_D ;

[0010] Step S4: Using the circular consistency model, I H_D and I L_E Steps S1 to S3 are executed again to exchange quality, ultimately resulting in a high-quality reconstructed fundus image I. ′H and low-quality fundus images I ′L At the same time, a consistency loss mechanism is constructed. Decoupling Enhancement Loss and feature-invariant loss The cycle consistency model is supervised and optimized.

[0011] Compared with the prior art, the present invention has the following advantages:

[0012] This invention discloses a structure preservation enhancement method for unpaired fundus images based on feature decoupling. Addressing the challenges of small structure and lesion sizes, numerous and complex low-quality factors in fundus images, and the difficulty in obtaining paired fundus images (i.e., paired low-quality / high-quality images), a cyclic consistency framework is constructed to decouple and enhance the structural and quality information of fundus images. Furthermore, a jump connection module based on instance normalization is built to ensure the consistency of the fundus image structure before and after enhancement. Attached Figure Description

[0013] Figure 1 This is a flowchart of a method for enhancing the structure preservation of unpaired fundus images based on feature decoupling, as described in an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of the basic model in an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of the decoupling enhancement module in an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram of the circular consistency framework in an embodiment of the present invention;

[0017] Figure 5 This is a structural block diagram of a structure preservation enhancement system for unpaired fundus images based on feature decoupling, according to an embodiment of the present invention. Detailed Implementation

[0018] This invention provides a method for enhancing the structure preservation of unpaired fundus images based on feature decoupling.

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below through specific implementations and in conjunction with the accompanying drawings.

[0020] Example 1

[0021] like Figure 1 As shown in the figure, an embodiment of the present invention provides a method for enhancing the structure preservation of unpaired fundus images based on feature decoupling, comprising the following steps:

[0022] Step S1: Acquire unpaired raw high-quality fundus images I H And the original low-quality fundus image I L After passing through an N-layer encoder, semantic features of each layer of fundus image are extracted. Where L and H represent low-quality fundus images and high-quality fundus images, respectively, and N is the number of layers of semantic features; semantic features include: intrinsic structural features and non-intrinsic quality features;

[0023] Step S2: Extract the semantic features from the highest layer, i.e., the Nth layer. and Input decoupling enhancement module QDM, decoupling enhancement and quality exchange, output semantic features and wherein, is the semantic feature after decoupling enhancement and quality enhancement of the low-quality fundus image semantic feature, is the semantic feature after decoupling enhancement and quality degradation of the high-quality fundus image semantic feature;

[0024] Step S3: the and respectively pass through N layers of decoders, and at the same time, the instance normalization-based skip connection I-norm-skip module is used between the first layer to the N-1 layer encoder and the corresponding decoder to fully utilize the structural information in the low-level semantic features, and the instance normalization is used to eliminate the extrinsic quality features in the low-level semantic features while retaining the intrinsic structural features thereof; after the normalized semantic features of each layer of the encoder are spliced with the features of the corresponding size in the corresponding decoder in the channel dimension, the quality enhanced image I L_E with the original low-quality fundus image structure and the quality degraded image I H_D with the original high-quality fundus image structure are finally obtained.

[0025] Step S4: using a cycle consistency model, I H_D and I L_E perform steps S1-S3 again to exchange quality, and finally obtain the reconstructed high-quality fundus image I ′H and the low-quality fundus image I ′L , and at the same time, construct consistency loss decoupling enhancement loss and feature invariance loss to supervise and optimize the cycle consistency model.

[0026] In one embodiment, the above step step S1: obtaining unpaired original high-quality fundus image I H and original low-quality fundus image I L pass through N layers of encoders to extract the semantic features of each layer of the fundus image wherein L and H represent the low-quality fundus image and the high-quality fundus image, respectively, and N is the number of layers of semantic features; the semantic features include intrinsic structural features and extrinsic quality features, which specifically include:

[0027] extracting I L and I H using an N-layer encoder based on a VGG network to extract semantic features at different levels Wherein, L and H are low-quality fundus images and high-quality fundus images respectively, and N is the number of semantic features;

[0028] Semantic features Including: intrinsic structural features And extrinsic quality features Wherein, intrinsic structural features Are defined as Wherein, μ and σ are the mean and variance of the channel, and extrinsic quality features Are defined as Wherein, M represents the batch size, and C is the number of channels of semantic features.

[0029] Since the VGG network has good image feature representation capability, the embodiment of the application adopts the VGG pre-trained on the image database ImageNet containing 14 million images of 2 million categories as the encoder. The VGG extracts the semantic features of the input fundus images (including high-quality fundus images and low-quality fundus images) at four levels. Through the exploration of the decoupling of different levels of semantic features, it is found that the highest layer, i.e. the fourth layer of semantic features has good decoupling characteristics, so the quality information of this level will be enhanced in the subsequent steps.

[0030] As Figure 2 The basic model diagram shows that the embodiment of the application uses a 4-layer encoder based on the VGG network to extract the intrinsic structural features And extrinsic quality features Of high-quality and low-quality fundus images at different levels, and realizes the enhancement of image quality by combining the intrinsic structural features of low-quality images and the extrinsic quality features of high-quality images.

[0031] In one embodiment, the step S2 described above: the semantic features of the highest layer, i.e. the Nth layer And Are input into the decoupling enhancement module QDM to perform decoupling enhancement and quality exchange, and output semantic features And Wherein, Is the semantic feature after decoupling strengthening and quality enhancement of the semantic feature of the low-quality fundus image, Is the semantic feature after decoupling strengthening and quality degradation of the semantic feature of the high-quality fundus image, and specifically includes:

[0032] Step S21: the highest layer semantic features And Are input into the QDM module to perform decoupling enhancement and quality exchange: the extrinsic quality representation Input into the classification network, wherein the classification network consists of a fully connected layer and an activation function RELU, and the enhanced extrinsic quality features are output by the RELU and The output vector of the final fully connected layer is classified into low quality and high quality by binary cross-entropy, and the enhanced semantic features are obtained consisting of low-quality intrinsic structural features and decoupled and enhanced high-quality extrinsic quality features Similarly, the degraded semantic features consisting of high-quality intrinsic structural features and decoupled and enhanced low-quality extrinsic quality features ;

[0033] The quality exchange is performed by adaptive instance normalization as follows:

[0034]

[0035] wherein x and y represent the semantic features of the low-quality image and the high-quality image respectively, the intrinsic quality features are first removed by channel-wise normalization, and then new extrinsic quality features are obtained by assigning new mean and variance;

[0036] Step S22: At the same time, the decoupled and enhanced loss is constructed consisting of cross-entropy loss and regularization loss

[0037]

[0038]

[0039] wherein is the highest N-layer low-quality extrinsic quality feature, is the decoupled and enhanced extrinsic quality feature; is the highest N-layer high-quality extrinsic quality feature, is the decoupled and enhanced extrinsic quality representation; ||.||2 is L2 norm.

[0040] In this step, the fourth layer semantic features of the high-quality fundus image and the low-quality fundus image extracted in step S1 are decoupled, and two feature vectors of structural information and quality information are respectively decoupled. The decoupled and enhanced module QDM is used, and its structure is as follows: Figure 3 ​As shown, the decoupling enhancement is completed while the quality exchange is completed: first, the extrinsic quality feature vector is mapped by the full connection layer, activated by the RELU activation function, and then mapped by the full connection layer again to realize dimension reduction. The extrinsic quality feature vectors of the high-quality fundus image and the low-quality fundus image after dimension reduction are calculated by cross-entropy loss, the two types of extrinsic quality feature vectors decoupled are strengthened, and the regularization loss is used to constrain the extrinsic quality feature to change within a proper range. The strengthened extrinsic quality features are exchanged, thereby completing the quality exchange.

[0041] In one embodiment, the above step S3: the and respectively pass through N layers of decoders, and at the same time, the instance normalization-based skip connection I-norm-skip module is used between the first layer to the N-1 layer encoder and the corresponding decoder to make full use of the structural information in the low-level semantic features, and the instance normalization is used to eliminate the extrinsic quality features in the low-level semantic features while retaining the intrinsic structural features thereof; after the normalized semantic features of each layer of the encoder are spliced with the features of the corresponding size in the corresponding decoder in the channel dimension, the quality enhanced image I L_E with the original low-quality fundus image structure and the quality degraded image I H_D with the original high-quality fundus image structure are finally obtained.

[0042] Step S31: the I-norm-skip module eliminates the extrinsic quality features in the semantic features by the instance normalization shown in formula (1):

[0043]

[0044] Wherein, F j is the semantic feature of the first layer to the N-1 layer fundus image, j ∈ [1, N-1]; μ nc (F j ) represents the mean value of the semantic feature by channel, and σ nc (F j ) represents the variance of the semantic feature by channel; both of which constitute the extrinsic quality features of the semantic features.

[0045] Step S32: the structural information contained in the normalized semantic feature INF(F j ) is transmitted to the decoder in the form of channel splicing; therefore, the semantic feature G th of the i i layer decoder is:

[0046]

[0047] Wherein, UP is the up-sampling operation.

[0048] In this step, the embodiment of the application designs a jump connection module based on instance normalization (I-norm-skip) to ensure the structural consistency of the enhanced image. The low-level semantic features with the quality elimination information are transmitted to the decoder using the jump connection, that is, the low-level semantic features of the encoder stage are spliced with the high-level semantic features of the corresponding decoder stage in the channel, so that the detailed structural information in the low-level semantic features can be fully utilized. The semantic feature information of the fundus image after splicing is input into the decoder, and finally the output image consistent with the size of the original low / high quality fundus image is obtained, that is, the quality enhanced image with the structure of the original low quality fundus image I L_E and the quality degraded image with the structure of the original high quality fundus image I H_D .

[0049] In one embodiment, the above step S4: using a cycle consistency model, I H_D and I L_E perform steps S1-S3 again to exchange quality, and finally obtain the reconstructed high quality fundus image I ′H and the low quality fundus image I ′L At the same time, a consistency loss Decoupling reinforcement loss and feature invariant loss The cycle consistency model is supervised and optimized, specifically including:

[0050] Step S41: For training the quality exchange model, a consistency loss is designed between the reconstructed image and the original image

[0051]

[0052] Wherein, L represents the original low quality fundus image, represents the reconstructed low quality fundus image;

[0053] H represents the original high quality fundus image, represents the reconstructed high quality fundus image; ||.||1 represents L1 norm;

[0054] Step S42: constructing feature invariant loss The enhanced image and the original image are supervised in the intrinsic structural feature and the extrinsic quality feature respectively, to ensure the invariance of the structure of the enhanced image and the expected quality;

[0055]

[0056]

[0057]

[0058] wherein, λ e is a preset parameter, f represents an encoder used for extracting semantic features, g represents a decoder used for outputting corresponding images from the semantic features, represents the extracted semantic features of the enhanced image.

[0059] As Figure 4 shown, in order to realize the enhancement task of unpaired images, the application realizes supervision on the image pix-pix through the reciprocal process of enhancement-degradation, and a cycle consistency framework is constructed to complete the reconstruction of the input image through continuous quality enhancement and degradation of the input image, so as to realize pixel-pixel supervision on the image. The cycle consistency framework includes two stages, the first stage is quality exchange, the above steps S1-S3 are used to obtain the enhanced low-quality image and the degraded high-quality image, and input them into the image reconstruction of the second stage, and then repeat steps S1-S3 for quality exchange, so as to finally complete the reconstruction of the input image.

[0060] The application discloses a structure preservation enhancement method for unpaired fundus images based on feature decoupling.

[0061] Embodiment two

[0062] As Figure 5 shown, the embodiment of the application provides a structure preservation enhancement system for unpaired fundus images based on feature decoupling, which comprises the following modules:

[0063] A semantic feature extraction module 51 is used to obtain unpaired original high-quality fundus images I H and original low-quality fundus images I L , and extract semantic features of each layer of fundus images through N layers of encoders. wherein, L and H respectively represent low-quality fundus images and high-quality fundus images, and N is the number of layers of semantic features; the semantic features include intrinsic structural features and extrinsic quality features.

[0064] A decoupling strengthening and quality enhancement module 52 is used to input the semantic features of the highest layer, i.e. the Nth layer and into a decoupling enhancement module QDM to perform decoupling enhancement and quality exchange, and output semantic features and wherein, is the semantic feature after decoupling enhancement and quality enhancement for the low-quality fundus image, is the semantic feature after decoupling enhancement and quality degradation for the high-quality fundus image;

[0065] the structure preservation module 53 is configured to perform the following operations: and pass through the N-layer decoder respectively, and meanwhile, use the instance normalization-based skip connection I-norm-skip module between the 1st-layer to the N-1st-layer encoder and the corresponding decoder to make full use of the structural information in the low-level semantic feature, the instance normalization is used to eliminate the non-intrinsic quality features in the low-level semantic feature while retaining its intrinsic structural features; after the normalized semantic feature of each layer of the encoder is spliced with the corresponding size feature in the corresponding decoder in the channel dimension, the quality-enhanced image I L-E with the original low-quality fundus image structure and the quality-degraded image I H-D with the original high-quality fundus image structure are finally obtained.

[0066] the cycle consistency module 54 is configured to perform the following operations: H_D and L_E perform the above modules again to exchange the quality, and finally obtain the reconstructed high-quality fundus image I ′H and the low-quality fundus image I ′L , and meanwhile, construct the consistency loss the decoupling enhancement loss and the feature invariance loss to supervise and optimize the cycle consistency model.

[0067] The above embodiments are provided only for the purpose of describing the present application, and are not intended to limit the scope of the present application. The scope of the present application is defined by the appended claims. Various equivalent replacements and modifications made without departing from the spirit and principles of the present application shall be encompassed within the scope of the present application.

Claims

1. A feature decoupling based unpaired fundus image structure preservation enhancement method, characterized in that, Comprise: Step S1: obtaining an original high-quality fundus image that is not paired and an original low-quality fundus image , passing through N layers of encoders to extract semantic features of fundus images of each layer } wherein L and H represent low-quality fundus images and high-quality fundus images, respectively, is the number of layers of the semantic features; the semantic features include intrinsic structural features and extrinsic quality features; Step S2: the semantic features of the highest layer, i.e. the Nth layer, are input into a decoupling enhancement module QDM for decoupling enhancement and quality exchange, and output semantic features and input into a decoupling enhancement module QDM for decoupling enhancement and quality exchange, and output semantic features and wherein, is the semantic feature after decoupling enhancement and quality enhancement on the low-quality fundus image semantic feature, is the semantic feature after decoupling enhancement and quality degradation on the high-quality fundus image semantic feature; Step S3: obtaining the low-quality image with the original structure of the low-quality image by performing the following steps on the low-quality image: and respectively passing through N-layer decoders, while using instance normalization-based skip connections I-norm-skip modules between the encoders and their corresponding decoders from the 1st layer to the N-1th layer to take full advantage of the structural information in the low-level semantic features, instance normalization being used to eliminate the extrinsic quality features in the low-level semantic features while retaining their intrinsic structural features; after the normalized semantic features of each layer of the encoder are concatenated with the corresponding size features in the corresponding decoder in the channel dimension, the quality-enhanced image with the original structure of the low-quality fundus image is finally obtained and the quality-degraded image with the original structure of the high-quality fundus image ; Step S4: Employ the circular consistency model to... and Steps S1-S3 are executed again to exchange quality, ultimately resulting in a high-quality reconstructed fundus image. and low-quality fundus images At the same time, a consistency loss mechanism is constructed. Decoupling enhances loss and feature-invariant loss The supervision and optimization of the cycle consistency model specifically includes: Step S41: For the training quality exchange model, a consistency loss is designed between the low-quality fundus image of the reconstructed high-quality fundus image and the original image ; wherein, denotes the original low-quality fundus image, denotes the reconstructed low-quality fundus image; denotes the original high-quality fundus image, denotes the reconstructed high-quality fundus image; denotes the L1 norm; Step S42: constructing feature-invariant loss The enhanced image and the original image are supervised in the intrinsic structural features and the extrinsic quality features respectively, so as to ensure the invariance of the structure of the enhanced image and the expected quality. wherein, is a predetermined parameter, represents the decoder for outputting the semantic feature corresponding to the image, represents the semantic feature of the extracted enhanced image.

2. The method of claim 1, wherein the method is performed by a computer system. The step S1: acquiring an unpaired original high-quality fundus image and an original low-quality fundus image , passing through N-layer encoders to extract semantic features of each layer of fundus images , wherein L and H represent low-quality fundus images and high-quality fundus images, respectively, is the number of layers of the semantic features; the semantic features include intrinsic structural features and extrinsic quality features, specifically including extracted using an N-layer encoder based on a VGG network and at different levels of semantic features wherein L and H are low and high quality fundus images, respectively, is the number of layers of semantic features; the semantic feature comprises an intrinsic structural feature and an extrinsic quality feature wherein the intrinsic structural feature is defined as wherein μ and σ are the mean and variance per channel, the extrinsic quality feature is defined as where M denotes the batch size and C is the number of channels of the semantic feature.

3. The method of claim 2, wherein the method is performed by a computer system comprising a processor and a memory, and the method comprises: receiving the first and second images; and storing the first and second images in the memory. The step S2: the highest layer, namely the semantic feature of the Nth layer And The input decoupling enhancement module QDM, decoupling enhancement and quality exchange, output semantic features And Wherein, The semantic feature after decoupling enhancement and quality enhancement of the low-quality fundus image semantic feature, The semantic feature after decoupling enhancement and quality degradation of the high-quality fundus image semantic feature, specifically including: Step S21: Transfer the semantic features from the highest layer and The inputs are respectively fed into the QDM module for decoupling enhancement and mass exchange: non-intrinsic mass characterization. The input is fed into a classification network, which consists of fully connected layers and the ReLU activation function, with the ReLU outputting enhanced intrinsic quality features. Finally, the output vector of the fully connected layer is used for binary cross-entropy classification to distinguish between low-quality and high-quality features, resulting in enhanced semantic features. = Due to low-quality intrinsic structural features High-quality eigenquality characteristics enhanced by decoupling Composition; similarly, degenerate semantic features Features of high-quality intrinsic structure and low-mass eigenmass characteristics enhanced by decoupling composition; The quality exchange is carried out by adaptive instance normalization as follows: Wherein, x, y respectively represent the semantic features of low-quality images and high-quality images, first, the non-intrinsic quality features are removed by channel normalization, and then new non-intrinsic quality features are obtained by assigning new mean and variance. Step S22: Meanwhile, construct decoupled reinforcement loss comprised of a cross-entropy loss and a regularization loss ​ wherein, is a low quality extrinsic quality feature of the highest Nth layer, is the decoupled reinforced extrinsic quality feature; is a high quality extrinsic quality feature of the highest Nth layer, is the decoupled reinforced extrinsic quality feature; is an L2 norm.

4. The method of claim 3, wherein the method further comprises: The step S3: the and respectively through N layers of decoders, while using instance normalization-based skip connections I-norm-skip modules between the 1st to N-1st layers of the encoders and their corresponding decoders to take full advantage of the structural information in the low-level semantic features, instance normalization being used to eliminate the extrinsic quality features in the low-level semantic features while retaining their intrinsic structural features; after the normalized semantic features of each layer of the encoders are spliced with the corresponding size features in the corresponding decoders in the channel dimension, the quality-enhanced image with the original low-quality fundus image structure is finally obtained and the quality-degraded image with the original high-quality fundus image structure , specifically comprising: Step S31: The I-norm-skip module removes the non-intrinsic quality features in the semantic features by the instance normalization as shown in the following formula: wherein, is a semantic feature of the 1st to N-1st layer fundus images, ; denotes the mean of the semantic feature by channel, denotes the variance of the semantic feature by channel; both together constitute the extrinsic quality features of the semantic feature; Step S32: normalizing the semantic features The structure information contained in the semantic features is delivered to the decoder in a channel concatenation manner; therefore, the semantic features of the first layer decoder are The semantic features of the second layer decoder are : Wherein, UP is an up-sampling operation.

5. A feature decoupled based unpaired fundus image structure preservation enhancement system, characterized in that, Comprise the following modules: The semantic feature extraction module is used for obtaining an unpaired original high-quality fundus image and an original low-quality fundus image After N-layer encoders, semantic features of each layer of fundus images are extracted }, wherein L and H respectively represent a low-quality fundus image and a high-quality fundus image, is the number of layers of the semantic features; the semantic features include intrinsic structure features and extrinsic quality features; decoupling and quality enhancement module for decoupling and quality enhancement of the highest layer, i.e. the Nth layer semantic feature and input decoupling enhancement module QDM for decoupling and quality exchange, output semantic feature and wherein, is the semantic feature after decoupling and quality enhancement of the low-quality fundus image semantic feature, is the semantic feature after decoupling and quality degradation of the high-quality fundus image semantic feature; a structure preserving module for and respectively pass through N layers of decoders, while using instance normalization based skip connections I-norm-skip modules between the encoders and their corresponding decoders from the 1st to the N-1st layers to take advantage of the structural information in the low-level semantic features, instance normalization is used to eliminate the extrinsic quality features in the low-level semantic features while retaining their intrinsic structural features; after the normalized semantic features of each layer of the encoder are concatenated with the corresponding size features in the corresponding decoder in the channel dimension, the final quality enhanced image with the original low-quality fundus image structure is obtained and the quality degraded image with the original high-quality fundus image structure ; a cycle consistency module, configured to and performing the above modules again for quality exchange, and finally obtaining a reconstructed high-quality fundus image and a low-quality fundus image Meanwhile, a consistency loss is constructed , a reinforcement loss is decoupled and a feature invariant loss The cycle consistency module is supervised and optimized, specifically including: Step S41: For the training quality exchange model, a consistency loss is designed between the low-quality fundus image of the reconstructed high-quality fundus image and the original image ; wherein, denotes the original low-quality fundus image, denotes the reconstructed low-quality fundus image; denotes the original high-quality fundus image, denotes the reconstructed high-quality fundus image; denotes the L1 norm; Step S42: constructing feature-invariant loss The enhanced image and the original image are supervised in the intrinsic structural features and the extrinsic quality features respectively, so as to ensure the invariance of the structure of the enhanced image and the expected quality. wherein, is a predetermined parameter, represents the decoder for outputting the semantic feature corresponding to the image, represents the semantic feature of the extracted enhanced image.

Images