Fundus image reflection repair and completion method and related device

Abstract

The application belongs to an image repairing method, aiming at the problems of weak edge recognition ability and unnatural repairing result of the current fundus image completion method, and provides a fundus image reflection repairing and completion method and related device, after obtaining a to-be-processed fundus image, first performing enhancement processing on the to-be-processed fundus image, then inputting the strengthened input image into an optimized ResNet50, wherein the optimized ResNet50 introduces a mixed attention mechanism composed of a channel attention mechanism and a pixel attention mechanism in the shallow layer of the network, and introduces a maximum pooling operation in the deep layer, finally, inputting the feature image into a generative adversarial network to obtain a fundus image without a reflection area. The application significantly improves the effect of image completion of the reflection area, and has good performance in the structural consistency, detail definition and semantic consistency of image completion.

Description

TECHNICAL FIELD

[0001] The application belongs to an image repairing method, and particularly relates to an eye fundus image reflection repairing and completing method and a related device. BACKGROUND

[0002] Eye fundus tessellation and parapapillary atrophy are common imaging manifestations in the early stage of pathological myopia. In the eye fundus image, the eye fundus tessellation is usually characterized by clear visibility of choroidal blood vessels and obvious tessellation texture structure of the retinal background. Parapapillary atrophy is characterized by different degrees of atrophy of the retina around the optic nerve head, accompanied by pigmentation or retinal blood vessel changes, presenting a ring or curved shape, and even spreading to the macular area. Accurate identification and segmentation of the eye fundus tessellation area and the parapapillary atrophy area in the eye fundus image have important clinical significance for early screening and disease progression evaluation of pathological myopia.

[0003] At present, during the eye fundus image acquisition process, reflection areas often appear due to uneven lighting, lens reflection and other reasons, resulting in missing of eye fundus image information, which seriously affects the subsequent automatic diagnosis and analysis. The existing eye fundus image completion method generally has problems such as insufficient extraction of structural information, weak edge recognition ability, and unnatural repair results. SUMMARY

[0004] In view of the problems of weak edge recognition ability and unnatural repair results in the existing eye fundus image completion method, the application provides an eye fundus image reflection repairing and completing method and a related device.

[0005] To achieve the above-mentioned purpose, the application adopts the following technical solutions:

[0006] In a first aspect, the application provides an eye fundus image reflection repairing and completing method, comprising:

[0007] obtaining an eye fundus image to be processed;

[0008] performing enhancement processing on the eye fundus image to be processed to obtain an enhanced input image;

[0009] inputting the enhanced input image into an optimized ResNet50 to obtain a feature image; the optimized ResNet50 comprises:

[0010] introducing a hybrid attention mechanism composed of a channel attention mechanism and a pixel attention mechanism into a shallow layer of the ResNet50, and introducing a max-pooling operation into a deep layer of the ResNet50;

[0011] inputting the feature image into a generative adversarial network to obtain an eye fundus image without reflection areas.

[0012] Further, the method for performing enhancement processing on the to-be-processed fundus image comprises the following steps.

[0013] A red channel is extracted from the to-be-processed fundus image and histogram equalization processing is performed to obtain a processed red channel image.

[0014] The processed red channel image and the to-be-processed fundus image are spliced in the channel dimension to obtain a multi-channel input image as an enhanced input image.

[0015] Further, the method for performing enhancement processing on the to-be-processed fundus image further comprises the following steps.

[0016] A multi-level filter is used to extract high-frequency information in the multi-channel input image in combination with a Sobel operator and a Laplacian operator to obtain the enhanced input image.

[0017] Further, the channel attention mechanism introduced in the shallow layer of the ResNet50 network comprises the following steps.

[0018] Global average pooling and maximum pooling are respectively performed on the enhanced input image as input in the spatial dimension to obtain two channel description vectors.

[0019] The two channel description vectors are input into a two-layer fully connected network sharing weights to generate a channel attention weight map through nonlinear transformation and a Sigmoid activation function.

[0020] Further, the pixel attention mechanism introduced in the shallow layer of the ResNet50 network comprises the following steps.

[0021] Each pixel position of the enhanced input image is modeled by a convolution kernel to learn the importance distribution of each pixel point to obtain a convolution feature map.

[0022] A pixel-by-pixel attention map is generated from the convolution feature map through a Sigmoid activation function.

[0023] The hybrid attention mechanism in the shallow layer of the ResNet50 network further comprises the following steps.

[0024] The enhanced input image, the channel attention weight map and the pixel-by-pixel attention map are spliced in the channel dimension to form a fused feature representation.

[0025] Further, the generative adversarial network comprises a generator and a discriminator.

[0026] The generator comprises six convolution layers and an upsampling operation.

[0027] The discriminator comprises five convolution layers.

[0028] Further, the loss function used in the training of the generative adversarial network includes:

[0029] a joint adversarial loss, a reconstruction loss and a perception loss.

[0030] In a second aspect, the application provides an eye fundus image reflection repair and completion system, comprising:

[0031] An acquisition module is configured to acquire an eye fundus image to be processed.

[0032] An enhancement module is configured to perform enhancement processing on the eye fundus image to be processed to obtain an enhanced input image.

[0033] A feature module is configured to input the enhanced input image into an optimized ResNet50 to obtain a feature image; the optimized ResNet50 comprises:

[0034] A hybrid attention mechanism composed of a channel attention mechanism and a pixel attention mechanism is introduced in a shallow layer of the ResNet50, and a max-pooling operation is introduced in a deep layer of the ResNet50.

[0035] A completion module is configured to input the feature image into a generative adversarial network to obtain an eye fundus image without a reflection area.

[0036] In a third aspect, the application provides an electronic device, comprising a memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code comprises computer instructions, when the computer instructions are executed by the processor, the electronic device executes the steps of the eye fundus image reflection repair and completion method.

[0037] In a fourth aspect, the application provides a computer readable storage medium, the computer readable storage medium stores a computer program, when the computer program is executed by a processor, the steps of the eye fundus image reflection repair and completion method are implemented.

[0038] Compared with the prior art, the application has the following beneficial effects:

[0039] The application provides an eye fundus image reflection repair and completion method. After an eye fundus image to be processed is acquired, the eye fundus image to be processed is subjected to enhancement processing, and then the enhanced input image is input into an optimized ResNet50, wherein the optimized ResNet50 introduces a mixed attention mechanism composed of a channel attention mechanism and a pixel attention mechanism in a shallow layer of the network, and introduces a max-pooling operation in a deep layer. Finally, the feature image is input into a generative adversarial network to obtain an eye fundus image without a reflection area. The application significantly improves the effect of image completion in the reflection area and has good performance in structural consistency, detail definition and semantic consistency of image completion. Compared with the prior art, the application can not only effectively repair the reflection area when processing the eye fundus image containing reflection, but also retain the key structural features of the image, improve the recognition accuracy of the lesion area, has high practical application value and broad application prospect.

[0040] The application also provides an eye fundus image reflection repair and completion system, an electronic device and a computer readable storage medium, which have all the advantages of the eye fundus image reflection repair and completion method. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0042] Figure 1 FIG. 1 is a flowchart of an eye fundus image reflection repair and completion method according to an embodiment of the application;

[0043] Figure 2 FIG. 2 is another flowchart of an eye fundus image reflection repair and completion method according to an embodiment of the application;

[0044] Figure 3 FIG. 3 is an eye fundus image to be processed in an embodiment of the application;

[0045] Figure 4 FIG. 4 is an eye fundus image after red channel extraction and histogram equalization processing; Figure 3

[0046] Figure 5 FIG. 5 is a schematic diagram of an eye fundus image reflection repair and completion system according to an embodiment of the application. DETAILED DESCRIPTION

[0047] ​In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings for the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0049] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0050] In the description of the embodiments of the present application, it should be noted that, if the orientation or position relationship indicated by the terms "upper", "lower", "horizontal", "inner" and the like is based on the orientation or position relationship shown in the drawings, or is the orientation or position relationship when the product of the present application is usually placed, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, therefore, it cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only used for differentiation in description, and cannot be understood as indicating or implying relative importance.

[0051] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly inclined. For example, "horizontal" only means that its direction is relatively more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined.

[0052] In the description of the embodiments of the present application, it should also be noted that, unless otherwise explicitly specified and limited, if the terms "arrange", "mount", "connect", "connect" appear, they should be understood in a broad sense, for example, can be fixedly connected, can be detachably connected, or integrally connected; can be mechanically connected, can be electrically connected; can be directly connected, can be indirectly connected through an intermediate medium, or can be the communication inside two elements. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0053] Pathological myopia is a significant eye disease leading to irreversible vision loss, and its development is accompanied by various characteristic fundus changes. Fundus mosaicism and perioptic disc atrophy are the "dual-core" imaging markers for early screening of pathological myopia. The former reflects retinal-choroidal microcirculatory disturbances, while the latter suggests mechanical damage to the optic nerve. Combined analysis of both can comprehensively assess the disease status, providing crucial evidence for early intervention and vision protection. In clinical practice, a dynamic monitoring system combining fundus images needs to be established to optimize management strategies for pathological myopia.

[0054] However, during the acquisition of fundus images, reflective areas often appear due to uneven lighting, lens reflection, and other reasons, leading to missing image information and seriously affecting subsequent automatic diagnosis and analysis. To solve this problem, this application proposes a method and related apparatus for fundus image reflection repair and completion, which will be described in detail below with reference to embodiments and accompanying drawings.

[0055] like Figure 1 The diagram shown is a flowchart illustrating one method for repairing and completing fundus image reflections according to this application, which may include:

[0056] S101, Obtain the fundus image to be processed.

[0057] In practical applications, fundus photographs containing reflective areas can be taken. Reflections are usually caused by uneven illumination or reflections from the surface of the equipment, manifesting as localized overexposure and loss of detail, such as blurred blood vessels and invisible optic disc structures.

[0058] S102, perform enhancement processing on the fundus image to be processed to obtain the enhanced input image.

[0059] The fundus images to be processed mainly suffer from the following problems: loss of detail due to overexposure in reflective areas, low overall contrast of the fundus, and noise interference. Therefore, enhancement processing can be performed on the fundus images to improve image quality and provide clearer input for subsequent feature extraction.

[0060] S103, the enhanced input image is input into the optimized ResNet50 to obtain the feature image.

[0061] The optimized ResNet50 includes:

[0062] A hybrid attention mechanism consisting of channel attention and pixel attention is introduced in the shallow layers of ResNet50, and max pooling is introduced in the deep layers of ResNet50.

[0063] In the channel attention mechanism, global average pooling is used to compress spatial information and generate channel weights, enhancing the channel response to key structures such as blood vessels and the optic disc. In the pixel attention mechanism, pixel-level weighting is applied to reflective areas to suppress interference from overexposed areas. The channel attention mechanism improves the semantic feature representation capability, while the pixel attention mechanism refines spatial details; the two complement each other to enhance feature robustness. Max pooling reduces the number of parameters, expands the receptive field, and captures global contextual information. The shallow layers of ResNet50 can capture details such as edges and textures, while the deeper layers can obtain semantic information, such as the location of reflective areas and the overall structure of the fundus.

[0064] S104, the feature image is input into the generative adversarial network to obtain a fundus image with no reflective area.

[0065] Generative Adversarial Networks (GANs) achieve the ability to learn from data distributions and generate realistic samples through an adversarial game mechanism between a generator and a discriminator. The generator can produce samples with the same dimensionality as the real data, while the discriminator can output a result representing the probability that the input is a real sample.

[0066] like Figure 2 The diagram shown is another flowchart illustrating the fundus image reflection repair and restoration method of this application, which may include:

[0067] S201, Image Input and Fusion.

[0068] This embodiment first acquires a fundus image containing reflective areas, extracts the red channel from the fundus image, and performs histogram equalization to obtain a processed red channel image, thereby enhancing structural contrast and detail. The optical properties of fundus tissue determine that blood vessels and lesion areas have the best contrast performance in the red channel, which is determined by the absorption spectrum characteristics of hemoglobin.

[0069] Subsequently, the processed red channel image is stitched together with the fundus image to be processed along the channel dimension to form a multi-channel input image, which serves as the enhanced input image. This multi-channel input image not only retains the original color information but also incorporates enhanced structural features, facilitating the subsequent extraction of image structural information through deep learning models and providing stronger support for the later completion task.

[0070] Automated segmentation of fundus images often relies on image analysis techniques such as deep learning. The red channel of fundus images contains more pathological information related to choroidal structures. During image preprocessing, the red channel is prioritized to enhance the discriminability of mosaic regions. Histogram equalization is widely used to enhance image contrast, which can improve the visibility of image textures, thereby improving the accuracy of subsequent segmentation models.

[0071] As shown in Figure 3 As shown in Figure 4 As shown in Figure 3 The red channel is extracted and the fundus image after histogram equalization processing.

[0072] S202, high frequency information extraction.

[0073] Before the multi-channel input image is sent into the deep network, in order to enhance the structural edge features in the image, in the embodiment, a multi-level filter is used, Sobel operator (first derivative) and Laplacian operator (second derivative) are used to perform parallel high frequency information extraction on the multi-channel input image, respectively capturing the gradient change and second order structure change features of the image, and then fusing the results of the two to obtain more rich high frequency structure information, strengthening the structural features, especially the edge profile of the reflection area, to obtain the enhanced input image. Sobel operator effectively extracts edge information by calculating the horizontal and vertical gradients of the multi-channel input image, highlighting the structural change area, but Sobel operator is sensitive to noise and difficult to detect complex details or non-obvious edges. The Laplacian operator further enhances the details and rapid change parts of the image by calculating the second derivative of the image, improving the boundary definition, but it does not have directionality and is more prone to amplify noise. The combination of the two retains the directional gradient information of the Sobel operator and uses the enhancement ability of the Laplacian operator to supplement the detail expression, effectively suppressing the shortcomings of each other, highlighting the edges and structures while reducing noise interference. This joint strategy can significantly improve the recognizability of the reflection area and its surrounding structure, thereby improving the recognition ability of the edge of the reflection area.

[0074] The present application combines histogram equalization, Sobel operator and Laplacian operator to facilitate fine extraction and subsequent high-quality completion of the reflection boundary.

[0075] S203, feature encoding and attention mechanism enhancement.

[0076] In the embodiment, ResNet50 (Residual Network 50, a kind of deep convolutional neural network architecture) is used as the backbone network, which effectively captures the complex structure and texture information in the enhanced input image by using its powerful multi-scale feature extraction capability.

[0077] It should be noted that the overall structure of ResNet50 can be divided into network shallow layer and network deep layer. The network shallow layer is mainly responsible for extracting low-level features, preserving spatial details, and is computationally efficient and versatile, which can be used for transfer learning or tasks requiring fine positioning. The network deep layer is mainly responsible for extracting high-level features and understanding semantic information, and has strong semantic abstraction ability, which can be used for tasks requiring global understanding.

[0078] In this embodiment, a hybrid attention mechanism composed of a channel attention mechanism and a pixel attention mechanism is introduced in the shallow layer of the ResNet50 network to enhance the expression ability of key areas.

[0079] The channel attention mechanism in the hybrid attention mechanism can be implemented by the following steps:

[0080] (1) First, the enhanced input image as input is subjected to global average pooling and maximum pooling respectively in the spatial dimension (HxW), and two channel description vectors with a size of (Cxlxl) are obtained. Wherein, H represents height, W represents width, and C represents channel number.

[0081] (2) Then, the two channel description vectors are sent to a two-layer fully connected network (i.e. Multilayer Perceptron, MLP) with shared weights, and a channel attention weight map is generated through nonlinear transformation and Sigmoid activation function. The channel attention weight map is used to weight the response of each channel, strengthen the important feature channels related to image semantics, and suppress irrelevant or redundant channel information.

[0082] The pixel attention mechanism in the hybrid attention mechanism can be implemented by the following steps:

[0083] (1) Each pixel position of the enhanced input image as input is modeled by a 1x1 convolution kernel, and the importance distribution of each pixel point is learned to obtain a feature map after convolution.

[0084] (2) A pixel-by-pixel attention map with a size of (1xHxW) is generated through Sigmoid activation function for the feature map after convolution, which is used to highlight local significant details such as edges and texture changes.

[0085] The pixel attention mechanism helps to improve the perception and expression ability of ResNet50 for detail areas such as reflective edges and lesion areas, thereby improving the overall image restoration effect.

[0086] Finally, the enhanced input image, channel attention weight map and pixel-by-pixel attention map are spliced in the channel dimension to form a fused feature representation, which has both global semantic perception and local detail expression ability, enhances the attention to key areas, and highlights the structural features of reflective or lesion areas.

[0087] In this embodiment, a maximum pooling operation is introduced in the deep layer of the ResNet50 network, which compresses the feature map size while preserving important features, improves the global structure understanding ability, and obtains a feature image.

[0088] In this embodiment, the improved ResNet50 overall design makes the network have stronger feature expression ability, and provides rich and reliable feature support for subsequent image completion tasks.

[0089] The improved ResNet50 can improve the attention of the network to the key structure and the reflection area, which is better than the traditional method. The ResNet50 backbone and the attention mechanism are used to realize deep feature extraction and hierarchical information integration, and to ensure the consistency of local details and global structure.

[0090] In S204, the image completion is performed by the generative adversarial network.

[0091] In this embodiment, the generative adversarial network is introduced to realize the completion of the reflection area of the fundus image on the basis of feature extraction and attention mechanism.

[0092] In the generative adversarial network, the overall structure is composed of two core parts: generator and discriminator. Specifically:

[0093] The generator receives the feature image extracted by the optimized ResNet50, and gradually restores the fundus image without the reflection area through six layers of convolution layers with a stride of 2 and a convolution kernel size of 7x7 and upsampling operation.

[0094] The discriminator includes five layers of convolution layers with a stride of 2 and a convolution kernel size of 4x4, which gradually extracts image features, and finally outputs a probability value for judging the authenticity of the image, so as to evaluate the difference between the generated image and the real image, and provide optimization feedback for the generator.

[0095] In actual application, the generative adversarial network needs to be trained in advance. During the training process, the generator and the discriminator are constantly optimized through the game of the two. In actual application, the weighted method can be used to guide the combination of the adversarial loss, the reconstruction loss and the perception loss, so as to improve the authenticity and structural consistency of the completed image. Finally, a complete structure, clear details and more natural fundus image without reflection is generated, which is suitable for subsequent medical image analysis tasks.

[0096] Specifically, the adversarial loss is obtained by the game mechanism of the generative adversarial network, which forces the generator to generate realistic images that are difficult to be distinguished by the discriminator, directly improving the overall authenticity of the completion result. The reconstruction loss can be quantified by, for example, mean square error or mean absolute error, to quantify the difference between the generated image and the target image at the pixel level, to ensure the continuity of the completed area and the surrounding pixels, and to avoid structure rupture. The perception loss is used to capture the consistency of image content and structure, so that the completion result is more consistent with human visual perception, especially suitable for medical image scenes sensitive to details.

[0097] The generative adversarial network can generate natural and real completed images through adversarial training, which is significantly better than traditional regression or interpolation methods. The comprehensive reconstruction loss, perception loss and adversarial loss can guide the model to improve the completion quality at the pixel and semantic levels.

[0098] The present application first extracts the red channel of the image and performs histogram equalization to enhance the structural contrast and fully extract the structural information. Then, the Sobel operator and the Laplacian operator are used to extract high-frequency edge features, which significantly improves the recognition ability of the edge of the reflection area. Then, a deep network with ResNet50 as the backbone is used to extract multi-scale features, and spatial attention and pixel attention mechanisms are introduced to enhance the focusing ability of key areas. Finally, the generative adversarial network is used to realize image completion, and the adversarial loss, reconstruction loss and perception loss are used to jointly optimize the generation effect during training, which improves the naturalness and structural consistency of the completed area, making the repair result more natural. Compared with the prior art, the present application can effectively repair the reflection area while maintaining the integrity of the semantic information of the image, generate a medical image with clear structure and rich details, and has stronger practicality and clinical application value.

[0099] The present application is particularly aimed at the problem of image information loss caused by the reflection area, and combines multi-channel image input, high-frequency feature extraction, feature encoding enhancement and generative adversarial network technology to realize the structured reconstruction and completion of the reflection area in the fundus image. It can effectively repair the area disturbed by reflection in the image, significantly improve the structural integrity and detail clarity of the image, and thus enhance the recognition accuracy of the lesion area, providing a more accurate and reliable image repair method for medical image processing and auxiliary diagnosis applications.

[0100] As shown in Figure 5 , it is a schematic diagram of the fundus image reflection repair and completion system of the present application, which can include:

[0101] The acquisition module is configured to acquire a fundus image to be processed.

[0102] The enhancement module is configured to perform enhancement processing on the fundus image to be processed to obtain an enhanced input image.

[0103] The feature module is configured to input the enhanced input image into an optimized ResNet50 to obtain a feature image; the optimized ResNet50 includes:

[0104] The hybrid attention mechanism composed of the channel attention mechanism and the pixel attention mechanism is introduced in the shallow layer of the ResNet50 network, and the max-pooling operation is introduced in the deep layer of the ResNet50 network.

[0105] The completion module is configured to input the feature image into a generative adversarial network to obtain a fundus image without a reflection area.

[0106] It should be noted that in several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the above-described system embodiments are only illustrative, for example, the division of each module is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component can be or can not be physically separated, and the component displayed as a module can be one physical unit or multiple physical units, that is, it can be located in one place or can be distributed to multiple different places. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0107] In addition, each module in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present separately, or two or more modules can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0108] The embodiments of the present application also provide an electronic device, which can include one or more processors, memories and communication interfaces.

[0109] The memory, the communication interface and the processor are coupled. For example, the memory, the communication interface and the processor can be coupled together through a bus.

[0110] The communication interface is configured to perform data transmission with other devices. The memory stores computer program codes. The computer program codes include computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the fundus image reflection repair and completion method.

[0111] The processor can be a processor or a controller, for example, can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, transistor logic device, hardware component or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the present disclosure. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The processor can be used to support the electronic device to perform the method steps provided in the above embodiments.

[0112] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0113] The computer readable storage medium provided by the embodiments of the present application stores a computer program, and the computer program is executed by the processor to implement the steps of the fundus image reflection repair and completion method.

[0114] The computer readable storage medium involved in the present application includes random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD ROM, or any other form of storage medium known in the technical field.

[0115] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for reflection repair and completion of an eye fundus image, characterized by, The method comprises the following steps: An eye fundus image to be processed is acquired; An enhancement processing is performed on the eye fundus image to be processed to obtain an enhanced input image; The enhanced input image is input into an optimized ResNet50 to obtain a feature image; The optimized ResNet50 comprises: A hybrid attention mechanism composed of a channel attention mechanism and a pixel attention mechanism is introduced in a shallow layer of the ResNet50, and a max-pooling operation is introduced in a deep layer of the ResNet50; The feature image is input into a generative adversarial network to obtain an eye fundus image without a reflection area.

2. The method of claim 1, wherein the method further comprises: The method for performing the enhancement processing on the eye fundus image to be processed comprises the following steps: A red channel is extracted from the eye fundus image to be processed and subjected to a histogram equalization processing to obtain a processed red channel image; The processed red channel image and the eye fundus image to be processed are spliced in a channel dimension to obtain a multi-channel input image as the enhanced input image.

3. The method of claim 2, wherein the method further comprises: The method for performing the enhancement processing on the eye fundus image to be processed further comprises the following steps: A multi-level filter is used to extract high-frequency information in the multi-channel input image by combining a sobel operator and a Laplacian operator to obtain the enhanced input image.

4. The method of claim 1, wherein the method further comprises: The channel attention mechanism introduced in the shallow layer of the ResNet50 comprises the following steps: Global average pooling and max-pooling are respectively performed on the enhanced input image as an input in a spatial dimension to correspondingly obtain two channel description vectors; The two channel description vectors are input into a two-layer fully connected network sharing weights to generate a channel attention weight map through a nonlinear transformation and a Sigmoid activation function.

5. The method of claim 4, wherein the reflection of the fundus image is repaired and completed by using a reflection of a fundus image of a patient who has undergone a cataract surgery. The pixel attention mechanism introduced in the shallow layer of the ResNet50 comprises the following steps: Each pixel position of the enhanced input image is modeled by a convolution kernel to learn the importance distribution of each pixel point to obtain a feature map after convolution; A pixel-by-pixel attention map is generated from the feature map after convolution through a Sigmoid activation function. The hybrid attention mechanism in the shallow layer of the ResNet50 further comprises the following steps: The enhanced input image, the channel attention weight map and the pixel-by-pixel attention map are spliced in a channel dimension to form a fused feature representation.

6. The method of claim 1, wherein the method further comprises: The generative adversarial network comprises a generator and a discriminator; The generator comprises six convolutional layers and an up-sampling operation; The discriminator comprises five convolutional layers.

7. The method of claim 1, wherein the method further comprises: determining a reflection of the fundus image; and removing the reflection of the fundus image. When the generative adversarial network is trained, a loss function used comprises: A joint adversarial loss, a reconstruction loss and a perception loss.

8. A fundus image reflection repair and completion system characterized by comprising: The method comprises the following steps: An acquisition module is configured to acquire an eye fundus image to be processed; An enhancement module is configured to perform an enhancement processing on the eye fundus image to be processed to obtain an enhanced input image; A feature module is configured to input the enhanced input image into an optimized ResNet50 to obtain a feature image; The optimized ResNet50 comprises: A hybrid attention mechanism composed of a channel attention mechanism and a pixel attention mechanism is introduced in a shallow layer of the ResNet50, and a max-pooling operation is introduced in a deep layer of the ResNet50; A completion module is configured to input the feature image into a generative adversarial network to obtain an eye fundus image without a reflection area.

9. An electronic device, comprising: The method comprises the following steps: The electronic device comprises a memory, one or more processors; the memory is coupled with the processor; wherein the memory has computer program code stored therein, the computer program code comprises computer instructions, when the computer instructions are executed by the processor, the electronic device executes the steps of the fundus image reflection repair and completion method as claimed in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium has a computer program stored therein, when the computer program is executed by the processor, the steps of the fundus image reflection repair and completion method as claimed in any one of claims 1-7 are implemented.

Images