Fundus image training method, enhancement method and related device
Through the combination method of generator network and discriminator network, the problem of low-quality images during fundus image acquisition is solved, and the generation of high-quality images is realized, the number of acquisitions is reduced, the diagnostic efficiency is improved and the harm to patients is avoided.
Patent Information
- Application Number
- CN202210124602.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-10
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2042-02-10
AI Technical Summary
During the fundus image acquisition process, due to insufficient light, excessive exposure, eyeball displacement, etc., the fundus image has deteriorated types such as dummy blur, motion blur, overexposed and dark. It needs to be repeated multiple times, which affects the diagnostic efficiency and treats the patient. The eyes cause unnecessary damage.
Using the combination method of generator network and discriminator network, by obtaining low-quality and high-quality fundus image training sets, calculating loss function values and iteratively optimized, the generator network can directly generate high-quality images from low-quality images, reducing acquisition frequency.
The quality of fundus images is improved, blurred images are clarified, unnecessary damage to the patient's eyes is reduced, and diagnostic efficiency is improved.
Smart Images
Figure CN114611579B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image processing, and in particular to a fundus image training method, an enhancement method and related devices. Background Art
[0002] Fundus image detection is currently one of the most important means of screening for ophthalmic diseases. Doctors can preliminarily diagnose whether they have common ophthalmic diseases such as pathological myopia, glaucoma, diabetic retinopathy, cataracts, etc. from fundus images. Clear fundus images can help doctors make quick and accurate diagnoses, and to a certain extent avoid misdiagnosis. However, in the actual fundus image acquisition process, especially for young children, due to insufficient light, overexposure, eye displacement and other reasons, the fundus image will show degraded types such as out-of-focus blur, motion blur, overexposure and darkness. In order to obtain clear and usable fundus images, it is often necessary to repeat dozens of times, which not only greatly affects the hospital's diagnostic efficiency, but also repeatedly exposing and shooting the eyeballs will cause unnecessary damage to the patient's eyes. Summary of the invention
[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
[0004] In a first aspect, the present invention provides a fundus image training method, comprising:
[0005] Acquire a fundus image training set, wherein the fundus image training set includes multiple groups of training fundus images, and each group of training fundus images includes a low-quality training fundus image and a corresponding high-quality training fundus image;
[0006] Passing the low-quality training fundus image through a generator network to obtain a predicted high-quality fundus image;
[0007] Passing the high-quality training fundus image and the predicted high-quality fundus image through a discriminator network to obtain a discriminant matrix;
[0008] Calculating a loss function value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix;
[0009] The generator network and the discriminator network are iteratively optimized according to the loss function value, and the optimized generator network is used as a fundus image training model.
[0010] The fundus image training method provided in accordance with the embodiment of the first aspect of the present invention has at least the following beneficial effects: the fundus image training set is passed through a generator network and a discriminator network to obtain a loss function value, and the generator network and the discriminator network are iteratively optimized according to the loss function value, wherein the generator network can obtain a predicted high-quality fundus image, and the discriminator network is used to judge the authenticity of the predicted high-quality fundus image, and iterative optimization can improve the performance of the generator network and the discriminator network, improve the quality of the predicted high-quality fundus image, and make the blurred fundus image clear. In addition, low-quality fundus images can directly generate high-quality fundus images through the fundus image training model obtained by iterative optimization. This process reduces the number of fundus image acquisition times and effectively avoids unnecessary damage to the patient's eyes.
[0011] Some embodiments according to the first aspect of the present invention further include:
[0012] Before the low-quality training fundus image is passed through a generator network to obtain a predicted high-quality fundus image, the method includes:
[0013] The low-quality training fundus image is preprocessed, and the preprocessing result is updated into the low-quality training fundus image.
[0014] According to some embodiments of the first aspect of the present invention, the step of passing the low-quality training fundus image through a generator network to obtain a predicted high-quality fundus image includes:
[0015] Passing the low-quality training fundus image through an input projection layer to obtain a first feature map;
[0016] Passing the first feature map through a plurality of first sampling modules to obtain a second feature map, wherein the first sampling module includes a plurality of local transformer layers and a downsampling layer;
[0017] Passing the second feature map through multiple local transformer layers to obtain a third feature map;
[0018] Passing the third feature map through a plurality of second sampling modules to obtain a fourth feature map, wherein the second sampling module includes a plurality of local transformer layers and an upsampling layer, and the first sampling module and the second sampling module are channel-cascaded;
[0019] Passing the fourth feature map through an output projection layer to obtain a residual fundus image;
[0020] The residual fundus image is added to the low-quality training fundus image to obtain a predicted high-quality fundus image.
[0021] According to some embodiments of the first aspect of the present invention, the step of passing the high-quality training fundus image and the predicted high-quality fundus image through a discriminant network to obtain a discriminant matrix includes:
[0022] Passing the high-quality training fundus image and the predicted high-quality fundus image through an input mapping layer to obtain a fifth feature map;
[0023] Passing the fifth feature map through a plurality of third sampling modules to obtain a sixth feature map, wherein the third sampling module includes a plurality of local transformer layers and a downsampling layer;
[0024] The sixth feature map is passed through the output mapping layer to obtain a discriminant matrix.
[0025] According to some embodiments of the first aspect of the present invention, the loss function value is calculated based on the Charbonnier Loss value, the fundus quality perception loss value, the adversarial loss value and the edge loss value;
[0026] Correspondingly, the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix, and the calculation of the loss function value, include:
[0027] Calculating the CharbonnierLoss value according to the predicted high-quality fundus image and the high-quality training fundus image;
[0028] Calculating the fundus quality perception loss value according to the predicted high-quality fundus image and the high-quality training fundus image;
[0029] Calculating the adversarial loss value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix, wherein the adversarial loss value includes the adversarial loss value of the generator network and the adversarial loss value of the discriminator network;
[0030] Calculating the edge loss value according to the predicted high-quality fundus image and the high-quality training fundus image;
[0031] The Charbonnier Loss value, the fundus quality perception loss value, the adversarial loss value and the edge loss value are summed according to the weights to obtain the loss function value.
[0032] According to some embodiments of the first aspect of the present invention, calculating the Charbonnier Loss value according to the predicted high-quality fundus image and the high-quality training fundus image includes:
[0033] The Charbonnier Loss value L1 is calculated according to the predicted high-quality fundus image, the high-quality training fundus image and the Charbonnier Loss function, wherein the Charbonnier Loss function is:
[0034]
[0035] Among them, I R is the predicted high-quality fundus image, y is the high-quality training fundus image, and ε is a fixed constant, which is 10 -3 ;
[0036] The calculating the fundus quality perception loss value according to the predicted high-quality fundus image and the high-quality training fundus image comprises:
[0037] The fundus quality perception loss value L is calculated based on the predicted high-quality fundus image, the high-quality training fundus image and the fundus quality perception function. p , wherein the fundus quality perception function is:
[0038]
[0039] Among them, W and H are the dimensions of the feature map, and φ() is the feature map extracted by the VGG-19 network pre-trained on the fundus image quality assessment dataset Eyes-Q;
[0040] The calculating the adversarial loss value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix includes:
[0041] The adversarial loss value L is calculated based on the predicted high-quality fundus image, the high-quality training fundus image, the discriminant matrix and the adversarial loss function. adv , where the adversarial loss function is:
[0042]
[0043]
[0044]
[0045] in, is the adversarial loss of the discriminator network, is the adversarial loss of the generator network, N is the order of the discriminant matrix output by the discriminant network, D represents the discriminant network, G represents the generator network, D() represents the probability that the discriminant network discriminates the input image as a real image, and i is the point in the discriminant matrix;
[0046] The calculating the edge loss value according to the predicted high-quality fundus image and the high-quality training fundus image comprises:
[0047] The edge loss value L is calculated based on the predicted high-quality fundus image, the high-quality training fundus image and the edge loss function. edge , where the edge loss function is:
[0048]
[0049] Among them, LOG() is the Gaussian Laplace operator;
[0050] The step of summing the Charbonnier Loss value, the fundus quality perception loss value, the adversarial loss value, and the edge loss value according to the weights to obtain the loss function value includes:
[0051] L=L1+λ1L p +λ2L edg +λ3L adv
[0052] Wherein, L is the loss function value, λ1, λ2, λ3 are hyperparameters used to control the weights of different losses.
[0053] According to some embodiments of the first aspect of the present invention, iteratively optimizing the generator network and the discriminator network according to the loss function value, and using the optimized generator network as a fundus image training model, comprises:
[0054] According to the loss function value, the generator network and the discriminator network are alternately trained by back propagation to update and optimize the parameters of the generator network and the discriminator network;
[0055] Using a plurality of sets of the training fundus images, the generator network and the discriminator network are trained multiple times to iteratively optimize the parameters of the generator network and the discriminator network;
[0056] The optimized generator network is used as a fundus image training model, wherein, as the generator network and the discriminator network are optimized, the loss function value decreases, the probability that the discriminator network identifies the input image as a real image tends to 0.5, and the quality of the predicted high-quality fundus image is close to that of the high-quality training fundus image.
[0057] Some embodiments according to the first aspect of the present invention further include:
[0058] Acquire a fundus image test set, wherein the fundus image test set includes a plurality of low-quality test fundus images and corresponding high-quality test fundus images;
[0059] Passing the low-quality test fundus image through the fundus image training model to obtain a synthetic high-quality fundus image;
[0060] The performance of the fundus image training model is obtained according to the similarity between the high-quality synthetic fundus image and the high-quality test fundus image.
[0061] In a second aspect, the present invention provides a fundus image enhancement method, comprising:
[0062] Acquire original fundus images;
[0063] The original fundus image is passed through the fundus image training model as described in the first aspect to obtain a target fundus image.
[0064] Since the fundus image enhancement method of the second aspect applies any fundus image training method of the first aspect, it has all the beneficial effects of the first aspect of the present invention.
[0065] In a third aspect, the present invention provides a fundus image training device, comprising:
[0066] An image acquisition module, wherein the image acquisition module is used to acquire a fundus image training set;
[0067] A generator network module, wherein the generator network module is used to pass the low-quality training fundus image through a generator network to obtain a predicted high-quality fundus image;
[0068] A discriminator network module, wherein the discriminator network module is used to obtain a discriminant matrix by passing the high-quality training fundus image and the predicted high-quality fundus image through a discriminator network;
[0069] A loss function value calculation module, the loss function value calculation module is used to calculate the loss function value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix;
[0070] A network optimization module, wherein the network optimization module is used to iteratively optimize the generator network and the discriminator network according to the loss function value, and use the optimized generator network as a fundus image training model.
[0071] Since the fundus image training device of the third aspect can execute any fundus image training method of the first aspect and / or the fundus image enhancement method as described in the second aspect, it has all the beneficial effects of the first aspect of the present invention.
[0072] In a fourth aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program according to the fundus image training method described in the first aspect and / or the fundus image enhancement method described in the second aspect.
[0073] Since the processor of the fourth aspect executes the computer program as the class activation map generation method of any one of the first aspect and / or the neural network interpretable method of any one of the second aspect, it has all the beneficial effects of the first aspect of the present invention.
[0074] In a fifth aspect, the present invention provides a computer storage medium, comprising computer executable instructions stored therein, wherein the computer executable instructions are used to execute the fundus image training method as described in the first aspect and / or the fundus image enhancement method as described in the second aspect.
[0075] Since the computer storage medium of the fifth aspect can execute any of the class activation map generation methods of the first aspect and / or any of the neural network interpretable methods of the second aspect, it has all the beneficial effects of the first aspect of the present invention.
[0076] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0078] Figure 1 is a schematic diagram of the structure of a fundus image training device provided by an embodiment of the present application;
[0079] Figure 2 This is a diagram of the main steps of a fundus image training method provided by an embodiment of the present application;
[0080] Figure 3 is a diagram of the preprocessing steps of the fundus image training method provided by one embodiment of the present application;
[0081] Figure 4 It is a step diagram of generating a predicted high-quality fundus image in a fundus image training method provided by an embodiment of the present application;
[0082] Figure 5It is a step diagram of generating a discriminant matrix of a fundus image training method provided by an embodiment of the present application;
[0083] Figure 6 It is a step diagram of loss function calculation of a fundus image training method provided by an embodiment of the present application;
[0084] Figure 7 It is a step diagram of iterative optimization of a fundus image training method provided by an embodiment of the present application;
[0085] Figure 8 It is a step diagram of a fundus image training model for testing a fundus image training method provided by an embodiment of the present application;
[0086] Fig. 9 is a step diagram of a fundus image enhancement method provided by an embodiment of the present application;
[0087] Fig.10 This is an overall framework diagram of a fundus image training network provided by an embodiment of the present application;
[0088] Fig.11 This is an overall framework diagram of a fundus image enhancement network provided by an embodiment of the present application;
[0089] Fig.12 is a framework diagram of a generator network provided by an embodiment of the present application;
[0090] Fig.13 is a framework diagram of a discriminator network provided by an embodiment of the present application;
[0091] Fig.14 is a framework diagram of a local transformer module provided in one embodiment of the present application;
[0092] Fig.15 It is a comparison diagram and a detailed magnified diagram of a test fundus image training model provided by an embodiment of the present application;
[0093] Fig.16 It is a comparison diagram of a test fundus image training model and a retinal blood vessel segmentation diagram provided by an embodiment of the present application;
[0094] Fig.17 It is a comparison diagram of a test fundus image training model and an optic cup and optic disc segmentation diagram provided by an embodiment of the present application. DETAILED DESCRIPTION
[0095] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the embodiments of the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the embodiments of the present application.
[0096] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0097] It should also be understood that the references to "one embodiment" or "some embodiments" described in the specification of the embodiments of the present application mean that one or more embodiments of the embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0098] Fundus image detection is one of the most important means of screening for ophthalmic diseases. Doctors can preliminarily diagnose common ophthalmic diseases such as pathological myopia, glaucoma, diabetic retinopathy, cataracts, etc. from fundus images. Clear fundus images can help doctors make quick and accurate diagnoses, and to a certain extent avoid misdiagnosis. However, in the actual fundus image acquisition process, especially for young children, due to insufficient light, overexposure, eye displacement and other reasons, the fundus image will show degraded types such as blur, motion blur, overexposure and dark. In order to obtain clear and usable fundus images, it is often necessary to repeat dozens of times, which not only greatly affects the hospital's diagnostic efficiency, but also repeatedly exposes the eyeball for multiple times and causes unnecessary damage to the patient's eyes. Although tasks such as deblurring, denoising and super-resolution are all image restoration tasks, and all require the restoration of detailed information in the original image as much as possible, there are great differences between different restoration tasks and different types of training images. Methods that work well on natural images often perform poorly on fundus images.
[0099] Based on this, the embodiments of the present application provide a fundus image training method, an enhancement method and related devices. The embodiments of the present application are based on transformer and generative adversarial network, and propose a fundus image training method. A fundus image training model is obtained through the fundus image training method. The fundus image training model can directly generate high-quality fundus images based on low-quality fundus images, which can reduce the number of fundus image acquisitions and effectively avoid unnecessary damage to the patient's eyes.
[0100] The embodiments of the present application are further described below in conjunction with the accompanying drawings.
[0101] like Figure 1 As shown, Figure 1 FIG. 1 is a schematic diagram of the structure of a fundus image training device provided by an embodiment of the present application. Figure 1 In the example, the fundus image training device includes an image acquisition module 100, a generator network module 200, a discriminator network module 300, a loss function value calculation module 400 and a network optimization module 500.
[0102] Among them, the image acquisition module 100 is connected to the generator network module 200 and the discriminator network module 300, and the image acquisition module 100 is used to acquire a fundus image training set, which includes multiple groups of training fundus images, and each group of training fundus images includes a low-quality training fundus image and a corresponding high-quality training fundus image.
[0103] The generator network module 200 is connected to the image acquisition module 100 and the discriminator network module 300 respectively. The generator network module 200 is used to pass the received low-quality training fundus image through the generator network to obtain a predicted high-quality fundus image, and send the predicted high-quality fundus image to the discriminator network module 300.
[0104] The discriminator network module 300 is respectively connected to the image acquisition module 100, the generator network module 200, and the loss function value calculation module 400. The discriminator network module 300 is used to pass the received high-quality training fundus image and the predicted high-quality fundus image through the discriminator network to obtain a discriminant matrix, and send the discriminant matrix to the loss function value calculation module 400.
[0105] The loss function value calculation module 400 is respectively connected to the image acquisition module 100, the discriminator network module 300 and the network optimization module 500. The loss function value calculation module 400 is used to calculate the loss function value based on the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix.
[0106] The network optimization module 500 is respectively connected to the loss function value calculation module 400, the generator network module 200 and the discriminator network module 300. The network optimization module 500 is used to iteratively optimize the generator network and the discriminator network according to the loss function value, and use the optimized generator network as a fundus image training model.
[0107] In this embodiment, the image acquisition module 100 sends the acquired fundus image training set to the generator network module 200, the discriminator network module 300 and the loss function value calculation module 400 respectively; the generator network module 200 passes the low-quality training fundus image received from the image acquisition module 100 through the generator network to obtain a predicted high-quality fundus image, and sends the predicted high-quality fundus image to the discriminator network module 300; the discriminator network module 300 is used to pass the high-quality training fundus image received from the image acquisition module 100 and the predicted high-quality fundus image from the generator network module 200 through the discriminator network to obtain a discriminant matrix, and send the discriminant matrix to the loss function value calculation module 400; the loss function value calculation module 400 calculates the loss function value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix, and sends the loss function value to the network optimization module 500; the network optimization module 500 iteratively optimizes the generator network and the discriminator network according to the loss function value, and uses the optimized generator network as the fundus image training model.
[0108] The devices and application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0109] It can be understood by those skilled in the art that Figure 1 The device structure shown in the figure does not constitute a limitation on the embodiments of the present application, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0110] exist Figure 1 In the device structure shown, each module can respectively call the fundus image training program stored therein to execute the fundus image training method.
[0111] Based on the above-mentioned device, various embodiments of the fundus image training method of the embodiment of the present application are proposed.
[0112] like Figure 2 As shown, Figure 2This is a diagram of the main steps of a fundus image training method provided by an embodiment of the present application. The fundus image training method includes but is not limited to the following steps:
[0113] Step S100: Acquire a fundus image training set, wherein the fundus image training set includes multiple groups of training fundus images, and each group of training fundus images includes a low-quality training fundus image and a corresponding high-quality training fundus image.
[0114] It is understandable that the degradation types of low-quality training fundus images include out-of-focus blur, motion blur, insufficient illumination, artifacts, etc. The degradation types contained in each low-quality training fundus image are nonlinear superposition and mixture of multiple degradations.
[0115] Step S200: Pass the low-quality training fundus image through the generator network to obtain a predicted high-quality fundus image.
[0116] Step S300: The high-quality training fundus image and the predicted high-quality fundus image are passed through a discriminator network to obtain a discriminant matrix.
[0117] Step S400: Calculate the loss function value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix.
[0118] Step S500, iteratively optimize the generator network and the discriminator network according to the loss function value, and use the optimized generator network as a fundus image training model.
[0119] It should be understood that the purpose of optimizing the generator network is to make the quality of the predicted high-quality fundus image output by the generator network as close as possible to the quality of the high-quality training fundus image; the purpose of optimizing the discriminator network is to enable the discriminator network to distinguish whether the input image is a predicted high-quality fundus image or a high-quality training fundus image. When the performance of the generator network and the discriminator network are both optimal, the quality of the predicted high-quality fundus image generated by the generator network is optimal.
[0120] It should be understood that by passing the fundus image training set through the generator network and the discriminator network to obtain the loss function value, and iteratively optimizing the generator network and the discriminator network according to the loss function value, the performance of the generator network and the discriminator network can be improved, thereby improving the quality of predicting high-quality fundus images and making blurry fundus images clear. In addition, low-quality fundus images can be directly generated into high-quality fundus images through the fundus image training model obtained by iterative optimization. This process reduces the number of fundus image collection times and effectively avoids unnecessary damage to the patient's eyes.
[0121] In addition, in one embodiment, referring to Figure 3The preprocessing is located between step S100 and step S200, and the preprocessing includes but is not limited to the following steps:
[0122] Step S600: preprocess the low-quality training fundus image, and update the preprocessing result into the low-quality training fundus image.
[0123] In some embodiments, assuming that the spatial resolution of the low-quality training fundus image is 2560×2560, the low-quality training fundus image is preprocessed to be cropped into non-overlapping and non-intersecting square image blocks of size 128×128, the cropped image blocks are randomly rotated and translated to achieve image data expansion, and the preprocessing result is updated to the low-quality training fundus image.
[0124] In addition, in one embodiment, referring to Figure 4 , Fig.12 and Fig.14 Step S200 includes but is not limited to the following steps:
[0125] Step S210: Pass the low-quality training fundus image through the input projection layer to obtain a first feature map.
[0126] In some embodiments, the input projection layer is a convolution layer with a convolution kernel size of 3×3 and a stride of 1, which can convert a 128×128×3 image into a 128×128×32 feature map.
[0127] Step S220: Pass the first feature map through multiple first sampling modules to obtain a second feature map, wherein the first sampling module includes multiple local transformer layers and a downsampling layer.
[0128] It should be understood that the number of first sampling modules is not limited, but the number of first sampling modules should be consistent with the number of second sampling modules.
[0129] In some embodiments, the number of times the first feature map passes through the first sampling module is 3, and the first sampling module sequentially includes two local transformer layers and one downsampling layer.
[0130] In some embodiments, the local transformer layer includes a series of non-overlapping local transformer modules, which can calculate the multi-head self-attention in the local feature maps in all non-overlapping local feature maps of the segmentation, so as to extract feature information in the non-overlapping feature maps. Assuming that the first local feature map is obtained by the local transformer module to obtain the fifth local feature map, this process is that the first local feature map is obtained by layer normalization and multi-head self-attention to obtain the second local feature map, the first local feature map is added to the second local feature map to obtain the third local feature map, the third local feature map is sequentially obtained by layer normalization, convolution layer, activation layer, convolution layer, activation layer, convolution layer, activation layer, to obtain the fourth local feature map, the fourth local feature map is added to the third local feature map to obtain the fifth local feature map.
[0131] In some embodiments, the downsampling layer is a convolution layer with a convolution kernel size of 4×4 and a stride of 2. The downsampling layer can reduce the size of the feature map to half of the original size.
[0132] Step S230: Pass the second feature map through multiple local transformer layers to obtain a third feature map.
[0133] Step S240: Pass the third feature map through multiple second sampling modules to obtain a fourth feature map, wherein the second sampling module includes multiple local transformer layers and an upsampling layer, and the first sampling module and the second sampling module are channel-cascaded.
[0134] It should be understood that channel cascading of the first sampling module and the second sampling module can improve the performance of the generator network and make the generator network more easily converged.
[0135] It should be understood that the number of the second sampling modules is not limited, but the number of the second sampling modules should be consistent with the number of the first sampling modules.
[0136] In some embodiments, the number of times the third feature map passes through the second sampling module is 3, and the second sampling module sequentially includes an upsampling layer and two local transformer layers.
[0137] In some embodiments, the upsampling layer includes two parts. The feature map size is first doubled by bilinear interpolation, and then feature information is extracted by a convolution layer with a convolution kernel size of 3×3 and a step size of 1.
[0138] Step S250: Pass the fourth feature map through the output projection layer to obtain a residual fundus image.
[0139] In some embodiments, the output projection layer is a convolution layer with a convolution kernel size of 3×3 and a stride of 1, which can convert a 128×128×32 feature map into a 128×128×3 RGB image.
[0140] Step S260: Add the residual fundus image to the low-quality training fundus image to obtain a predicted high-quality fundus image.
[0141] In addition, in one embodiment, referring to Figure 5 and Fig.13 Step S300 includes but is not limited to the following steps:
[0142] Step S310: The high-quality training fundus image and the predicted high-quality fundus image are input through a mapping layer to obtain a fifth feature map.
[0143] It should be understood that the input mapping layer is a convolution layer with a convolution kernel size of 3×3 and a stride of 1, which can convert a 128×128×3 image into a 128×128×32 feature map.
[0144] Step S320: Pass the fifth feature map through multiple third sampling modules to obtain a sixth feature map, wherein the third sampling module includes multiple local transformer layers and a downsampling layer.
[0145] It should be understood that the number of the third sampling modules is 3, and the third sampling module sequentially includes two local transformer layers and one downsampling layer.
[0146] Step S330: Pass the sixth feature map through the output mapping layer to obtain a discriminant matrix.
[0147] In some embodiments, the output projection layer is a convolution layer with a convolution kernel size of 1×1, which can reduce the number of channels of the feature map to one dimension.
[0148] It should be understood that the discriminant matrix is a square matrix with the same number of rows and columns.
[0149] In some embodiments, the discriminant matrix is a 16×16 matrix, and whether the input image is a real clear picture can be determined based on all the points in the discriminant matrix.
[0150] It should be understood that the preset label is 1 / 0, where the result obtained by passing the high-quality training fundus image through the discriminator network is 1, that is, Real, and the result obtained by passing the low-quality training fundus image through the discriminator network is 0, that is, Fake. If the quality of the predicted high-quality fundus image is similar to that of the low-quality training fundus image, the discriminator determines that the predicted high-quality fundus image is Fake and the output is 0; if the quality of the predicted high-quality fundus image is similar to that of the high-quality training fundus image, the discriminator determines that the predicted high-quality fundus image is Real and the output is 1.
[0151] In addition, in one embodiment, the loss function value is calculated based on the Charbonnier Loss value, the fundus quality perception loss value, the confrontation loss value and the edge loss value, referring to Figure 6 Step S400 includes but is not limited to the following steps:
[0152] Step S410: Calculate the Charbonnier Loss value according to the predicted high-quality fundus image and the high-quality training fundus image.
[0153] In some embodiments, a Charbonnier Loss value L1 is calculated based on the predicted high-quality fundus image, the high-quality training fundus image and the Charbonnier Loss function, where the Charbonnier Loss function is:
[0154]
[0155] Among them, I R To predict high-quality fundus images, y is a high-quality training fundus image, and ε is a fixed constant, which is 10 -3 .
[0156] It should be understood that the Charbonnier Loss function can preserve edge details, provide pixel space regularization for iterative optimization, and contribute to quality improvement.
[0157] Step S420: Calculate the fundus quality perception loss value according to the predicted high-quality fundus image and the high-quality training fundus image.
[0158] In some embodiments, the fundus quality perception loss value L is calculated based on the predicted high-quality fundus image, the high-quality training fundus image and the fundus quality perception function. p , where the fundus quality perception function is:
[0159]
[0160] Among them, W and H are the dimensions of the feature map, and φ() is the feature map extracted by the VGG-19 network pre-trained on the fundus image quality assessment dataset Eyes-Q.
[0161] It should be understood that, through the fundus quality perception loss, the generated restored image can be made close to the real clear image in terms of high-level features.
[0162] Step S430, calculating the adversarial loss value based on the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix, wherein the adversarial loss value includes the adversarial loss value of the generator network and the adversarial loss value of the discriminator network.
[0163] In some embodiments, the adversarial loss value L is calculated based on the predicted high-quality fundus image, the predicted high-quality fundus image, the discriminant matrix, and the adversarial loss function. adv , where the adversarial loss function is:
[0164]
[0165]
[0166]
[0167] in, is the adversarial loss of the discriminator network, is the adversarial loss of the generator network, N is the order of the discriminant matrix output by the discriminant network, D represents the discriminator network, G represents the generator network, D() represents the probability that the discriminator network identifies the input image as a real image, and i is the point in the discriminant matrix.
[0168] It should be understood that the adversarial loss function is a measure of the semantic perceptual similarity between the predicted high-quality fundus image and the predicted high-quality fundus image.
[0169] Step S440: Calculate the edge loss value according to the predicted high-quality fundus image and the high-quality training fundus image.
[0170] In some embodiments, the edge loss value L is calculated based on the predicted high-quality fundus image, the high-quality training fundus image and the edge loss function. edge , where the marginal loss function is:
[0171]
[0172] Among them, LOG() is the Laplace operator of Gaussian.
[0173] It should be understood that, through the edge loss function of the embodiment of the present application, it is possible to effectively suppress the generator network from generating false non-existent textures, enhance real textures, and greatly ensure the authenticity of the predicted high-quality fundus images.
[0174] Step S450: sum the Charbonnier Loss value, the fundus quality perception loss value, the adversarial loss value and the edge loss value according to the weights to obtain a loss function value.
[0175] In some embodiments, the Charbonnier Loss value, the fundus quality perception loss value, the adversarial loss value, and the edge loss value are summed according to the weights to obtain a loss function value, including:
[0176] L=L1+λ1L p +λ2L edg +λ3L adv
[0177] Among them, L is the loss function value, λ1, λ2, λ3 are hyperparameters used to control the weights of different losses.
[0178] In addition, in one embodiment, referring to Figure 7 Step S500 includes but is not limited to the following steps:
[0179] Step S510: According to the loss function value, the generator network and the discriminator network are alternately trained by back propagation to update and optimize the parameters of the generator network and the discriminator network.
[0180] Step S520: Use multiple groups of training fundus images to train the generator network and the discriminator network multiple times to iteratively optimize the parameters of the generator network and the discriminator network.
[0181] Step S530: Use the optimized generator network as a fundus image training model, wherein, as the generator network and the discriminator network are optimized, the loss function value decreases, the probability that the discriminator network identifies the input image as a real image tends to 0.5, and the quality of the predicted high-quality fundus image is close to the high-quality training fundus image.
[0182] It should be understood that in order to ensure the authenticity of the predicted high-quality fundus images, it is necessary to optimize the performance of the generator network and the discriminator network, that is, to improve the authenticity of the generator network to generate predicted high-quality fundus images and improve the discriminative ability of the discriminator network. Since D() represents the probability that the discriminator network discriminates the input image as a real image, that is, the probability that the discriminator outputs 1. In order to improve the authenticity of the generator network to generate predicted high-quality fundus images, it is necessary to improve the probability that the generator network discriminates the predicted high-quality fundus images as real images, that is, to find the minimum value of the adversarial loss of the generator network; in order to improve the discriminative ability of the discriminator network, it is necessary to improve the probability that the generator network discriminates the high-quality training fundus images as real images and reduce the probability that the generator network discriminates the predicted high-quality fundus images as real images, that is, to find the maximum value of the adversarial loss of the discriminator network. According to the adversarial loss of the generator network and the discriminator network, the probability that the discriminator network discriminates the input image as a real image is 0.5. Therefore, as the generator network and the discriminator network are optimized, the loss function value decreases, and the probability that the discriminator network identifies the input image as a real image tends to 0.5.
[0183] In addition, in one embodiment, referring to Figure 8 ,Testing the performance of the fundus image training model includes but is not limited to the following steps:
[0184] Step S710: Acquire a fundus image test set, wherein the fundus image test set includes a plurality of low-quality test fundus images and corresponding high-quality test fundus images.
[0185] Step S720: The low-quality test fundus image is passed through the fundus image training model to obtain a synthesized high-quality fundus image.
[0186] Step S730: Obtain the performance of the fundus image training model according to the similarity between the synthesized high-quality fundus image and the high-quality test fundus image.
[0187] It should be understood that in order to measure the performance of the fundus image training model, it is necessary to test the performance of the fundus image training model. The low-quality test fundus image is passed through the fundus image training model to obtain a high-quality test fundus image, and the performance of the fundus image training model is measured based on the similarity between the high-quality test fundus image and the high-quality test fundus image.
[0188] It should be understood that reference Figures 15 to 17 , Fig.15 The pictures in the middle and upper part are low-quality test fundus images, synthetic high-quality fundus images, and high-quality test fundus images. Fig.15 The pictures in the middle and lower parts are the enlarged details of the low-quality test fundus image, the synthesized high-quality fundus image, and the high-quality test fundus image; Fig.16The pictures in the middle and upper part are low-quality test fundus images, synthetic high-quality fundus images, and high-quality test fundus images. Fig.16 The pictures in the middle and lower parts are the retinal blood vessel segmentation maps corresponding to the low-quality test fundus image, the synthesized high-quality fundus image, and the high-quality test fundus image; Fig.17 The pictures in the middle and upper part are low-quality test fundus images, synthetic high-quality fundus images, and high-quality test fundus images. Fig.17 The images in the middle and lower parts are the optic cup and optic disc segmentation images corresponding to the low-quality test fundus image, the synthesized high-quality fundus image, and the high-quality test fundus image. Figures 15 to 17 , the performance of the fundus image training model generated by the generator network is excellent, in which the synthesized high-quality fundus image is almost identical to the high-quality test fundus image, and the quality of the synthesized high-quality fundus image is significantly improved compared with the low-quality test fundus image. The detailed magnified image, retinal vascular segmentation map, and optic cup and optic disc segmentation map are usually used as the basis for doctors to diagnose whether the patient has an ophthalmic disease. The detailed magnified image, retinal vascular segmentation map, and optic cup and optic disc segmentation map of the synthesized high-quality fundus image are highly similar to the high-quality test fundus image. Therefore, the synthesized high-quality fundus image can be used as the basis for doctors' diagnosis, which can reduce the number of fundus image acquisitions and avoid unnecessary damage to the patient's eyes.
[0189] It should be understood that the fundus image training set is passed through the generator network and the discriminator network to obtain the loss function value, and the generator network and the discriminator network are iteratively optimized according to the loss function value, so that the predicted high-quality fundus image generated by the generator network is as close as possible to the high-quality training fundus image, and the generator network is used as the fundus image training model, and the low-quality fundus image can directly generate a high-quality fundus image through the fundus image training model. The embodiment of the present application can significantly improve the quality of low-quality fundus images through the powerful ability of the local transformer layer to extract long-distance feature information and the powerful ability of the generator network and the adversary network to generate realistic images, so that the blurred fundus image is clear, and thanks to the ingenious design of the loss function in the embodiment of the present application, the spatial structure of the fundus image has not changed, the image details are richer, and the restored fundus image is consistent with the real clear fundus image color, without color distortion. This process reduces the number of fundus image acquisitions and effectively avoids unnecessary damage to the patient's eyes.
[0190] In addition, an embodiment of the present application also provides a fundus image enhancement method, referring to Fig. 9 The fundus image enhancement method includes but is not limited to the following steps:
[0191] Step S810: Acquire an original fundus image.
[0192] Step S820: The original fundus image is trained through the fundus image training model to obtain a target fundus image.
[0193] It should be understood that for the fundus image training model obtained by the fundus image training method, the original fundus image can be directly passed through the fundus image training model to obtain the target fundus image. The embodiment of the present application reduces the number of fundus image acquisitions and effectively avoids unnecessary damage to the patient's eyes.
[0194] In addition, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a fundus image training method such as any one of steps S100 to S500 and / or a fundus image enhancement method such as steps S810 and S820.
[0195] The processor and the memory may be connected via a bus or other means.
[0196] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0197] The non-transient software program and instructions required to implement the class activation map generation method and / or the neural network interpretable method of the above-mentioned embodiment are stored in the memory. When executed by the processor, the fundus image training method of the above-mentioned embodiment is executed, for example, the above-described Figure 2 Alternatively, the fundus image enhancement method in the above embodiment is performed, for example, the method described above is performed. Fig. 9 Method steps S810 to S820.
[0198] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0199] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are executed by a processor or a controller, which enables the processor to execute the fundus image training method in the above embodiment, for example, to execute the above described Figure 2 The method steps S100 to S500, Figure 3 In the method step S600, Figure 4 Steps S210 to S260 of the method, Figure 5 Steps S310 to S330 of the method, Figure 6 Steps S410 to S450 of the method, Figure 7 Steps S510 to S530 of the method, Figure 8 In another example, the method steps S710 to S730 in the embodiment are executed by a processor in the above embodiment, so that the processor can execute the fundus image enhancement method in the above embodiment, for example, executing the above described Fig. 9 Method steps S810 to S820 in.
[0200] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0201] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.
Claims
1. A fundus image training method, characterized in that: include: Acquire a fundus image training set, wherein the fundus image training set includes multiple groups of training fundus images, and each group of training fundus images includes a low-quality training fundus image and a corresponding high-quality training fundus image; Passing the low-quality training fundus image through a generator network to obtain a predicted high-quality fundus image; Passing the high-quality training fundus image and the predicted high-quality fundus image through a discriminator network to obtain a discriminant matrix; Calculating a loss function value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix; Iteratively optimizing the generator network and the discriminator network according to the loss function value, and using the optimized generator network as a fundus image training model; The step of passing the low-quality training fundus image through a generator network to obtain a predicted high-quality fundus image includes: Passing the low-quality training fundus image through an input projection layer to obtain a first feature map; Passing the first feature map through a plurality of first sampling modules to obtain a second feature map, wherein the first sampling module includes a plurality of local transformer layers and a downsampling layer; Passing the second feature map through multiple local transformer layers to obtain a third feature map; Passing the third feature map through a plurality of second sampling modules to obtain a fourth feature map, wherein the second sampling module includes a plurality of local transformer layers and an upsampling layer, and the first sampling module and the second sampling module are channel-cascaded; Passing the fourth feature map through an output projection layer to obtain a residual fundus image; Adding the residual fundus image to the low-quality training fundus image to obtain a predicted high-quality fundus image; The step of passing the high-quality training fundus image and the predicted high-quality fundus image through a discriminator network to obtain a discriminant matrix includes: Passing the high-quality training fundus image and the predicted high-quality fundus image through an input mapping layer to obtain a fifth feature map; Passing the fifth feature map through a plurality of third sampling modules to obtain a sixth feature map, wherein the third sampling module includes a plurality of local transformer layers and a downsampling layer; The sixth feature map is passed through the output mapping layer to obtain a discriminant matrix.
2. The fundus image training method according to claim 1, characterized in that: Also includes: Before the low-quality training fundus image is passed through a generator network to obtain a predicted high-quality fundus image, the method includes: The low-quality training fundus image is preprocessed, and the preprocessing result is updated into the low-quality training fundus image.
3. The fundus image training method according to claim 1, characterized in that: The loss function value is calculated based on the Charbonnier Loss value, the fundus quality perception loss value, the confrontation loss value and the edge loss value; Correspondingly, calculating the loss function value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix includes: Calculating the CharbonnierLoss value according to the predicted high-quality fundus image and the high-quality training fundus image; Calculating the fundus quality perception loss value according to the predicted high-quality fundus image and the high-quality training fundus image; Calculating the adversarial loss value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix, wherein the adversarial loss value includes the adversarial loss value of the generator network and the adversarial loss value of the discriminator network; Calculating the edge loss value according to the predicted high-quality fundus image and the high-quality training fundus image; The Charbonnier Loss value, the fundus quality perception loss value, the adversarial loss value and the edge loss value are summed according to the weights to obtain the loss function value.
4. The fundus image training method according to claim 3, characterized in that: The calculating the CharbonnierLoss value according to the predicted high-quality fundus image and the high-quality training fundus image comprises: The Charbonnier Loss value is calculated based on the predicted high-quality fundus image, the high-quality training fundus image and the Charbonnier Loss function. L 1, wherein the Charbonnier Loss function is: in, For predicting a high-quality fundus image, For the high-quality training fundus image, is a fixed constant, take 10 -3 ; The calculating the fundus quality perception loss value according to the predicted high-quality fundus image and the high-quality training fundus image comprises: Calculating the fundus quality perception loss value according to the predicted high-quality fundus image, the high-quality training fundus image and the fundus quality perception function L p , wherein the fundus quality perception function is: in, , is the dimension of the feature map, It is a feature map extracted by the VGG-19 network pre-trained on the fundus image quality assessment dataset Eyes-Q; The calculating the adversarial loss value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix includes: The adversarial loss value is calculated according to the predicted high-quality fundus image, the high-quality training fundus image, the discriminant matrix and the adversarial loss function. L adv , where the adversarial loss function is: in, is the adversarial loss of the discriminator network, is the adversarial loss of the generator network, N is the order of the discriminant matrix output by the discriminator network, D represents the discriminator network, G represents the generator network, D () represents the probability that the discriminator network identifies the input image as a real image, i is a point in the discriminant matrix; The calculating the edge loss value according to the predicted high-quality fundus image and the high-quality training fundus image comprises: Calculate the edge loss value according to the predicted high-quality fundus image, the high-quality training fundus image and the edge loss function L edge , where the edge loss function is: in, is the Laplace operator of Gaussian; The step of summing the Charbonnier Loss value, the fundus quality perception loss value, the adversarial loss value, and the edge loss value according to the weights to obtain the loss function value includes: in, L is the loss function value, , , is a hyperparameter used to control the weights of different losses.
5. The fundus image training method according to claim 1, characterized in that: The iterative optimization of the generator network and the discriminator network according to the loss function value, and using the optimized generator network as a fundus image training model, comprises: According to the loss function value, the generator network and the discriminator network are alternately trained by back propagation to update and optimize the parameters of the generator network and the discriminator network; Using a plurality of sets of the training fundus images, the generator network and the discriminator network are trained multiple times to iteratively optimize the parameters of the generator network and the discriminator network; The optimized generator network is used as a fundus image training model, wherein, as the generator network and the discriminator network are optimized, the loss function value decreases, the probability that the discriminator network identifies the input image as a real image tends to 0.5, and the quality of the predicted high-quality fundus image is close to that of the high-quality training fundus image.
6. The fundus image training method according to claim 1, characterized in that: Also includes: Acquire a fundus image test set, wherein the fundus image test set includes a plurality of low-quality test fundus images and corresponding high-quality test fundus images; Passing the low-quality test fundus image through the fundus image training model to obtain a synthetic high-quality fundus image; The performance of the fundus image training model is obtained according to the similarity between the synthesized high-quality fundus image and the high-quality test fundus image.
7. A fundus image enhancement method, characterized in that: include: Acquire original fundus images; The original fundus image is passed through the fundus image training model as described in any one of claims 1 to 6 to obtain a target fundus image.
8. A fundus image training device, characterized in that: include: An image acquisition module, the image acquisition module is used to acquire a fundus image training set, wherein the fundus image training set includes multiple groups of training fundus images, each group of training fundus images includes a low-quality training fundus image and a corresponding high-quality training fundus image; A generator network module, wherein the generator network module is used to pass the low-quality training fundus image through a generator network to obtain a predicted high-quality fundus image; A discriminator network module, wherein the discriminator network module is used to obtain a discriminant matrix by passing the high-quality training fundus image and the predicted high-quality fundus image through a discriminator network; A loss function value calculation module, the loss function value calculation module is used to calculate the loss function value according to the predicted high-quality fundus image, the high-quality training fundus image and the discriminant matrix; A network optimization module, the network optimization module is used to iteratively optimize the generator network and the discriminator network according to the loss function value, and use the optimized generator network as a fundus image training model; The generator network module can be used to: Passing the low-quality training fundus image through an input projection layer to obtain a first feature map; Passing the first feature map through a plurality of first sampling modules to obtain a second feature map, wherein the first sampling module includes a plurality of local transformer layers and a downsampling layer; Passing the second feature map through multiple local transformer layers to obtain a third feature map; Passing the third feature map through a plurality of second sampling modules to obtain a fourth feature map, wherein the second sampling module includes a plurality of local transformer layers and an upsampling layer, and the first sampling module and the second sampling module are channel-cascaded; Passing the fourth feature map through an output projection layer to obtain a residual fundus image; Adding the residual fundus image to the low-quality training fundus image to obtain a predicted high-quality fundus image; The discriminator network module can be specifically used for: Passing the high-quality training fundus image and the predicted high-quality fundus image through an input mapping layer to obtain a fifth feature map; Passing the fifth feature map through a plurality of third sampling modules to obtain a sixth feature map, wherein the third sampling module includes a plurality of local transformer layers and a downsampling layer; The sixth feature map is passed through the output mapping layer to obtain a discriminant matrix.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it is the fundus image training method according to any one of claims 1 to 6 and / or the fundus image enhancement method according to claim 7.
10. A computer storage medium, characterized in that: The method comprises storing computer executable instructions, wherein the computer executable instructions are used to execute the fundus image training method according to any one of claims 1 to 6 and / or the fundus image enhancement method according to claim 7.