Ultra-wide-field fundus image (UFI) conversion system and UFI conversion method

Abstract

According to an embodiment of the present invention, a UFI conversion system comprises: a collection unit for collecting a plurality of fundus images; a memory for storing a program for generating, from UFI, a CFI generation image corresponding to a CFI; and a processor for generating the CFI generation image on the basis of the plurality of fundus images by executing the program, wherein the processor comprises: a scale adjustment unit that identifies the positions of an optic disk and a macula from a first UFI from among the plurality of fundus images, and cutting the first UFI and adjusting the size thereof on the basis of the position of the optic disk; a matching unit for matching the first UFI with a first CFI matched to the first UFI from among the plurality of fundus images; and an image conversion unit for training a deep learning model for generating the CFI generation image from the UFI on the basis of the first UFI and the first CFI.

Description

UFI (ULTRA-WIDE-FIELD FUNDUS IMAGE) conversion system and UFI conversion method

[0001] The present invention relates to a UFI (Ultra-wide-field Fundus Image) conversion system and a UFI conversion method, and more particularly, to a system and method for converting a UFI image to generate an image that looks like a CFI image.

[0002] Most ophthalmic screening systems utilize images captured using conventional fundus imaging (CFI) techniques. However, obtaining a CFI requires dilation of the patient's pupils, requiring several minutes of waiting after drug administration. This dilation can blur vision and increase sensitivity to light, and prolonged dilation can be physically burdensome for the patient.

[0003] Ultra-wide field fundus image (UFI) is one of the types of examinations that does not require pupil dilation, and the imaging area is very wide, including the area captured by CFI, but the quality may be lower than CFI in terms of resolution or image artifacts.

[0004] The purpose of the present invention is to solve the above problems, and to provide a UFI (Ultra-wide-field Fundus Image) conversion system and UFI conversion method that can convert a UFI image to generate an image that looks like a CFI image.

[0005] In order to achieve the above object, according to an embodiment of the present invention, a UFI conversion system includes a collection unit that collects a plurality of fundus images, a memory that stores a program that generates a CFI generation image corresponding to a CFI from the UFI, and a processor that executes the program to generate the CFI generation image based on the plurality of fundus images, wherein the processor includes a scale adjustment unit that identifies the positions of the optic nerve head and the macula from a first UFI among the plurality of fundus images, crops the first UFI based on the position of the optic nerve head, and adjusts the size of the first UFI, an alignment unit that aligns the first UFI with a first CFI that matches the first UFI among the plurality of fundus images, and an image conversion unit that trains a deep learning model that generates the CFI generation image from the UFI based on the first UFI and the first CFI.

[0006] The above first UFI and the above first CFI may represent the same fundus of the same subject.

[0007] The above scale adjustment unit may include an optic nerve head detection unit that detects an optic nerve head from the first UFI using deep learning, and an image cropping unit that adjusts and crops the first UFI to a size corresponding to the first CFI based on the size of the detected optic nerve head.

[0008] The above matching unit can position common features between the first intermediate UFI image, which is resized and cropped from the first UFI, and the first CFI on the same coordinate line, extract features from the first intermediate UFI image and the first CFI using deep learning, and match the extracted features to generate a UFI matching image from the first intermediate UFI image.

[0009] The image conversion unit may include a gradient estimation branch that enhances detail information in the UFI matching image to improve image quality, a generator that generates a fake CFI based on the UFI matching image and an image output from the gradient estimation branch, and a discriminator that sets the first CFI as a real CFI and determines a real from a fake CFI and the real CFI.

[0010] The above fake CFI may be an image similar in color and size to the above real CFI.

[0011] The above gradient estimation branch includes a plurality of convolution blocks and can estimate an edge map that has passed through a Laplacian filter of the real CFI from the UFI matching image.

[0012] According to another embodiment of the present invention, a UFI conversion method is provided, wherein a processor executes a program to generate a CFI generated image corresponding to a CFI from a UFI stored in a memory, and the method generates the CFI generated image based on a plurality of fundus images, the method comprising the steps of collecting a plurality of fundus images, identifying the positions of an optic disc and a macula from a first UFI among the plurality of fundus images, cropping and resizing the first UFI based on the position of the optic disc, matching the first UFI with a first CFI matched to the first UFI among the plurality of fundus images, and training a deep learning model that generates the CFI generated image from the UFI based on the first UFI and the first CFI, wherein the first UFI and the first CFI represent the same fundus of the same subject.

[0013] The method may further include a step of detecting an optic nerve head from the first UFI using deep learning, and a step of adjusting and cropping the first UFI to correspond to the first CFI based on the size of the detected optic nerve head.

[0014] The method may further include a step of positioning common features between the first intermediate UFI image, which is resized and cropped from the first UFI, and the first CFI on the same coordinate line, a step of extracting features from the first intermediate UFI image and the first CFI using deep learning, and a step of generating a UFI matching image from the first intermediate UFI image by matching the extracted features.

[0015] The method further includes a step of generating a fake CFI based on the UFI matching image and an image output from a gradient estimation branch that enhances detail information in the UFI matching image to improve image quality, and a step of setting the first CFI as a real CFI and determining the real from the fake CFI and the real CFI, wherein the fake CFI may be an image having a similar color and size to the real CFI.

[0016] The step of estimating an edge map that has passed through a Laplacian filter of the real CFI from the UFI alignment image may be further included.

[0017] According to the present invention, a CFI-generated image corresponding to a CFI can be obtained from a UFI that is relatively easy to obtain, thereby providing a high-quality CFI-generated image generated for an ophthalmic examination system.

[0018] According to the present invention, the negative impacts such as time and cost of CFI examination of a patient can be reduced.

[0019] According to the present invention, not only can the diagnostic accuracy be improved through simple CFI generation image acquisition, but also the cost and time required for retinal disease management can be reduced, effective treatment and intervention can be achieved through early diagnosis of the disease, and ultimately, the prognosis and quality of life of the patient can be improved.

[0020] FIG. 1 is a block diagram schematically illustrating the configuration of a UFI conversion system according to one embodiment of the present invention.

[0021] Figure 2 is a block diagram schematically showing the detailed configuration of the processor illustrated in Figure 1.

[0022] Figure 3 is an example diagram showing UFI and CFI for the same fundus of the same subject.

[0023] FIG. 4 is a framework for generating a CFI generation image corresponding to CFI from UFI according to one embodiment.

[0024] Figure 5 is a Faster RCNN detection framework according to one embodiment.

[0025] FIG. 6 is a block diagram schematically illustrating two steps of a UFI and CFI transformation learning method according to one embodiment.

[0026] Figure 7 is a Gradient-guided GAN framework according to one embodiment.

[0027] Figure 8 is a flowchart of a UFI conversion method according to one embodiment.

[0028] The present invention can be implemented with various modifications without departing from the spirit or scope of the invention, and may have one or more embodiments. Furthermore, the embodiments described in the "Specific Description for Carrying Out the Invention" and "Drawings" sections of the present invention are examples intended to specifically illustrate the invention and do not limit or restrict the scope of the invention.

[0029] Accordingly, anything that a person with ordinary knowledge in the technical field to which the present invention belongs can easily infer from the “specific contents for carrying out the invention” and “drawings” of the present invention can be interpreted as falling within the scope of the present invention.

[0030] In addition, the size and shape of each component shown in the drawing may be exaggerated for the purpose of explaining the embodiment, and do not limit the size and shape of the invention actually implemented.

[0031] Unless specifically defined, terms used in the specification of the present invention may have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention pertains.

[0032] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0033] FIG. 1 is a block diagram schematically illustrating the configuration of a UFI conversion system according to one embodiment of the present invention.

[0034] Referring to FIG. 1, a UFI conversion system (1) may include a UFI conversion server (10) and a user terminal (20). The UFI conversion server (10) may include a collection unit (100), a memory (200), and a processor (300).

[0035] The collection unit (100) can collect multiple fundus images (hereinafter, "multiple fundus images") taken of the fundus of multiple subjects. The collection unit (100) can receive multiple fundus images from the user terminal (20) by communicating with the user terminal (20) via a wired or wireless network.

[0036] The multiple fundus images may include ultra-wide-field fundus images (UFIs) and conventional fundus images (CFIs). Each of the multiple fundus images may be an image of the fundus of a patient (subject) diagnosed with multiple ophthalmic diseases. The multiple fundus images may be images labeled with at least one of the multiple ophthalmic diseases. The multiple ophthalmic diseases may include diabetic retinopathy (DR), epiretinal membrane (ERM), glaucoma suspect (GS), macular degeneration (MD), retinal detachment (RB), retinal vein occlusion (RVO), etc.

[0037] The UFI and CFI included in the multiple fundus images may be images representing the same fundus of the same subject. For example, the multiple fundus images may include a first UFI and a first CFI, and each of the first UFI and the first CFI may represent the first fundus of the first subject.

[0038] The memory (200) may include volatile memory and / or non-volatile memory. The memory (200) may store, for example, commands or data related to the collection unit (100) and the processor (300), one or more programs and / or software, an operating system, etc., to implement and / or provide operations, functions, etc. provided by the UFI conversion system (1).

[0039] The program stored in the memory (200) may include a UFI conversion program (hereinafter, "UFI conversion program") that generates a CFI-generated image corresponding to a CFI from a UFI. The UFI conversion program may provide a service for converting a CFI-generated image that looks like a CFI from a UFI. Hereinafter, the CFI-generated image may be generated from a UFI and may represent an image that looks like a CFI that was taken of the same fundus of the same subject as the subject of the UFI.

[0040] Figure 2 is a block diagram schematically showing the detailed configuration of the processor illustrated in Figure 1.

[0041] Referring to FIG. 2, the processor (300) may include a scale adjustment unit (310), a matching unit (320), and an image conversion unit (330).

[0042] The scale adjustment unit (310) can identify the positions of the optic nerve head and macula from the first UFI among the plurality of fundus images, and crop and adjust the size of the first UFI based on the position of the optic nerve head.

[0043] The alignment unit (320) can align the first UFI, which has been cropped and resized by the scale adjustment unit (310), with the first CFI that matches the first UFI among the plurality of fundus images. Here, the first UFI and the first CFI can represent the same fundus of the same subject.

[0044] The image conversion unit (330) can train a deep learning model that generates a CFI generation image from the UFI based on the first UFI and the first CFI. The CFI generation image may be an image converted from the UFI to a CFI format. Hereinafter, the deep learning model may be a model that generates a CFI generation image from the UFI.

[0045] Referring to FIG. 2, the scale adjustment unit (310) may include an optic nerve head detection unit (311) and an image cropping unit (312).

[0046] The optic nerve head detection unit (311) can detect the optic nerve head from the first UFI using deep learning.

[0047] The image cropping unit (312) can adjust the size and crop the first UFI to correspond to the first CFI based on the size of the detected optic nerve head.

[0048] Figure 3 is an example diagram showing UFI and CFI for the same fundus of the same subject.

[0049] Referring to FIG. 3, the CFI ((b) of FIG. 2) may include the optic disc, the macula, and the periphery. The UFI ((a) of FIG. 2) may include a wider area in addition to the area included in the CFI.

[0050] Since it is difficult to obtain CFI from ophthalmic examination, one embodiment of the present invention proposes a method of generating CFI from UFI.

[0051] FIG. 4 is a framework for generating a CFI generation image corresponding to CFI from UFI according to one embodiment.

[0052] Since the CFI includes the optic disc, macula, and periphery, the optic disc detection unit (311) can detect the optic disc region in the UFI using a Faster RCNN detector that is robust to image size and deformation. Thereafter, based on the detected optic disc, the image cropping unit (312) can crop a portion of the UFI region including the actual CFI region to a similar scale. The actual CFI region may be an area included in a CFI that matches the UFI among multiple fundus images.

[0053] The alignment unit (320) can utilize the common features of the cropped UFI and the actual CFI to align the common features of the two images on a pixel-by-pixel basis. The image conversion unit (330) uses the CFI and the aligned UFI to learn the transformation relationship between the two modalities by a gradient-guided GAN model. The deep learning model can be implemented using a gradient-guided GAN method. While both the actual UFI and CFI are required for training, only the UFI is used as input during the testing phase. After completing training for the deep learning model, the image conversion unit (330) can generate a CFI-generated image that directly converts the CFI from the UFI using the trained deep learning model.

[0054] Referring to FIG. 4, the processor (300) can train a deep learning model to generate a CFI from a UFI using a first CFI matched with a first UFI. Multiple pairs of UFIs and CFIs representing the same fundus of the same subject among multiple fundus images can be utilized as a training dataset.

[0055] In the training phase, the processor (300) can generate a registered UFI by performing scale-aware image registration using the first UFI included in the training dataset and the first CFI matched to the first UFI. The processor (300) can perform OD scale adjustment based on the position of the optic nerve head (OD) and generate the registered UFI through image registration. The deep learning model can learn a method of generating a CFI generated image corresponding to the first UFI through gradient-guided translation using the registered UFI and the first CFI. The deep learning model can learn weights for generating the first CFI generated image corresponding to the first UFI using the registered UFI and the first CFI.

[0056] Once learning is complete, in the test phase, the processor (300) can generate a scale-matched UFI by performing scale-aware image registration using the second UFI included in the test dataset. The deep learning model can generate a second CFI generated image corresponding to the second UFI by performing gradient-guided translation on the scale-matched UFI using the weights learned in the learning phase.

[0057] Hereinafter, a first step of detecting and cropping the optic nerve head from each image by a processor (300) according to one embodiment, a second step of matching the cropped image with a CFI, and a third step of training a deep learning model using the two matched images are sequentially described.

[0058] The first step is described below with reference to Fig. 5.

[0059] Figure 5 is a Faster RCNN detection framework according to one embodiment.

[0060] Referring to FIG. 5, the scale adjustment unit (310) can utilize a Fast RCNN detector, which is reported to be effective in detecting small objects for optic nerve head detection. Referring to FIG. 5, the scale adjustment unit (310) can pass the input UFI through a Convolutional Neural Network (CNN) for feature extraction. The feature map extracted from the CNN can be input to a Region Proposal Network (RPN). The RPN can output 256 regions of interest (ROIs), which can include the foreground or background, from the feature map. Since the RPN proposes a region of interest as a raw prediction, the proposed ROIs can be input to a classifier for accurate classification. The classifier classifies each ROI into one of two classes, optic nerve head or background, and can also provide the coordinates of a bounding box. Since each UFI and each CFI includes exactly one optic nerve head, the scale adjustment unit (310) can select a bounding box with the highest probability. The scale adjustment unit (310) can crop and enlarge the UFI to include the CFI area by calculating the ratio of the size of the bounding box selected in each image to the size of the entire image. The scale adjustment unit (310) can crop the first UFI based on the size of the optic nerve head detected from each of the first UFI and the first CFI.

[0061] The second step is described below.

[0062] The alignment unit (320) can position common features of a UFI-CFI pair on the same coordinate line to apply a pixel-based image conversion technology. For example, the alignment unit (320) can position common features between an image resized and cropped from a first UFI (hereinafter, "UFI first intermediate image") and a first CFI matched to the first UFI on the same coordinate line.

[0063] To this end, the matching unit (320) can convert both images to grayscale, extract features from each image using a feature extraction network, and match the extracted features so that the same features of the two images can be on the same position line. Here, the feature extractor and feature matcher used by the matching unit (320) utilize a deep learning method. For example, the matching unit (320) can extract features from the UFI first intermediate image and the first CFI using deep learning, and match the extracted features to generate a matched UFI (hereinafter, “UFI matched image”) from the UFI first intermediate image. In this process, an image perspective transformation technique is used, and the method for calculating the coordinate transformation relationship is as shown in [Mathematical Formula 1] below.

[0064] [Mathematical Formula 1]

[0065] or

[0066] Here, H is a 3x3 matrix representing the coordinate transformation, and (u, v) and (u', v') are the keypoint coordinates of UFI and CFI, respectively. These can be calculated based on the transformation relationship between the two images.

[0067] The minimum number of matching points required for the matching unit (320) to calculate the homography matrix is ​​4, and the more matching points there are, the lower the error rate and the higher the registration performance. Afterwards, the cropped UFI is converted into the CFI coordinate system based on the coordinate transformation relationship between the matched coordinates. This process is called "image registration" and is performed through image perspective transformation. The result of the performance can be expressed as a registered UFI, and the registered UFI can indicate the coordinate matching state of the common feature points with the CFI.

[0068] FIG. 6 is a block diagram schematically illustrating two steps of a UFI and CFI transformation learning method according to one embodiment.

[0069] Referring to FIG. 6, in the first step, the scale adjustment unit (310) can detect the optic nerve head (OD detection) from the first UFI to generate a UFI in which the optic nerve head is detected, calculate the scale of the optic nerve head (OD scale calculation), and generate a first intermediate image of the UFI (scaled UFI) whose size is adjusted and cropped through cropping and resizing.

[0070] Also, in the second step, the alignment unit (320) converts each of the scaled and cropped first intermediate UFI image (Scaled UFI) and the CFI matched to the first UFI into grayscale (Grayscale Conversion), extracts features therefrom (Feature extraction), and matches each feature to generate a matrix H using multiple matching points. The alignment unit (320) can generate a registered UFI from the matrix H through perspective transformation and masking.

[0071] The third step is described below with reference to Fig. 7.

[0072] Figure 7 is a Gradient-guided GAN framework according to one embodiment.

[0073] Referring to FIG. 7, the image conversion unit (330) can generate a CFI generation image by a Gradient-guided GAN model framework that improves the performance of a pixel-based image conversion technique using a Gradient-guided Branch module. The image conversion unit (330) can make the features of the first CFI generation image generated from the first UFI and the actual first CFI similar by forcibly aligning the colors and sizes of the first UFI and the first CFI, which are two different image modalities, to be similar. Referring to FIG. 7, the deep learning model trained by the image conversion unit (330) can include a gradient estimation branch (331), a generator (332), and a discriminator (333).

[0074] The slope estimation branch (331) can enhance the image quality by reinforcing the detail information in the UFI alignment image generated by the alignment unit (320).

[0075] The generator (332) and the discriminator (333) may be used to translate images between the UFI and CFI domains. The generator (332) may generate a fake CFI based on the UFI matching image and the image output from the gradient estimation branch (331). The discriminator (333) may use the first CFI as the real CFI and discriminate between the real and fake CFIs based on the fake CFI and the real CFI. The fake CFI may be an image similar in color and size to the real CFI.

[0076] One embodiment of the present invention can learn the transformation relationship between two images most accurately by minimizing errors when matching two images when the actual UFI and the actual CFI are images taken from the same patient.

[0077] The generator (332) generates a CFI, and the discriminator (333) can classify the real CFI from the fake CFI generated by the generator (332). The main idea of ​​GAN is to convert the real UFI into a CFI, and then separately train a discriminator to discriminate it, thereby learning a relationship that allows the trained generator (332) to create an image identical to the real CFI. The real UFI and the real CFI illustrated in FIG. 7 can represent the UFI and CFI matched in FIG. 6, respectively.

[0078] The gradient estimation branch (331) primarily focuses on improving the conversion capability for small detail areas in low-quality UFI. The gradient estimation branch (331) includes multiple convolution blocks and can estimate an edge map that passes through a Laplacian filter of a true CFI from a UFI-matched image. For example, the gradient estimation branch (331) can be implemented as a shallow network module consisting of four convolution blocks.

[0079] The gradient estimation branch (331) can receive the registered UFI as input and estimate the edge map that has passed through the Laplacian filter of the real CFI. Since the UFI has relatively low resolution compared to the real CFI, it may lack gradient information. Therefore, in one embodiment, the image conversion unit (330) can use a gradient-guided UFI-to-CFI conversion model. The gradient-guided UFI-to-CFI conversion model can enhance detail information using a gradient loss function and provide the detail information to the generator (332) as a guideline for the final output by learning the gradient of the CFI. Through this, the gradient-guided UFI-to-CFI conversion model can improve the clarity and texture of the optic nerve, blood vessel boundaries, etc. of the generated fake CFI, and correct distorted areas such as the macular region or blood vessel structures.

[0080] Each of the generator (332) and the discriminator (333) may include a plurality of convolution blocks.

[0081] The generator (332) can learn the weights of the convolution block to minimize the loss function value according to the cGAN loss value (cGAN loss) according to the discrimination result of the discriminator (333) and the loss function value according to the L1 loss value between the fake CFI and the real CFI.

[0082] The discriminator (333) can learn the weights of the convolution block to minimize the loss function value according to the cGAN loss value (cGAN loss) according to the discrimination result of the discriminator (333).

[0083] The gradient estimation branch (331) can learn the weights of the convolution blocks to minimize the loss function value according to the gradient loss between the edge map (x') extracted from the UFI matching image (x) and the result (y') passed through the Laplacian filter from the real CFI (y). Through this, the gradient estimation branch (331) can estimate the edge map passed through the Laplacian filter of the real CFI (y) from the UFI matching image (x).

[0084] A typical conversion model reflects the pixel differences between the output image and the ground truth in the loss function during the data learning process, and can partially correct small distortions that occur in various areas during the image conversion process. However, since the characteristic that has the greatest influence on model learning is the color spectrum, an objective function that simply reduces pixel value differences tends to ignore small details in the image. In contrast, in one embodiment, the gradient-guided UFI-to-CFI conversion model used by the image conversion unit (330) can solve the problem of ignoring details that typical conversion models suffer from through the gradient loss function, and provide better image conversion performance.

[0085] Figure 8 is a flowchart of a UFI conversion method according to one embodiment.

[0086] Below, in the description of each component of the UFI conversion system (1), any description that overlaps with the description described above may be omitted.

[0087] Referring to FIG. 8, the collection unit (100) can collect multiple fundus images (S100).

[0088] The processor (300) can detect the position of the optic nerve head and macula from the first UFI among multiple fundus images (S200).

[0089] The processor (300) can crop and resize the first UFI based on the location of the optic nerve head (S300). The processor (300) can determine one ROI with the highest probability among multiple ROIs, crop the image through calculations based on the optic nerve head scale, and resize the image to generate a first intermediate UFI image (scaled UFI).

[0090] The processor (300) can align the first UFI with the first CFI that matches the first UFI among the plurality of fundus images (S400). The processor (300) can convert the first intermediate UFI (scaled UFI) and the first CFI into gray scale to extract features, match the features, calculate an H matrix based on the matching points, and generate a registered UFI through perspective transformation and masking.

[0091] The processor (300) can train a deep learning model that generates a CFI generation image from the UFI based on the first UFI and the first CFI (S500). The processor (300) can train a gradient estimation branch (331), a generator (332), and a discriminator (333) through GAN learning.

[0092] When learning is completed, the processor (300) can input a second UFI other than the first UFI among a plurality of fundus images into the learned deep learning model and output a second CFI generation image having the form of a CFI from the second UFI.

[0093] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0094] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0095] While the embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications may be made within the scope of the detailed description and the attached drawings, as long as the spirit of the present invention is not deviated from or the effects are not impaired. Furthermore, it is obvious that such embodiments fall within the scope of the present invention.

Claims

1. A collection unit that collects multiple fundus images; Memory storing a program that generates a CFI generation image corresponding to CFI from UFI; and A processor that executes the above program to generate the CFI generation image based on the plurality of fundus images, The above processor, A scale adjustment unit that determines the positions of the optic nerve head and macula from the first UFI among the plurality of fundus images, and crops and adjusts the size of the first UFI based on the position of the optic nerve head; A matching unit that aligns the first UFI with a first CFI that matches the first UFI among the plurality of fundus images; and An image conversion unit that trains a deep learning model to generate the CFI generation image from the UFI based on the first UFI and the first CFI, UFI conversion system.

2. In paragraph 1, The above first UFI and the above first CFI, Representing the same fundus of the same subject, UFI conversion system.

3. In paragraph 1, The above scale adjustment unit, An optic disc detection unit that detects the optic disc from the first UFI using deep learning; and An image cropping unit that adjusts and crops the first UFI to correspond to the first CFI based on the size of the detected optic nerve head. UFI conversion system.

4. In paragraph 3, The above matching part is, Common features between the first intermediate image of the UFI resized and cropped from the first UFI and the first CFI are positioned on the same coordinate line, features are extracted from the first intermediate image of the UFI and the first CFI using deep learning, and the extracted features are matched to generate a UFI matching image from the first intermediate image of the UFI. UFI conversion system.

5. In paragraph 4, The above image conversion unit, A gradient estimation branch that enhances detail information in the above UFI alignment image to improve image quality; A generator for generating a fake CFI based on the UFI matching image and the image output from the gradient estimation branch; and The first CFI is a real CFI, and includes a discriminator that determines the real from the fake CFI and the real CFI. UFI conversion system.

6. In paragraph 5, The above fake CFI is, An image similar in color and size to the above real CFI, UFI conversion system.

7. In paragraph 5, The above slope estimation branch is, A method comprising: including a plurality of convolution blocks, and estimating an edge map that passes through a Laplacian filter of the real CFI from the UFI registration image; UFI conversion system.

8. A method for generating a CFI generation image based on a plurality of fundus images by executing a program that generates a CFI generation image corresponding to a CFI from a UFI stored in a memory by a processor, A step of collecting multiple fundus images; A step of identifying the position of the optic nerve head and macula from the first UFI among the plurality of fundus images; A step of cutting and resizing the first UFI based on the position of the optic nerve head; A step of aligning the first UFI with a first CFI matched to the first UFI among the plurality of fundus images; and A step of training a deep learning model to generate the CFI generation image from the UFI based on the first UFI and the first CFI, The above first UFI and the above first CFI, Representing the same fundus of the same subject, How to convert UFI.

9. In paragraph 8, A step of detecting the optic nerve head from the first UFI using deep learning; and Further comprising a step of adjusting and cropping the first UFI to correspond to the first CFI based on the size of the detected optic nerve head. How to convert UFI.

10. In paragraph 9, A step of positioning common features between the first intermediate UFI image, which is resized and cropped from the first UFI, and the first CFI on the same coordinate line; A step of extracting features from the first intermediate image of the UFI and the first CFI using deep learning; and Further comprising a step of generating a UFI matching image from the UFI first intermediate image by matching the extracted features. How to convert UFI.

11. In paragraph 10, A step of generating a fake CFI based on the UFI matching image and an image output from a gradient estimation branch that enhances detail information in the UFI matching image to improve image quality; and The above first CFI is a real CFI, and further includes a step of determining the real and the fake from the fake CFI and the real CFI, The above fake CFI is, An image similar in color and size to the above real CFI, How to convert UFI.

12. In paragraph 11, Further comprising a step of estimating an edge map passed through a Laplacian filter of the real CFI from the UFI matching image. How to convert UFI.

Images