Ophthalmological image processing program

Abstract

To generate a color fundus image that gives less discomfort to an examiner.SOLUTION: Provided is an ophthalmologic image processing program that causes a computer to execute: a first image acquisition step of acquiring a first image as a color fundus image by being executed by a processor of the computer; and a color gamut corrected image generation step of correcting pixel values of at least one of color components in the first image to generate a color gamut corrected image, on the basis of color gamut information for specifying a predetermined color gamut to be applied to the color fundus image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an ophthalmologic image processing program for processing a fundus image of a subject's eye. [Background technology]

[0002] In the field of ophthalmology, color fundus images (hereinafter referred to as fundus photographs) taken with a fundus camera have been used in fundus examinations for a long time.

[0003] In recent years, devices with a higher confocality than conventional fundus cameras have become popular in ophthalmology facilities. Color fundus images taken with these devices are also used in fundus examinations. Color fundus images taken with devices with a higher confocality depict each part with higher contrast, but have different colors from fundus photographs.

[0004] As a technique for making the color tone of a color fundus image captured by a highly confocal device closer to that of a fundus photograph, Patent Document 1 discloses a technique for correcting the color tone of a color fundus image captured by a highly confocal device based on a predetermined target pattern of a histogram for each color. According to this technique, a highly confocal device generates a color fundus image with a constant color tone regardless of the wavelength of the illumination light, individual differences in fundus shape, and alignment state. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2020-054479 Summary of the Invention [Problem to be solved by the invention]

[0006] However, due to the difference between the photographing method of a fundus camera and another photographing method, fundus images obtained by a device employing the latter method may have unnatural coloring in parts of the image when compared with a fundus photograph, even if color correction is performed using the method described in Patent Document 1. While this coloring can be a useful indicator for understanding the state of the fundus, it can sometimes cause discomfort to examiners who are accustomed to fundus photographs.

[0007] The present disclosure has been made in consideration of at least one of the problems of the conventional techniques, and has as its technical object to generate a color fundus image that is less uncomfortable for the examiner. [Means for solving the problem]

[0008] Book Disclosure No. 1 The ophthalmologic image processing program according to this embodiment is executed by a computer processor to cause the computer to perform a first image acquisition step of acquiring a first image as a color fundus image using a first imaging method, and a color gamut correction image generation step of correcting pixel values of at least one color component in the first image based on color gamut information that specifies a predetermined color gamut to be applied to the color fundus image, the color gamut indicating a color gamut corresponding to a color fundus image obtained using a second imaging method different from the first imaging method, to generate a color gamut correction image. [Effects of the Invention]

[0009] According to the present disclosure, a color fundus image that is less uncomfortable for the examiner can be generated. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing a schematic configuration of a fundus imaging apparatus according to an embodiment. [Figure 2] 1 is a flowchart showing an ophthalmologic image processing method according to the present embodiment. [Figure 3]FIG. 10 is a diagram showing a GUI for color correction and a display layout of a fundus image. [Figure 4] 10A and 10B are diagrams for explaining a color tone correction method, showing fundus images and histograms of each channel before and after correction when a target pattern is used. [Figure 5] 10A and 10B are diagrams for explaining clipping as a method of correcting a color gamut in an embodiment. [Figure 6] FIG. 10 is a diagram showing a GUI for manually adjusting a target color gamut. [Figure 7] FIG. 10 is a diagram showing an example of a target color gamut after adjustment. [Figure 8] FIG. 10 is a diagram showing a case where a second image is simultaneously displayed with a monochromatic fundus image as an example of a display mode of the second image, and shows the monochromatic fundus image displayed in combination with the second image. [Figure 9] FIG. 10 is a diagram for explaining an outline of color gamut correction in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0011] "overview" The present disclosure will be described below based on embodiments. For convenience, unless otherwise specified, the processing contents of the ophthalmic image processing program and the ophthalmic image processing method according to the embodiments will be described below as being executed by a computer. The computer may be an ophthalmic imaging device including an image processing device, or may be separate from the ophthalmic imaging device.

[0012] The ophthalmologic image processing program of this embodiment causes a computer to execute at least a first image acquisition step and a color gamut-corrected image generation step. In the first image acquisition step, a color fundus image is acquired as a first image. In the color gamut-corrected image generation step, pixel values of at least one color component in the first image are corrected based on color gamut information. As a result, a color gamut-corrected image is generated from the first image. The color gamut information is information that specifies a predetermined color gamut to be applied to the color fundus image. Therefore, in the color gamut-corrected image, the color of each tissue is expressed in the predetermined color gamut.

[0013] Note that the color gamut specified in the color gamut information in this embodiment is different from the color reproduction gamut (color gamut) of each image output device. In other words, the color gamut specified in the color gamut information does not originate from an image output device such as a display or printer. Therefore, the color gamut-corrected image is expressed in a predetermined color gamut that is different from both the original color gamut of the first image and the color gamut originating from an image output device such as a display or printer.

[0014] In this embodiment, the color gamut information may be information that specifies the target hue range after correction (after color gamut conversion). More specifically, the color gamut information may be information that defines the target color gamut itself after correction (after color gamut conversion), information that indicates the correspondence between the color gamuts before and after correction, such as a mapping table, or other information. In this embodiment, pixel values of at least any color component in the first image are corrected so that at least colors that fall outside the target hue range in the first image are included within the target hue range or so that the difference from the target hue range is reduced. As a result, unnatural colors relative to the target can be appropriately reduced in the color gamut-corrected image.

[0015] The first image may also be a color fundus image captured using a first imaging method. The target hue interval in the color gamut information may be determined as appropriate. For example, the hue interval of a color fundus image captured using a second imaging method different from the first imaging method may be determined as the target color gamut. In this case, the target color gamut is determined to reduce the difference in color tone between the color fundus image captured using the second imaging method and the first image, and does not necessarily need to match the color gamut of the color fundus image captured using the second imaging method. The color gamut corresponding to the second imaging method may be determined based on the average and variance of the hue of each pixel in the color fundus image captured using the second imaging method.

[0016] Furthermore, the first imaging method may be a confocal method, and the second imaging method may be a non-confocal method. There are several differences between the confocal and non-confocal methods, as follows. For example, the confocal method reduces the amount of unnecessary scattered light being guided to the light-receiving element compared to non-confocal methods such as fundus cameras. This results in a first image with excellent contrast. Furthermore, for example, the confocal method uses a light source with a narrow wavelength range (e.g., a monochromatic light source) compared to white light sources such as flash lamps, which are widely used in fundus cameras. Since the spectral reflection characteristics of each tissue (e.g., each layer) of the fundus differ, the tissues that are emphasized may differ between color fundus images obtained by the two imaging methods.

[0017] When the photographing optical system is a confocal system, the color tone of the entire fundus image (balance of each color component) may differ for each photograph depending on the photographing conditions such as alignment and focus. Therefore, even when photographing a specific subject's eye, it is difficult to stabilize the color tone of the entire image for each photograph.

[0018] In contrast, in this embodiment, the first image may be acquired by converting a color fundus image (referred to as an original image) captured by a confocal method using a process different from the color gamut correction image generation step. For example, the conversion may be performed manually or automatically.

[0019] As a specific example, pixel values for each color component in the original image may be corrected based on a predetermined histogram target pattern for each color component. As a result, regardless of differences in photographing conditions, the first image becomes an image with a color tone defined by the predetermined histogram target pattern for each color component. The target pattern may be, for example, a target pattern designed to approximate the color tone of a fundus photograph.

[0020] However, even if the color tone of the first image has been corrected in advance, there may be cases where the color tone of a portion of the image differs from the target color tone.

[0021] For example, even if the color tone of the first image is corrected in advance to approximate the color of the fundus photograph, the color of tissues, such as the optic disc, that have shapes different from those of the surrounding tissues, or lesions, may appear unnatural and different from that of the fundus photograph. More specifically, the red component of some of the tissues or lesions may be relatively low, resulting in a greenish appearance. This phenomenon is thought to be due to the above-mentioned or other differences between the confocal and non-confocal methods.

[0022] In contrast, the first image obtained using the confocal method is corrected based on the color gamut corresponding to the color fundus image obtained using the non-confocal method, thereby reducing the occurrence of areas with unnatural colors compared to the color fundus image obtained using the non-confocal method.

[0023] In this case, it is preferable that pixel values of at least the red component in the first image are corrected based on the color gamut information. That is, since the red component is relatively small in the greenish portion of the first image, the pixel values of the red component are corrected to the high luminance side, so that the tissue is depicted in a more natural color when compared with a color fundus image obtained by a non-confocal method.

[0024] In this embodiment, a GUI for accepting an operation to change the color gamut specified by the color gamut information may be displayed. The color gamut information may be changeable based on the change operation via the GUI. This allows the examiner to adjust the color gamut-corrected image to the desired color tone.

[0025] In this embodiment, the color gamut corrected image and the color component images corresponding to the first image may be output to an image output device so that the color component images, which are fundus images for each color component in the first image, are displayed together with the color gamut corrected image. Here, the fundus image for each color component in the color gamut corrected image may depict images that do not actually exist in pixels where the color components have been corrected as a result of the color gamut correction. Therefore, when displaying the color component images together with the color gamut corrected image, the color component images of the first image are used as the color component images, allowing the examiner to properly confirm the color component images.

[0026] In the above embodiment, the color gamut information has been described as information indicating a color gamut corresponding to the second imaging method, but this is not necessarily limited to this. For example, the color gamut information may be set based on the first image itself. For example, if it is empirically clear that natural colors (i.e., colors included in a desired hue range) are dominant in the first image, it is considered possible to eliminate unnatural colors that are outliers from the hue range by determining the hue range based on the average, variance, etc. of the hues of each pixel in the first image. Therefore, it is considered possible to generate a gamut-corrected image in which unnatural colors in the first image are reduced, even if color gamut correction is performed using color gamut information based on the first image itself in this manner.

[0027] "Example" <Overall structure> The fundus imaging device 1 according to the embodiment (hereinafter abbreviated as "the device 1") comprises at least imaging optical systems 10, 20 and an image processor 80. By including the image processor 80, the device 1 becomes a computer that executes various image processing operations. The image processor 80 may be shared by a processor that controls the operation of the entire device. The image processor 80 may be separate from the processor that controls the operation of the entire device. An ophthalmologic image processing program may be stored in a memory accessible from the processor of the image processor 80.

[0028] As shown in FIG. 1, the present device 1 is roughly divided into an optical unit 1a and a control unit 1b, and the photographing optical systems 10 and 20 may be stored in the optical unit 1a, and the image processor 80 may be stored in the control unit 1b. The control unit 1b has a processor (CPU) 71 and various memories 72. An ophthalmological image processing program may be stored in the memory 72. An operation unit 75 (user interface) may be connected to the control unit 1b. The operation unit 75 may be a pointing device such as a mouse or a touch panel, or may be another user interface. For example, a PC may be used as the control unit 1b.

[0029] The device 1 may also have a monitor 90. For example, a captured fundus image is displayed on the monitor 90. In addition, various GUIs may also be displayed on the monitor 90.

[0030] <Photographing optical system> The photographing optical systems 10 and 20 capture at least a color fundus image, which is a type of fundus image. In this embodiment, the fundus image is a front image of the fundus. In addition, in a color fundus image, each pixel has color information. The photographing optical systems 10 and 20 include an illumination optical system 10 and a light-receiving optical system 20 (see FIG. 1). Additionally, the photographing optical systems 10 and 20 may capture a monochrome fundus image.

[0031] The illumination optical system 10 illuminates the fundus with a plurality of monochromatic light beams having different wavelengths. The light-receiving optical system 20 includes at least a light-receiving element 25 that receives the fundus reflection light beams of the plurality of monochromatic light beams. A signal from the light-receiving element 25 is input to an image processor 80, which then generates a color fundus image. The light-receiving optical system 20 may include a plurality of light-receiving elements 25, one for each wavelength of the monochromatic light beam. In this case, the plurality of fundus reflection light beams of the plurality of monochromatic light beams may be simultaneously received by the plurality of light-receiving elements 25. Alternatively, the plurality of fundus reflection light beams of the different wavelengths may be received by a single light-receiving element 25 in a time-division manner (in other words, at different timings). The light-receiving element 25 may be a point light-receiving element, a one-dimensional image sensor (line sensor), a two-dimensional image sensor, or the like. The type of light-receiving element to be used is selected appropriately depending on the imaging method.

[0032] The irradiation optical system 10 may have a light source 11 for each wavelength. For example, the light source 11 may be any of various light sources, such as a monochromatic LED or a laser light source. Here, "monochromatic light" has a narrow meaning of light that cannot be spectrally resolved, but this disclosure is not necessarily limited to this. "Monochromatic light" in this disclosure may have a wavelength distribution width that is considered to be a specific color in the technical field of fundus imaging devices. However, the wavelength distribution width of monochromatic light is sufficiently narrow so that it can be clearly distinguished from white light in the technical field of fundus imaging devices.

[0033] For example, the photographing optical systems 10 and 20 may irradiate the fundus with three colors, R (red), G (green), and B (blue), when capturing a color fundus image. In this case, IR (infrared) light may be irradiated onto the fundus in addition to or instead of the R light. Here, three or four colors of light are irradiated onto the fundus when capturing a color fundus image, but this is not necessarily limited to this. The light irradiated onto the fundus may be two colors, or five or more colors. Furthermore, the above color combinations are merely examples, and color fundus images may be captured using other color combinations.

[0034] The photographing optical systems 10 and 20 may be confocal optical systems. In the case of a confocal optical system, the illumination optical system 10 has an optical scanner that focuses illumination light (here, monochromatic light) into a point or line on the fundus, and the optical scanner scans the illumination light across the fundus. The light-receiving optical system 20 also has a harmful light removal unit (e.g., a pinhole or slit aperture) at a position conjugate with the fundus. The harmful light removal unit guides fundus reflected light from the area irradiated with the illumination light to the light-receiving element 25 and removes other light. A frontal image of the fundus is acquired as a result of sequential light reception by the light-receiving element 25. In the line-scanning method, the light-receiving element 25 may also function as the harmful light removal unit. In this case, a line sensor used as the light-receiving element 25 is positioned at a position conjugate with the fundus. An example of a non-confocal photographing optical system is the optical system of a general fundus camera. For more details about the confocal optical system, please refer to, for example, Japanese Patent Application Laid-Open No. 2016-059539 filed by the present applicant, which discloses a scanning laser ophthalmoscope, which is an example of a fundus imaging device having a confocal optical system.

[0035] <Image processor> As described above, the image processor 80 generates a fundus image based on the signal from the light receiving element 25. In this embodiment, at least a color fundus image is generated as the fundus image. However, this is not necessarily limited to this, and the image processor 80 may additionally generate a monochrome fundus image based on the signal from the light receiving element 25.

[0036] The image processor also performs correction processing on the color fundus image captured by the device 1 employing a confocal system to approximate the color of a fundus photograph (a color fundus image captured using a non-confocal system and white light). For example, when a color fundus image is captured, the image processor generates a first image (a color-tone-corrected image) and a second image (a color-gamut-corrected image). The color fundus image obtained as a result of the capture is used as the processing target (i.e., the original image), and pixel values of the original image are corrected to generate the first and second images. Note that the first and second images in this embodiment are both color fundus images having R, G, and B color components.

[0037] For ease of explanation, hereinafter, colors that do not exist empirically in fundus photographs will be referred to as “unnatural colors,” and colors that do exist will be referred to as “natural colors.” However, it should be noted that the distinction between natural and unnatural is merely a convenient way of expressing differences in color between different photographing methods, and is unrelated to the question of whether or not the characteristics that should be present are properly depicted in each photographing method.

[0038] A method of color correction for a color fundus image in this embodiment will be described with reference to FIGS.

[0039] In this embodiment, as shown in Fig. 2, first, a color fundus image serving as an original image is captured, and then the color tone of the original image is corrected to generate and acquire a first image. Furthermore, the color gamut of the first image is corrected to generate and acquire a second image.

[0040] 3, a GUI 210 for color tone correction and GUIs 220 and 230 for color gamut correction are displayed on the screen together with a color fundus image 200. In response to operations on the GUIs 210 to 230, the colors in the color fundus image 200 displayed on the screen are corrected.

[0041] First, the original image is displayed on the screen as a color fundus image 200. The examiner first operates the GUI 210 for color correction, which corrects the color tone of the original image to generate a first image, which is then displayed on the screen.

[0042] <Color correction> The color fundus images obtained by the photographing optical systems 10 and 20 in this embodiment use monochromatic light as illumination. Therefore, compared to images captured using white light, such as fundus photography, the color tone is more susceptible to the influence of the fundus's reflection characteristics, making the color tone less stable. Furthermore, because the photographing optical systems 10 and 20 in this embodiment have a higher degree of confocality than fundus cameras, individual differences in fundus shape and alignment conditions significantly affect the color tone. As a result, the color fundus images obtained by the photographing optical systems 10 and 20 in this embodiment are less likely to reproduce a consistent color tone. Furthermore, in the field of fundus examinations, color fundus images captured using white light have been used for many years, and the color tone of color fundus images captured using white light is widely known among examiners. In contrast, it is difficult to say that the color tone of color fundus images captured using monochromatic light is sufficiently known among examiners.

[0043] In contrast, in this embodiment, the first image (color-tone corrected image) is a color fundus image expressed in a predetermined color tone regardless of the eye to be examined and the photographing conditions. The first image is generated by correcting the gradation value of each pixel in the original image so that the feature value for each color component, which is based on the distribution of gradation values of the pixels in the original image, fits to the feature value predetermined as a target pattern for each color component.

[0044] The target pattern in this embodiment represents a histogram pattern of each color component (R, G, B) in a fundus photograph. The light source emitting white light may be a xenon lamp, a white LED, or the like. The color fundus image may be, for example, an image captured by a fundus camera. For example, the target pattern may be predetermined based on the average luminance distribution in an actual fundus photograph.

[0045] The feature values determined as the target pattern are values that define a histogram based on the distribution of gradation values of each pixel. As an example, in this embodiment, a target value of brightness and a target value of contrast may be determined in advance as feature values for each color component. As an example, the target value of brightness may be any one of the mean, median, and mode of gradation values in the target gradation distribution. Furthermore, the target value of contrast may be any one of the standard deviation, variance, and half-width in the target gradation distribution.

[0046] Here, the characteristics of the color-tone corrected image in which the color tone has been corrected by applying the target pattern in this embodiment will be described in detail with reference to Fig. 4. Each graph in Fig. 4 shows a histogram for each color component (each of the R, G, and B color components).

[0047] In the color-tone corrected image shown in FIG. 4, the histogram for each color component is corrected so as to fit into a target pattern defined by the following target values. [Target brightness (average value of gradation distribution)] R:122 G:66 B:32 Contrast target value (standard deviation) R:40 G:27.5 B:16.6 However, each target value is expressed in units of 256 gradation levels.

[0048] These target values, shown as examples, are determined based on fundus photographs. The first image obtained in this way is meaningful because it has a color tone familiar to the examiner and ensures reproducibility.

[0049] <Color gamut correction> However, although the first image was depicted in colors that did not look out of place when compared with fundus photographs, etc., some areas of some images were depicted in unnatural colors that were unfamiliar to fundus photographs. For example, unnatural colors were likely to appear in tissues with shapes different from those around the optic disc, such as lesions. Areas where unnaturalness occurred tended to have less red components than in fundus photographs, and as a result, the optic disc, etc., were depicted in a greenish (or bluish) color that was unfamiliar to fundus photographs, causing a sense of incongruity.

[0050] However, the greenish color of the nipple or lesion in the first image can be used as an indicator of the shape of the nipple or the presence or absence of a lesion. Therefore, the examiner may be able to select whether to display the first image or the second image. Therefore, in this embodiment, color gamut correction can be switched between enabled and disabled via the GUI 220.

[0051] When the GUI 220 is operated and color gamut correction is enabled, the color information of the first image is corrected and the second image is displayed on the screen as a color fundus image 200.

[0052] According to the inventor's research, the unnaturalness of coloring can be reduced in areas with unnatural colors (areas expressed in greenish colors) by adding red components. However, if the balance of red components in the entire image is changed, the colors of most of the image that were expressed in natural colors in the first image will change.

[0053] In contrast, in this embodiment, the second image is generated by correcting pixel values of at least one color component in the first image based on a predetermined color gamut. In this embodiment, the correction is performed by clipping the color gamut of the first image based on color gamut information indicating the predetermined color gamut (details will be described later). Here, the color gamut information is a color gamut predetermined to be applied to the color fundus image, and in this embodiment, a target color gamut (hereinafter, for convenience, referred to as a "target color gamut") is specified. The target color gamut does not depend on the hardware performance of the image output device (i.e., for example, the color reproduction gamut of the monitor 90, etc.).

[0054] The target color gamut indicated by the color gamut information may be set empirically. For example, the target color gamut in this embodiment may be set based on the hue (average hue variance) in the first image in which no unnatural color regions occur. In this way, since the first image is an image whose color tone is closer to that of a fundus photograph, the target color gamut is set indirectly taking into account the hue range of the color fundus image. The target color gamut may also be set based on the hue (average hue variance) in the fundus photograph itself. Unnatural colors in the first image may cause some examiners to feel uncomfortable because they do not exist in the target fundus photograph, and such colors are unlikely to be included in the hue range of the fundus photograph. On the other hand, since most pixels in the first image are depicted in natural colors, most of these pixels are likely to be included in the hue range of the fundus photograph. Therefore, unnatural colors in the first image can be appropriately excluded from the target color gamut set taking into account the hue range of the fundus photograph, and most of the colors used in the first image are included almost directly in the target color gamut in this embodiment.

[0055] Here, the color gamut correction method in this embodiment will be described in detail with reference to Figure 5. In this embodiment, the color gamut is corrected by clipping. In the clipping shown in this embodiment, pixels having color information that falls outside the target color gamut are corrected to the color of the color gamut boundary in the target color gamut.

[0056] An example of color gamut information in a color space is shown in the graph on the left side of Figure 5. In Figure 5, the cubic area surrounded by the dotted line indicates the original color gamut in which an image can be expressed.

[0057] Meanwhile, the quadrangular pyramid area surrounded by a solid line indicates the target color gamut. As an example, the quadrangular pyramid area in FIG. 5 corresponds to a hue range in the HSV color space where the hue H ranges from -60° to +60°. In this hue range, an image is expressed in colors ranging from magenta to yellow. Each pixel in a typical fundus photograph is expressed in this hue range. However, the target color gamut is not necessarily limited to this. For example, the target color gamut may be set based on the average and variance of the hues of each pixel in an actual fundus photograph, etc.

[0058] In Figure 5, the fundus image on the right side of the graph is the fundus image to be subjected to color gamut correction (i.e., the first image in this embodiment). In the color gamut correction process of this embodiment, for each pixel of the first image, it is determined whether the color information of the pixel is outside the target color gamut. In this determination, when comparing with the target color gamut, the R, G, and B pixel values of each pixel may be referenced, or the H value of each pixel may be referenced after conversion to HSV.

[0059] The value of hue H can be calculated from R, G, and B using the conversion formula shown below in "Equation 1." In other words, hue H is derived based on the ratio of R, G, and B.

[0060]

number

[0061] Here, the number of unnatural colors in the first image is sufficiently small. In particular, in this embodiment, this is sufficiently small, combined with the color correction performed in advance. For example, pixels with such a small number of colors, such as the pixels in the optic disc in the fundus image of Figure 5, are determined to be outside the target color gamut in the first image, and the pixel values of those pixels are changed to fall within the target color gamut. At this time, in this embodiment, the pixel values of at least the red component are changed to the high-luminance side. In this embodiment, the pixel values are changed based on the following "Equation 2."

[0062]

number

[0063] Here, R' on the left side of "Equation 2" is the pixel value of the R component after correction. R, G, and B on the right side of "Equation 2" are the pixel values for each color component before correction. According to "Equation 2," for pixels whose color information is determined to be outside the target color gamut, the pixel value of only the R component is changed, thereby correcting the color to the color on the color gamut boundary.

[0064] In this embodiment, when the target color gamut is defined as a hue interval in the HSV color space where the hue H is from -60° to +60°, outside the target color gamut, the gradation value of the R component is always on the low-luminance side relative to at least one of the gradation values of G and B. Therefore, even if "Equation 2" is applied to all pixels of the first image without determining whether they are outside the target color gamut, the same results as when a determination is made can be obtained. Therefore, depending on the setting of the target color gamut, the above determination may not necessarily be necessary.

[0065] On the other hand, pixels determined to be inside the target color gamut in the first image, such as pixels corresponding to blood vessels and many fundus tissues in the fundus image of Figure 5, maintain their pixel values as they are. Since the first image contains a large number of such pixels, the color information of those pixels is maintained in the second image.

[0066] As described above, the color correction process limits the target to pixels in the first image that have unnatural colors, and replaces the colors expressed by those pixels with colors in the range from magenta to yellow (i.e., colors that are less unnatural when compared with the fundus photograph). As a result, the color fundus image captured by the device 1 is appropriately expressed using only natural colors.

[0067] <Manual correction> In this embodiment, the target color gamut can be further set (changed) based on a user operation. For example, a setting (change) operation for the target color gamut is accepted via the GUI 230. In this embodiment, the GUI 230 is a slider, and the hue interval is set by moving the knob along the arrow. In this embodiment, the hue interval of the hue H generally has two endpoints, one on the blue side and one on the green side. However, since there is almost no blue in the first image, the blue side is fixed (-60° in this embodiment), and the green side endpoint is changed via the GUI 230. Note that the adjustment range of the green side endpoint in the GUI 230 is set between 0° and 60°. Generally, the hue interval of a certain hue ranges from -180° to 180°, but the hue interval of a fundus photograph, as described above, is only a part of that range. In this embodiment, the adjustment range is limited to a range necessary and sufficient for reproducing the color of the fundus photograph, allowing the examiner to appropriately adjust the target hue interval.

[0068] The second image, in which the hue interval has been changed based on the operation of the GUI 230, is displayed sequentially on the screen. In this embodiment, values indicating the endpoints of the set hue interval are displayed in conjunction with the movement of the knob on the GUI 230. The GUI 230 also displays a slider in a color (color gradation) corresponding to the set hue interval. This makes it easy for the examiner to intuitively understand the set color gamut. This allows the user to intuitively change the color fundus image to the desired color.

[0069] Furthermore, instead of the GUI 230, a GUI 300 shown in FIG. 6 may be used. The GUI 300 shown in FIG. 6 resembles a color wheel, and the user moves the endpoints 310a and 310b between colors by operation. The hue interval between the two endpoints 310a and 310b is set as the target color gamut. In this embodiment, the second image based on the hue interval changed based on the operation is sequentially displayed on the screen. This allows the user to intuitively change the color fundus image to the desired color.

[0070] For example, in the description of FIG. 5, the preset target color gamut was a hue range where the hue H was from -60° to +60°, but actual fundus photographs tend to be expressed in a hue range that is narrower than that. Therefore, in order to depict a color fundus image captured by this device 1 in colors closer to those of the fundus photograph, the target range may be narrowed from the hue range where the hue H is from -60° to +60°. In this case, the pixel values are changed based on the following "Equation 3."

[0071]

number

[0072] Here, f(H) is the target hue (target value of hue) of the color after conversion by "Equation 3." f(H) is expressed by Equation 4.

[0073]

number

[0074] Here, H is the hue of the pixel before conversion. B is closer to blue than the endpoints 310a and 310b of the hue interval, while H G The remaining color is closer to green. The target color gamut in this case is shown in Figure 7. Note that depending on the combination of the hue interval setting and the pixel value before conversion, the pixel value after conversion may exceed the maximum value that can be expressed (R' > 255), in which case it may be replaced with the maximum value (R' = 255).

[0075] As a result, even if the target color gamut is changed, pixels having color information that falls outside the target color gamut are appropriately corrected to the color of the color boundary in the target color gamut by changing at least the pixel value of the red component.

[0076] <Simultaneous display of the second image and each color component image> In this embodiment, the color fundus image may be displayed together with monochrome fundus images (color component images) corresponding to each color component. In this case, the color fundus image and the color component images may be displayed side by side on the same screen, or may be displayed alternately in one window.

[0077] The spectral reflectance of each of the R (red), G (green), and B (blue) lights irradiated from the device 1 varies for each tissue (e.g., each layer). Therefore, the R (red), G (green), and B (blue) color component images can depict different characteristics. Therefore, by displaying color component images corresponding to each color component along with the color fundus image, the examiner can grasp the structure of the fundus from multiple angles.

[0078] 8(a) and 8(b) show a color fundus image (before correction: 401, after correction: 402) before and after color gamut correction, and color component images (before correction: 401R, 401G, 401B, after correction: 402R, 402G, 402B) of each color component of the color fundus image. In the color component image 402R of the R component of the second image shown in Fig. 8(b), the gradation value of the R component has been corrected by the hue correction process, and as a result, images that do not actually exist have been depicted in the areas where the gradation values have been corrected (especially the optic disc and lesions) surrounded by dotted lines.

[0079] Therefore, in this embodiment, when the second image is displayed, the combination shown in Fig. 8(c) is used, i.e., the second image 402 and color component images 401R, 401G, and 401B of each color component in the first image are simultaneously displayed. As a result, in this embodiment, an appropriate color component image is displayed for at least the R component. As a result, even when the second image is displayed, the examiner can grasp the structure of the fundus using the appropriate color component image.

[0080] "Example of transformation" In the above embodiment, one method for correcting the color gamut (compressing the color gamut) has been described, in which pixels having color information outside the target color gamut are corrected to the color on the color gamut boundary of the target color gamut. However, the method for correcting the color gamut is not necessarily limited to this. For example, the second image may be generated by compressing the color gamut nonlinearly as follows:

[0081] A method for compressing the color gamut nonlinearly will be described with reference to FIG. In FIG. 9, for a certain hue (for example, hue H), the hue of the first image is shown on the horizontal axis, and the target hue (target value of the hue) specified by the color gamut information is shown on the vertical axis.

[0082] In the method of the embodiment described above, colors within the target color gamut are left as they are, and colors outside the target color gamut are replaced with colors on the boundary of the target color gamut. In contrast, the method of this modified example replaces colors within the target color gamut as well, so that differences in hues outside the target color gamut are reflected in the corrected image. In this case, the following "Equation 5" may be applied to the hue f(H) in the above "Equation 3".

[0083]

number

[0084] Here, the boundary of the color gamut is set as a combination of a straight line with a slope of 1 and a sigmoid curve, but it is not necessarily limited to this combination. G′ indicates the transition point between the straight line and the curve on the green side, and H B ′ indicates the transition point between straight and curved lines on the blue side.

[0085] In the above embodiment, the conversion from the original image to the first image is performed based on a predetermined target pattern of the histogram for each color component, but this is not necessarily limited to this, and at least one of the brightness, contrast, gamma, etc. of each color component may be manually adjustable. These parameters may be preset for each device.

[0086] Similarly, a value determined in advance by the examiner may be used as the target color gamut in the conversion from the first image to the second image.

[0087] In the above embodiment, the color gamut of the entire first image is corrected, but this is not necessarily limited to this. The color gamut of a partial region of the first image may also be corrected. As a partial region, the color gamut of at least the optic optic disc region may be corrected. For example, the color gamut of the optic optic disc region in the first image may be corrected using the method of this embodiment, while the color of the first image may be maintained outside the optic optic disc region. Since the difference in color of the optic optic disc is easily noticeable when comparing the first image with a fundus photograph, correcting the color gamut of the optic optic disc region using the method of this embodiment and expressing it in the color of the fundus photograph reduces the sense of discomfort felt by the examiner. Furthermore, by maintaining the color of the first image outside the optic optic disc region, lesions, etc., can be expressed in a different color from the fundus photograph, making them stand out. In this case, the optic optic disc region may be detected by image processing, set manually, or set at a predetermined position on the image. [Explanation of symbols]

[0088] 1. Computer 71 CPU

Claims

1. An ophthalmic image processing program, comprising: When executed by a computer processor, a first image acquisition step of acquiring a first image as a color fundus image by a first imaging method; and a color gamut-corrected image generation step of correcting pixel values of at least any color component in the first image based on color gamut information that specifies a predetermined color gamut to be applied to a color fundus image, the color gamut information indicating a color gamut corresponding to a color fundus image obtained by a second imaging method different from the first imaging method, to generate a color gamut-corrected image.

2. 2. The ophthalmologic image processing program according to claim 1, wherein the first imaging method is a confocal method and the second imaging method is a non-confocal method.

3. 3. The ophthalmologic image processing program according to claim 1, wherein the color gamut information is defined as a range of hues.

4. The ophthalmologic image processing program according to claim 1 , wherein the color gamut information is set based on the hue of each pixel in the first image.

Images