Information processing device, image processing method, and program

The information processing device addresses the challenge of evaluating retinal layer thickness symmetry by deforming fundus images to align the angle between the macula and papilla lines to 180 degrees, enabling accurate symmetry assessment in both nasal and temporal regions, thus enhancing diagnostic accuracy.

WO2024232238A1PCT designated stage expired Publication Date: 2024-11-14CANON KK
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2024/015515
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-11
Filing Date
2024-04-19
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

Existing methods for evaluating the symmetry of retinal layer thickness in fundus images, particularly in the temporal region, are inadequate due to the use of a straight line that is not approximately horizontal, making it difficult to assess symmetry accurately.

Method used

The proposed solution involves an information processing device that includes a first straight line passing through the macula and a second straight line passing through the papilla, with the device deforming the fundus image to rotate the second straight line so that the angle between the two lines approaches 180 degrees, allowing for appropriate evaluation of symmetry in both nasal and temporal regions.

Benefits of technology

This approach enables accurate evaluation of symmetry in both nasal and temporal regions of the retinal layer thickness, improving diagnostic capabilities in ophthalmological assessments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2024015515_14112024_PF_FP_ABST
    Figure JP2024015515_14112024_PF_FP_ABST
Patent Text Reader

Abstract

An information processing device according to the present disclosure comprises a deformation means for deforming a first region and a second region in an ocular fundus image, the second region being larger than the first region, the first region and the second region being defined by a first straight line and a second straight line, such that the angle becomes close to180 degrees, such angle being formed between the first straight line, which passes through a first feature part relating to macula in the ocular fundus image and is positioned on the ear side, and the second straight line, which passes through the first feature part and a second feature part relating to papilla in the ocular fundus image and is positioned on the nose side.
Need to check novelty before this filing date? Find Prior Art

Description

Information processing device, image processing method, and program

[0001] The present disclosure relates to an information processing device, an image processing method, and a program.

[0002] Optical coherence tomography (OCT) devices that acquire tomographic images of the fundus of the eye are widely used as ophthalmic devices. Doctors diagnose the presence or absence of ophthalmic diseases such as glaucoma by evaluating the thickness of the retina (layer thickness) obtained from the tomographic images.

[0003] Here, Patent Document 1 discloses a method for evaluating the symmetry of layer thickness with respect to a line passing through the macula and the optic disc.

[0004] JP 2022-38751 A

[0005] Here, the thickness of the retina (layer thickness) is symmetrical with respect to a line passing through the macula and the optic disc in the region from the macula to the nose, but may not be symmetrical with respect to a line passing through the macula and the optic disc in the region from the macula to the ear. In other words, because the line serving as the reference for symmetry is different, for example, it may be possible to properly evaluate the symmetry of the layer thickness in the nasal region, but not in the ear region.

[0006] Therefore, an object of the present disclosure is to provide a method capable of appropriately evaluating the symmetry of layer thickness in the nose-side region and the symmetry of layer thickness in the ear-side region.

[0007] The information processing device of the present disclosure includes a deformation means for deforming a first region and a second region in the fundus image, the first region being defined by the first straight line and the second straight line, so that an angle formed by a first straight line passing through a first feature related to the macula in the fundus image and located on the temporal side, and a second straight line passing through the first feature and a second feature related to the papilla in the fundus image and located on the nasal side, approaches 180 degrees.

[0008] According to the present disclosure, the symmetry of the layer thickness in the nose-side region and the symmetry of the layer thickness in the ear-side region can be appropriately evaluated.

[0009] 17. A block diagram of an information processing device 100. A flowchart showing the operation of a control unit 102. A diagram showing features of a fundus image. A diagram showing features of a fundus image. A diagram showing a method for deforming a fundus image. A diagram showing a method for deforming a fundus image. A diagram showing a grid. A diagram showing a method for deforming a fundus image according to a second embodiment. A diagram showing a method for deforming a fundus image according to the second embodiment. A diagram showing a method for deforming a fundus image according to the second embodiment. A diagram showing a method for deforming a fundus image according to the second embodiment. A diagram showing a method for deforming a fundus image. A diagram showing a fundus image with a grid superimposed thereon. A diagram showing a fundus image with a grid superimposed thereon according to a third embodiment. A diagram showing features of a fundus image according to a fourth embodiment. A diagram showing features of a fundus image according to the fourth embodiment. A flowchart showing the operation of a control unit 102 according to the third embodiment. A diagram showing a GUI displayed on a monitor 1700. A diagram showing a GUI displayed on the monitor 1700. A diagram showing a thickness map 1900. A diagram showing a method for deforming a fundus image. A diagram showing a method for deforming a fundus image. A diagram showing a method for deforming a fundus image.

[0010] Hereinafter, exemplary embodiments for carrying out the present disclosure will be described in detail with reference to the drawings. In the drawings, the same reference numerals are used to indicate identical or functionally similar elements. In addition, in each drawing, some components, members, and processes that are not important for the explanation may be omitted.

[0011] In this specification, a fundus image is defined as an image of the fundus acquired by various imaging techniques (modalities) such as an OCT device, a fundus camera, or a scanning laser ophthalmoscope (SLO). For example, fundus images include tomographic images, thickness maps, EnFace images, and Projection images. Furthermore, in this specification, the depth direction of the fundus is defined as the Z direction. Furthermore, a plane perpendicular to the Z direction is defined as the XY plane.

[0012] An embodiment of the present disclosure will be described.

[0013] 1 is a block diagram of an information processing device 100 according to this embodiment. The information processing device 100 includes an input unit 101, a control unit 102, a display control unit 103, an operation unit 104, and a storage unit 105. The information processing device 100 is, for example, a computer or a tablet terminal (portable information terminal). The information processing device 100 does not need to be separate from an ophthalmic device such as an OCT device. For example, the information processing device 100 may be incorporated into an information processing system that includes an ophthalmic device such as an OCT device.

[0014] The input unit 101 is, for example, a connector compatible with USB (Universal Serial Bus, registered trademark). Data is input to the input unit 101 from an ophthalmic device such as an OCT device. The input unit 101 and the ophthalmic device are connected, for example, by a cable. The data input from the ophthalmic device includes a fundus image. The data input from the ophthalmic device may also include data about the subject (such as identification number, axial length, age, visual acuity, race, medical history, and whether or not the subject has severe myopia) and an image of a medical record. If an image of the medical record is included, data about the subject may be extracted from the image of the medical record using text mining technology or the like. Here, data may be input to the input unit 101 from an external device such as a server in addition to the ophthalmic device. The input unit 101 and the external device may be connected directly by a cable or via a network. The input unit 101 and the external device may be connected not only by a wired connection but also wirelessly. As long as the input unit 101 and the external device can communicate with each other, the communication method is not important.

[0015] The data input from the ophthalmologic apparatus is output to the control unit 102 .

[0016] The control unit 102 is, for example, a CPU (Central Processing Unit). The control unit 102 performs image processing on the data input from the input unit 101. The image processing includes deformation processing of the fundus image. The control unit 102 is executed in accordance with a program stored in the storage unit 105. Here, the control unit 102 may be a CPU, a micro processing unit (MPU), a graphic processing unit (GPU), or a field-programmable gate array (FPGA).

[0017] The display control unit 103 controls a display unit such as a display. The display control unit 103 is, for example, one of the functions provided in a CPU. The display control unit 103 controls the display unit to display data input from the control unit 102. The data displayed on the display unit includes, for example, data on the subject, various images, and estimated results on ophthalmic diseases.

[0018] The operation unit 104 is, for example, a mouse or a keyboard. The operation unit 104 is operated by an examiner. Operation information by the examiner is input to the control unit 102.

[0019] The storage unit 105 is, for example, a non-volatile memory. The storage unit 105 stores programs for executing the control unit 102. The storage unit 105 also stores programs for implementing various application software, including an operating system (OS), device drivers for peripheral devices, and programs for performing processes described below. The storage unit 105 also stores data necessary for various calculations.

[0020] [Regarding Fundus Image Characteristics] First, the characteristics of a general fundus image will be described using FIGS. 3(a0) and 3(c0). FIG. 3(a0) shows a schematic diagram of a fundus image 300 of a right eye. The fundus image 300 includes a macula 301, a optic disc 302, and blood vessels 303. Here, in the case of the fundus image 300 of the right eye, the right side of the fundus image 300 is the nose side, and the left side of the fundus image 300 is the ear side. Also, the top of the fundus image 300 is the head side, and the bottom of the fundus image 300 is the leg side. FIG. 3(c0) shows a schematic diagram of a fundus image 300 of a left eye. Like the right eye, the fundus image 300 includes a macula 301, a optic disc 302, and blood vessels 303. Here, in the case of the fundus image 300 of the left eye, unlike the right eye, the right side of the fundus image 300 is the ear side, and the left side of the fundus image 300 is the nose side. As with the right eye, the upper side of the fundus image 300 is the head side, and the lower side of the fundus image 300 is the leg side. In this specification, the ear side is described as the left side in the fundus image of the right eye and the right side in the fundus image of the left eye. The nose side is described as the right side in the fundus image of the right eye and the left side in the fundus image of the left eye.

[0021] When photographing using an ophthalmologic apparatus, the examiner sets the position of the fixation light so that the macular region 301 is located at the center of the fundus image. Then, the examiner photographs the fundus image after confirming that the subject's eye is looking directly at the fixation light. In a fundus image photographed in this manner, for the right eye, the macular region 301 is generally located approximately at the center of the fundus image 300, and the optic disc 302 is generally located above and on the nasal side of the macular region 301, as shown in the fundus image 300 of FIG. 3( a0).

[0022] Next, the characteristics of general retinal layers will be described with reference to Figures 3 and 4. Here, the retinal layers refer to the retinal nerve fiber layer, ganglion cell layer, inner plexiform layer, and the like.

[0023] The fundus image 300(a1) in FIG. 3(a1) depicts a straight line 304 extending from the macular region 301 toward the nasal and temporal sides. The straight line 304 extends approximately horizontally relative to the fundus image 300(a1). The straight line 304 is depicted as a solid line extending from the macular region 301 toward the temporal side. The straight line 304 is depicted as a dashed line extending from the macular region 301 toward the nasal side. The fundus image 300(a2) in FIG. 4(a2) depicts a straight line 401 passing through the macular region 301 and perpendicular to the line 304. It is known that, among the four regions separated by the lines 304 and 401, the retinal layer thickness (layer thickness) of the upper temporal region 403 and the layer thickness of the lower temporal region 404 tend to be symmetrical with respect to the line 304.

[0024] The fundus image 300 (b1) in FIG. 3 (b1) depicts a straight line 305 connecting the macular region 301 and the optic papilla 302. The straight line 305 is a solid line extending from the macular region 301 toward the nasal side. The straight line 305 is a dashed line extending from the macular region 301 toward the temporal side. The fundus image 300 (b2) in FIG. 4 (b2) depicts a straight line 402 that passes through the macular region 301 and intersects the straight line 305 at right angles. Here, it is known that, of the four regions separated by the lines 305 and 402, the retinal layer thickness (layer thickness) of the upper nasal region 405 and the layer thickness of the lower nasal region 406 tend to be symmetrical with respect to the line 305.

[0025] A conventional method for evaluating symmetry involves geometrically transforming (e.g., enlarging, reducing, rotating, translating, or affine transforming) a fundus image 300 so that a line 305 connecting a macula 301 and an optic papilla 302 is approximately horizontal, and then using the geometrically transformed fundus image to evaluate the symmetry of layer thickness. However, while the geometrically transformed fundus image can appropriately evaluate the symmetry between the layer thickness of a nasal upper region 405 and a nasal lower region 406, it has a problem in that it cannot appropriately evaluate the symmetry between the layer thickness of a temporal upper region 403 and a temporal lower region 404 because the line 304 is not approximately horizontal.

[0026] In this embodiment, a modification method is described that enables appropriate evaluation of the symmetry between the layer thickness of the upper nose-side region 405 and the layer thickness of the lower nose-side region 406, and the symmetry between the layer thickness of the upper ear-side region 403 and the layer thickness of the lower ear-side region 404.

[0027] First Embodiment In this embodiment, the fundus image of the right eye will be mainly used for explanation, but the same applies to the fundus image of the left eye.

[0028] The flowchart in FIG. 2 shows the operation of the control unit 102 according to the first embodiment.

[0029] [Step S1: Image Input] In step S1, a tomographic image of the fundus acquired by the OCT device is input from the input unit 101 to the control unit 102. Here, the tomographic image includes images captured at multiple B-scan positions. The operation of step S1 allows the control unit 102 to acquire three-dimensional data regarding the fundus of the subject's eye. Here, an A-scan refers to acquiring tomographic information from a single point on the subject's eye. Furthermore, a B-scan refers to performing A-scans multiple times in any transverse direction (main scanning direction) to acquire two-dimensional tomographic information of the subject's eye E in the transverse direction and depth direction. In this embodiment, tomographic images of both the right and left eyes of the same subject are input. If there is a difference in axial length between the right and left eyes, a process to correct this may be performed.

[0030] [Step S2: Image Processing] In step S2, the control unit 102 performs image processing on the tomographic image input from the input unit 101. Step S2 is composed of the processes of step S21 (thickness map generation), step S22 (thickness map deformation), and step S23 (image analysis).

[0031] [Step S21: Thickness Map Generation] In step S21, the control unit 102 generates a thickness map using the tomographic image input in step S1. The thickness map is a map (map image) that represents the thickness (layer thickness) of an anatomically defined observation layer in the retina using brightness values ​​or the like for any coordinate (XY coordinate) in an XY plane perpendicular to the depth direction (Z direction) of the subject's eye. Examples of observation layers include the nerve fiber layer (NFL), ganglion cell layer (GCL), and inner plexiform layer (IPL). The thickness map can be generated by integrating the thicknesses of these layers. Alternatively, the thickness map may be generated from the layer thickness of the nerve fiber layer (NFL) alone, or an arbitrary layer may be selected and the thickness map generated based on the layer thickness of the selected layer. The arbitrary layer may be selected by manually drawing a line on the tomographic image to specify the boundary of each layer, or by selecting the layer from the boundary of each layer obtained by segmentation processing. Segmentation processing is a process of dividing each layer of a tomographic image. Specifically, a layer structure is extracted from a tomographic image of the eye to be examined, each layer is divided, and the thickness of each layer is calculated. Note that any known method may be used for the segmentation processing and the thickness calculation method.

[0032] In the thickness map, the layer thickness at any coordinate (XY coordinate) may be displayed in pseudocolor. Alternatively, in the thickness map, the layer thickness at any coordinate (XY coordinate) may be displayed as a grayscale brightness value. Alternatively, a thickness map displayed in pseudocolor and a thickness map displayed as a grayscale brightness value may be displayed side by side. Alternatively, the thickness map displayed in pseudocolor and the thickness map displayed as a grayscale brightness value may be displayed so that they can be switched between. FIG. 19 is an example of a thickness map 1900 corresponding to the fundus image 300 of FIG. 3. In the thickness map 1900, the layer thickness is displayed as a grayscale brightness value. In FIG. 19, the lighter the color (the higher the brightness value), the thinner the layer thickness.

[0033] Here, the control unit 102 may perform a reduction process or a trimming process on the input tomographic image before generating the thickness map. This method is expected to have the effect of reducing the processing load.

[0034] Next, the control unit 102 performs a correction process on the generated thickness map. An example of the correction process is described below. One example of the correction process is a process (error correction process) that corrects the brightness values ​​of regions in the thickness map that cannot be used for diagnosis. For example, if an error occurs in the layer segmentation process, the thickness at that position will have an unusual value and will be displayed as blown-out highlights or crushed shadows in the thickness map. The control unit 102 determines that the region having the unusual value is a region that cannot be used for diagnosis. Then, the control unit 102 performs a process of replacing the brightness values ​​of the region that cannot be used for diagnosis with the average value of the surrounding area. Alternatively, the control unit 102 may remove the region that cannot be used for diagnosis from the thickness map. The control unit 102 may also similarly replace or remove the brightness values ​​of regions that are considered unnecessary for diagnosis, such as the inside of the nipple.

[0035] Furthermore, if it is determined that the image data input from the input unit 101 has low image quality and cannot be used for diagnosis, the control unit 102 can cause the display control unit 103 to display a message prompting the examiner to retake an image. This determination may be made visually by the examiner, or may be made based on rules based on image brightness, the success rate of segmentation, or the like, or may be made using a method using machine learning.

[0036] Furthermore, when generating the thickness map, the layer thickness values ​​can be corrected based on the axial length data of the subject's eye. Here, the axial length data of the subject's eye may be acquired in step S1 or may be input by the examiner. If the axial length of the subject's eye is longer than that of the normal eye database, the retina may be stretched, resulting in a thinner overall layer thickness. Therefore, the control unit 102 can correct the thickness map to reduce the effect of the layer thickness depending on the axial length. The correction of the layer thickness depending on the axial length may be performed using any known method. This correction makes it possible to distinguish between cases where the layer thickness has been thinned due to a pathology and cases where the layer thickness has been thinned since normal. Furthermore, because different axial lengths change the fundus imaging range relative to the scan angle, the control unit 102 may perform a process to correct the scale of the thickness map. Furthermore, subjects with severe myopia tend to have longer axial lengths. Therefore, if the subject has severe myopia, the control unit 102 may perform a similar correction on the thickness map. The axial length of the subject's eye and whether or not the subject has severe myopia may be determined based on input data relating to the subject.

[0037] In this embodiment, the explanation is mainly given using a thickness map, but it is not limited to a thickness map. Alternatively, a difference map using the difference when comparing the layer thickness with a normal eye database, or a blood vessel density map generated using an OCT Angiography image may be used.

[0038] [Step S22: Thickness Map Deformation] In step S22, the control unit 102 deforms the thickness map generated in step S21. In this embodiment, an example will be described in which an EnFace image is generated from a tomographic image input from the input unit 101, and the generated EnFace image is deformed to deform the thickness map. Here, the EnFace image is an image generated by projecting the tomographic image in the depth direction (Z direction). Generally, when an EnFace image and a thickness map are generated from the same tomographic image, the thickness map is similarly deformed by deforming the EnFace image. Note that the control unit 102 that deforms the thickness map is an example of a deformation means.

[0039] A specific modification method will be described with reference to FIG.

[0040] First, the fundus image is segmented.

[0041] The fundus image 300(a3) in FIG. 5 is an EnFace image generated from a tomographic image input from the input unit 101. In addition to the lines 304 and 401 shown in the fundus image 300(a2) in FIG. 4, a line 305 connecting the macular region 301 and the optic papilla 302 is added to the fundus image 300(a3). The upper nasal region of the region divided by the lines 401 and 305 is designated as region 501. The angle between the lines 401 and 305 in region 501 is designated as α1. The lower nasal region of the region divided by the lines 401 and 305 is designated as region 502. The angle between the lines 401 and 305 in region 502 is designated as β1. The positions of the macular region 301 and the optic papilla 302 may be set by an operator's instruction or automatically by analyzing the fundus image. Here, the straight lines 401 and 305 do not necessarily have to pass through the macular region 301 itself, but may pass through the periphery of the macular region 301. Furthermore, the straight line 305 does not necessarily have to pass through the optic optic region 302 itself, but may pass through the periphery of the optic optic region 302. The macular region 301 or the periphery of the macular region 301 is an example of a first characteristic portion related to the macula. Furthermore, the optic optic region 302 or the periphery of the optic optic region 302 is an example of a second characteristic portion related to the optic optic region. Furthermore, the solid line portion of the straight line 304 is an example of a first straight line. Furthermore, the straight line 305 is an example of a second straight line.

[0042] Any straight line may be set manually instead of the straight line 401 and the straight line 305, or the values ​​of the angles α1 and β1 may be manually specified. The method of specifying each of these is not important, and they may be specified directly on the fundus image, or by inputting coordinates or numerical values ​​using a user interface (not shown), or by selecting from preset numerical values.

[0043] Next, the fundus image is deformed.

[0044] In this embodiment, an example will be described in which ear-side regions 403 and 404 (the dotted regions in FIG. 5 ) are not deformed, and nose-side regions 501 and 502 are deformed. Fundus image 300(a4) is the image obtained after the control unit 102 has deformed fundus image 300(a3). Regions 501b and 502b in fundus image 300(a4) are the regions obtained after the control unit 102 has deformed regions 501 and 502. In this embodiment, the control unit 102 enlarges region 501 and reduces region 502 so that the angle (= 90 degrees + α1) between lines 304 and 305 approaches 180 degrees. In other words, the control unit 102 performs deformation by rotating line 305 around the macular region 301 as a fulcrum so that line 305 approaches the dashed line portion of line 304. The deformed straight line 305 is a straight line 305b in the fundus image 300(a4). The fundus image 300(a4) shows an example in which the angle between the straight lines 304 and 305b as a result of the deformation is 90 degrees + α2. Note that the angle between the straight lines 304 and 305b (90 degrees + α2) does not necessarily have to be deformed to 180 degrees. The effects of the present disclosure can be achieved as long as the angle between the straight lines 304 and 305b (90 degrees + α2) is closer to 180 degrees than the angle between the straight lines 304 and 305 (= 90 degrees + α1).

[0045] As a result of this deformation, the nose-side upper region 501 is deformed so as to be stretched in the circumferential direction, becoming region 501b. That is, the nose-side upper region 501 is enlarged. Furthermore, angle α1 is deformed to angle α2 (>α1). On the other hand, the nose-side lower region 502 is deformed in a direction such that it is compressed in the circumferential direction, becoming region 502b. That is, the nose-side lower region 502 is reduced. Furthermore, angle β1 is deformed to angle β2 (<β1). Note that the region consisting of region 403 and region 501 is an example of a first region. The first region is a region defined by lines 304 and 305 in the fundus image. Note that the region consisting of region 404 and region 502 is an example of a second region. The second region is also a region defined by lines 304 and 305 in the fundus image.

[0046] A more specific deformation method will be described with reference to Figure 6. In a coordinate system with macular region 301 as the origin, line 304 as the x-axis, and line 401 as the y-axis, the angle θ (β1-β2) between line 305 and the x-axis is calculated, and an arbitrary point P1 on line 305 is deformed to point P2 on the x-axis corresponding to that point based on the angle θ. The method for calculating the coordinates of point P2 is not limited, but the coordinates of point P2 can also be easily calculated by using a formula for a rotating coordinate system with θ as the rotation angle of point P1.

[0047] Similarly, a method of transforming the nasal-side upper region 501 into region 501b and the nasal-side lower region 502 into region 502b will be described with reference to Figure 7. If the angle between the y-axis and a line (not shown) connecting an arbitrary point Q1 on region 501 and the macula 301 is θ1, the rotation angle θ2 of point Q2 corresponding to point Q1 can be calculated by θ2 = θ1(α2 / α1) - θ1 = θ1((α2 / α1) - 1). Furthermore, if the angle between the line (not shown) connecting an arbitrary point R1 on region 502 and the macula 301 is θ3, the rotation angle θ4 of point R2 corresponding to point R1 can be calculated by θ4 = θ3(β2 / β1) + (β1 - β2 - θ3) = θ3((β2 / β1) - 1) + (β1 - β2).

[0048] As described above, by applying a similar transformation process to the coordinates of regions 501 and 502, the regions can be transformed into regions 501b and 502b. In the transformed regions 501b and 502b, if a point S' (not shown) in the Cartesian coordinate system does not correspond to an original point S (not shown) (i.e., if no value is assigned), the data of adjacent coordinates may be used as is, or interpolated from nearby data. Furthermore, if an original point does not exist and interpolation from nearby points is not possible, such as at the edge of the image, the absence of data may be indicated by a black outline or diagonal lines. Conversely, if the transformed coordinates at the edge of the image are outside the image, the image size may be increased to prevent information loss. Furthermore, if multiple original points (points S1 and S2) correspond to a certain point S', the data of either point S1 or point S2 may be used for point S', or the data of points S1 and S2 may be averaged and used.

[0049] In addition, in this example, points P2, Q2, and R2 are deformed using a rotating coordinate system centered on the macular region 301, but a similar deformation process may be applied with a point other than the macular region 301 as the center, and each straight line used in the explanation may be replaced with a straight line that does not pass through the macular region 301.

[0050] Alternatively, coordinate transformation may be performed using a different transformation method without using the rotation angles α1 and β1. For example, transformation may be performed using only the y-coordinate component in a Cartesian coordinate system. For example, a line 1201 perpendicular to the dashed line portion of the line 304 is extended from an arbitrary point P1 on the line 305 of the fundus image 300 in FIG. 12 , and transformation is performed on the line 1201 so that the intersection of the line 1201 and the line 304 becomes P2. Similarly, an arbitrary point Q1 on the region 501 and an arbitrary point R1 on the region 502 may be transformed to points Q2 and P2 on the line 1201, thereby transforming the region 501 and the region 502. Furthermore, although the description in this example uses the line 1201 perpendicular to the line 304, a line having an arbitrary inclination relative to the line 304 may also be used as the line 1201.

[0051] Furthermore, weighting may be performed using characteristics such as axial length and retinal distortion during coordinate transformation. In this example, the upper ear-side region 403 and the lower ear-side region 404 are assumed to be symmetrical with respect to a straight line 304 extending substantially horizontally from the macular region 301. However, the entire fundus image may be rotated depending on the circumstances at the time of capture, and the regions 403 and 404 may not necessarily be symmetrical with respect to the straight line 304. In this case, a straight line 304' (not shown) may be used that is angled so as to reduce the difference between the regions 403 and 404. In this case, it is preferable to rotate the entire fundus image 300 so that the straight line 304' is substantially horizontal before performing the transformation process.

[0052] The fundus image 300 may also be divided into multiple fundus images. For example, FIGS. 20A and 20B show partial images 2000(a) and 2000(b) obtained by cutting out the stippled portions of the fundus images 300(a2) and 300(b2) shown in FIG. 4 . In this case, a portion of the original fundus image 300 may overlap both partial images, or may not be included in either partial image. For example, FIG. 21 shows an overlay of partial images 2000(a) and 2000(b). In this case, the dark gray region 2101 overlaps both partial images, and the white region 2102 is not included in either partial image. An arbitrary transformation process may be applied to each of the partial images 2000(a) and 2000(b), and the subsequent step S23 (image analysis) may be performed. Furthermore, partial images 2000(a) and 2000(b) do not necessarily need to be transformed, and subsequent steps may be performed without transformation processing. Furthermore, when subsequent step S23 (image analysis) is performed on each partial image to obtain multiple analysis results, the multiple analysis results may be displayed individually, or a single result that combines the multiple analysis results may be obtained.

[0053] In addition, although this example describes an example in which the EnFace image and the Projection image are deformed to deform the corresponding thickness map, it is also possible to deform only the thickness map without deforming the EnFace image and the Projection image. Furthermore, instead of the EnFace image, a fundus image acquired by another device, such as a fundus photograph or an SLO, may be used. In this case, it is preferable to align the fundus photograph or SLO before deformation with the thickness map in advance, and then apply a similar deformation process to the thickness map in accordance with the deformation of the fundus photograph or SLO.

[0054] Furthermore, the object to be deformed is not limited to a two-dimensional image; three-dimensional OCT data may also be deformed. The two-dimensional images (e.g., EnFace images, Projection images, and thickness maps) described in this embodiment are images generated by projecting information in an OCT A-scan in the Z direction. Therefore, if coordinate transformations of each XY in a two-dimensional image can be calculated, coordinate transformation of an A-scan in three-dimensional OCT is also possible. Deformed three-dimensional OCT data is generated by changing the XY coordinates of the A-scan. Then, any deformed two-dimensional image (EnFace image, Projection image, and thickness map) can be generated using the deformed three-dimensional OCT data.

[0055] [Step S23: Image Analysis] In step S23, the control unit 102 analyzes the fundus image using the fundus image and thickness map deformed in step S22. Specifically, the control unit 102 superimposes grids defining multiple evaluation regions in the fundus image and evaluation indices on the deformed fundus image 300 (a4). As an example of a grid, FIG. 8 shows a square grid 801 and a concentric grid 802 for the right eye. The average or median value of the thickness map within each evaluation region (e.g., A1, B1) included in each grid is assigned as an evaluation index. For convenience, the evaluation regions are assigned symbols. For example, in a square grid 801 with four rows and five columns, rows A, B, C, and D are arranged from top to bottom, and columns 1 to 5 are arranged from the ear side to the left, with symbols A1 to E5 assigned to each evaluation region. Furthermore, the inner and outer concentric grids 802 are divided into four sections, with the inner grid being I and the outer grid being E, with the upper ear side being 1, the upper nose side being 2, the lower ear side being 3, and the lower nose side being 4, and the evaluation regions being assigned symbols I1 to I4 and E1 to E4. Meanwhile, the symbols of the square grid 803 for the left eye and the concentric grid 804 for the left eye are in a left-right reverse relationship with the symbols of the grids for the right eye. The mean and median values ​​of the thickness map within the evaluation region are examples of evaluation indices generated using fundus thickness information. The control unit 102 is also an example of an image generating unit that generates a fundus image on which the grid and evaluation indices are superimposed. The evaluation indices may also be the standard deviation or variance of the thickness map.

[0056] FIG. 13 shows fundus images 1301 and 1302 in which a square grid 801 and a concentric circular grid 802 are superimposed on the fundus image 300 (a4) in FIG. 5 . The fundus images 1301 and 1302 are examples of a second fundus image. Consider the symmetry of each evaluation region. In the case of the square grid 801 in the fundus image 1301, there is a tendency for symmetry in the evaluation regions in the same column of rows A and D (for example, there is a tendency for symmetry between A1 and D1). There is also a tendency for symmetry in the evaluation regions in the same column of rows B and C (for example, there is a tendency for symmetry between B1 and C1). In the case of the concentric circular grid 802 in the fundus image 1302, there is a tendency for symmetry between I1 and I3. There is also a tendency for symmetry between I2 and I4. There is also a tendency for symmetry between E1 and E3. There is also a tendency for symmetry between E2 and E4. Therefore, the examiner evaluates the symmetry of the fundus image by comparing grids that have a symmetrical correspondence. Various methods for evaluating symmetry are conceivable. For example, one method is to normalize the values ​​of each evaluation region by the average value and subtract the value (thickness) of the evaluation region located below (e.g., D1) from the value (thickness) of the evaluation region located above (e.g., A1) in the symmetrical evaluation regions to calculate the difference information between the thickness of the evaluation region located above and the thickness of the evaluation region located below. If the calculated difference information is R and the reference value is N, then if R<-N (group 1), the upper region is thinner; if -N≦R≦N (group 2), there is no difference between the upper and lower regions (symmetric); and if N<R (group 3), the lower region is thinner. Furthermore, if there are many evaluation regions in group 1 or group 3, it can be evaluated that the symmetry is broken. However, the method for evaluating symmetry is not limited to this; it is also possible to use the sum of squares of the differences or weight each grid. Furthermore, the reference value N may be set manually or automatically based on the average value of each evaluation region. Furthermore, the calculated difference information R or the information of groups 1 to 3 classified using the reference value N may be further superimposed on the fundus image 1301 or the fundus image 1302. The fundus image 1301 or the fundus image 1302 on which these are superimposed is an example of a third fundus image.

[0057] Furthermore, although this example describes an example in which symmetry is determined using a grid and a thickness map, symmetry may also be analyzed by assigning other analytical values ​​to the map, such as a difference map obtained by comparison with a normal eye database or a blood vessel density map obtained from an OCT Angiography image. Furthermore, when comparing the deformed fundus image with another fundus image or a normal eye database, it is advisable to apply the same deformation process as in step S22 to the fundus image or normal eye database to be compared before comparison. Of course, a difference map or the like may be generated before deforming the fundus image in step S22, and then the difference map may be deformed in step S22.

[0058] [Step S3: Display] In step S3, the control unit 102 outputs the fundus image generated in step S23 to the display control unit 103. The image to be output may be the fundus image generated in step S23, or an image generated by arbitrarily combining a fundus image obtained by deformation, a thickness map, a grid, analysis results, and the like. When the fundus image is displayed on a monitor, the fundus image and thickness map before deformation and the fundus image and thickness map after deformation may be displayed side by side. Alternatively, the fundus image and thickness map before deformation and the fundus image and thickness map after deformation may be displayed by switching between them. Furthermore, when displaying fundus images of left and right eyes or multiple identical eyes side by side or by switching between them for comparison or follow-up observation, it is preferable to apply the same deformation processing to all fundus images before and after deformation. Furthermore, the fundus image may be output to a monitor, a printer, or the like. Alternatively, it may be output to the storage unit 105 or an external storage medium (not shown).

[0059] For example, FIG. 17 shows an example of a GUI for displaying fundus images of the left and right eyes on a monitor 1700. First, a deformed fundus image 1701(c) is generated and displayed by manually specifying the coordinates on the screen of the macular region 301(a) and the optic optic disc 302(a) for a fundus image 1701(a) of the right eye. Similarly, a deformed fundus image 1701(d) is generated and displayed by manually specifying the coordinates on the screen of the macular region 301(b) and the optic optic disc 301(b) for a fundus image 1701(b) of the left eye. Furthermore, when specifying the macular region 301 and the optic disc 302, the macular region 301 and the optic disc 302 may be automatically detected and specified by pressing an automatic selection button (not shown). Furthermore, information on the macular region 301 and the optic disc 302 specified by an operator for the fundus image of one of the left and right eyes may be used to automatically detect and specify the macular region 301 and the optic disc 302 for the other fundus image. For example, the macular region 301(b) and the optic ...

[0060] Furthermore, square maps 801 and 803 are superimposed on the deformed fundus images 1701(c) and 1701(d), respectively. In each of the square grids 801 and 803, symmetry may be evaluated by comparing corresponding rows A and D, and corresponding rows B and C, respectively, and the results may be displayed. Furthermore, in the square grids 801 and 803, grids with the same reference numerals may be compared to evaluate the symmetry of the fundus images of the left and right eyes, and the results may be displayed. The method for evaluating the symmetry of the fundus images of the left and right eyes may be the same as the evaluation method used for one eye, or a different evaluation method. Furthermore, concentric grids 802 and 804 may be used instead of the square grids 801 and 803.

[0061] 18 also shows an example of a GUI that deforms fundus images of a subject's eye captured at different times for follow-up observation and displays the analysis results on the monitor 1700. Fundus images captured at different times refer to, for example, a fundus image captured today, a fundus image captured one year ago, and a fundus image captured two years ago. Fundus images 1801(a) to (d) and corresponding deformed fundus images 1801(e) to (h) are generated and displayed on the monitor 1700. Each fundus image may be displayed individually or alternately. Furthermore, if the values ​​for deformation (such as the position of the macula 301 and the optic disc 302) are changed for any of the fundus images 1801(a) to (d), the same values ​​may be applied to the other fundus images, thereby deforming all of the fundus images together. For example, if the position of the macular region 301 and the position of the optic optic papilla 302 in the fundus image 1801(a) are changed by the operator, the positions of the macular region 301 and the optic optic papilla 302 in the other fundus images (b) to (h) may also be changed in the same way. In this case, the positions of the macular region 301 and the optic optic papilla 302 in the fundus images (b) to (h) may be obtained by image analysis of the fundus images. For example, based on the feature amount of the macular region 301 in 1801(a), a position having a similar feature amount in each of the fundus images (b) to (h) may be identified as the macular region 301 in each of the fundus images. Furthermore, deformation may be performed based on individually set values ​​for each fundus image. Furthermore, when comparing thickness difference information during follow-up observation, a comparison result of the thickness difference information for each of the fundus images 1801(a) to (d) before deformation may be displayed, or a comparison result of the thickness difference information for each of the deformed fundus images 1801(e) to (h) may be displayed.

[0062] Second Embodiment In the first embodiment, a case was described in which, of the regions divided by the straight lines 304, 305, and 401 as shown in Fig. 5(a3), the temporal regions 403 and 404 were not deformed, and the nasal regions 501 and 502 were deformed. In the first embodiment, the straight line 401 was set as a line perpendicular to the straight line 304, under the assumption that the region on the temporal side of the macular region 301 was symmetric. However, in actual fundus images, the region on the temporal side of the straight line 401 is not necessarily symmetric due to factors such as individual differences.

[0063] Therefore, in this embodiment, the problem is solved by partially modifying the first embodiment. The information processing device according to this embodiment has the same configuration as in FIG. 1, and therefore the same reference numerals will be used and a description thereof will be omitted. Furthermore, since the information processing device according to this embodiment performs the same process as in the flowchart of FIG. 2, the same reference numerals will be used and a description thereof will be given. Below, the control unit 102 according to this embodiment will be described, focusing on the differences from the first embodiment.

[0064] [Step S22: Thickness Map Deformation] In step S22, the input fundus image is deformed. A specific method will be described with reference to FIGS. 9A and 9B . A line 304 is approximately horizontal to the macular region 301 of the fundus image 300 in FIG. 9A , and a line 305 connects the macular region 301 and the optic optic disc 302. Lines 901 and 902 are extended from the macular region 301, with angles γ1 and γ2 relative to the line 304. In the regions divided by the lines 304, 305, 901, and 902, the temporal superior region is defined as region 403, the temporal inferior region is defined as region 404, the nasal superior region is defined as region 501, and the nasal inferior region is defined as region 502. Here, the angle of the corner of the macular region 301 in region 501 is defined as α1, the angle of the corner of the macular region 301 in region 502 is defined as β1, and the angle of the corner of the macular region 301 in the combined region of regions 403 and 404 is defined as γ = γ1 + γ2. In this case, similarly to the first embodiment, the regions 403 and 404 of the fundus image 300 are not deformed, but the straight line 305 is deformed so as to coincide with the dashed line portion of the straight line 304, thereby deforming the regions 501 and 502. The result after the deformation is shown in Fig. 9B. The straight line 901 is an example of the third straight line, and the straight line 902 is an example of the fourth straight line.

[0065] The arbitrary angles γ1 and γ2 may be set manually, or may be automatically set so as to maximize the index of symmetry relative to the line 304. The index of symmetry may be calculated, for example, from the difference between thickness maps at the same distance in the perpendicular direction relative to a reference line. As shown in FIG. 10 , the arbitrary angles γ1 and γ2 may be zero. As shown in FIG. 11 , a line 1101 having an arbitrary inclination from the macular region 301 may be used instead of the line 304. The line 1101 may be set manually, or may be automatically set so as to maximize the index of symmetry relative to the line 1101. The temporal side of the line 1101 relative to the macular region 301 is indicated by a solid line, and the nasal side is indicated by a dashed line. However, by deforming the line 305 so that it coincides with the dashed line portion of the line 1101, the regions 501 and 502 can be deformed. Alternatively, the entire fundus image 300 may be rotated in advance so that the straight line 1101 becomes substantially horizontal (matches the straight line 304), and then the image may be deformed.

[0066] Furthermore, when the values ​​of γ1 and γ2 are different, the ear-side regions 403 and 404 may be deformed by rotating the line 1101 so that γ1 = γ2 = γ / 2, or the regions 403, 404, 501, and 502 may be deformed by rotating the lines 901 and 902. When rotating the lines 901 and 902, the nose-side regions 501 and 502 may not be deformed except in the vicinity of the lines 901 and 902, or the ear-side regions 403 and 404 may be left undeformed while the nose-side regions 501 and 502 are compressed (reduced) or stretched (expanded) in the direction of rotation. Furthermore, the ear-side regions 403 and 404 and some or all of the nose-side regions 501 and 502 may be deformed by compressing (reduced) or stretching (expanded) in the direction of rotation. Furthermore, it is not necessary that γ1=γ2; the transformation process may be applied even if γ1=γ2 before transformation, and γ1≠γ2 after transformation may also be acceptable.

[0067] 11 do not necessarily pass through the macular region 301 and the optic disc 302, and any position may be specified on the fundus image or by coordinates, etc. The angles formed by the lines may also be freely set.

[0068] Furthermore, when this processing is applied to fundus images of the left and right eyes, the arbitrary angles γ1 and γ2 and the line 1101 may be set to individual values ​​for each fundus image, or may be synchronized between the left and right eyes. Furthermore, when displaying fundus images captured with different devices or multiple fundus images during follow-up observation, changing the arbitrary angles α1, β1, γ1, and γ2 and the lines 305, 901, 902, and 1101 for one fundus image may also change the other fundus images in synchronization. Furthermore, when the examiner changes the type of map to be displayed, such as by switching the layer targeted by the thickness map, the arbitrary angles and lines may retain the same values ​​as before the map change, or different values ​​may be stored for each type of map. Furthermore, values ​​applied to one fundus image may be transformed by applying a common value to another fundus image. Furthermore, these images may be arbitrarily combined, arranged, switched, or superimposed for display.

[0069] In the first and second embodiments, a method for superimposing a square grid 801 or a concentric grid 802 on a deformed fundus image or thickness map has been described. In this embodiment, a method for superimposing a grid on an undeformed fundus image or thickness map will be described by partially modifying the first and second embodiments.

[0070] The information processing device according to this embodiment has the same configuration as in Fig. 1, and therefore the same reference numerals will be used and a description thereof will be omitted. Also, the information processing device according to this embodiment performs the same process as in Fig. 2, and therefore the same reference numerals will be used and a description thereof will be given. The control unit 102 according to this embodiment will be described below, focusing on the differences from the first embodiment.

[0071] In FIG. 16, step S24 (thickness map inverse transformation) is added after step S23 in FIG.

[0072] [Step S24: Thickness Map Inverse Transformation] In step S24, the thickness map obtained by transformation in step S22 is inversely transformed to return to its pre-transformation state. At this time, the results of the analysis in step S23 are also inversely transformed. Specifically, fundus images 1301 and 1302 with grids superimposed thereon shown in Fig. 13 are transformed by inverse transformation into fundus images 1401 and 1402 shown in Fig. 14, and superimposed grids 801 and 802 are transformed into grids 801b and 802b.

[0073] In this embodiment, for the sake of convenience, a method has been described in which a deformed fundus image is returned to its original state by inverse deformation. However, the fundus image may not be deformed, and the deformed grids 801b and 802b may be directly superimposed on the fundus image before deformation.

[0074] In the first and second embodiments, the straight lines 304 and 305 are set as straight lines, and deformation is performed by dividing the image into ear-side and nose-side regions, as shown in Fig. 3. However, as shown in Fig. 15A, a fundus image may have symmetry with respect to a curve 1501 rather than a straight line. In such cases, dividing the image into ear-side and nose-side regions alone may not be sufficient.

[0075] Therefore, in this embodiment, the problem is solved by partially modifying the first and second embodiments. The information processing device according to this embodiment has the same configuration as in FIG. 1, so the same reference numerals will be used and a description thereof will be omitted. Furthermore, since the information processing device according to this embodiment performs the same process as in the flowchart of FIG. 2, the same reference numerals will be used and a description thereof will be given. Below, the control unit 102 according to this embodiment will be described, focusing on the differences from the first embodiment.

[0076] [Step S22: Thickness Map Deformation] In step S22, the input fundus image is deformed. A specific method will be described with reference to Figures 15A and 15B. Figures 15A and 15B show an example in which a fundus image 300b is generated by deforming a curve 1501 in the fundus image 300 so that the curve 1501 becomes a straight line 1501b.

[0077] 15A shows an auxiliary line 1502 that intersects the curve 1501 at a given point on the curve 1501. When the curve 1501 is transformed into a line 1501b, the auxiliary line 1502 is transformed into an auxiliary line 1502b that intersects the line 1501b at a given point on the curve 1501. Similarly, by transforming an arbitrary point on the auxiliary line 1502 onto the line 1502b, the blood vessel 303 is transformed into a blood vessel 303b.

[0078] Here, the arbitrary curve 1501 may be set manually. Alternatively, the curve 1501 may be set automatically so that the index of symmetry with respect to the transformed straight line 1501b is maximized. In addition, in the present embodiment, a method for transforming a fundus image by setting an auxiliary straight line 1502 has been described, but a method for dividing the fundus image into a plurality of regions shown in FIG. 15A and transforming each region may also be used.

[0079] Although the preferred embodiments of the present disclosure have been described above, it goes without saying that the present disclosure is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present disclosure.

[0080] (Other Embodiments) The disclosed technology can also be realized by executing the following process. That is, the disclosed technology can also be realized by providing software (programs) that realize one or more functions of the various embodiments described above to a system or device via a network or a storage medium, and having a computer (or a CPU, MPU, etc.) of the system or device read and execute the programs. The computer has one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions. In this case, the processor or circuit may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or a field-programmable gateway (FPGA). The processor or circuit may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

[0081] The disclosure of this embodiment includes the following configuration, method, and program.

[0082] (Configuration 1) An information processing device including a deformation means for deforming a first region and a second region in a fundus image defined by the first straight line and the second straight line, the first region being larger than the first region, so that an angle formed by a first straight line passing through a first feature related to the macula in the fundus image and located on the ear side, and a second straight line passing through the first feature and a second feature related to the papilla in the fundus image and located on the nose side, approaches 180 degrees.

[0083] (Configuration 2) The information processing device according to Configuration 1, wherein the deformation means deforms the first region and the second region by rotating the second line around the first characteristic portion as a fulcrum so that an angle between the first line and the second line approaches 180 degrees.

[0084] (Configuration 3) The information processing device according to configuration 1 or 2, wherein the deformation means deforms the first area and the second area by expanding the first area and shrinking the second area so that the angle between the first straight line and the second straight line approaches 180 degrees.

[0085] (Configuration 4) The information processing device according to any one of configurations 1 to 3, wherein the deformation means deforms, in the first region, a region on the ear side of a third straight line that passes through the first characteristic portion and is located in the first region, but deforms a region on the nose side of the third straight line, and in the second region, deforms, in the second region, a region on the ear side of a fourth straight line that passes through the first characteristic portion and is located in the second region, but deforms a region on the nose side of the fourth straight line.

[0086] (Configuration 5) An information processing device according to any one of configurations 1 to 4, wherein the deformation means deforms, in the first region, a region on the ear side of a third straight line that passes through the first characteristic portion and is located in the first region, without deforming a region on the nose side of the third straight line, and in the second region, deforms a region on the ear side of the fourth straight line that passes through the first characteristic portion and is located in the second region, without deforming a region on the nose side of the fourth straight line.

[0087] (Configuration 6) An information processing device described in any one of configurations 1 to 5, further comprising an image generation means for generating a second fundus image in which a grid defining a plurality of evaluation areas and an evaluation index in the evaluation areas are superimposed on the fundus image obtained by deformation by the deformation means.

[0088] (Configuration 7) An information processing device according to any one of configurations 1 to 6, wherein the deformation means inversely deforms the second fundus image so that the angle between the first straight line and the second straight line in the second fundus image approaches the angle between the first straight line and the second straight line in the fundus image.

[0089] (Configuration 8) The information processing device according to Configuration 6, wherein the evaluation index is at least one of the mean value, median value, standard deviation, and variance of the thickness information of the fundus included in the evaluation region.

[0090] (Configuration 9) The information processing device described in Configuration 6, wherein the image generating means generates a third fundus image in which difference information of the evaluation index of an evaluation area that is symmetrical with respect to the first straight line and difference information of the evaluation index of an evaluation area that is symmetrical with respect to the second straight line are further superimposed.

[0091] (Configuration 10) The information processing device according to any one of configurations 1 to 9, wherein the fundus image is a thickness map.

[0092] (Configuration 11) The information processing device according to any one of Configurations 1 to 10, wherein at least one of the first feature part and the second feature part is designated by an instruction from an operator or is designated based on an analysis result of the fundus image.

[0093] (Configuration 12) An information processing device described in any of configurations 1 to 11, in which, when at least one of the first feature portion and the second feature portion is specified by an instruction from an operator in a fundus image of one of the right eye or left eye of a subject, the specified information is used to specify at least one of the first feature portion and the second feature portion in the other fundus image of the subject.

[0094] (Configuration 13) The information processing device according to any one of configurations 1 to 12, further comprising a display control unit that controls the display unit to display at least one of the fundus image and a fundus image obtained by deformation by the deformation means.

[0095] (Configuration 14) The information processing device according to Configuration 13, wherein the display control unit controls the display unit to display a fundus image of the right eye obtained by deformation by the deformation means and a fundus image of the left eye obtained by deformation by the deformation means side by side.

[0096] (Configuration 15) The information processing device according to Configuration 13, wherein the display control unit controls the display unit to display, side by side, a plurality of fundus images obtained by photographing the subject's eye at different times and a plurality of fundus images obtained by deforming the plurality of fundus images using the deformation means.

[0097] (Configuration 16) An information processing device according to configuration 14 or 15, wherein when at least one of the first feature portion and the second feature portion is specified by an instruction from an operator in any one of the fundus images displayed on the display unit, the deformation means uses the specified information to specify at least one of the first feature portion and the second feature portion in another fundus image.

[0098] (Configuration 17) An information processing system comprising: an imaging device that captures the fundus image; and the information processing device according to any one of configurations 1 to 16, which is communicably connected to the imaging device.

[0099] (Method 1) A deformation method for deforming a first region and a second region in a fundus image, the first region and the second region being larger than the first region, so that an angle formed by a first straight line passing through a first feature related to the macula in the fundus image and located on the temporal side, and a second straight line passing through the first feature and a second feature related to the papilla in the fundus image and located on the nasal side, approaches 180 degrees.

[0100] (Program 1) A program that causes a computer to execute the transformation method described in Method 1.

[0101] The present invention is not limited to the above-described embodiments, and various modifications and variations can be made without departing from the spirit and scope of the present invention. Therefore, the following claims are appended to apprise the public of the scope of the present invention.

[0102] This application claims priority based on Japanese Patent Application No. 2023-078905, filed on May 11, 2023, the entire contents of which are incorporated herein by reference.

[0103] 101 Input unit 102 Control unit 103 Display control unit 104 Operation unit 105 Storage unit

Claims

1. An information processing device comprising: a deformation means for deforming a first region and a second region in a fundus image, the first region and the second region being defined by the first straight line and the second straight line, so that an angle formed by a first straight line passing through a first characteristic feature relating to the macula in the fundus image and located on the ear side, and a second straight line passing through the first characteristic feature and a second characteristic feature relating to the optic disc in the fundus image and located on the nasal side, approaches 180 degrees.

2. An information processing device as described in claim 1, wherein the transformation means transforms the first area and the second area by rotating the second straight line around the first characteristic portion as a fulcrum so that the angle between the first straight line and the second straight line approaches 180 degrees.

3. An information processing device as described in claim 1, wherein the transformation means transforms the first area and the second area by expanding the first area and shrinking the second area so that the angle between the first straight line and the second straight line approaches 180 degrees.

4. An information processing device as described in claim 1, wherein the deformation means deforms, in the first region, a region on the ear side of a third straight line that passes through the first characteristic part and is located in the first region, but deforms a region on the nose side of the third straight line, and in the second region, deforms, in the second region, a region on the ear side of a fourth straight line that passes through the first characteristic part and is located in the second region, but deforms a region on the nose side of the fourth straight line, without deforming a region on the ear side of the fourth straight line.

5. An information processing device as described in claim 1, wherein the deformation means deforms, in the first region, a region on the ear side of a third straight line that passes through the first characteristic portion and is located in the first region, without deforming a region on the nose side of the third straight line, and in the second region, deforms a region on the ear side of the fourth straight line that passes through the first characteristic portion and is located in the second region, without deforming a region on the nose side of the fourth straight line.

6. An information processing device as described in claim 1, further comprising an image generating means for generating a second fundus image in which a grid defining a plurality of evaluation areas and an evaluation index in the evaluation areas are superimposed on the fundus image obtained by deformation by the deformation means.

7. An information processing device as described in claim 6, wherein the transformation means inversely transforms the second fundus image so that the angle between the first straight line and the second straight line in the second fundus image approaches the angle between the first straight line and the second straight line in the fundus image.

8. An information processing device according to claim 6, wherein the evaluation index is at least one of the mean, median, standard deviation, and variance of the thickness information of the fundus included in the evaluation region.

9. An information processing device as described in claim 6, wherein the image generating means generates a third fundus image in which difference information of the evaluation index of an evaluation area that is symmetrical with respect to the first straight line and difference information of the evaluation index of an evaluation area that is symmetrical with respect to the second straight line are further superimposed.

10. An information processing apparatus according to claim 1, wherein the fundus image is a thickness map.

11. An information processing device according to claim 1, wherein at least one of the first feature portion and the second feature portion is specified by an instruction from an operator, or is specified based on the analysis result of the fundus image.

12. An information processing device as described in claim 1, which, when at least one of the first feature portion and the second feature portion is specified by an instruction from an operator in a fundus image of one of the right eye or left eye of a subject, uses the specified information to specify at least one of the first feature portion and the second feature portion in the other fundus image of the subject.

13. An information processing device according to claim 1, further comprising a display control unit that controls the display unit to display at least one of the fundus image and a fundus image obtained by deformation by the deformation means.

14. An information processing device according to claim 13, wherein the display control unit controls the display unit so as to display a fundus image of the right eye obtained by deformation using the deformation means and a fundus image of the left eye obtained by deformation using the deformation means side by side.

15. An information processing device as described in claim 13, wherein the display control unit controls the display unit to display side by side a plurality of fundus images obtained by photographing the subject's eye at different times and a plurality of fundus images obtained by deforming the plurality of fundus images by the deformation means.

16. An information processing device as described in claim 14 or 15, wherein when at least one of the first feature portion and the second feature portion is specified by an instruction from an operator in any one of the fundus images displayed on the display unit, the deformation means uses the specified information to specify at least one of the first feature portion and the second feature portion in another fundus image.

17. A method for deforming a first region and a second region in a fundus image, the first region and the second region being defined by the first line and the second line, so that an angle formed by a first line passing through a first characteristic part related to the macula in the fundus image and located on the temporal side, and a second line passing through the first characteristic part and a second characteristic part related to the optic disc in the fundus image and located on the nasal side, approaches 180 degrees.

18. A program for causing a computer to execute the transformation method according to claim 17.

19. An information processing system comprising: an imaging device that captures the fundus image; and an information processing device according to any one of claims 1 to 16 that is communicatively connected to the imaging device.

Citation Information

Patent Citations

  • Image processing device, and control method and program of image processing device

    JP2011110158A

  • Image processing device, estimation method, system and program

    JP2017127397A

  • Ocular fundus image analysis device, ocular fundus image analysis method, and program

    JP2017196041A

  • Ophthalmology imaging apparatus and imaging control program

    JP2018083106A