Ophthalmologic image processing device and ophthalmologic image processing program

Abstract

To provide an ophthalmologic image processing device and an ophthalmologic image processing program capable of appropriately processing data on a tomographic image of a tissue of an eye to be examined.SOLUTION: In an image acquisition step, a control part acquires an ophthalmologic image 62 captured by an ophthalmologic image capturing device. In an extraction step, the control part extracts data on an image in a target range for storing data from each of a plurality of partial images 70 arranged in a predetermined direction in the ophthalmologic image 62. In a position information acquisition step, the control part acquires position information indicating each position of a plurality of extraction images 80 extracted from each of the plurality of partial images 70 in the extraction step. In a storage step, the control part causes storage means to store the data on the plurality of extraction images 80 and the position information on each of the plurality of extraction images 80.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to an ophthalmic image processing device that processes data of ophthalmic images, which are tomographic images of tissues of a subject's eye, and an ophthalmic image processing program executed in the ophthalmic image processing device. [Background technology]

[0002] There are known techniques for capturing tomographic images of tissues of a subject's eye. For example, the OCT device disclosed in Patent Document 1 captures two-dimensional tomographic images of cross sections passing through a scanning line by scanning a measurement light along the scanning line. Furthermore, a three-dimensional tomographic image is acquired by arranging a plurality of two-dimensional tomographic images captured along a plurality of scanning lines. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-150532 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, captured tomographic image data is stored in a storage device as is. As a result, problems such as a storage capacity shortage in the storage device and a decrease in the processing speed when processing the stored data are likely to occur. In particular, in recent years, the capture of tomographic images has become more sophisticated, with a wider angle of view, greater depth, and faster speed, and the frequency with which tomographic images are captured and used for diagnosis and the like has also increased. Therefore, reducing the amount of tomographic image data stored in a storage device has become an important issue.

[0005] A typical object of the present disclosure is to provide an ophthalmic image processing device and an ophthalmic image processing program capable of appropriately processing data of a tomographic image of tissue of a subject's eye. [Means for solving the problem]

[0006] An ophthalmic image processing device provided by a typical embodiment of the present disclosure is an ophthalmic image processing device that processes data of ophthalmic images, which are tomographic images of the tissues of a test eye, and a control unit of the ophthalmic image processing device executes an image acquisition step of acquiring an ophthalmic image captured by an ophthalmic image capturing device, an extraction step of extracting image data within a target range in which data is to be saved from each of a plurality of partial images arranged in a predetermined direction within the ophthalmic image, a position information acquisition step of acquiring position information indicating the position of each of a plurality of extracted images extracted from each of the plurality of partial images in the extraction step, and a storage step of storing the data of the plurality of extracted images and the position information of each of the plurality of extracted images in a storage means.

[0007] An ophthalmic image processing program provided by a typical embodiment of the present disclosure is an ophthalmic image processing program executed by an ophthalmic image processing device that processes data of ophthalmic images, which are tomographic images of the tissues of a test eye.The ophthalmic image processing program is executed by a control unit of the ophthalmic image processing device, causing the ophthalmic image processing device to execute the following steps: an image acquisition step that acquires an ophthalmic image captured by an ophthalmic image capturing device; an extraction step that extracts image data within a target range in which data is to be saved from each of a plurality of partial images arranged in a predetermined direction within the ophthalmic image; a position information acquisition step that acquires position information indicating the position of each of a plurality of extracted images extracted from each of the plurality of partial images in the extraction step; and a storage step that stores the data of the plurality of extracted images and the position information of each of the plurality of extracted images in a memory means.

[0008] According to the ophthalmological image processing device and ophthalmological image processing program of the present disclosure, data of a tomographic image of tissue of a subject's eye is appropriately processed. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of an ophthalmologic image processing system 100. FIG. [Figure 2]1 is an explanatory diagram for explaining a method in which the ophthalmologic image capturing device 1 captures an ophthalmologic image of tissue 50 of a subject's eye. [Figure 3] 1 is a diagram showing an example of an ophthalmologic image 61 captured by the ophthalmologic image capturing device 1. FIG. [Figure 4] 10 is a flowchart showing an example of ophthalmologic image processing executed by the ophthalmologic image processing device 40. [Figure 5] 1 is a diagram showing a captured ophthalmological image 61 and an ophthalmological image 62 obtained by extracting an image region from the ophthalmological image 61. FIG. [Figure 6] 8 is an explanatory diagram for explaining an example of a method for extracting an image 80 extracted from a partial image 70 in an ophthalmologic image 62. [Figure 7] FIG. 10 is a diagram showing a reference image 80A in which a plurality of extracted images 80 extracted from each of a plurality of partial images 70 are arranged in the X direction without taking into consideration the position of each image. [Figure 8] 10A and 10B are diagrams showing examples of reconstructed image data 90 and background-complemented image data 91. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Summary> The ophthalmic image processing device exemplified in the present disclosure processes data of ophthalmic images, which are tomographic images of tissues of a subject's eye. A control unit of the ophthalmic image processing device executes an image acquisition step, an extraction step, a position information acquisition step, and a storage step. In the image acquisition step, the control unit acquires an ophthalmic image captured by an ophthalmic image capturing device. In the extraction step, the control unit extracts image data within a target range for saving data from each of multiple partial images arranged in a predetermined direction within the ophthalmic image. In the position information acquisition step, the control unit acquires position information indicating the position of each of the multiple extracted images extracted from each of the multiple partial images in the extraction step. In the storage step, the control unit stores the data of the multiple extracted images and the position information of each of the multiple extracted images in a storage means.

[0011] According to the technology disclosed herein, an extracted image, which is an image within a target range for saving data, is extracted from each of a plurality of partial images constituting an ophthalmic image. Furthermore, position information for each extracted image is saved along with the data for the plurality of extracted images. That is, image data for an appropriate range of the ophthalmic image is extracted and saved. Furthermore, if each extracted image is positioned based on the position information, the image of the extracted portion is appropriately reconstructed. Therefore, compared to saving the entire data of the ophthalmic image, the storage capacity of the storage device is appropriately reduced.

[0012] Various devices can be used as an ophthalmic imaging apparatus for capturing (generating) a tomographic image. For example, an OCT apparatus can be used that captures a tomographic image of the tissue of a subject's eye using the principle of optical coherence tomography. In this case, the tomographic image may be, for example, a motion contrast image (e.g., an OCT angiography image) obtained by acquiring multiple OCT signals from the same position of the retinal layer of the fundus at different times. Also, an MRI (magnetic resonance imaging) apparatus, a CT (computed tomography) apparatus, or the like may be used as the ophthalmic imaging apparatus. The tomographic image acquired in the image acquisition step may be a two-dimensional tomographic image or a three-dimensional tomographic image. Furthermore, the tomographic image in the present disclosure is a tomographic image of the fundus of the subject's eye. However, the tomographic image may also be a tomographic image of tissue other than the fundus of the subject's eye (e.g., the anterior segment of the eye).

[0013] Various devices can function as an ophthalmic image processing device. For example, an ophthalmic imaging device that captures tomographic images may itself function as the ophthalmic image processing device of the present disclosure. In this case, the ophthalmic imaging device can capture tomographic images of the subject's eye while appropriately storing the captured tomographic images in a storage device. Also, a device (e.g., a personal computer (PC) or a mobile terminal) that can exchange data with the ophthalmic imaging device may function as the ophthalmic image processing device. Multiple control units (e.g., a control unit of the ophthalmic imaging device and a control unit of the PC) may cooperate to perform processing.

[0014] The control unit may further execute a reference site identification step of identifying at least one of multiple sites within the tissue shown in the ophthalmologic image as a reference site. In the extraction step, the control unit may extract, as an extracted image, image data within a target range based on the reference site from each of multiple partial images arranged in a predetermined direction within the ophthalmologic image. In this case, image data within an appropriate range based on the identified reference site is extracted and saved. Thus, image data within unnecessary ranges is appropriately excluded, and image data within the necessary ranges is saved.

[0015] A method for identifying a reference site within tissue depicted in an ophthalmic image can be selected as appropriate. For example, the control unit may identify the reference site within the ophthalmic image using known image processing (e.g., edge detection, etc.). Alternatively, a mathematical model trained by a machine learning algorithm may be used to identify the reference site within the input ophthalmic image. In this case, the control unit may input the ophthalmic image to the mathematical model and obtain the reference site identification result output by the mathematical model, thereby identifying the reference site.

[0016] The ophthalmologic image may be a tomographic image of a layered structure of tissue of the subject's eye (e.g., the layered structure of the fundus). In the reference portion identification step, the control unit may identify at least one of the layers included in the layered structure and the layer boundaries (hereinafter also referred to as "layer boundaries") as the reference portion. The layer boundaries included in the layered structure tend to have a shape that follows the overall shape of the layered structure. Therefore, even if the layered structure is not flat (e.g., if the layered structure is curved or has deformation due to disease, etc.), extracting an extracted image using the layer boundaries as the reference portion makes it easier to save an image of an appropriate range according to the shape of the layered structure.

[0017] In the reference portion identification step, the control unit may identify at least the most superficial boundary or layer of the layer structure shown in the ophthalmic image as the reference portion. The most superficial layer of the layer structure is the boundary between an area where the layer structure exists and an area where the layer structure does not exist. Therefore, by using the most superficial layer / boundary of the layer structure as the reference portion, an extracted image including the layer structure can be more appropriately extracted. Note that when the tissue shown in the ophthalmic image is retinal tissue of the fundus, the most superficial layer / boundary is the retinal surface layer or the boundary of the retinal surface layer (e.g., the boundary on the surface side).

[0018] In the extraction step, the control unit may include a predetermined range from the reference portion on the most superficial side of the layer structure toward the deeper side of the layer structure in the target range for extracting an extracted image from each of the multiple partial images. In this case, the extracted image is likely to include as wide a range as possible of the layer structure. In addition, it is less likely that an image of the tissue on the superficial side of the layer structure will not be extracted. Therefore, an image of the layer structure of the subject's eye is more appropriately extracted and saved.

[0019] The size of the image extraction range extending from the reference site on the most superficial side of the layer structure toward the deeper layers (i.e., the aforementioned "size of the predetermined range toward the deeper layers") can be set appropriately. For example, the control unit may set the size of the image extraction range extending from the reference site toward the deeper layers (hereinafter, also referred to as the "offset amount toward the deeper layers") according to an instruction input by the user. In this case, the user can efficiently and appropriately store image data of a desired range of the layer structure of the subject's eye. Furthermore, the control unit may adjust the offset amount toward the deeper layers according to the imaging conditions of the ophthalmologic image (e.g., at least one of the imaging magnification, imaging angle of view, imaging depth, etc.). In this case, image data of an appropriate range according to the imaging conditions is extracted. Furthermore, the offset amount toward the deeper layers may be a predetermined fixed value.

[0020] In the extraction step, the control unit may include a predetermined range from the most superficial reference portion in the layer structure toward the side opposite the deeper side of the layer structure (hereinafter referred to as the "near side" for convenience) in the target range for extracting an extracted image from each of the multiple partial images. In this case, even if the accuracy of identifying the most superficial reference portion decreases, an image of the tissue on the superficial side of the layer structure becomes more likely to be properly extracted. Furthermore, even if an object is captured on the surface side of the boundary between the most superficial layer due to various influences (e.g., the influence of a disease, etc.), the object captured on the surface side also becomes more likely to be extracted. Therefore, images can be more easily extracted appropriately.

[0021] Similarly to the amount of offset toward the deeper layer, the size of the image extraction range from the most superficial reference site toward the near side (hereinafter, sometimes referred to as the "amount of offset toward the near side") can also be set appropriately. For example, the control unit may set the amount of offset toward the near side according to an instruction input by the user. The control unit may adjust the amount of offset toward the near side according to the photographing conditions of the ophthalmologic image (for example, at least one of the photographing magnification, the photographing angle of view, the photographing depth, etc.). Furthermore, the amount of offset toward the near side may be a predetermined fixed value.

[0022] It is also possible to use a layer or boundary located deeper than the most superficial layer or boundary among the layer structures shown in an ophthalmic image as the reference site. For example, if the tissue shown in an ophthalmic image is retinal tissue at the fundus, at least one of the layers or boundaries located deeper than the most superficial retinal layer (e.g., the ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, external limiting membrane, photoreceptor inner-outer segment junction, retinal pigment epithelium, Bruch's membrane, choroid, etc., and their boundaries) may be used as the reference site. For example, by using a layer or structure that is more clearly visible among the layer structures shown in an ophthalmic image as the reference site, the accuracy of identifying the reference site may be improved, and the image may be extracted and saved more appropriately.

[0023] Furthermore, among the layer structures shown in an ophthalmological image, the layer or boundary identified as the reference portion may be changed according to various conditions. For example, there are layers or boundaries whose visibility in an ophthalmological image varies depending on the position of the tissue. Therefore, the layer or boundary identified as the reference portion may be changed according to the position of the tissue shown in the ophthalmological image.

[0024] The control unit may also specify multiple layers / boundaries shown in the ophthalmic image as reference regions. In this case, in the extraction step, the control unit may extract an extracted image from each of the multiple partial images, using a range surrounded by one of the multiple specified reference regions (e.g., the most superficial layer / boundary) and another reference region (e.g., a layer / boundary located deeper in the layer structure) as a target range. As a result, image data within a specific range of the layer structure shown in the ophthalmic image is appropriately extracted and saved.

[0025] Each of the plurality of partial images may be an elongated image extending in a direction intersecting the layers in the layer structure depicted in the ophthalmic image. The plurality of partial images may be arranged consecutively in the ophthalmic image in a direction intersecting the direction in which each partial image extends. In this case, it becomes easier to identify the layer boundary serving as the reference portion in each partial image. As a result, the extracted image is more appropriately extracted and saved.

[0026] For example, if the ophthalmic image is an OCT image, the multiple partial images may be A-scan images extending in a direction along the optical axis of the OCT measurement light. The OCT measurement light is irradiated onto the tissue of the test eye (e.g., fundus tissue) from the front side. Therefore, the A-scan image extending in a direction along the optical axis of the OCT measurement light becomes an image (pixel row) extending in a direction intersecting the layers in the tissue. Therefore, by using multiple A-scan images constituting the ophthalmic image as partial images, it becomes easier to extract an extracted image including an appropriate region of the layer structure. Note that each A-scan image may be used as a partial image as is, or multiple A-scan images may be used as components of each partial image.

[0027] However, it is also possible to change the partial images. For example, if the ophthalmologic image is an OCT image, multiple pixel rows (so-called C-scan images) that intersect perpendicularly with the A-scan image may be used as the partial images. Alternatively, the direction in which at least one layer / boundary (e.g., a layer / boundary serving as a reference site) extends may be detected, and multiple partial images that intersect with the detected direction may be appropriately set. Furthermore, if the layer structure is curved, a set of multiple partial images may be arranged in a fan shape or the like so that each of the multiple partial images intersects with the curved shape of the layer structure.

[0028] Furthermore, among multiple regions shown in an ophthalmological image, a region other than a layer boundary may be used as the reference region. For example, a diseased region shown in an ophthalmological image may be specified as the reference region, and an extracted image may be extracted based on the specified reference region. In this case, an extracted image is appropriately extracted and saved according to the diseased region. Furthermore, a blood vessel region shown in an ophthalmological image may be specified as the reference region.

[0029] If the reference region identification step results in missing reference region identification results at certain locations, a supplementation step may be further performed to supplement the reference region identification results at those missing locations based on the reference region identification results at other locations. In this case, even if a portion of the reference region is not identified due to various factors (e.g., image quality or disease), an extracted image is extracted from multiple partial images based on the reference region identification results supplemented by the supplementation step. Therefore, images may be extracted and saved more appropriately.

[0030] The specific method of the interpolation step can also be selected as appropriate. For example, when there is a missing portion where the identification result is missing, the control unit may complement the identification result of the missing portion based on the identification result of the reference portion in a portion adjacent to the missing portion (for example, a partial image adjacent to the partial image including the missing portion). For example, the control unit may complement, as the position of the reference portion, a position between a specific position in a partial image adjacent to one side of the partial image including the missing portion and a specific position in a partial image adjacent to the other side.

[0031] It should be noted that instead of the method of extracting an extracted image from an ophthalmic image using a reference portion as a reference, it is also possible to extract an extracted image by other methods. For example, the control unit may perform image processing on the acquired ophthalmic image to detect an image area in the ophthalmic image that includes an image, and extract image data within a target range including the detected image area as the extracted image. The control unit may also extract image data of an area in the ophthalmic image that is likely to include an image (for example, a predetermined area in the ophthalmic image that is expected to be likely to include an image) as the extracted image.

[0032] An image (reference image) in which the multiple extracted images extracted in the extraction step are arranged in the arrangement direction of the multiple partial images without taking into account the positional information may be an image in which at least a portion of the area other than the layer structure is deleted from the ophthalmic image acquired in the image acquisition step and the curvature of the layer structure is flattened. In this case, in the reference image, by deleting at least a portion of the area other than the layer structure from the ophthalmic image, the amount of image data is appropriately reduced and the curvature information of the layers reproduced by the positional information is removed, thereby flattening the layer structure. Therefore, necessary data is appropriately extracted and saved from the data of the ophthalmic image.

[0033] The extracted images may be extracted from the ophthalmic images and stored while maintaining the thickness of the layer structure in the ophthalmic images. The extracted image data may be stored without compression or after lossless compression. In this case, if each extracted image is arranged based on the position information, the layer structure in the original ophthalmic images is appropriately reconstructed while maintaining the thickness.

[0034] The control unit may further execute an image reconstruction step of reconstructing the area of ​​the ophthalmic image extracted in the extraction step by arranging data of the plurality of extracted images stored in the storage means based on the position information of each of the extracted images. In this case, the extracted images stored with a data amount smaller than the data amount of the original ophthalmic image are appropriately reconstructed.

[0035] The control unit may store the differences between the data of multiple extracted images and the data of other extracted images, rather than storing the data of the multiple extracted images as is. For example, the control unit may reduce the amount of data for some extracted images by storing the differences between the data of some extracted images and nearby (e.g., adjacent) extracted images. When reconstructing the image, the control unit may combine the data of some extracted images based on the differences between the data of the nearby extracted images.

[0036] <Embodiment> A typical embodiment according to the present disclosure will be described below. In this embodiment, a case will be illustrated in which a tomographic image of the fundus tissue of the subject's eye E, captured by an OCT device, which is a type of ophthalmic imaging device, is processed. However, the ophthalmic image to be processed may be an image of tissue other than the fundus tissue. For example, the ophthalmic image to be processed may be an image of tissue other than the fundus of the subject's eye E (e.g., the anterior segment of the eye, etc.). Furthermore, an image of biological tissue other than the subject's eye E (e.g., the skin, digestive organs, or brain, etc.) (i.e., an image other than an ophthalmic image) may be processed. Furthermore, as described above, the imaging device for capturing the image is not limited to the OCT device.

[0037] The schematic configuration of an ophthalmic image processing system 100 according to this embodiment will be described with reference to FIG. 1. The ophthalmic image processing system 100 according to this embodiment includes an ophthalmic image capturing device 1 and an ophthalmic image processing device 40. The ophthalmic image capturing device 1 captures a tomographic image of a living body (the fundus of the subject's eye in this embodiment). Specifically, the ophthalmic image capturing device (OCT device) 1 according to this embodiment captures a two-dimensional image extending in a first direction (scanning direction) and a second direction (depth direction, which is the direction of the optical axis of the measurement light) intersecting the first direction by scanning the measurement light on each of multiple scan lines within a two-dimensional measurement area in the living body to capture multiple two-dimensional tomographic images. The ophthalmic image processing device 40 processes data of the ophthalmic image captured (captured) by the ophthalmic image capturing device 1.

[0038] The configuration of the ophthalmologic imaging device 1 of this embodiment will be described. The ophthalmologic imaging device (OCT device) 1 includes an OCT section 10 and a control unit 30. The OCT section 10 includes an OCT light source 11, a coupler (light splitter) 12, a measurement optical system 13, a reference optical system 20, and a light receiving element 22.

[0039] The OCT light source 11 emits light (OCT light) for acquiring ophthalmologic image data. The coupler 12 splits the OCT light emitted from the OCT light source 11 into measurement light and reference light. The coupler 12 of this embodiment also combines the measurement light reflected by tissue (the fundus of the subject's eye E in this embodiment) with the reference light generated by the reference optical system 20 to cause interference. That is, the coupler 12 of this embodiment serves both as a branching optical element that branches the OCT light into measurement light and reference light, and as a combining optical element that combines the reflected light of the measurement light with the reference light. Note that the configuration of at least one of the branching optical element and the combining optical element may be changed. For example, an element other than a coupler (e.g., a circulator, a beam splitter, etc.) may be used.

[0040] The measurement optical system 13 guides the measurement light split by the coupler 12 to the subject and returns the measurement light reflected by the tissue to the coupler 12. The measurement optical system 13 includes a scanning unit (scanner) 14, an irradiation optical system 16, and a focus adjustment unit 17. The scanning unit 14 is driven by a driving unit 15 to scan the spot-shaped measurement light in a two-dimensional direction intersecting the optical axis of the measurement light. In this embodiment, two galvanometer mirrors capable of deflecting the measurement light in different directions are used as the scanning unit 14. However, another device for deflecting light (e.g., at least one of a polygon mirror, a resonant scanner, an acousto-optical element, etc.) may also be used as the scanning unit 14. The irradiation optical system 16 is located downstream of the scanning unit 14 in the optical path (i.e., on the subject side) and irradiates the tissue with the measurement light. The focus adjustment unit 17 adjusts the focus of the measurement light by moving an optical member (e.g., a lens) included in the irradiation optical system 16 in a direction along the optical axis of the measurement light.

[0041] The reference optical system 20 generates reference light and returns it to the coupler 12. In this embodiment, the reference optical system 20 generates the reference light by reflecting the reference light split by the coupler 12 using a reflective optical system (e.g., a reference mirror). However, the configuration of the reference optical system 20 can also be changed. For example, the reference optical system 20 may transmit the light incident from the coupler 12 without reflecting it and return it to the coupler 12. The reference optical system 20 includes an optical path length difference adjustment unit 21 that changes the optical path length difference between the measurement light and the reference light. In this embodiment, the optical path length difference is changed by moving the reference mirror in the optical axis direction. Note that the configuration for changing the optical path length difference may be provided in the optical path of the measurement optical system 13.

[0042] The light receiving element 22 detects an interference signal by receiving interference light between the measurement light and the reference light generated by the coupler 12. In this embodiment, the principle of Fourier domain OCT is adopted. In Fourier domain OCT, the spectral intensity of the interference light (spectral interference signal) is detected by the light receiving element 22, and a complex OCT signal is acquired by Fourier transforming the spectral intensity data. Examples of Fourier domain OCT that can be adopted include spectral-domain OCT (SD-OCT) and swept-source OCT (SS-OCT). It is also possible to adopt, for example, time-domain OCT (TD-OCT).

[0043] The control unit 30 is responsible for various controls of the ophthalmologic image capturing device 1. The control unit 30 includes a CPU 31, a RAM 32, a ROM 33, and a non-volatile memory (NVM) 34. The CPU 31 is a controller that performs various controls. The RAM 32 temporarily stores various information. The ROM 33 stores programs executed by the CPU 31, various initial values, etc. The NVM 34 is a non-transitory storage medium that can retain its contents even when the power supply is cut off.

[0044] A monitor 37 and an operation unit 38 are connected to the control unit 30. The monitor 37 is an example of a display unit that displays various images. The operation unit 38 is operated by a user to input various operation instructions to the ophthalmologic image capturing device 1. The operation unit 38 may be implemented by various devices such as a mouse, a keyboard, a touch panel, or a foot switch. Note that various operation instructions may be input to the ophthalmologic image capturing device 1 by inputting sound into a microphone.

[0045] The following describes the schematic configuration of the ophthalmic image processing device 40. In this embodiment, a personal computer (hereinafter referred to as a "PC") is used as the ophthalmic image processing device 40. However, a device other than a PC may be used as the ophthalmic image processing device. For example, the ophthalmic image capturing device 1 itself may function as the ophthalmic image processing device that performs the ophthalmic image processing described below. The ophthalmic image processing device 40 includes a CPU 41, a RAM 42, a ROM 43, and an NVM 44. The CPU 41 is a controller that performs various controls. The RAM 42, the ROM 43, and the NVM 44 can store various pieces of information as described above. An ophthalmic image processing program for performing the ophthalmic image processing (see FIG. 4) described below may be stored in the NVM 44. A monitor 47 and an operation unit 48 are also connected to the ophthalmic image processing device 40. The monitor 47 is an example of a display unit that displays various images. The operation unit 48 is operated by a user to input various operation instructions to the ophthalmic image processing device 40. The operation unit 48 may use various devices such as a mouse, a keyboard, a touch panel, etc., similar to the operation unit 38 of the ophthalmologic image capturing device 1. Furthermore, various operation instructions may be input to the ophthalmologic image processing device 40 by inputting sound into the microphone 46.

[0046] The ophthalmic image processing device 40 can acquire various data (e.g., data of ophthalmic images captured by the ophthalmic image capturing device 1) from the ophthalmic image capturing device 1. The various data may be acquired, for example, by at least one of wired communication, wireless communication, and a removable storage device (e.g., a USB memory).

[0047] 2 and 3, a method for capturing an ophthalmic image to be processed by the ophthalmic image processing device 40 of this embodiment and an example of the configuration of the ophthalmic image will be described. Note that this embodiment illustrates a case where data of a two-dimensional tomographic image is processed and stored in a storage device (e.g., NVM 44, etc.). However, the technology exemplified in this disclosure can also be applied to a case where data of a three-dimensional tomographic image is processed and stored in a storage device.

[0048] 2, the ophthalmologic imaging device 1 of this embodiment scans a spot of light (measurement light) on a living tissue 50 (a fundus tissue in the example shown in FIG. 2). In detail, the ophthalmologic imaging device 1 of this embodiment scans light on a scan line 52 extending in a predetermined direction on the tissue 50, thereby capturing an ophthalmologic image (two-dimensional tomographic image) 61 (see FIG. 3) extending in a Z direction along the optical axis of the light and an X direction intersecting the Z direction (perpendicular to the Z direction in this embodiment).

[0049] As shown in FIG. 3 , in this embodiment, the direction in which the spot-shaped measurement light scans the tissue (referred to as the “B scan direction”) is defined as the X direction. The depth direction of the tissue 50 along the optical axis of the measurement light (i.e., the direction perpendicular to the X direction, referred to as the “A scan direction”) is defined as the Z direction. The ophthalmic image 61 is a tomographic image of a layered structure of living tissue, in which multiple layers are stacked. Specifically, the ophthalmic image 61 of this embodiment captures the layered structure of the fundus tissue of the subject's eye. As described above, the ophthalmic imaging device 1 captures the two-dimensional ophthalmic image 61 extending in the X direction (B scan direction) and the Z direction (A scan direction) by scanning the measurement light irradiated from the front side onto the fundus tissue in the X direction. Therefore, all of the layers in the layered structure captured in the ophthalmic image 61 intersect with the Z direction (A scan direction).

[0050] The ophthalmic image 61 of this embodiment is configured by arranging a plurality of elongated A-scan images extending in the Z direction (A-scan direction) (i.e., pixel rows extending in the Z direction along the optical axis of the measurement light) in a predetermined X direction (a direction intersecting the Z direction). The plurality of A-scan images extend in a direction intersecting the layers of the layer structure depicted in the ophthalmic image 61. As will be described in detail later, in this embodiment, each of the plurality of A-scan images is treated as a partial image, which is a unit for extracting an image to be saved.

[0051] The ophthalmic image processing in this embodiment will be described with reference to FIGS. 4 to 8. In this embodiment, the ophthalmic image processing device 40, which is a PC, acquires data of an ophthalmic image 61 (hereinafter, sometimes simply referred to as "ophthalmic image 61") from the ophthalmic image capturing device 1, processes the acquired data of the ophthalmic image 61, and stores the processed data in a storage device. However, as described above, another device may function as the ophthalmic image processing device. For example, the ophthalmic image capturing device 1 itself may perform the ophthalmic image processing. Furthermore, multiple control units (for example, the CPU 31 of the ophthalmic image capturing device 1 and the CPU 41 of the ophthalmic image processing device 40) may cooperate to perform the ophthalmic image processing. The CPU 41 of the ophthalmic image processing device 40 performs the ophthalmic image processing shown in FIG. 4 in accordance with an ophthalmic image processing program stored in the NVM 44.

[0052] First, the CPU 41 acquires an ophthalmological image 61 of the tissue of the subject's eye (the retinal tissue of the fundus in this embodiment) (S1). As illustrated in Figures 3 and 5, the ophthalmological image 61 of this embodiment shows the layer structure of the retinal tissue of the fundus.

[0053] The CPU 41 preliminarily extracts a region (image region) containing a tissue image from the ophthalmic image 61 acquired in S1 (S2). As shown in FIG. 5, the ophthalmic imaging device 1 of this embodiment can capture tissue at a deep depth, even deeper than the deepest end of the retinal tissue (the bottom end of the image in the ophthalmic image 61 in FIG. 5). However, when the ophthalmic image 61 is used for diagnosis or the like, it is often sufficient to be able to observe the retinal tissue. Therefore, in this embodiment, the CPU 41 identifies an image region (the region indicated by the dotted-line frame in the ophthalmic image 61 in FIG. 5) containing a tissue image from the ophthalmic image 61 acquired in S1, and preliminarily extracts an ophthalmic image 62 containing the detected image region from the ophthalmic image 61 acquired in S1. As a result, the processing volume of various processes to be executed subsequently (e.g., the process of identifying the reference site in S3) and the data volume of image data to be processed are reduced.

[0054] A method for identifying an image region in the ophthalmological image 61 can be selected as appropriate. For example, the CPU 41 may identify the image region by performing known image processing on the ophthalmological image 61. Alternatively, a mathematical model pre-trained by a machine learning algorithm may be used to identify the image region in the ophthalmological image 61. In this case, the CPU 41 may input the ophthalmological image 61 to the mathematical model, and obtain the result of identifying the image region output by the mathematical model. Note that the processing of S2 may be omitted.

[0055] Next, the CPU 41 identifies at least one of the multiple regions in the tissue captured in the ophthalmological image 61 acquired in S1 as a reference region (S3). As will be described in detail later, the reference region serves as a reference for extracting images within the range to be saved from the ophthalmological image 61 acquired in S1 (in this embodiment, an ophthalmological image 62 obtained by extracting an image region from the ophthalmological image 61).

[0056] As shown in FIG. 6 , the ophthalmologic image 62 shows a layered structure of tissues of the subject's eye, in which multiple layers are stacked. In S3 of this embodiment, at least one of the multiple layers and layer boundaries included in the layered structure shown in the ophthalmologic image 62 is identified as the reference portion RS. Each layer / boundary included in the layered structure tends to have a shape that follows the overall shape of the layered structure. Therefore, even if the layered structure is not flat (for example, if the layered structure is curved), an extracted image is extracted using the layer / boundary as the reference portion RS in S6 and S7 (described later), which makes it easier to save an image of an appropriate range according to the shape of the layered structure.

[0057] In detail, in S3 of this embodiment, the boundary or layer (the retinal superficial layer in this embodiment) on the most superficial side (the top end in FIG. 6) of the layer structure shown in the ophthalmologic image 62 is identified as the reference portion RS. The most superficial side of the layer structure is the boundary between an area where the layer structure exists and an area where the layer structure does not exist. Therefore, by using the layer / boundary on the most superficial side of the layer structure as the reference portion RS, it becomes easier to more appropriately extract an image including the layer structure.

[0058] In S3 of the present embodiment, the CPU 41 identifies the reference portion RS in the ophthalmological image 62 by performing known image processing (e.g., edge detection) on the ophthalmological image 62. However, the method for identifying the reference portion RS may be changed. For example, the CPU 41 may input the ophthalmological image 62 to a mathematical model trained by a machine learning algorithm, and obtain the identification result of the reference portion RS output by the mathematical model.

[0059] Next, if there is a portion in the ophthalmologic image 62 where the identification result of the reference region RS is missing, the CPU 41 complements the identification result of the reference region RS at the missing portion based on the identification results of the reference region RS at other portions (portions where identification results were successfully acquired) (S4). As a result, even if a portion of the reference region RS is not identified in S3, images within the saved range are appropriately extracted using the reference region RS as a reference in S6 and S7, which will be described later. As an example, in S4 of this embodiment, the identification result of the missing portion is complemented based on the identification result of the reference region RS at a portion adjacent to the missing portion (for example, an A-scan image adjacent to the A-scan image containing the missing portion). In detail, the CPU 41 complements the position between the identified position of an A-scan image adjacent to one side of the A-scan image containing the missing portion (which may be a plurality of consecutive A-scan images) and the identified position of an A-scan image adjacent to the other side as the position of the reference region RS at the missing portion. However, the interpolation method may be changed.

[0060] Next, the CPU 41 identifies the extraction target range of the Nth partial image 70 (the initial value of N is "1") among the multiple partial images 70 arranged in a predetermined direction in the ophthalmic image 62, based on the reference region RS (S6). As shown in FIG. 6, the multiple partial images 70 are arranged consecutively in a predetermined direction in the ophthalmic image 62. In this embodiment, each of the multiple partial images 70 is an elongated image extending in a direction intersecting the layers in the layer structure depicted in the ophthalmic image 62 (the Z direction in this embodiment), and the multiple partial images 70 are arranged consecutively in a direction intersecting the direction in which the partial images 70 extend (the X direction in this embodiment). Therefore, the layer boundary (the outermost layer boundary in this embodiment) that serves as the reference region can be easily identified in each partial image 70. As described above, in this embodiment, the multiple partial images 70 are A-scan images extending in a direction along the optical axis of the OCT measurement light.

[0061] An example of a method for specifying the extraction target range of each partial image 70 based on the reference portion RS will be described in detail. In this embodiment, the CPU 41 includes a predetermined range from the reference portion RS, which is the most superficial portion in the layer structure of the ophthalmologic image 62, toward the deeper side of the layer structure (the +Z direction, or downward in the drawing, in the example shown in FIG. 6 ), in the extraction target range. That is, the CPU 41 specifies, in the partial image 70 extending in the Z direction, a position obtained by adding an offset amount (+offset1) toward the deeper side (+Z direction) to the Z coordinate of the reference portion RS as the deeper end DE of the extraction target range in the partial image 70. As a result, the range from the most superficial reference portion RS to the deeper side of the layer structure is extracted next, making it easier to extract as wide a range as possible of the layer structure.

[0062] Furthermore, the CPU 41 includes a predetermined range in the extraction target range from the most superficial reference portion RS in the layer structure of the ophthalmologic image 62 toward the near side (the upper side of the drawing, in the -Z direction, in the example shown in FIG. 6 ), which is opposite to the deep side of the layer structure. That is, the CPU 41 identifies the position of the partial image 70 extending in the Z direction, obtained by adding an offset amount (-offset2) toward the near side (+Z direction) to the Z coordinate of the reference portion RS, as the near end SE of the extraction target range in the partial image 70. As a result, even if the accuracy of identifying the most superficial reference portion RS decreases, for example, it becomes easier to properly extract an image of the tissue on the superficial side of the layer structure. Furthermore, even if an object is captured on the surface side of the boundary between the most superficial layer due to various influences (e.g., the influence of a disease, etc.), the object captured on the surface side is also easily extracted.

[0063] In this embodiment, the CPU 41 sets the offset amount toward the deeper layer (+offset1) and the offset amount toward the nearer layer (-offset2) according to an instruction input by the user. Therefore, the user can efficiently and appropriately store image data of a desired range of the layer structure of the subject's eye. Furthermore, the CPU 41 may adjust at least one of the two offset amounts according to the photographing conditions of the ophthalmologic image 61 (for example, at least one of the photographing magnification, the photographing angle of view, the photographing depth, etc.). In this case, image data of an appropriate range according to the photographing conditions is extracted. Furthermore, at least one of the two offset amounts may be a predetermined fixed value.

[0064] Next, the CPU 41 extracts (S7) data of an image (extracted image 80) within the extraction target range from the Nth partial image 70. As shown in Fig. 6, the extracted image 80 extracted from each partial image 70 is likely to include an appropriate range based on the reference portion RS, and is likely to exclude unnecessary ranges.

[0065] FIG. 7 shows a reference image 80A in which multiple extracted images 80 extracted from multiple partial images 70 are arranged in the X direction (i.e., the arrangement direction of the multiple partial images 70) without considering the positions of the multiple extracted images 80. As shown in FIG. 7, the reference image 80A, which is a collection of multiple extracted images 80, includes an appropriate range of tissue (layer structure in this embodiment) based on the identified reference portion RS. Furthermore, it can be seen that the amount of image data is smaller than that of the ophthalmological images 61 and 62. That is, in the reference image 80A, at least a portion of the area other than the layer structure is erased from the ophthalmological images 61 and 62, and the curvature of the layer structure is flattened. By erasing at least a portion of the area other than the layer structure, the amount of image data is appropriately reduced, and the layer curvature information reproduced based on position information is removed, resulting in a flattened layer structure. In the reference image 80A shown in FIG. 7, the background portion (white portion in FIG. 8) in the reconstructed image data 90 shown in FIG. 8 (details of which will be described later) is erased from the ophthalmological image 62 shown in FIG. 5, and the curvature is flattened.

[0066] Furthermore, the extracted images 80 of this embodiment are extracted from the ophthalmologic image 62 while maintaining the thickness of the layer structure. Therefore, as will be described in detail later, each extracted image 80 is reconstructed based on position information, so that the layer structure of the original ophthalmologic image 62 is appropriately reconstructed while maintaining the thickness.

[0067] The CPU 41 acquires position information indicating the position of the Nth extracted image 80 (i.e., the extracted image 80 extracted from the Nth partial image 70) (S8). Various information can be used as the position information. For example, coordinate information on at least one of the position of the reference portion RS in the Nth partial image 70, the position of the deep end DE of the extraction target range, and the position of the near end SE may be acquired as the position information of the Nth extracted image 80.

[0068] The CPU 41 stores data of the extracted image 80 extracted from the Nth partial image 70 and position information indicating the position of the Nth extracted image 80 in storage means (for example, the NVM 44) (S9). The CPU 41 stores the data of the extracted image 80 in the storage means either without compression or after performing lossless compression that allows images including layer structures to be reproduced after decoding. Next, the CPU 41 determines whether processing has been completed for all partial images 70 in the ophthalmologic image 62 (S10). If processing has not yet been completed for some partial images 70 (S10: NO), "1" is added to the counter N indicating the order of the partial images 70 (S11), and the process returns to S6, where the processes of S6 to S10 are repeated. When processing has been completed for all partial images 70, the process for saving the extracted image 80 and position information ends.

[0069] 8, an image reconstruction process for reconstructing an image based on the saved extracted image 80 and position information will be described. The CPU 41 of the ophthalmologic image processing device 40 executes the image reconstruction process in accordance with an ophthalmologic image processing program stored in the NVM 44.

[0070] When the user specifies data for which an image is to be reconstructed, the CPU 41 acquires the data specified by the user from among the plurality of data stored in the storage means (data on the plurality of extracted images 80 and position information for each extracted image 80). The CPU 41 arranges the data on the plurality of extracted images 80 based on the position information for each extracted image, thereby generating reconstructed image data 90 in which the image of the extracted region is reconstructed. As shown in FIG. 8, in the reconstructed image data 90, the region extracted from the ophthalmologic images 61 and 62 is appropriately reconstructed.

[0071] Note that the reconstructed image data 90 lacks information about the areas that were not extracted as the extracted image 80. The CPU 41 can also generate background-complemented image data 91 by complementing pixel information of the background areas that are missing from the reconstructed image data 90. As shown in Fig. 8, the background-complemented image data 91 is an image that is closer to the actually captured ophthalmologic images 61 and 62 by complementing black pixel information as the background.

[0072] The techniques disclosed in the above embodiments are merely examples. Therefore, it is possible to modify the techniques exemplified in the above embodiments. First, it is possible to omit some of the processes exemplified in the above embodiments. For example, the process (S2) of extracting the image area in advance may be omitted. Furthermore, the process (SS4) of filling in missing parts of the result of identifying the reference portion RS may also be omitted.

[0073] In the above embodiment, the most superficial layer / boundary of the layer structure captured in the ophthalmic image 62 is set as the reference region RS. However, it is also possible to change the reference region. For example, if the tissue captured in the ophthalmic image 62 is retinal tissue at the fundus, at least one of the layers / boundaries located deeper than the most superficial retinal layer (e.g., the ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, external limiting membrane, photoreceptor inner / outer segment junction, retinal pigment epithelium, Bruch's membrane, choroid, etc., and their boundaries) may be set as the reference region. For example, by setting a layer / structure that is more clearly visible among the layer structures captured in the ophthalmic image 62 as the reference region, the accuracy of identifying the reference region may be improved, and the image may be extracted and saved more appropriately.

[0074] Furthermore, the layer / boundary identified as the reference portion of the layer structure shown in the ophthalmologic image 62 may be changed according to various conditions. For example, there are layers / boundaries whose visibility in the ophthalmologic image changes depending on the position of the tissue. Therefore, the layer / boundary identified as the reference portion may be changed according to the position of the tissue shown in the ophthalmologic image.

[0075] Furthermore, the CPU 41 may specify multiple layers / boundaries shown in the ophthalmologic image 62 as reference regions. In this case, the CPU 41 may extract an extracted image from each of multiple partial images, using a range surrounded by one of the specified multiple reference regions (e.g., the most superficial layer / boundary) and another reference region (e.g., a layer / boundary located deeper in the layer structure) as the extraction target range. As a result, image data within a specific range of the layer structure shown in the ophthalmologic image 62 is appropriately extracted and saved.

[0076] Furthermore, among multiple regions shown in an ophthalmological image, a region other than a layer boundary may be used as the reference region. For example, a diseased region shown in an ophthalmological image may be specified as the reference region, and an extracted image may be extracted based on the specified reference region. In this case, an extracted image is appropriately extracted and saved according to the diseased region. Furthermore, a blood vessel region shown in an ophthalmological image may be specified as the reference region.

[0077] The process of acquiring an ophthalmologic image in S1 of FIG. 4 is an example of an "image acquisition step." The process of identifying the reference portion RS in S3 of FIG. 4 is an example of a "reference portion identification step." The process of extracting image data within the extraction target range from the partial image 70 in S6 and S7 of FIG. 4 is an example of an "extraction step." The process of acquiring position information of the extracted image 80 in S8 of FIG. 4 is an example of a "position information acquisition step." The process of saving the data and position information of the extracted image 80 in S9 of FIG. 4 is an example of a "storage step." The process of complementing the identification result of the reference portion RS in S4 of FIG. 4 is an example of an "complementation step." The process of reconstructing the image shown in FIG. 8 is an example of an "image reconstruction step."

[0078] 1. Ophthalmic imaging device 40 Ophthalmic image processing device 41 CPU 44 NVM 61,62 Ophthalmological images 70 partial images 80 Extracted Images 90 Reconstructed image data 91 Background Complement Image Data RS reference site

Claims

1. An ophthalmologic image processing device that processes data of an ophthalmologic image which is a tomographic image of tissue of a subject's eye, The control unit of the ophthalmologic image processing device an image acquiring step of acquiring an ophthalmic image captured by an ophthalmic image capturing device; An extraction step of extracting image data within a target range for storing data from each of a plurality of partial images arranged in a predetermined direction within the ophthalmic image; a position information acquiring step of acquiring position information indicating a position of each of a plurality of extracted images extracted from each of the plurality of partial images in the extracting step; a storing step of storing data of the plurality of extracted images and position information of each of the plurality of extracted images in a storage device; 13. An ophthalmologic image processing apparatus comprising:

2. 2. The ophthalmologic image processing device according to claim 1, The control unit is A reference site specifying step is further performed to specify at least one site among a plurality of sites in the tissue shown in the ophthalmologic image as a reference site; An ophthalmologic image processing device characterized in that in the extraction step, image data within a target range based on the reference portion is extracted as the extracted image from each of the multiple partial images arranged in a predetermined direction within the ophthalmologic image.

3. 3. The ophthalmologic image processing apparatus according to claim 2, the ophthalmologic image is a tomographic image of a layered structure of a tissue of a subject's eye, in which a plurality of layers are stacked; The ophthalmologic image processing apparatus, wherein in the reference portion specifying step, at least one of a plurality of layers and layer boundaries included in the layer structure is specified as the reference portion.

4. 4. The ophthalmologic image processing device according to claim 2, An ophthalmologic image processing device characterized in that, when a portion where the reference portion identification result is missing exists as a result of execution of the reference portion identification step, a complementation step is further executed to complement the reference portion identification result at the missing portion based on the reference portion identification result at other portions.

5. 5. The fundus image processing device according to claim 1, the ophthalmologic image is a tomographic image of a layered structure of a tissue of a subject's eye, in which a plurality of layers are stacked; A fundus image processing device characterized in that an image in which the multiple extracted images extracted in the extraction step are arranged in the arrangement direction of the multiple partial images without taking into account the positional information is an image in which at least a portion of the area other than the layer structure is erased from the ophthalmic image and the curvature of the layer structure is flattened.

6. 6. The fundus image processing device according to claim 1, the ophthalmologic image is a tomographic image of a layered structure of a tissue of a subject's eye, in which a plurality of layers are stacked; The fundus image processing apparatus is characterized in that the extracted image is extracted from the ophthalmic image and stored in a state in which the thickness of the layer structure in the ophthalmic image is maintained.

7. 7. An ophthalmologic image processing apparatus according to claim 1, An ophthalmologic image processing device characterized by further executing an image reconstruction step of reconstructing the area of ​​the ophthalmologic image extracted in the extraction step by arranging the data of the multiple extracted images stored in the memory means based on each of the position information.

8. An ophthalmic image processing program executed by an ophthalmic image processing device that processes data of an ophthalmic image, which is a tomographic image of tissue of a subject's eye, comprising: The ophthalmologic image processing program is executed by a control unit of the ophthalmologic image processing device, an image acquiring step of acquiring an ophthalmic image captured by an ophthalmic image capturing device; An extraction step of extracting image data within a target range for storing data from each of a plurality of partial images arranged in a predetermined direction within the ophthalmic image; a position information acquiring step of acquiring position information indicating a position of each of a plurality of extracted images extracted from each of the plurality of partial images in the extracting step; a storing step of storing data of the plurality of extracted images and position information of each of the plurality of extracted images in a storage device; An ophthalmologic image processing program causing the ophthalmologic image processing device to execute the above-mentioned.

Images