Ophthalmic image processing device, OCT device, ophthalmic image processing program and mathematical model construction method
The mathematical model trained by machine learning algorithms performs addition and average processing of ophthalmic images, which solves the problem of signal elimination caused by long shooting time and noise suppression in the prior art, and realizes the generation of ophthalmic images with high image quality and high frame rate.
Patent Information
- Application Number
- CN201880096917.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2018-06-29
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2038-06-29
AI Technical Summary
When improving the image quality of ophthalmic images, the prior art requires a lot of repeated shooting of the same part to suppress noise, resulting in an increase in shooting time and it is difficult to distinguish between weak signals and noise, resulting in the removal of weak signals.
The mathematical model trained by machine learning algorithm is used to add and average multiple ophthalmic images to generate target images with higher quality than the basic image. The ophthalmic images are used as input data and the H-added average image is used as output data. Perform training to suppress noise effects and output weak parts of the image signal if necessary.
It is achieved while shortening the shooting time, and obtaining high-quality ophthalmic images, avoiding weak signal elimination caused by suppression of noise influence, appropriately obtaining high-quality ophthalmic images, and maintaining the frame rate in dynamic image generation.
Smart Images

Figure CN113365547B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an ophthalmic image processing device that processes an ophthalmic image of an eye to be inspected, an OCT device, an ophthalmic image processing program executed in the ophthalmic image processing device, and a mathematical model construction method for constructing a mathematical model used by the ophthalmic image processing device. Background Art
[0002] Various techniques for obtaining high-quality images have been proposed. For example, an ophthalmic imaging device described in Patent Document 1 obtains a high-quality tomographic image by performing an addition and averaging process on a plurality of tomographic images to suppress speckle noise.
[0003] Prior art literature
[0004] Patent Literature
[0005] Patent Document 1: Japanese Patent Application Publication No. 2015-9108 Summary of the invention
[0006] In the technology of adding and averaging multiple images, the more accuracy is desired in suppressing the influence of noise, the more images of the same part need to be repeatedly photographed, which causes a problem that the photographing time increases in proportion to the number of images to be added.
[0007] A typical object of the present disclosure is to provide an ophthalmic image processing device, an OCT device, an ophthalmic image processing program, and a mathematical model construction method for appropriately acquiring high-quality ophthalmic images.
[0008] An ophthalmic image processing device provided in a typical embodiment of the present disclosure processes an ophthalmic image that is an image of a tissue of an eye to be inspected. A control unit of the ophthalmic image processing device obtains an ophthalmic image captured by an ophthalmic image capturing device, or an averaged image obtained by averaging a plurality of ophthalmic images captured by the ophthalmic image capturing device at the same part of the tissue as a base image, and obtains a target image with higher image quality than the base image by inputting the base image into a mathematical model trained using a machine learning algorithm. The mathematical model is trained using data of L (L≥1) training ophthalmic images out of a plurality of training ophthalmic images captured at the same part of the tissue as input training data, and using data of an averaged image obtained by averaging H (H>L) training ophthalmic images as output training data.
[0009] The OCT device provided in a typical embodiment of the present disclosure captures an ophthalmic image of the tissue by processing an OCT signal formed by a reference light and reflected light of a measurement light irradiated onto the tissue of the eye to be inspected. The control unit of the OCT device obtains the captured ophthalmic image of the tissue, or an averaged image obtained by averaging a plurality of ophthalmic images obtained by capturing the same portion of the tissue as a base image, and obtains a target image with higher image quality than the base image by inputting the base image into a mathematical model trained using a machine learning algorithm. The mathematical model is trained using data of L (L≥1) training ophthalmic images out of a plurality of training ophthalmic images obtained by capturing the same portion of the tissue as input training data, and using data of an averaged image obtained by averaging H (H>L) training ophthalmic images as output training data.
[0010] An ophthalmic image processing program provided in a typical embodiment of the present disclosure is executed by an ophthalmic image processing device that processes an ophthalmic image that is an image of a tissue of an eye to be inspected. The ophthalmic image processing program is executed by a control unit of the ophthalmic image processing device, thereby causing the ophthalmic image processing device to execute the following steps: a basic image acquisition step, acquiring an ophthalmic image captured by an ophthalmic image capturing device, or an averaged image obtained by averaging a plurality of ophthalmic images captured of the same portion of the tissue by the ophthalmic image capturing device as a basic image; and a target image acquisition step, acquiring a target image with higher image quality than the basic image by inputting the basic image into a mathematical model trained using a machine learning algorithm, the mathematical model being trained using data of L (L≥1) training ophthalmic images out of a plurality of training ophthalmic images captured of the same portion of the tissue as input training data, and using data of an averaged image obtained by averaging H (H>L) training ophthalmic images as output training data.
[0011] A typical embodiment of the present disclosure provides a mathematical model construction method which is executed by a mathematical model construction device for constructing a mathematical model, wherein the mathematical model is constructed by training using a machine learning algorithm, and a target image is output by inputting a basic image, and the mathematical model construction method comprises the following steps: an image set acquisition step, in which a plurality of training ophthalmic images are acquired by photographing the same part of the tissue of the eye to be inspected; and a training step, in which the mathematical model is trained by using data based on L (L≥1) of the training ophthalmic images among the plurality of training ophthalmic images as input training data, and using data of an added average image obtained by adding and averaging H (H>L) of the training ophthalmic images as output training data, thereby constructing the mathematical model.
[0012] According to the ophthalmic image processing device, OCT device, ophthalmic image processing program, and mathematical model construction method involved in the present disclosure, high-quality ophthalmic images can be appropriately acquired.
[0013] The control unit of the ophthalmic image processing device exemplified in the present disclosure obtains an ophthalmic image obtained by an ophthalmic image shooting device, or an averaged image obtained by averaging a plurality of ophthalmic images obtained by the ophthalmic image shooting device at the same part of the tissue as a base image. The control unit obtains a target image with higher image quality than the base image by inputting the base image into a mathematical model trained by a machine learning algorithm, and the mathematical model is trained by a training data set, which is a collection of input training data and output training data. Data of L (L≥1) training ophthalmic images among a plurality of training ophthalmic images obtained by shooting the same part of the tissue is used as input training data. In addition, data of an averaged image obtained by averaging H (H>L) training ophthalmic images among a plurality of training ophthalmic images is used as output training data (sometimes also referred to as correct answer data). In other words, H is a number greater than the number of training ophthalmic images used for input training data.
[0014] That is, a mathematical model construction device for constructing a mathematical model executes an image set acquisition step and a training step. In the image set acquisition step, a plurality of training ophthalmic images obtained by photographing the same part of the tissue of the eye to be examined are acquired. In the training step, the mathematical model is trained by using data of L (L≥1) training ophthalmic images among the plurality of training ophthalmic images as input training data and using data of an added average image of H (H>L) training ophthalmic images as output training data, thereby constructing a mathematical model.
[0015] In this case, the constructed mathematical model can output a target image in which the influence of noise is suppressed to the same extent as an image obtained by averaging a large number of ophthalmic images, even when a small number of ophthalmic images or an averaged image of a small number of ophthalmic images is input as a base image. Therefore, it is easy to shorten the shooting time when shooting the base image. In addition, in the existing image processing, the influence of noise can also be suppressed without performing the averaging process. However, in the existing image processing, sometimes it is impossible to distinguish between weak signals and noise, resulting in the elimination of weak signals that are originally required. In contrast, the mathematical model exemplified in the present disclosure can output a target image based on the base image in which the part of the image signal that becomes weak due to influences other than noise is not eliminated. Therefore, high-quality ophthalmic images can be appropriately obtained.
[0016] In addition, the data used as input training data of the training data set (i.e., "data based on L training ophthalmic images") may be data of the captured ophthalmic image itself (i.e., the original image without averaging) or data of an averaged image obtained by averaging a plurality of ophthalmic images. The data of the original image and the data of the averaged image may also be used together as input training data. When the input training data is the data of the original image, it is preferred that the base image input to the mathematical model is also the original image without averaging. In addition, when the input training data is the data of the averaged image, it is preferred that the base image is also the averaged image. In addition, the number of images (at least one of the original image and the averaged image) used for the input training data may be one or more (e.g., about 2 to 5). In addition, the number H of training ophthalmic images used when generating the averaged image used as output training data is at least greater than the number of training ophthalmic images used for the input training data. For example, when L original images are used as input training data, H>L. Furthermore, when an image obtained by adding and averaging L' images is used as input training data, H>L'. In addition, the number of ophthalmic images used as the base image is smaller than H images.
[0017] The ophthalmic image may also be a two-dimensional tomographic image of the tissue of the eye to be inspected obtained by an OCT device. The control unit of the ophthalmic image processing device may also obtain a plurality of base images obtained by respectively photographing a plurality of different positions in the tissue of the eye to be inspected. The control unit may also obtain target images of each of the plurality of positions by inputting each of the plurality of base images into a mathematical model. The control unit may also generate at least one of a three-dimensional tomographic image of the tissue and a two-dimensional frontal image when the tissue is observed from the front (i.e., the line of sight of the eye to be inspected) based on the plurality of target images. That is, the ophthalmic image capturing device may also perform at least one of a three-dimensional tomographic image generating step of generating a three-dimensional tomographic image of the tissue based on a plurality of target images and a frontal image generating step of generating a two-dimensional frontal image of the tissue based on a plurality of target images. In this case, a three-dimensional tomographic image or a two-dimensional frontal image of the tissue with the influence of noise suppressed can be appropriately generated without photographing a large number of two-dimensional tomographic images.
[0018] The two-dimensional frontal image may also be a so-called "enface image". For example, the data of the enface image may be, for example, cumulative image data obtained by accumulating brightness values at each position in the XY direction along the depth direction (Z direction), cumulative values of spectral data at each position in the XY direction, brightness data at each position in the XY direction at a certain depth direction, brightness data at each position in the XY direction in any layer of the retina (for example, the surface layer of the retina), etc.
[0019] The ophthalmic image may also be a two-dimensional tomographic image of the fundus of the subject's eye captured by an OCT device. The control unit of the ophthalmic image processing device may also identify at least one of the multiple layers included in the fundus by analyzing the target image as a two-dimensional tomographic image. That is, the ophthalmic image processing device may also perform a segmentation step of identifying the layers of the fundus from the acquired target image. In this case, a segmentation result with suppressed noise effects can be appropriately obtained without capturing a large number of two-dimensional tomographic images.
[0020] In addition, as an ophthalmic image, an image other than a two-dimensional tomographic image can also be used. For example, the ophthalmic image may not be a two-dimensional tomographic image, but a two-dimensional frontal image obtained by photographing a tissue (such as the fundus) from the front (i.e., the line of sight of the eye to be examined). In this case, the two-dimensional frontal image may also be an OCT angiography image. The OCT angiography image may be, for example, a motion contrast image obtained by processing at least two OCT signals obtained at different times for the same position. In addition, an image obtained by photographing the autofluorescence (FAF) of the fundus of the eye to be examined by a laser scanning fundus examination device (SLO) (hereinafter referred to as a "FAF image") may also be used as an ophthalmic image. The OCT angiography image and the FAF image can also appropriately suppress the influence of noise by performing an addition and averaging process on multiple images, similar to the OCT two-dimensional tomographic image. In addition, the shooting site of the ophthalmic image may be, for example, the fundus of the eye to be examined, or the anterior ocular part. As described above, the ophthalmic image of the present disclosure can use various images that can suppress the influence of noise by performing an addition and averaging process on multiple images.
[0021] In addition, the control unit of the ophthalmic image processing device may also perform an analysis process different from the segmentation process of the identification layer on the target image. For example, the control unit may obtain a frontal image of the fundus showing blood vessels (for example, the aforementioned OCT angiography image) as the target image, and perform a process of analyzing the blood vessel density of the fundus on the obtained target image. In addition, the control unit may also perform an analysis process on an image generated based on a plurality of target images (for example, the aforementioned three-dimensional tomographic image or Enface image, etc.).
[0022] The control unit of the ophthalmic image processing device may also perform at least one of a simultaneous display process and a switching display process. In the simultaneous display process, the control unit causes the base image and the target image to be displayed on the display device at the same time. In the switching display process, the control unit switches the base image and the target image according to an instruction input by the user and displays them on the display device. In this case, the user can easily confirm both the target image in which the influence of noise is suppressed and the base image (the original image or the added average image of the original image) which is the basis of the target image. Therefore, the user can make a diagnosis more appropriately. In addition, the base image and the target image can be displayed on the same display device or on different display devices.
[0023] The control unit may also obtain a plurality of base images continuously taken by the ophthalmic image shooting device. The control unit may also obtain a plurality of target images by sequentially inputting the obtained plurality of base images into the mathematical model. The control unit may also generate dynamic image data of the tissue based on the obtained plurality of target images. That is, the ophthalmic image processing device may also perform a dynamic image generation step of generating dynamic image data of the tissue based on the plurality of target images. In this case, the user can confirm the dynamic image of the tissue displayed in a state where the influence of noise is suppressed. That is, in the prior art, the influence of noise can also be suppressed by adding and averaging a plurality of ophthalmic images. However, in the prior art, since an addition and average image is generated based on a plurality of ophthalmic images, when the addition and average image is used to display the dynamic image of the tissue, the frame rate is likely to drop significantly. In particular, in the case where the shooting position changes over time, it is difficult to generate a good dynamic image using the addition and average image. In contrast, the ophthalmic image processing device disclosed in the present invention can generate a dynamic image of the tissue with the influence of noise suppressed while suppressing the decrease in the frame rate. Therefore, the user can make a diagnosis more appropriately.
[0024] In addition, the control unit may display the dynamic image of the base image and the dynamic image of the target image on the display device at the same time. In addition, the control unit may switch the dynamic image of the base image and the dynamic image of the target image according to an instruction input by the user and display them on the display device. In this case, the user can easily confirm both the dynamic image of the target image and the dynamic image of the base image. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a block diagram showing a schematic configuration of the mathematical model construction device 1 , the ophthalmologic image processing device 21 , and the ophthalmologic image capturing devices 11A and 11B.
[0026] Figure 2 4 is a flowchart of a mathematical model construction process executed by the mathematical model construction device 1 .
[0027] Figure 3 This is an explanatory diagram for explaining a first acquisition mode of input training data and output training data.
[0028] Figure 4 This is an explanatory diagram for explaining a second acquisition mode of input training data and output training data.
[0029] Figure 5 This is an explanatory diagram for explaining a third acquisition mode of input training data and output training data.
[0030] Figure 6 This is an explanatory diagram for explaining a fourth acquisition mode of input training data and output training data.
[0031] Figure 7 This is a flowchart of the high-quality moving image generation process executed by the ophthalmologic image processing device 21 .
[0032] Figure 8 4 is a flowchart of the high-quality three-dimensional image generation process executed by the ophthalmologic image processing device 21 .
[0033] Fig. 9 1 is a diagram showing an example of a high-quality three-dimensional image 40 . DETAILED DESCRIPTION
[0034] (Device Structure)
[0035] Hereinafter, one of the typical embodiments of the present disclosure will be described with reference to the accompanying drawings. Figure 1 As shown, in this embodiment, a mathematical model construction device 1, an ophthalmic image processing device 21, and ophthalmic image shooting devices 11A and 11B are used. The mathematical model construction device 1 constructs a mathematical model by training the mathematical model using a machine learning algorithm. The constructed mathematical model outputs a target image with higher image quality than the base image (in this embodiment, the influence of noise is appropriately suppressed) based on the input base image. The ophthalmic image processing device 21 uses the mathematical model to obtain the target image based on the base image. The ophthalmic image shooting devices 11A and 11B shoot ophthalmic images that are images of tissues of the eye to be examined.
[0036] As an example, the mathematical model construction device 1 of the present embodiment uses a personal computer (hereinafter referred to as "PC"). The details will be described later, and the mathematical model construction device 1 constructs a mathematical model by training the mathematical model using ophthalmic images obtained from the ophthalmic image capturing device 11A. However, the device that can function as the mathematical model construction device 1 is not limited to the PC. For example, the ophthalmic image capturing device 11A can also function as the mathematical model construction device 1. In addition, the control units of multiple devices (for example, the CPU of the PC and the CPU 13A of the ophthalmic image capturing device 11A) can also cooperate to construct a mathematical model.
[0037] In addition, the ophthalmic image processing device 21 of the present embodiment uses a PC. However, the device that can function as the ophthalmic image processing device 21 is not limited to the PC. For example, the ophthalmic image capturing device 11B or a server, etc. can also function as the ophthalmic image processing device 21. When the ophthalmic image capturing device (OCT device in the present embodiment) 11B functions as the ophthalmic image processing device 21, the ophthalmic image capturing device 11B can capture ophthalmic images and appropriately obtain ophthalmic images with higher quality than the captured ophthalmic images. In addition, mobile terminals such as tablet terminals or smart phones can also function as the ophthalmic image processing device 21. The control units of multiple devices (for example, the CPU of a PC and the CPU 13B of the ophthalmic image capturing device 11B) can also cooperate to perform various processes.
[0038] In addition, in this embodiment, a case where a CPU is used as an example of a controller for performing various processes is illustrated. However, at least a portion of the various devices may also use a controller other than a CPU. For example, a GPU may be used as a controller to achieve high-speed processing.
[0039] The mathematical model construction device 1 is described. The mathematical model construction device 1 is configured, for example, in an ophthalmic image processing device 21 or a manufacturer that provides an ophthalmic image processing program to a user. The mathematical model construction device 1 includes a control unit 2 and a communication I / F 5 that perform various control processes. The control unit 2 includes a CPU 3 as a controller responsible for control and a storage device 4 that can store programs and data. The storage device 4 stores data for executing the mathematical model construction process described later (see Figure 2 ) of the mathematical model building program. In addition, the communication I / F 5 connects the mathematical model building device 1 to other devices (for example, the ophthalmic image capturing device 11A and the ophthalmic image processing device 21, etc.).
[0040] The mathematical model building device 1 is connected to an operation unit 7 and a display device 8. The operation unit 7 is operated by a user so that the user can input various instructions to the mathematical model building device 1. The operation unit 7 can use at least any one of a keyboard, a mouse, a touch panel, etc., for example. In addition, a microphone for inputting various instructions can also be used together with or instead of the operation unit 7. The display device 8 displays various images. The display device 8 can use various devices that can display images (for example, at least any one of a monitor, a display, a projector, etc.). In addition, the "image" in the present disclosure includes both static images and dynamic images.
[0041] The mathematical model construction device 1 can obtain ophthalmic image data (hereinafter sometimes referred to as "ophthalmic image") from the ophthalmic image capturing device 11A. The mathematical model construction device 1 can obtain ophthalmic image data from the ophthalmic image capturing device 11A through at least one of wired communication, wireless communication, and a removable storage medium (such as a USB memory).
[0042] The ophthalmic image processing device 21 is described. The ophthalmic image processing device 21 is, for example, disposed in a facility (e.g., a hospital or health diagnosis facility) that performs diagnosis or examination of a subject. The ophthalmic image processing device 21 includes a control unit 22 that performs various control processes and a communication I / F 25. The control unit 22 includes a CPU 23 that is a controller in charge of control and a storage device 24 that can store programs and data. The storage device 24 stores programs for executing ophthalmic image processing (e.g., Figure 7 The high-quality dynamic image generation process shown and Figure 8 The ophthalmic image processing program includes a program for realizing the mathematical model constructed by the mathematical model construction device 1. The communication I / F 25 connects the ophthalmic image processing device 21 with other devices (for example, the ophthalmic image shooting device 11B and the mathematical model construction device 1, etc.).
[0043] The ophthalmologic image processing apparatus 21 is connected to an operation unit 27 and a display device 28. As the operation unit 27 and the display device 28, various devices can be used, similarly to the operation unit 7 and the display device 8 described above.
[0044] The ophthalmic image processing device 21 can obtain ophthalmic images from the ophthalmic image capturing device 11B. The ophthalmic image processing device 21 can obtain ophthalmic images from the ophthalmic image capturing device 11B, for example, through at least one of wired communication, wireless communication, and a removable storage medium (e.g., a USB memory). In addition, the ophthalmic image processing device 21 can also obtain a program or the like for realizing the mathematical model constructed by the mathematical model construction device 1 through communication or the like.
[0045] The ophthalmic image capturing devices 11A and 11B are described. As an example, in the present embodiment, a case where an ophthalmic image capturing device 11A that provides an ophthalmic image to the mathematical model building device 1 and an ophthalmic image capturing device 11B that provides an ophthalmic image to the ophthalmic image processing device 21 are used is described. However, the number of ophthalmic image capturing devices used is not limited to two. For example, the mathematical model building device 1 and the ophthalmic image processing device 21 may also obtain ophthalmic images from a plurality of ophthalmic image capturing devices. In addition, the mathematical model building device 1 and the ophthalmic image processing device 21 may also obtain ophthalmic images from a common ophthalmic image capturing device. In addition, the two ophthalmic image capturing devices 11A and 11B illustrated in the present embodiment have the same structure. Therefore, the two ophthalmic image capturing devices 11A and 11B are collectively described below.
[0046] In addition, in this embodiment, an OCT device is exemplified as the ophthalmic image capturing device 11 (11A, 11B). However, an ophthalmic image capturing device other than the OCT device (for example, a laser scanning fundus examination device (SLO), a fundus camera, or a corneal endothelial cell imaging device (CEM)) may also be used.
[0047] The ophthalmic image capturing device 11 (11A, 11B) includes a control unit 12 (12A, 12B) for performing various control processes and an ophthalmic image capturing unit 16 (16A, 16B). The control unit 12 includes a CPU 13 (13A, 13B) as a controller in charge of control and a storage device 14 (14A, 14B) capable of storing programs, data, etc.
[0048] The ophthalmic image capturing unit 16 has various structures required for capturing an ophthalmic image of the eye to be inspected. The ophthalmic image capturing unit 16 of this embodiment includes an OCT light source, a branching optical element for branching the OCT light emitted from the OCT light source into a measurement light and a reference light, a scanning unit for scanning the measurement light, an optical system for irradiating the measurement light to the eye to be inspected, and a light receiving element for receiving a composite light of the light reflected by the tissue of the eye to be inspected and the reference light.
[0049] The ophthalmic image capturing device 11 is capable of capturing two-dimensional tomographic images and three-dimensional tomographic images of the fundus of the eye to be examined. In detail, the CPU 13 captures a two-dimensional tomographic image of a cross section intersecting the scan line by scanning the OCT light (measurement light) on the scan line. In addition, the CPU 13 can capture a three-dimensional tomographic image in the tissue by scanning the OCT light two-dimensionally. For example, the CPU 13 obtains a plurality of two-dimensional tomographic images by scanning the measurement light on a plurality of scan lines at different positions in a two-dimensional region when the tissue is observed from the front. Then, the CPU 13 obtains a three-dimensional tomographic image by combining the captured plurality of two-dimensional tomographic images.
[0050] Moreover, the CPU 13 scans the measuring light multiple times on the same part of the tissue (in the present embodiment, on the same scanning line), thereby capturing multiple ophthalmic images of the same part. The CPU 13 can obtain an averaged image in which the influence of speckle noise is suppressed by performing an addition averaging process on multiple ophthalmic images of the same part. The addition averaging process can be performed, for example, by averaging the pixel values of pixels at the same position in multiple ophthalmic images. The more images that are subjected to the addition averaging process, the easier it is to suppress the influence of speckle noise, but the shooting time becomes longer. In addition, while shooting multiple ophthalmic images of the same part, the ophthalmic image shooting device 11 performs a tracking process in which the scanning position of the OCT light follows the movement of the eye to be examined.
[0051] (Mathematical model building process)
[0052] Reference Figure 2 to Figure 6 , the mathematical model construction process executed by the mathematical model construction device 1 will be described. The mathematical model construction process is executed by the CPU 3 according to the mathematical model construction program stored in the storage device 4.
[0053] In the mathematical model construction process, the mathematical model is trained by using the training data set, thereby constructing a mathematical model for outputting a high-quality target image based on the base image. The details will be described later, and the training data set includes input training data and output training data. In the present embodiment, as input training data, data based on one to multiple ophthalmic images (in the present embodiment, two-dimensional tomographic images of the fundus) are used. In addition, as output training data, data based on an additive average image of a number of ophthalmic images (H images) greater than the number of ophthalmic images (L images) used for the input training data is used. The input training data and the output training data use the same part of the tissue as the imaging object.
[0054] like Figure 2 As shown, the CPU 3 obtains a set 30 (see FIG. 1 ) of a plurality of training ophthalmic images 300A to 300X (X is an arbitrary integer greater than or equal to 2) obtained by photographing the same part of the tissue. Figure 3 to Figure 6 )(S1). Specifically, in the present embodiment, a set of a plurality of two-dimensional tomographic images captured by scanning the OCT light on the same scanning line on the fundus multiple times is obtained from the ophthalmic image capturing device 11A as a set of ophthalmic images for training. However, the CPU 3 may also obtain a two-dimensional tomographic image by obtaining a signal (e.g., an OCT signal) that is a basis for generating a two-dimensional tomographic image from the ophthalmic image capturing device 11A and generating a two-dimensional tomographic image based on the obtained signal.
[0055] Next, the CPU 3 obtains input training data from a portion of the plurality of training ophthalmic images 300A to 300X in the set 30 (S2). The specific method of obtaining the input training data can be selected as appropriate. For details, please refer to Figure 3 to Figure 6 This will be described later.
[0056] Next, the CPU 3 obtains an average image 31 (see FIG. 3 ) of the plurality of ophthalmic training images 300A to 300X in the set 30. Figure 3 to Figure 6 ) as the output training data (S3). As described above, the number H of training ophthalmologic images used as the added average image 31 as the output training data is greater than the number L of training ophthalmologic images used as the input training data. As an example, in this embodiment, Figure 3 to Figure 6 As shown, all (e.g., 120) of the plurality of training ophthalmic images 300A to 300X in the set 30 are used for the average image 31. The average image 31 may also be generated by the mathematical model construction device 1. The mathematical model construction device 1 may also obtain the average image 31 generated by the ophthalmic image capturing device 11A.
[0057] Reference Figure 3 , the first acquisition mode of input training data is described. In the first acquisition mode, the CPU 3 obtains data of a training ophthalmic image 300A itself (i.e., the original image that has not been subjected to the addition averaging process) among the multiple training ophthalmic images 300A to 300X included in the set 30 as input training data. In this case, the ophthalmic image processing device 21 preferably inputs an original image as a base image into the mathematical model constructed when inputting the base image into the constructed mathematical model to obtain the target image. The ophthalmic image processing device 21 can obtain the target image based on one base image, so it is easy to shorten the shooting time of the base image. In addition, when generating a dynamic image of the target image, it is easy to increase the frame rate of the dynamic image.
[0058] Reference Figure 4 , the second acquisition mode of the input training data is described. In the second acquisition mode, the CPU 3 acquires original image data of a plurality of training ophthalmologic images 300A to 300X included in the set 30 as input training data. Figure 4In the example shown, data of two training ophthalmic images 300A and 300B are obtained as input training data. However, the number of training ophthalmic images obtained as input training data can be changed within a range less than the number of training ophthalmic images used for the added average image 31. When the second acquisition mode is adopted, the ophthalmic image processing device 21 preferably inputs original images of a number as close as possible to the number of training ophthalmic images used as input training data into the mathematical model as basic images when inputting the basic images into the constructed mathematical model. In this case, it is easy to improve the image quality of the obtained target image compared to the case where one original image is input as the basic image into the mathematical model. However, even when the second acquisition mode is adopted, one original image can be input as the basic image into the mathematical model. As described above, when the original image is used as the input training data, the number L of original images used is a number greater than 1 and less than the number used for the added average image 31.
[0059] Reference Figure 5 , the third acquisition mode of input training data is described. In the third acquisition mode, the CPU 3 acquires data of an average image 32 obtained by adding and averaging L (L≥2) training ophthalmic images among the plurality of training ophthalmic images 300A to 300X included in the set 30 as input training data. Figure 5 In the example shown, data of an average image 32 of two training ophthalmic images 300A and 300B is obtained as input training data. However, the number of training ophthalmic images used for the average image 32 can be changed within a range less than the number of training ophthalmic images used for the average image 31. When the third acquisition mode is adopted, the ophthalmic image processing device 21 preferably inputs the average image as the base image when inputting the base image into the constructed mathematical model. The number of ophthalmic images used for the average of the base image is preferably as close as possible to the number of training ophthalmic images used for the average image 32. In this case, it is easy to improve the quality of the obtained target image compared to the case where one original image is input as the base image to the mathematical model. However, even when the third acquisition mode is adopted, one original image can be input as the base image to the mathematical model.
[0060] Reference Figure 6 , the fourth acquisition mode of input training data is described. In the fourth acquisition mode, the CPU 3 acquires a plurality of average images 32 obtained by averaging L (L ≥ 2) training ophthalmic images, and uses the acquired data of the plurality of average images 32 as input training data. Figure 6In the example shown, data of two averaged images 32A and 32B are obtained as input training data. However, the number of averaged images 32 can also be changed. When the fourth acquisition mode is adopted, the ophthalmic image processing device 21 preferably inputs a plurality of averaged images as the base image when inputting the base image to the constructed mathematical model. In this case, it is easy to improve the quality of the obtained target image compared to the case where one original image is input as the base image. However, even when the fourth acquisition mode is adopted, one original image can be input as the base image to the mathematical model.
[0061] The four modes described above are examples, and the above modes can also be changed. For example, the CPU 3 can also use the data of the original images of P training ophthalmological images and the data of the added average image 32 of P' training ophthalmological images as input training data. In this case, the number of training ophthalmological images used as the added average image 31 of the output training data is greater than the number of training ophthalmological images used as the input training data (for example, the sum of P and P').
[0062] In addition, multiple training data sets can also be obtained from a set 30 of training ophthalmic images 300A to 300X. Figure 3 In the example shown, a set of training data sets is obtained, in which the data of the training ophthalmic image 300A is used as input training data and the data of the average image 31 is used as output training data. Here, the CPU 3 can also obtain a training data set in which the data of the training ophthalmic image 300B is used as input training data and the data of the average image 31 is used as output training data from the same set 30.
[0063] return Figure 2 CPU 3 uses a machine learning algorithm to perform training of a mathematical model using a training data set (S4). As a machine learning algorithm, generally known ones include neural networks, random forests, boosting, support vector machines (SVM), etc.
[0064] Neural network is a method of simulating the behavior of biological neural cell networks. Neural networks include, for example, feedforward (forward propagation) neural networks, RBF networks (radial basis functions), spike neural networks, convolutional neural networks, recursive neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Bayesian networks, etc.).
[0065] Random forest is a method for generating a large number of decision trees by learning based on randomly sampled training data. When using random forest, the branches of multiple decision trees that have been pre-learned as recognizers are tracked, and the average (or majority vote) of the results obtained from each decision tree is taken.
[0066] Boosting is a method of generating a strong classifier by combining multiple weak classifiers. A strong classifier is constructed by learning only the weak classifiers one by one.
[0067] SVM is a method of constructing two types of pattern recognizers using linear input elements. SVM learns the parameters of linear input elements based on, for example, the principle of finding a margin-maximizing hyperplane that maximizes the distance to each data point based on training data (hyperplane separating theorem).
[0068] The mathematical model refers to, for example, a data structure for predicting the relationship between input data and output data. The mathematical model is constructed by training using a training data set. The training data set is a collection of input training data and output training data. The mathematical model is trained so that when a certain input training data is input, the corresponding output training data is output. For example, the relevant data (e.g., weight) of each input and output can be updated through training.
[0069] In this embodiment, a multi-layer neural network is used as a machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data to be predicted, and one or more hidden layers between the input layer and the output layer. Multiple nodes (also called neurons) are configured in each layer. In detail, in this embodiment, a convolutional neural network (CNN) is used as a type of multi-layer neural network. However, other machine learning algorithms may also be used. For example, a generative adversarial network (GAN) using two competing neural networks may be used as a machine learning algorithm.
[0070] The processes of S1 to S4 are repeated until the mathematical model is constructed (S5: No). When the mathematical model is constructed (S5: Yes), the mathematical model construction process ends. The program and data for realizing the constructed mathematical model are incorporated into the ophthalmic image processing device 21.
[0071] (Ophthalmic Image Processing)
[0072] Reference Figure 7 to Figure 9 , the ophthalmologic image processing performed by the ophthalmologic image processing device 21 will be described. The ophthalmologic image processing is performed by the CPU 23 according to the ophthalmologic image processing program stored in the storage device 24.
[0073] Reference Figure 7 The high-quality dynamic image generation process is described as an example of ophthalmic image processing. In the high-quality dynamic image generation process, a static image of a high-quality target image is obtained using a mathematical model, and a high-quality dynamic image of a tissue is generated.
[0074] First, the CPU 23 obtains a base image as an ophthalmologic image (static image) (S11). The base image of this embodiment is a two-dimensional tomographic image of the fundus similar to the above-mentioned training ophthalmologic images 300A to 300X. The CPU 23 outputs the base image to the above-mentioned mathematical model, thereby obtaining a target image with higher image quality than the base image (S12).
[0075] exist Figure 7 In the example shown, the ophthalmic image capturing device 11B captures a dynamic image by continuously capturing a two-dimensional tomographic image of the fundus. In S11, the CPU 23 sequentially obtains the static images constituting the dynamic image captured by the ophthalmic image capturing device 11B. The CPU 23 obtains at least one of an original image of the acquired static image and an averaged image obtained by averaging a plurality of original images as a basic image input to the mathematical model. The CPU 23 sequentially inputs the continuously acquired basic images to the mathematical model, thereby continuously obtaining the target image.
[0076] The type of the basic image obtained in S11 (at least one of the original image and the average image) and the number of original images used as the basic image input to the mathematical model can be appropriately set according to the acquisition mode of the input training data described above. When the average image is used as the basic image input to the mathematical model, the CPU 23 can obtain the average image by averaging a plurality of ophthalmic images obtained by photographing the same photographing part. Alternatively, the CPU 23 can obtain the average image generated by the ophthalmic image photographing device 11B.
[0077] In addition, Figure 7 In the illustrated high-quality dynamic image generation process, a dynamic image of the tissue is generated. Here, in order to generate a good dynamic image, it is preferable to increase the frame rate (the number of static images per unit time) as much as possible. That is, in order to increase the frame rate of the dynamic image of the target image, it is necessary to increase the number of target images (static images) obtained per unit time as much as possible. Therefore, in Figure 7 In S11, it is preferred to obtain as few original images as possible as basic images to be input to the mathematical model. For example, the CPU 23 generates a target image by inputting the obtained original image into the mathematical model each time it obtains an original image from the ophthalmic image capturing device 11B, thereby generating a dynamic image with the same frame rate as the dynamic image captured by the ophthalmic image capturing device 11B using the high-quality target image.
[0078] Next, the CPU 23 determines whether an instruction to display the target image on the display device 28 has been input (S14). In the present embodiment, the user can input an instruction to display the target image, the base image, or one of the target image and the base image on the display device 28 to the ophthalmic image processing device 21 by operating the operation unit 27. If no instruction to display the target image has been input (S14: No), the process directly transfers to S17. In the case where an instruction to display the target image has been input (S14: Yes), the CPU 23 displays the target image on the display device 28 (S15). In detail, in S15 of the present embodiment, a plurality of target images continuously acquired in S12 are sequentially displayed on the display device 28, thereby displaying a dynamic image of the tissue. That is, based on the dynamic image of the original image being captured by the ophthalmic image capturing device 11B at that time, a high-quality dynamic image of the tissue being captured is displayed in real time on the display device 28. In addition, the CPU 23 may generate data of a moving image of the target image based on the plurality of target images continuously acquired in S12 , and store the data in the storage device.
[0079] In addition, the ophthalmic image processing device 21 can input instructions for displaying the target image, the base image, or both the target image and the base image on the display device 28 in various ways. For example, the CPU 23 can display an icon for inputting a display instruction for the target image, an icon for inputting a display instruction for the base image, and an icon for inputting a display instruction for both images on the display device 28. The user can input a display instruction to the ophthalmic image processing device 21 by selecting a desired icon.
[0080] Next, the CPU 23 determines whether an instruction to display the base image on the display device 28 has been input (S17). If no instruction has been input (S17: No), the process directly transfers to S20. When an instruction to display the base image has been input (S17: Yes), the CPU 23 displays the base image on the display device 28 (S18). In detail, in S18 of the present embodiment, a plurality of base images continuously acquired in S11 are sequentially displayed on the display device 28, thereby displaying a dynamic image of the organization. As described above, the CPU 23 can switch the target image and the base image according to the instruction input by the user and display them on the display device 28. Therefore, the user can easily confirm both the target image and the base image.
[0081] Next, the CPU 23 determines whether an instruction to display the target image and the base image simultaneously on the display device 28 has been input (S20). If no instruction has been input (S20: No), the process directly transfers to S23. When a simultaneous display instruction has been input (S20: Yes), the CPU 23 displays the target image and the base image simultaneously on the display device 28. Specifically, in S20 of the present embodiment, the dynamic image of the target image and the dynamic image of the base image are displayed together on the display device 28.
[0082] In addition, in S15, S18, and S21, a static image may be displayed instead of a dynamic image. The CPU 23 may also switch between displaying a dynamic image and displaying a static image according to an instruction input by a user. In addition, the CPU 23 may store data of at least one of the dynamic image and the static image in the storage device 24 instead of displaying the dynamic image and the static image on the display device 28.
[0083] Furthermore, when the CPU 23 displays the dynamic image or static image of the target image on the display device 28, it can display the target image equivalent to the average image of several ophthalmic images on the display device 28. For example, when constructing a mathematical model, if the average image 32 used as the output training data is an average image based on H ophthalmic images, the target image obtained in S12 also becomes an image equivalent to the average image of H ophthalmic images. Therefore, the CPU 23 displays on the display device 28 that the target image being displayed is equivalent to the average image of H ophthalmic images. Therefore, the user can appropriately grasp the quality of the target image.
[0084] In addition, as the timing for displaying the target image, there are multiple timings, such as the timing for real-time display during the photographing process by the ophthalmic image photographing device 11B, the timing for the user to confirm whether the photographed image is good, and the timing for displaying it together with the analysis result or report after photographing. The CPU 23 can input an instruction for each of the multiple timings at which the target image can be displayed, namely, the target image, the base image, and which one of the target image and the base image is to be displayed on the display device 28. That is, the user can pre-set which one of the target image, the base image, and which one of the target image and the base image is to be displayed on the display device 28 according to the timing at which the target image can be displayed. As a result, the convenience of the user is further improved.
[0085] Next, CPU23 determines whether an instruction to end the display has been input (S23). If no instruction has been input (S23: No), the process returns to S11 and the processes of S11 to S23 are repeated. When an instruction to end the display has been input (S23: Yes), CPU23 performs a segmentation process on the target image (in the present embodiment, at least one of the multiple target images obtained continuously) (S24). The so-called segmentation process is a process for identifying at least one of the multiple layers of tissue shown in the image. The identification of the layer can be performed, for example, by image processing of the target image, or by using a mathematical model trained by a machine learning algorithm. By performing the segmentation process on the target image, a segmentation result that suppresses the influence of noise can be appropriately obtained. In addition, there is also a case where the user operates the operation unit 27 and manually performs the segmentation process. In this case, CPU23 can also display the target image on the display device 28 as an image to be segmented.
[0086] Reference Figure 8 and Fig. 9 , a high-quality three-dimensional tomographic image generation process as an example of ophthalmic image processing is described. In the high-quality three-dimensional tomographic image generation process, a mathematical model is used to obtain a high-quality two-dimensional tomographic image and a three-dimensional tomographic image. When the high-quality three-dimensional tomographic image generation process is performed, in the ophthalmic image shooting device 11B, a three-dimensional tomographic image in the tissue is captured by two-dimensionally scanning the OCT light (measurement light). In detail, the ophthalmic image shooting device 11B obtains a plurality of (first to Zth) two-dimensional tomographic images by scanning the measurement light on a plurality of scanning lines at different positions in a two-dimensional region when the tissue is observed from the front. A three-dimensional tomographic image is obtained by combining a plurality of two-dimensional tomographic images.
[0087] CPU23 obtains the Nth (the initial value of N is "1") two-dimensional tomographic image among the multiple (first to Zth) two-dimensional tomographic images captured by the ophthalmic image capturing device 1B as a base image (S31). CPU23 obtains the Nth target image by inputting the obtained base image into the mathematical model (S32). Then, CPU23 determines whether the processing of all the multiple (first to Zth) base images is completed (S33). If not completed (S33: No), "1" is added to the value of N (S34), and the processing returns to S31. The processing from S31 to S33 is repeated until the processing of all the base images is completed.
[0088] When the processing of all the basic images is completed (S33: Yes), a three-dimensional tomographic image 40 of the tissue is generated by combining the acquired plurality of target images (see Fig. 9 )(S35). According to Figure 8The illustrated high-quality three-dimensional tomographic image generation process can generate a three-dimensional tomographic image using a target image that suppresses the influence of noise to the same extent as the average image, even without using an average image as each of the two-dimensional tomographic images constituting the three-dimensional tomographic image.
[0089] In addition, the CPU 23 can also generate images other than the three-dimensional tomographic image based on the plurality of target images. For example, the CPU 23 can generate a two-dimensional frontal image (so-called "enface image") when observing the tissue from the front based on the plurality of target images acquired in S32.
[0090] In addition, the CPU 23 can also perform analysis processing on the three-dimensional tomographic image 40 generated based on the multiple target images. For example, the CPU 23 can perform segmentation processing on the three-dimensional tomographic image 40 to extract at least one of the multiple layers included in the fundus. The CPU 23 can also generate a thickness map that two-dimensionally represents the thickness distribution of at least one of the layers based on the result of the segmentation processing. In this case, the thickness map can be appropriately generated based on the three-dimensional tomographic image 40 in which the influence of noise is suppressed.
[0091] In addition, the CPU 23 may also generate a two-dimensional tomographic image of an arbitrary position in the tissue included in the shooting range of the three-dimensional tomographic image 40 based on the three-dimensional tomographic image 40 generated based on the plurality of target images. The CPU 23 may also display the generated two-dimensional tomographic image on the display device 28. As an example, the CPU 23 of the present embodiment inputs an instruction from the user for specifying the position for generating the tomographic image. For example, the user may specify the generating position of the tomographic image on the two-dimensional frontal image or the three-dimensional tomographic image 40 displayed on the display device 28 by operating the operation unit 27. The generating position of the tomographic image may be specified, for example, by lines of various shapes (lines on a straight line, circular lines, etc.). The CPU 23 may also generate (extract) a two-dimensional tomographic image of a position specified by the user from the three-dimensional tomographic image 40. In this case, the tomographic image of an arbitrary position desired by the user can be appropriately generated based on the three-dimensional tomographic image 40 in which the influence of noise is suppressed.
[0092] In addition, the CPU 23 may also determine whether the ophthalmic image captured by the ophthalmic image capturing device 11B is good each time one or more ophthalmic images are captured. The CPU 23 may input the good ophthalmic image as the base image into the mathematical model when it is determined that the captured ophthalmic image is good. In addition, the CPU 23 may output an instruction to re-record the ophthalmic image at the same position to the ophthalmic image capturing device 11B when it is determined that the captured ophthalmic image is not good. In this case, the target image can be obtained based on the good base image. In addition, the specific method for determining whether the captured ophthalmic image is good can be appropriately selected. For example, the CPU 23 may use an index (for example, SSI (Signal Strength Index: signal strength index) or QI (Quality Index: quality index) etc.) representing the strength of the signal of the captured ophthalmic image to determine whether the ophthalmic image is good. In addition, the CPU 23 may also determine whether the ophthalmic image is good by comparing the captured ophthalmic image with the template.
[0093] In addition, the CPU 23 may extract a good image from the multiple target images or the multiple base images that are the basis of the multiple target images when performing an analysis process (for example, a segmentation process) on any one of the multiple target images. The CPU 23 may also perform an analysis process on the extracted good target image or the target image based on the good base image. In this case, the accuracy of the analysis process is improved. In addition, the method of extracting a good image may adopt various methods such as the method using the aforementioned SSI.
[0094] In addition, when generating a motion contrast image, the CPU 23 may also use the mathematical model illustrated in the present disclosure to obtain a target image, and generate a motion contrast image based on the obtained target image. As described above, when generating a motion contrast image, multiple OCT signals are obtained at different times for the same position, and the multiple OCT signals are processed by calculation. The CPU 23 may also obtain a target image based on each of the multiple OCT signals (two-dimensional tomographic images), and obtain a motion contrast image by performing calculation processing on the multiple obtained target images. In this case, a motion contrast image in which the influence of noise is suppressed can be appropriately generated.
[0095] The technology disclosed in the above embodiment is only an example. Therefore, the technology illustrated in the above embodiment can also be changed. First, it is also possible to execute only a part of the multiple technologies illustrated in the above embodiment. For example, the ophthalmic image processing device 21 can also execute only the high-quality dynamic image generation process (see Figure 7 ) and high-quality 3D tomographic image generation processing (refer to Figure 8 ) in the case of a party.
[0096] In addition, Figure 7 S11 and Figure 8 The process of obtaining the base image in S31 is an example of the "base image obtaining step". Figure 7 S12 and Figure 8 The process of acquiring the target image in S32 is an example of the "target image acquisition step". Figure 8 The process of generating a three-dimensional tomographic image in S35 is an example of a “three-dimensional tomographic image generating step”. Figure 7 The segmentation process shown in S24 is an example of a "segmentation step". Figure 7 The process of switching the display target image and the base image in S14 to S18 is an example of "display switching process" and "display switching step". Figure 7 The process of simultaneously displaying the target image and the base image in S21 is an example of "simultaneous display processing" and "simultaneous display step". Figure 7 The process of generating dynamic image data of the target image in S15 is an example of a "dynamic image generating step". Figure 2 The process of acquiring the set 30 of ophthalmic images for training in S1 is an example of the "image set acquisition step". Figure 2 The process of constructing a mathematical model in S2 to S4 is an example of a “training step”.
[0097] Description of symbols
[0098] 1. Mathematical model construction device
[0099] 3 CPU
[0100] 4 Storage Devices
[0101] 11 Ophthalmic image capture device
[0102] 21 Ophthalmic image processing device
[0103] 23 CPU
[0104] 24 Storage Devices
[0105] 28 Display device
[0106] 30 Collections
[0107] 31 Additive average image
[0108] 32 Additive average image
[0109] 300 training ophthalmological images
Claims
1. An ophthalmic image processing device for processing an ophthalmic image which is an image of tissue of an eye to be inspected, characterized in that: The control unit of the ophthalmic image processing device obtains an ophthalmic image obtained by an ophthalmic image capturing device, or an averaged image obtained by averaging a plurality of ophthalmic images obtained by capturing the same part of a tissue by the ophthalmic image capturing device as a basic image, and obtains a target image with higher image quality than the basic image by inputting the basic image into a mathematical model trained by a machine learning algorithm, and extracts a two-dimensional tomographic image of a designated position in the tissue included in the capturing range of the target image and displays it on a display device, The mathematical model is trained using data of L training ophthalmic images among a plurality of training ophthalmic images obtained by photographing the same part of the tissue as input training data, and using data of an average image obtained by adding and averaging H training ophthalmic images as output training data, wherein L≥1, H>L, The ophthalmic image is a two-dimensional tomographic image of the eye to be examined obtained by taking an image with an OCT device. The control unit obtains a plurality of two-dimensional tomographic images obtained by photographing different positions of the tissue as the basic image, and obtains a plurality of target images by inputting each of the plurality of basic images into the mathematical model, and based on the plurality of target images, generates at least one of a three-dimensional tomographic image of the tissue and a two-dimensional frontal image when observing the tissue from the front.
2. The ophthalmic image processing device according to claim 1, characterized in that: The ophthalmological image is a two-dimensional tomographic image of the fundus of the eye to be examined obtained by taking an image with an OCT device. The control unit recognizes at least any one of a plurality of layers included in the fundus by performing analysis processing on the target image acquired using the mathematical model.
3. The ophthalmic image processing device according to claim 1 or 2, characterized in that: The control unit performs at least one of a simultaneous display process of displaying the base image and the target image on the display device at the same time, and a switching display process of switching and displaying the base image and the target image on the display device according to an input instruction.
4. The ophthalmic image processing device according to claim 1 or 2, characterized in that: The control unit obtains a plurality of the basic images continuously captured by the ophthalmic image capturing device, and sequentially inputs the obtained plurality of basic images into the mathematical model, thereby obtaining a plurality of the target images, and generates dynamic image data of the tissue based on the obtained plurality of target images.
5. The ophthalmic image processing device according to claim 1 or 2, characterized in that: It is preset that both the target image and the base image or which one of the target image and the base image is to be displayed on the display device.
6. An ophthalmic image processing device for processing an ophthalmic image which is an image of tissue of an eye to be inspected, characterized in that: The control unit of the ophthalmic image processing device obtains an ophthalmic image obtained by an ophthalmic image capturing device, or an averaged image obtained by averaging a plurality of ophthalmic images obtained by capturing the same part of a tissue by the ophthalmic image capturing device as a basic image, and obtains a target image with higher image quality than the basic image by inputting the basic image into a mathematical model trained by a machine learning algorithm, and extracts a two-dimensional tomographic image of a designated position in the tissue included in the capturing range of the target image and displays it on a display device, The mathematical model is trained using data of L training ophthalmic images among a plurality of training ophthalmic images obtained by photographing the same part of the tissue as input training data, and using data of an average image obtained by adding and averaging H training ophthalmic images as output training data, wherein L≥1, H>L, The ophthalmic image is an OCT angiography image.
7. An OCT device for capturing an ophthalmic image of a tissue of an eye to be inspected by processing an OCT signal formed by reference light and reflected light of a measurement light irradiated to the tissue of the eye to be inspected, characterized in that: The control unit of the OCT device obtains an ophthalmic image of the tissue taken, or an average image obtained by adding and averaging a plurality of ophthalmic images taken of the same part of the tissue as a basic image, and inputs the basic image into a mathematical model trained using a machine learning algorithm, thereby obtaining a target image with higher image quality than the basic image. The mathematical model is trained using data of L training ophthalmic images among a plurality of training ophthalmic images obtained by photographing the same part of the tissue as input training data, and using data of an average image obtained by adding and averaging H training ophthalmic images as output training data, wherein L≥1, H>L, The ophthalmic image is a two-dimensional tomographic image of the eye to be inspected obtained by taking the image with the OCT device. The control unit obtains a plurality of two-dimensional tomographic images obtained by photographing different positions of the tissue as the basic image, and obtains a plurality of target images by inputting each of the plurality of basic images into the mathematical model, and based on the plurality of target images, generates at least one of a three-dimensional tomographic image of the tissue and a two-dimensional frontal image when observing the tissue from the front.
8. A non-transitory computer-readable storage medium storing an ophthalmic image processing program, the ophthalmic image processing program being executed by an ophthalmic image processing device for processing an ophthalmic image as an image of tissue of an eye to be inspected, characterized in that: The ophthalmologic image processing program is executed by the control unit of the ophthalmologic image processing device, thereby causing the ophthalmologic image processing device to execute the following steps: a basic image acquisition step of acquiring an ophthalmic image captured by an ophthalmic image capturing device, or an averaged image obtained by averaging a plurality of ophthalmic images captured by the ophthalmic image capturing device at the same part of the tissue as a basic image; a target image acquisition step, wherein the base image is input into a mathematical model trained by a machine learning algorithm, thereby acquiring a target image having a higher quality than the base image; and a tomographic image acquisition step of extracting a two-dimensional tomographic image of a designated position in the tissue included in the imaging range of the target image and displaying the two-dimensional tomographic image on a display device, The mathematical model is trained using data of L training ophthalmic images among a plurality of training ophthalmic images obtained by photographing the same part of the tissue as input training data, and using data of an average image obtained by adding and averaging H training ophthalmic images as output training data, wherein L≥1, H>L, The ophthalmic image is a two-dimensional tomographic image of the eye to be examined obtained by taking an image with an OCT device. In the basic image acquisition step, a plurality of two-dimensional tomographic images obtained by photographing different positions of the tissue are acquired as the basic image. In the target image acquisition step, a plurality of target images are acquired by inputting each of the plurality of base images into the mathematical model. In the tomographic image acquisition step, at least one of a three-dimensional tomographic image of the tissue and a two-dimensional frontal image when the tissue is observed from the front is generated based on the plurality of target images.
9. A mathematical model construction method, executed by a mathematical model construction device for constructing a mathematical model, wherein the mathematical model is constructed by training using a machine learning algorithm and outputs a target image by inputting a base image, the mathematical model construction method comprising the following steps: an image set acquisition step of acquiring a plurality of ophthalmic training images obtained by photographing the same part of the tissue of the eye to be examined; and A training step, wherein the mathematical model is constructed by training the mathematical model using data based on L of the plurality of training ophthalmic images as input training data and using data of an averaged image obtained by adding and averaging H of the training ophthalmic images as output training data, wherein: L≥1, H>L, The basic image is a two-dimensional tomographic image of the eye to be inspected obtained by an OCT device. The mathematical model is used in the target image acquisition step of the ophthalmic image processing program, which performs the following steps: A basic image acquisition step, acquiring a plurality of two-dimensional tomographic images obtained by photographing different positions of the tissue as the basic image; The target image acquisition step is to acquire a plurality of target images by inputting each of the plurality of basic images into the mathematical model; and The tomographic image acquisition step generates at least one of a three-dimensional tomographic image of the tissue and a two-dimensional frontal image when the tissue is observed from the front, based on the plurality of target images.
