Ophthalmology image processing system, image processing program, learning apparatus

By combining OCT motion contrast images from different time intervals using a trained model with more data sets than used for training, the method addresses the challenge of obtaining high-quality OCT motion contrast composite images efficiently.

JP2024158728APending Publication Date: 2024-11-08CANON KK
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023074204
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing methods for obtaining OCT motion contrast composite images with a high signal-to-noise ratio require numerous OCT data sets, which are difficult to obtain easily.

Method used

The method involves generating OCT motion contrast images at different time intervals and using a trained model to combine these images, ensuring that the number of OCT data sets used for input exceeds the number used for training, thereby enhancing image quality.

Benefits of technology

This approach allows for the easy generation of high-quality OCT motion contrast composite images without increasing imaging time, improving image clarity and contrast.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024158728000001_ABST
    Figure 2024158728000001_ABST
Patent Text Reader

Abstract

To easily acquire OCT motion contrast composite images that have high SN ratios and have different time intervals.SOLUTION: A learning apparatus performs learning using learning data that includes input data that is an image based on a third OCT motion contrast image based on an OCT data set of M1 group at a first time interval and a fourth OCT motion contrast image based on an OCT data set of M2 group at a second time interval, and output data that is an OCT motion contrast combination image obtained by combining a fifth OCT motion contrast image based on an OCT data set of N1 group at the first time interval with a sixth OCT motion contrast image based on an OCT data set of N2 group at the second time interval and is subjected to learning using learning data satisfying the following formulas (1) to (3): (1) M1≤N1, (2) M2≤N2, (3) M1<N1 or / and M2<N2 (where all of M1, M2, N1, and N2 are natural numbers)SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an ophthalmologic image processing device, an image processing program, and a learning device. [Background technology]

[0002] As ophthalmic image processing devices, devices for acquiring two-dimensional images of the fundus of a test eye (hereinafter, referred to as a fundus camera device) and devices for acquiring tomographic images of a test eye using optical coherence tomography (OCT) with low-coherence light (hereinafter, referred to as an OCT device) have been put to practical use. Some of these devices can obtain blood vessel information of a test eye by generating motion contrast data based on OCT data obtained by photographing the same position of the same test eye multiple times.

[0003] Patent document 1 discloses an OCT signal processing device that selects multiple sets of OCT data with different time intervals, removes OCT motion contrasts with low S / N ratios from the multiple OCT motion contrasts calculated for each set, and performs statistical processing to obtain a composite motion contrast with a high S / N ratio. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] WO17 / 119437 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in order to obtain an OCT motion contrast composite image having a high S / N ratio at different time intervals using the method of Patent Document 1, many sets of OCT data must be obtained, which is problematic in that it is not easy to obtain. [Means for solving the problem]

[0006] In one embodiment of the present disclosure, an OCT motion contrast image acquisition unit acquires a first OCT motion contrast image generated using a set of OCT data for a first time interval, and a second OCT motion contrast image generated using a set of OCT data for a second time interval different from the first time interval; an OCT motion contrast synthetic image output unit that outputs an image quality improved OCT motion contrast synthetic image by inputting an image based on the first OCT motion contrast image and the second OCT motion contrast image to a trained model, The trained model is The image processing device includes: input data that is an image based on a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval; and output data that is an OCT motion contrast synthetic image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and is a trained model trained using training data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1 <N1もしくは / およびM2<N2 (where M1, M2, N1, and N2 are all natural numbers)

[0007] In one embodiment of the present disclosure, an OCT motion contrast image acquisition unit acquires a first OCT motion contrast image generated using M1 sets of OCT data at a first time interval, and a second OCT motion contrast image generated using M2 sets of OCT data at a second time interval different from the first time interval; an output unit that outputs a first OCT motion contrast image with improved image quality by inputting the first OCT motion contrast image to a first trained model, and outputs a second OCT motion contrast image with improved image quality by inputting the second OCT motion contrast image to a second trained model; a composite image generating unit that generates a composite image of a first OCT motion contrast image with improved image quality and a second OCT motion contrast image with improved image quality; An image processing device comprising: the first trained model is a trained model trained with training data including input data which is a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval, and output data which is a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval; The image processing device is a trained model, wherein the second trained model is trained using training data including input data which is a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and output data which is a sixth OCT motion contrast image generated using N1 sets of OCT data at the first time interval. (Note that M1, M2, N1, and N2 are all natural numbers, and M1 <N1かつM2<N2)

[0008] In one embodiment of the present disclosure, an OCT motion contrast image acquisition step includes acquiring a first OCT motion contrast image generated using a set of OCT data for a first time interval, and a second OCT motion contrast image generated using a set of OCT data for a second time interval different from the first time interval; and an output step of an OCT motion contrast synthetic image, which outputs an OCT motion contrast synthetic image having a high image quality by inputting an image based on the first OCT motion contrast image and the second OCT motion contrast image to a trained model, The trained model is The image processing program includes: input data that is an image based on a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval; and output data that is an OCT motion contrast synthetic image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and is a trained model trained using training data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1 <N1もしくは / およびM2<N2 (where M1, M2, N1, and N2 are all natural numbers)

[0009] In one embodiment of the present disclosure, an OCT motion contrast image acquisition step includes acquiring a first OCT motion contrast image generated using M1 sets of OCT data for a first time interval, and a second OCT motion contrast image generated using M2 sets of OCT data for a second time interval different from the first time interval; an output step of inputting the first OCT motion contrast image to a first trained model to output a first OCT motion contrast image with improved image quality, and inputting the second OCT motion contrast image to a second trained model to output a second OCT motion contrast image with improved image quality; a composite image generating step of generating a composite image of the enhanced first OCT motion contrast image and the enhanced second OCT motion contrast image; An image processing program comprising: the first trained model is a trained model trained using training data including input data that is a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval, and output data that is a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval; The image processing program is a trained model, wherein the second trained model is trained using training data including input data which is a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and output data which is a sixth OCT motion contrast image generated using N1 sets of OCT data at the first time interval. (Note that M1, M2, N1, and N2 are all natural numbers, and M1 <N1かつM2<N2)

[0010] In one embodiment of the present disclosure, the learning device includes input data that is an image based on a third OCT motion contrast image generated using M1 sets of OCT data at a first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at a second time interval different from the first time interval, and output data that is an OCT motion contrast synthetic image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and learns using learning data that satisfies the following equations (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1 <N1もしくは / およびM2<N2 (where M1, M2, N1, and N2 are all natural numbers) Effect of the Invention

[0011] According to one embodiment of the present disclosure, it is possible to easily obtain a high-quality OCT motion contrast composite image based on OCT motion contrast images taken at different time intervals. [Brief description of the drawings]

[0012] [Figure 1] 1(a) to 1(c) are diagrams illustrating an example of a configuration of an ophthalmologic image processing apparatus according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a processing flow of the ophthalmologic image processing apparatus according to the first embodiment. [Diagram 3] 5A and 5B are diagrams showing an example of a report screen of the ophthalmologic image-processing device according to the first embodiment. [Figure 4] 5A and 5B are diagrams illustrating an example of steps for generating an improved image quality OCT en face motion contrast image according to the first embodiment. [Diagram 5] FIG. 11 is a diagram showing an example of a processing flow of the ophthalmologic image processing apparatus according to the second embodiment. [Figure 6] 13A and 13B are diagrams illustrating an example of steps for generating an improved image quality OCT en face motion contrast image according to the second embodiment. [Figure 7] FIG. 11 is a diagram showing an example of a processing flow of an ophthalmologic image processing apparatus according to a modified example. [Figure 8] 13A is a diagram showing an example of a report screen according to Example 3. FIG. 13B is a diagram showing an example of an analysis screen according to Example 3. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Exemplary embodiments and examples of the present disclosure will be described in detail below with reference to the drawings. However, dimensions, materials, shapes, and relative positions of components used in the following description are arbitrary and can be changed according to the configuration of an apparatus to which the present disclosure is applied or various conditions. In addition, in the drawings, the same reference numerals are used between the drawings to indicate elements that are the same or functionally similar.

[0014] <Device configuration> An example of an ophthalmologic image processing apparatus according to this embodiment will be described.

[0015] As shown in FIG. 1(a), the ophthalmologic image processing apparatus of this embodiment includes an OCT imaging unit 1, a personal computer 102, a display unit 103, and an operation unit 104 such as a mouse and a keyboard, and the personal computer 102 is connected to the OCT imaging unit 1. The personal computer 102 also includes a storage unit 101 and a control unit 105. Note that, in this embodiment, the ophthalmologic image processing apparatus includes the OCT imaging unit 1 and is configured to acquire OCT images and OCT front motion contrast images using the OCT imaging unit 1, but the apparatus may not include the OCT imaging unit 1 and may acquire these images from a server or another image processing apparatus. Note that, in this embodiment, the display unit 103 and the operation unit 104 are configured separately, but they may be integrated into one unit such as a touch panel.

[0016] The CPU built into the control unit 105 reads a program stored in the memory unit 101, such as a hard disk, to generate images, and also performs functions such as processing image data imported from the memory unit 101 and generating screens to be displayed by the display unit 103, and also controls the OCT imaging unit 1, including the stage unit 111.

[0017] As shown in Fig. 1(c), the control unit 105 includes a data acquisition unit 1a, an optical system control unit 1b, a fundus front image acquisition unit 1c, an OCT data set acquisition unit 1d, an OCT motion contrast image generation unit 1e, an En-Face image generation unit 1f, a display control unit 1g (which can also be called an output unit), a composite image generation unit 1h, an image quality improvement unit 1i, an operation reception unit 1j, a report generation unit 1k, and the like. The control unit 105 may also include a determination unit that determines whether or not the scanning pattern of OCT data acquired in a past examination is included in the acquisition area of ​​the newly set three-dimensional volume OCT data (details will be described later in Example 3). The operation reception unit 1j receives an operation performed by the operator using the operation unit 104.

[0018] The OCT imaging unit 1 is composed of an optical head 110, which is a measurement optical system for acquiring anterior eye images, frontal images of the fundus, and tomographic images, a stage unit 111 that drives it in the up, down, left and right directions using a motor or the like, a base unit 112 that incorporates a spectrometer, etc., and a chin rest 113 that fixes the subject's chin and forehead.

[0019] 1(b), the optical configuration of the OCT imaging section 1 will be described. The OCT imaging section 1 includes an imaging optical system 10 and a spectroscope 20.

[0020] The photographing optical system 10 is composed of a measurement optical system for capturing front images and tomographic images of the anterior eye Ea of the subject's eye E and the fundus Er of the subject's eye. An objective lens 101-1 is installed facing the subject's eye E, and the optical paths are separated by a first dichroic mirror 102 and a second dichroic mirror 103 provided behind the objective lens 101-1 on the optical axis, and an anterior eye observation optical path L1, a fundus observation and fixation target optical path L2, and an OCT optical system measurement optical path L3 are formed for each wavelength band.

[0021] A lens 141 and an imaging element 142 for anterior eye observation having sensitivity to the wavelength of irradiation light for anterior eye observation (not shown), specifically, around 970 nm, are arranged on the anterior eye observation optical path L1.

[0022] In the fundus observation and fixation lamp optical path L2, a relay lens 101-2 is arranged, and further, an X scanner 117-1 (for the main scanning direction) and a Y scanner 117-2 (for the sub-scanning direction intersecting with the main scanning direction) are arranged behind the relay lens 101-2 at a position conjugate with the pupil of the subject's eye to scan the fundus Er of the subject's eye E with light emitted from an illumination light source for fundus observation (not shown). Behind the relay lens 101-2, a focus lens 111 and a lens 112 are arranged, and the light is separated by a third dichroic mirror 118 into optical paths to an APD (avalanche photodiode) 115 for fundus observation and a fixation lamp 116 for each wavelength band. The focus lens 111 is driven by a drive motor (not shown) for focus adjustment for the fixation lamp and fundus observation, and can adjust the focus (focus adjustment) between the diaphragm and the fixation lamp 116 placed in front of the APD (avalanche photodiode) 115 for fundus observation and the fundus of the subject's eye. The APD 115 is a light receiving element for detecting light scattered and reflected back from the fundus Er, and has sensitivity to the wavelength of an SLO light source (not shown), specifically, around 780 nm. Note that the SLO light source (not shown) and the APD 115 are usually arranged in a form that separates the light emitted from the light source and the light returned from the fundus Er using a prism with a holed mirror or a hollow mirror vapor-deposited thereon (not shown). On the other hand, the fixation lamp 116 is used to generate a fixation target that generates visible light to encourage the subject to fixate.

[0023] As described above, the OCT optical system measurement light path L3 forms an OCT optical system and is used to capture a tomographic image of the fundus Er of the test eye. Specifically, it is used to obtain an interference signal for forming a tomographic image. On the OCT optical system measurement light path L3, a lens 101-3, a mirror 121, and an X scanner 122-1 and a Y scanner 122-2 that function as a scanning optical system for scanning the fundus Er of the test eye E with light are arranged. The X scanner 122-1 and the Y scanner 122-2 are arranged so that the vicinity of the center is the focal position of the lens 101-3, and further, the vicinity of the center and the position of the pupil of the test eye E are in an optically conjugate relationship. In addition, the measurement light source 130 is an SLD (Super Luminescent Diode), which is a low coherent light source for making the measurement light incident on the measurement light path. The central wavelength is 855 nm, and the wavelength bandwidth is about 100 nm. Here, the bandwidth is an important parameter because it affects the resolution in the optical axis direction of the obtained tomographic image. Although an SLD is selected as the light source here, it is sufficient to emit low coherent light, and ASE (Amplified Spontaneous Emission) or the like may also be used. Considering that the central wavelength is to be measured on the eye, near-infrared light is suitable. In addition, since the central wavelength affects the lateral resolution of the obtained tomographic image, it is desirable that the wavelength be as short as possible. Reference numerals 123 and 124 denote lenses, and lens 123 is driven by a drive motor (not shown) for focus adjustment.

[0024] Next, a description will be given of the optical path from the measurement light source 130, the reference optical system, and the spectrometer 20. A Michelson interference system is configured by the measurement light source 130, the optical coupler 125, the optical fibers 125-1 to 125-4, the lens 151, the dispersion compensation glass 152, the reference mirror 153, and the spectrometer 20. The optical fibers 125-1 to 125-4 are single-mode optical fibers that are connected to and integrated with the optical coupler 125.

[0025] Light emitted from the measurement light source 130 passes through optical fiber 125-1 and is split into measurement light on the optical fiber 125-2 side and reference light on the optical fiber 125-3 side via the optical coupler 125. The measurement light passes through the above-mentioned OCT optical system measurement optical path L3, is irradiated onto the fundus Er of the subject's eye E, which is the observation target, and reaches the optical coupler 125 through the same optical path due to reflection and scattering by the retina.

[0026] On the other hand, the reference light passes through optical fiber 125-3, lens 151, and dispersion compensation glass 152 and reaches reference mirror 153 where it is reflected. Then, it travels along the same optical path and reaches optical coupler 125, where the measurement light and reference light are multiplexed to become interference light by optical coupler 125. Here, interference occurs when the optical path length of the measurement light and the optical path length of the reference light become approximately the same.

[0027] The position of the reference mirror 153 can be adjusted in the optical axis direction by a drive motor and a drive mechanism (not shown). The interference light is guided to the light receiving optical system 20 via an optical fiber 125-4.

[0028] The light receiving optical system 20 includes a lens 201, a diffraction grating 202, a lens 203, and a line sensor 204. The interference light emitted from the optical fiber 125-4 becomes approximately parallel light via the lens 201, is then dispersed by the diffraction grating 202, and is imaged on the line sensor 204 by the lens 203. The line sensor 204 is configured with a plurality of pixels, i.e., light receiving elements, arranged in a line, and can read out all the pixels at once according to a predetermined clock.

[0029] In this embodiment, a Michelson interferometer is used as the interferometer, but a Mach-Zehnder interferometer or the like may also be used.

[0030] <Acquisition of front image> Next, a method for acquiring a front image using the OCT imaging unit 1 will be described.

[0031] Light emitted from an SLO light source (not shown) is reflected by a third dichroic mirror 118, passes through lenses 112 and 111, and is deflected by an X scanner 117-1 and a Y scanner 117-2 to scan the fundus Er of the subject eye, and the deflected light passes through the pupil of the subject eye by a lens 101-2 and an objective lens 101-1 to scan the fundus Er of the subject eye. Return light from the fundus Er of the subject eye travels the same path as the projection light, is reflected by a dichroic mirror 118, and is received by an APD 115. The control unit 105 generates a front image (two-dimensional front image) of the fundus of the subject eye by associating this light reception signal with the scanning positions of the X scanner 117-1 and the Y scanner 117-2.

[0032] Meanwhile, the light emitted from the fixation light 115 also passes through the third dichroic mirror 118, similarly passes through the lenses 112 and 111, and is scanned on the fundus Er of the subject's eye by the X scanner 117-1 and the Y scanner 117-2. At this time, by blinking the fixation light 115 in accordance with the movements of the X scanner 117-1 and the Y scanner 117-2, a fixation light of any shape can be presented at any position on the fundus Er of the subject's eye.

[0033] In this embodiment, an SLO image is used as the front image, but the effect of the present invention is not impaired if a fundus camera image is acquired using a fundus camera, etc. When a fundus camera, etc. is used, the fixation light may take the form of a conventional fixation light for a fundus camera, and since this is a technology well known to those skilled in the art, a description thereof will be omitted.

[0034] <Acquisition of tomographic images> Next, a method for acquiring a tomographic image using the OCT imaging unit 1 will be described.

[0035] The control unit 105 controls the X scanner 122-1 and the Y scanner 122-2 to irradiate the measurement light to a desired position within the X imaging range of the fundus Er of the subject's eye E. The return light is combined with the reference light as described above to become interference light, which is then dispersed by the spectroscope 230 and imaged by the line sensor 204. The spectral distribution of the interference light acquired by the line sensor 231 is converted to a wave number distribution, and then Fast Fourier Transform (FFT) is performed to obtain an intensity distribution in the depth (Z) axis direction in the subject space. Here, the spectral distribution obtained by the line sensor 204 is called one-dimensional OCT data (A-scan data), and the data obtained by performing FFT and converting the data into pixel values ​​(BW density or color signal information) to display the obtained intensity distribution on a monitor is called a one-dimensional OCT image (A-scan image). Additionally, two-dimensional data consisting of multiple one-dimensional OCT data arranged in the main scanning (X) direction is called two-dimensional OCT data (B-scan data), and a two-dimensional image consisting of an array of one-dimensional OCT images is called a two-dimensional OCT image (B-scan image). After acquiring B-scan data, the scanning position is moved sequentially at equal intervals in the sub-scanning (Y) direction and scanning is performed again in the X direction (so-called raster scanning). The OCT data consisting of multiple two-dimensional OCT data obtained is called three-dimensional volume OCT data, and a three-dimensional image consisting of an array of multiple two-dimensional OCT images is called a three-dimensional OCT image.

[0036] In this embodiment, point scanning OCT is used as a method for acquiring three-dimensional volume OCT data, and raster scanning is used as the scanning pattern. However, the imaging method and scanning pattern may be any method that can densely acquire three-dimensional volume OCT data from a two-dimensional area on the fundus. For example, dense three-dimensional radial scanning or Lissajous scanning by point scanning OCT, line scanning imaging by line scanning OCT, or surface imaging by full-field OCT may be used. Line field OCT is a method for acquiring OCT data by irradiating a subject with measurement light spreading on a line and receiving signal light obtained by interfering with return light of this measurement light and a reference light also having a line shape. The measurement light is linearly scanned on the measurement area in a direction intersecting the measurement light to acquire three-dimensional volume OCT data. In this case, the OCT imaging unit 1 may be provided with only a scanner in one direction. In full-field OCT, a measurement light that spreads in two dimensions is irradiated onto the subject, and three-dimensional volume OCT data is acquired by receiving signal light that is the result of interference between the return light of this measurement light and a two-dimensional reference light, so no scanning means is required.

[0037] <Motion Contrast> The OCT motion contrast image generating means 1e processes multiple OCT data acquired by the OCT data acquiring means 1a at different times for the same position to generate motion contrast data, and then arranges the generated motion contrast data in an xyz coordinate system to construct a three-dimensional OCT motion contrast image.

[0038] The En-Face image generating means 1f generates an En-Face image (hereinafter, also referred to as a projected image), which is a two-dimensional image projected onto a plane based on two arbitrary reference planes in the depth direction (Z direction) from a three-dimensional image such as a three-dimensional OCT image or a three-dimensional OCT motion contrast image. Note that the En-Face image can be generated using various known methods, so detailed explanations will be omitted.

[0039] <General explanation of machine learning> A machine learning model is a model that has been trained (learned) in advance using appropriate teacher data (learning data) for an arbitrary machine learning algorithm. The teacher data is composed of one or more pairs of input data and output data (correct answer data). The format and combination of the input data and output data of the pairs that make up the teacher data may be suitable for a desired configuration, such as one being an image and the other being a numerical value, one being composed of a group of multiple images and the other being a character string, or both being images.

[0040] The image quality improvement engine (trained model for image quality improvement) is a module that improves the image quality of an input low-image quality image and outputs a high-image quality image. In this specification, image quality improvement refers to converting an input image into an image with image quality more suitable for image diagnosis, and a high-image quality image refers to an image converted into an image with image quality more suitable for image diagnosis. In addition, a low-image quality image is, for example, a two-dimensional image or a three-dimensional image acquired by OCT that is captured without being set to have a particularly high image quality. Specifically, low-image quality images include, for example, images acquired by short-time OCT imaging, and OCTA images acquired with a small number of imaging sessions.

[0041] The image processing method constituting the image quality improvement method in this embodiment performs processing using various machine learning algorithms such as deep learning. Note that the image processing method may perform any existing processing such as various image filter processing, matching processing using a database of high-quality images corresponding to similar images, and knowledge-based image processing in addition to processing using the machine learning algorithm.

[0042] In addition, when a high-image-quality engine that requires different image sizes for the image input to the high-image-quality engine and the image output from the high-image-quality engine is adopted, the image size is adjusted appropriately. Specifically, the image size is adjusted by padding or combining the surrounding shooting area of ​​the input image for an input image such as an image used as teacher data for training a machine learning model or an image input to the high-image-quality engine. In addition, the area to be padded is filled with a certain pixel value, filled with a neighboring pixel value, or mirror padded according to the characteristics of the high-image-quality method so that the image quality can be effectively improved.

[0043] Example 1 In step S101 in Fig. 2, the OCT data acquisition unit 1a acquires OCT data for generating at least three OCT images with different acquisition time intervals for the same position, with the time interval being t. In this embodiment, as shown in Fig. 4(a), four OCT data are acquired for the same position, O1 to O4, with a time interval Δt.

[0044] In step S102, the OCT data selection unit 1d selects two or more sets of OCT data with different time intervals from the four acquired OCT data. In this embodiment, three types of OCT data sets with time intervals of Δt, 2Δt, and 3Δt are selected. At this time, three sets of OCT data (O1 and O2, O2 and O3, and O3 and O4) are selected as combinations with a time interval of Δt, two sets (O1 and O3, O2 and O4) are selected as combinations with a time interval of 2Δt, and one set (O1 and O4) is selected as combinations with a time interval of 3Δt. Note that the OCT data used to generate the motion contrast data may be selected by the OCT data selection unit 1d upon receipt of an operation by the operator using the operation unit 104 by the operation reception unit 1j, or may be automatically selected by the OCT data selection unit 1d according to the shooting conditions.

[0045] In step S103, the OCT motion contrast image generating means 1e generates a three-dimensional OCT motion contrast image. In this embodiment, as shown in FIG. 4(b), OCT motion contrast images MC1 to MC3 are generated for each set of OCT data selected by the OCT data selecting means 1d. A similar process is performed on sets of OCT data at multiple positions to generate three-dimensional OCT motion contrast images for each time interval. When generating one OCT motion contrast image from multiple sets of OCT data at the same time interval, the OCT motion contrast image can be generated using statistics of the OCT motion contrast data generated for each data set. At this time, it is desirable to use the OCT motion contrast data generated from each set of OCT data after aligning them.

[0046] For example, a motion contrast image is generated from each of three sets of OCT data acquired at a time interval Δt, and the generated motion contrast images are averaged to generate MC1. Alternatively, MC1 may be generated by statistical processing using statistical values ​​such as the maximum, minimum, and median of the motion contrast data generated from each set. Alternatively, the image quality of the generated motion contrast images may be evaluated, and the image with the highest evaluation value may be used. Note that noise removal processing, etc. may be performed as appropriate.

[0047] In step S104, the En-Face image generating means 1f generates OCT front motion contrast images ME1 to ME3, which are En-Face images, from the three-dimensional OCT motion contrast image. Here, the OCT front motion contrast images generated from the sets of OCT data with time intervals Δt, 2Δt, and 3Δt are respectively designated ME1, ME2, and ME3. In this embodiment, the En-Face image is generated under the conditions that the reference plane in the depth direction range is the nerve fiber layer and the ganglion cell layer, the representative value at the time of generating the En-Face image is the maximum value between the reference planes, and no artifact removal is performed. Note that the generation conditions of the En-Face image are not limited to this. For example, it may be configured to be set based on the shooting conditions, or it may be configured to allow the operator to set the reference plane.

[0048] In step S105, the composite image generating means 1h combines the OCT en face motion contrast images ME1 to ME3 generated from the OCT data sets at different time intervals, and the display control unit 1g displays the generated composite image MEC.

[0049] The OCT front motion contrast image ME1 has a relatively short time interval (time interval of acquisition time). Therefore, blood vessels with slow blood flow are difficult to depict, but the image has high contrast. On the other hand, the OCT front motion contrast image ME3 has a relatively long time interval. Blood vessels with slow blood flow are also depicted, but the contrast is lower than other OCT front motion contrast images. The composite image generating means 1h generates one OCT front motion contrast composite image (variable interscan time analysis image) MEC in which more blood vessels are depicted with high contrast by combining these images. The composition method is performed, for example, by averaging ME1 to ME3. At this time, the composition process may be performed by assigning different weights to each of the OCT front motion contrasts ME1 to ME3. For example, the weighting may be performed so that the OCT data set with a shorter time interval is heavier. The composition method is not limited to averaging such as addition, and may be statistical processing such as using the median or maximum value. In addition, noise removal processing may be performed appropriately.

[0050] Then, the report generating means 1k generates a report screen including the generated composite image MEC, and the display control unit 1g displays the generated report screen on the display unit 103. FIG. 3(a) is an example of a report screen to be displayed. In this embodiment, the display control unit 1g displays a fundus image captured by a fundus camera or SLO (SLO: Scanning Laser Ophthalmoscope) in the display area 401, and superimposes indices 430 and 431 indicating the acquisition position of OCT data on the fundus image displayed in the display area 401. In addition, a stereoscopic image of the boundary surface of the layers is displayed in the display area 420. When the operator moves the indices 430 and 431 indicating the acquisition position of OCT data by operating the operation unit 104, the display control unit 1g displays the OCT tomographic image of the corresponding acquisition position in the display area 402, and the operator can check the OCT tomographic image at the position indicated by the indices 430 and 431. Furthermore, the display control means 1g displays the composite image MEC generated in step S205 in the display area 411. At this time, indices (indices indicating the acquisition positions of the B-scan image displayed in the display area 402) are similarly displayed in positions on the composite image MEC corresponding to the positions of the indices 430, 431 on the fundus image displayed in the display area 401. A color map showing the correspondence between the color information of the composite image MEC and the time interval of the OCT data is displayed in the display area 412.

[0051] In step S106, the image quality improvement means 1i accepts the synthetic image MEC generated in step S105 as an input image, and generates a synthetic image MEI by improving the image quality of the synthetic image MEC, and the display control unit 1g displays the generated synthetic image MEI on the display unit 103.

[0052] For example, when the composite image MEC is displayed in the display area 411 of the report screen displayed in step S105, the operator presses the intelligent denoising button 413, which is an operation area on the report screen for performing image quality improvement processing of the composite image shown in Figure 3 (a), whereby a high-image-quality composite image MEI is generated and the high-image-quality composite image MEI is displayed in the display area 411, switching from the composite image MEC.

[0053] The image quality improvement unit 1i inputs the composite image MEC, which is an input image, to the image quality improvement engine to generate a composite image MEI with improved image quality. The image quality improvement engine generates a high-image-quality image, which is a composite image obtained by synthesizing OCT front motion contrast images with different time intervals generated from a larger number of OCT data sets, based on a machine learning model that performs machine learning using teacher data. In other words, the image quality improvement engine can generate a composite image MEI, which is a high-image-quality image with reduced noise and enhanced contrast compared to the composite image MEC, which is an input image.

[0054] Here, the image quality improving unit 1i will be described in detail. The image quality improving unit 1i is provided with an image quality improving engine. The image quality improving engine according to this embodiment performs image quality improvement processing using a machine learning algorithm.

[0055] In this embodiment, training of a machine learning model related to a machine learning algorithm uses training data consisting of a group of pairs of input data, which is a composite image synthesized from an OCT frontal motion contrast image, which is assumed to be the processing target, and output data, which is a high-quality composite image corresponding to the input data.

[0056] Regarding the input data among the training data, from a group of OCT data acquired by repeatedly scanning the same position at a time interval Δt using the same model and settings as the OCT imaging unit 1, similar to steps S102 to 104, M1 sets of OCT data sets with a time interval Δt, M2 sets of OCT data sets with a time interval of 2Δt, and M3 sets of OCT data sets with a time interval of 3Δt are selected, and the OCT frontal motion contrast images generated from each data set are averaged for each OCT frontal motion contrast image with the same time interval, and then, similar to step S105, the synthesized OCT frontal motion contrast composite image is used as the input data.

[0057] On the other hand, for the output data, an image with higher image quality than the input image obtained by the settings and image processing provided by the same model as the OCT imaging unit 1 is used.

[0058] Specifically, from the OCT data group obtained by repeatedly scanning at time intervals Δt for the same position, acquired with the same model as the OCT imaging unit 1 and under the same settings as the OCT imaging unit 1, similar to S102 to 104, N1 sets of OCT data sets with a time interval of Δt, N2 sets of OCT data sets with a time interval of 2Δt, and N3 sets of OCT data sets with a time interval of 3Δt are selected. After averaging the OCT forward motion contrast images generated from each data set for the OCT forward motion contrast images with the same time interval, an OCT forward motion contrast composite image obtained by synthesizing in the same manner as step S105 is acquired and used as the output image. At this time, the number of sets of OCT data sets at any time interval for the same position where the output image is generated is equal to or greater than the number of sets of OCT data sets at the same time interval for generating the input image, and the number of sets of OCT data sets at at least one time interval is greater than the number of sets of OCT data sets at the same time interval for generating the input image. In other words, also, M1 to 3 and N1 to 3 all satisfy the following formulas (i) to (iii) and are all natural numbers. (i) M1 ≤ N1 (ii) M2 ≤ N2 (iii) M3 ≤ N3, and MX < NX (X is at least one of 1 to 3)

[0059] Thereby, the output image used as the teacher data is an image with higher image quality than the input image. Here, it is described for the case of generating a composite image from the OCT forward motion contrast data at the first to third three time intervals, but it may be generated from the OCT forward motion contrast data at two or more time intervals.

[0060] That is, it is sufficient to train the system using training data consisting of an input image which is an image based on a third OCT front motion contrast data generated using a set of M1 sets of OCT data for a first time interval among a plurality of time intervals (for example, among the above three time intervals), M2 sets of OCT front motion contrast data for a second time interval, and an output image which is an OCT front motion contrast synthetic image obtained by synthesizing a fifth OCT front motion contrast image generated using a set of N1 sets of OCT data for the first time interval, and a sixth OCT front motion contrast image generated using N2 sets of OCT data for the second time interval, in which case M1, M2, N1, and N2 satisfy all of the following (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1 <N1もしくは / およびM2<N2 (where M1, M2, N1, and N2 are all natural numbers)

[0061] In addition, considering that an OCT front motion contrast image with a long time interval often has a lower contrast than an OCT front motion contrast image with a short time interval, it is considered that the effect of improving image quality is greater by improving the image quality of a portion of the OCT front motion contrast synthetic image corresponding to an OCT front motion contrast image with a long time interval than that of a portion corresponding to an OCT front motion contrast image with a short time interval. Therefore, if the second time interval is longer than the first time interval, it is preferable that the OCT front motion contrast image with the second time interval is trained with training data having a larger number (or a higher ratio) of sets of OCT data for generating output data for input data than the first time interval.

[0062] In other words, the effect of improving image quality can be further improved by making the learning data further satisfy the following formula (4) and / or (5). (4)N1 / M1 <N2 / M2 (5)N1-M1 <N2-M2 (wherein M1, M2, N1, and N2 are the same as M1, M2, N1, and N2 in formulas (1) to (3), respectively.)

[0063] Note that, here, the composite image MEC generated and displayed in S105 is switched to the high-image-quality composite image MEI as the image to be displayed in the display area 411, but the composite image MEI may be displayed in a display area different from the display area 411. In that case, as shown in FIG. 3(b), the composite image MEC and the composite image MEI can be displayed side by side. In this case, in either the switching display or the side-by-side display, the composite image MEC and the composite image MEI can be displayed so as to be distinguishable. This allows the composite image MEC and the composite image MEI to be easily compared.

[0064] In this embodiment, the combination for generating motion contrast data is selected from the acquired OCT data, but the OCT data may be acquired in consideration of the combination for generating motion contrast data. For example, the imaging sequence may be determined based on the time interval of an arbitrary set of OCT data specified by the operator via the operation unit 104, and imaging may be performed. Alternatively, specifiable time intervals may be presented in advance, and the operator may select from them.

[0065] Furthermore, in this embodiment, a group of OCT data was acquired at a fixed time interval Δt, and OCT data sets with time intervals Δt, 2Δt, and 3Δt were created from the acquired OCT data groups, but at least one of the OCT data sets with time interval 2Δt and the OCT data set with time interval 3Δt may be acquired by scanning separately from the OCT data group acquired at time interval Δt.

[0066] The above method enables the operator to obtain a high-quality composite image by combining OCT front motion contrast images taken at different time intervals, without increasing the imaging time.

[0067] Example 2 In the first embodiment, En-Face images are generated for each time interval, and then synthesized and image-enhanced to obtain a high-quality synthesized image.

[0068] On the other hand, in this embodiment, En-Face images are generated for each time interval, and a composite image is obtained in which a portion corresponding to at least one of the En-Face images has been made high quality.

[0069] FIG. 5 shows the flow of this embodiment.

[0070] Steps S201 to S204 are the same as steps S101 to S104 in the first embodiment, respectively.

[0071] Next, in step S205, the En-Face images ME1 to ME3 generated in S204 are input to a trained model for improving image quality, as shown in FIG. 6(a), and a synthetic image MEI with improved image quality is output and displayed.

[0072] The image quality improving means 1i of this embodiment will now be described in detail.

[0073] As shown in FIG. 6(b), the teacher data used in training the machine learning model used in this embodiment is, as input data, a group of OCT data acquired by repeatedly scanning the same position at a time interval Δt using the same model and settings as the OCT imaging unit 1, and similarly to steps S102 to 104, from which M1 sets of OCT data sets with a time interval Δt, M2 sets of OCT data sets with a time interval of 2Δt, and M3 sets of OCT data sets with a time interval of 3Δt are selected (FIG. 6(b)(i)), and the OCT frontal motion contrast images (ME1 to 3) generated from each OCT data set are averaged for each OCT frontal motion contrast image with the same time interval (FIG. 6(b)(ii)).

[0074] On the other hand, the output data for the training data is a composite image obtained by combining OCT frontal motion contrast images with the same time intervals (Δt, 2Δt, 3Δt) used in the input data, and at least one of the OCT frontal motion contrast images for the time interval used to generate the composite image is an OCT frontal motion contrast image generated from a greater number of OCT data sets than the OCT frontal motion contrast images with the same time interval in the input data.

[0075] Specifically, from a group of OCT data acquired by repeatedly scanning the same position at time intervals Δt using the same model and settings as the OCT imaging unit 1, similar to S102 to 104, N1 sets of OCT data sets with a time interval Δt, N2 sets of OCT data sets with a time interval of 2Δt, and N3 sets of OCT data sets with a time interval of 3Δt are selected, and OCT front motion contrast images (ME1 to 3) generated from each data set are generated. Then, after averaging is performed for each OCT front motion contrast image with the same time interval, similar to step S105, an OCT front motion contrast composite image obtained by combining the OCT front motion contrast images with each time interval after averaging is used as output data.

[0076] In this case, similarly to the first embodiment, the OCT front motion contrast images of any time interval (Δt, 2Δt, 3Δt) for generating the OCT front motion contrast composite image used as output data are generated with OCT data sets with a number equal to or greater than the number of OCT data sets for generating the OCT front motion contrast image of the same time interval in the input data, and the OCT front motion contrast image of at least one time interval among the OCT front motion contrast images for generating the OCT front motion contrast composite image used as output data is generated with a number of OCT data sets greater than the number of OCT front motion contrast images of the same time interval in the input image. As a result, the output data used as the teacher data has a higher image quality in a portion corresponding to the OCT front motion contrast image of at least one time interval in the composite image than the OCT front motion contrast image of the same time interval that is the input image.

[0077] Then, the report generating means 1k generates a report screen including the generated OCT en face motion contrast synthetic image MEI, and the display control means 1g displays the generated report screen on the display means 103.

[0078] Note that, here, a configuration is described in which an improved image quality OCT frontal motion contrast composite image MEI is displayed when the report screen is displayed; however, similar to step S105 in the first embodiment, a configuration in which the composite image MEC before image quality enhancement is displayed when the report screen is displayed and then the improved image quality OCT frontal motion contrast composite image MEI is displayed may also be used.

[0079] In addition, the output data may be an OCT frontal motion contrast synthetic image based on an OCT frontal motion contrast image obtained by averaging OCT frontal motion contrast images of the same time interval and then further improving the image quality by smoothing filter processing or gradation processing, or a synthetic image obtained by combining the images and then performing these processes may be used.

[0080] In this embodiment, the combination for generating motion contrast data is selected from the acquired OCT data sets, but the OCT data sets may be acquired in consideration of the combination for generating motion contrast data. For example, the imaging sequence may be determined based on the time interval of any OCT data set specified by the operator via the operation unit 104, and imaging may be performed. Alternatively, the time intervals that can be specified may be presented in advance, and the operator may select one.

[0081] The above method enables the operator to obtain a high-quality frontal OCT motion contrast composite image in a short imaging time.

[0082] (Example of transformation) In Example 2, the OCT frontal motion contrast images generated for each time interval were input into one trained model to obtain a high-quality OCT frontal motion contrast composite image, but in this modified example, a trained model is constructed for each time interval, and the high-quality OCT frontal motion contrast images for each time interval are synthesized to obtain an OCT frontal motion contrast composite image.

[0083] Figure 7 shows the flow of this modified example.

[0084] Steps S301 to S304 are the same as steps S101 to S104 in the first embodiment, respectively.

[0085] In step S305, the OCT front motion contrast images generated for each time interval are input to a trained model corresponding to each OCT front motion contrast image, and high-quality OCT front motion contrast images for each time interval are output.

[0086] The teacher data used to train the learning model is a dataset consisting of pairs of input data, which is an OCT frontal motion contrast image generated from an OCT dataset of a specified time interval, and output data, which is an OCT frontal motion contrast image generated from an OCT dataset of the same time interval and with a greater number of sets than the OCT frontal motion contrast images that are the input data.

[0087] More specifically, for the input data of the training data, M1 sets of OCT data sets with a time interval of Δt are selected from a group of OCT data acquired by repeatedly scanning the same position at a time interval Δt using the same model and settings as the OCT imaging unit 1, and an image obtained by additively averaging the OCT frontal motion contrast images generated from each of the M1 sets of OCT data sets is used.

[0088] For the output data of the training data, N1 sets (more than M1 sets) of OCT data sets with a time interval of Δt are selected from the OCT data group acquired by repeatedly scanning the same position at a time interval Δt, and an image generated by averaging the OCT frontal motion contrast images generated from each set is used.

[0089] If the learning model for improving the image quality of the OCT frontal motion contrast image with a time interval of Δt is called the first learning model, in the second learning model for improving the image quality of the OCT frontal motion contrast image with a time interval of 2Δt, as input data, M2 sets of OCT data sets with a time interval of 2Δt are selected from a group of OCT data acquired by repeatedly scanning the same position at a time interval Δt, using the same model and settings as the OCT imaging unit 1, and OCT frontal motion contrast images generated from each of the M2 sets of OCT data sets are used as images obtained by additive averaging, and as output data, N2 sets of OCT data sets with a time interval of 2Δt, more than M2 sets, are selected from the group of OCT data with a time interval of 2Δt for the same position, and the learning data using images obtained by additive averaging the OCT frontal motion contrast images generated from each of the M2 sets of OCT data sets are used.

[0090] A learning model (third learning model) for improving the image quality of OCT frontal motion contrast images with a time interval of 3Δt is also constructed in the same manner as above.

[0091] In step S306, the OCT front motion contrast images acquired in step S305 and with improved image quality for each time interval are synthesized by the synthetic image generating means 1h to generate an OCT front motion contrast synthetic image MEI. This step is performed in the same manner as the synthesis method in S105 of the first embodiment.

[0092] Then, the report generating means 1k generates a report screen including the generated OCT en face motion contrast synthetic image MEI, and the display control means 1g displays the generated report screen on the display means 103.

[0093] Note that, here, a configuration is described in which an improved image quality OCT frontal motion contrast composite image MEI is displayed when the report screen is displayed; however, similar to step S105 in the first embodiment, a configuration may also be adopted in which the composite image MEC before image quality enhancement is displayed when the report screen is displayed, and the composite image MEC before image quality enhancement is switched to the improved image quality OCT frontal motion contrast composite image MEI.

[0094] Example 3 In Example 1, after generating En-Face images for each time interval, the images were input to a trained model to obtain a high-quality composite image.

[0095] On the other hand, in this embodiment, an example will be described in which an OCT en face motion contrast synthetic image MEC and an improved image quality OCT en face motion contrast synthetic image MEI are obtained and used for follow-up observation, similarly to steps S101 to 105 in embodiment 1. Note that, although an example will be described here in which an improved image quality synthetic image obtained by the method of embodiment 1 is used, the method of obtaining the improved image quality synthetic image may be any of embodiment 2, embodiment 3, modification example 1, and modification example 2.

[0096] The operator selects one examination to display on the report screen and presses the Progression tab as shown in Figure 8(a).The OCT motion contrast synthetic images MEC1 to 4 generated based on OCT data acquired on four different dates and times, including the selected examination, for the same test eye are then displayed on the report screen in order of acquisition date and time, for example, as shown in Figure 8(a).

[0097] If there are not four or more OCT data for the same examinee's eye, a message to that effect is displayed. The images of the four examinations displayed in the initial state are divided equally between the examination date and time of the oldest examination of the same examinee and the examination date and time selected by the operator, and the images of the examination with the examination date and time closest to the examination date and time are displayed. Alternatively, the images of the most recent four examinations of the same examinee are displayed. Alternatively, the images may be images of any four examinations.

[0098] Furthermore, the operator displays an operation area for selecting an ETDRS grid, which is a two-dimensional index indicating an analysis region, on any one of the composite images MEC1 to 4 in FIG. 8(a) by, for example, right-clicking the mouse, and performs an operation to select an ETDRS grid, and displays the ETDRS grid superimposed on each of the composite images MEC1 to 4 as shown in FIG. 8(b). At this time, for example, the OCT en face motion contrast composite images MEC1 to 4 acquired at different times and the results of analyzing the blood vessel area density using the ETDRS grid, which is a two-dimensional index indicating an analysis region, can be displayed side by side. The operator can adjust the position and inclination of the region to be analyzed (analysis region) by moving the displayed ETDRS grid on the composite image MEC.

[0099] When the operator moves the ETDRS grid on the OCT en face motion contrast composite image (MEC1-4) of any of the examinations displayed by the operator operating the operation unit 104, the display control unit 1g of this embodiment can automatically move the ETDRS grid to a corresponding position on the OCT en face motion contrast image of the other examination in conjunction with the ETDRS grid. As an analysis result, a time series graph, which is a type of time series data obtained by analyzing the blood vessel area density of the entire area in the ETDRS grid for the OCT en face motion contrast composite image (MEC1-4) at each date and time, is displayed in the display area 701. Note that, although the blood vessel area density of the entire area in the ETDRS grid is displayed as a time series graph here, the blood flow velocity may be calculated as an analysis result.

[0100] In the time series graph displayed in the display area 701, the analysis results of the examination at the date and time selected by the operator using the operation unit 104 are identifiable, and a regression line calculated from the analysis results plotted in the time series graph can also be displayed. Note that, in this embodiment, the blood vessel area density of all areas in the ETDRS grid is analyzed, but this is not limited to this. By operating the display area 702 using the operation unit 104, the operator can change the type of analysis result to be displayed to the blood vessel area density or statistics of a specific area in the ETDRS grid, the skeleton density for the reference examination, the amount of change in these densities depending on the acquisition time interval, etc.

[0101] On the above report screen, when the operator presses the intelligent denoising button 413 which improves the image quality of the OCT frontal motion contrast synthetic images (MEC1-4) and the operation reception unit 1j receives it, the display control unit 1g inputs the OCT frontal motion contrast synthetic images MEC1-4 respectively into the trained model of Example 1, and outputs and displays the improved image quality OCT frontal motion contrast synthetic images MEI1-4 (not shown).

[0102] As a result, the OCT front motion contrast synthetic image to be analyzed is changed from the OCT front motion contrast synthetic images MEC1-4 to the OCT front motion contrast synthetic images MEI1-4, and the display control unit 1g switches the time series data of the analysis result to be displayed in the display area 701 from the time series data of the analysis area based on the two-dimensional index superimposed on the first OCT front motion contrast synthetic image to the time series data of the analysis area based on the two-dimensional index superimposed on the image quality improved OCT front motion contrast synthetic image. At this time, the analysis result before the switch and the time series data after the switch may be displayed in a different display form so that it is possible to distinguish whether the data calculated based on the image quality improved synthetic image or the image quality improved synthetic image is not based on the image quality improved synthetic image, for example, by displaying the display items indicating that the image quality improved synthetic image is the image quality improved synthetic image together with the image quality improved synthetic image.

[0103] Furthermore, when the operator presses the intelligent denoising button 413, the intelligent denoising button 413 becomes, for example, a button for displaying the composite image before image quality improvement, and by pressing this button, the displayed image and the image to be analyzed are changed to OCT frontal motion contrast composite images MEC1-4, and the time series graph displayed in the display area 701 is also changed to time series data of the analysis area based on two-dimensional indices superimposed on the OCT frontal motion contrast composite images MEC1-4.

[0104] Here, an ETDRS grid is superimposed on each of the OCT frontal motion contrast synthetic images MEC1-4 and the enhanced image quality OCT frontal motion contrast synthetic images MEI1-4 from examinations on different dates and times, but it is also possible to superimpose an ETDRS grid on the OCT frontal motion contrast synthetic images MEC and the enhanced image quality OCT frontal motion contrast synthetic images MEI of Examples 1 and 2 and the modified examples for analysis.

[0105] In the above Examples 1 to 3 and the transformation example, the learning data used to train the learning model and the images input to and output from the trained model are described as OCT frontal motion contrast images, but the learning data used to train the learning model and the images input to and output from the trained model may also be 3D OCT motion contrast images.

[0106] <Other embodiments> Although the embodiment examples have been described in detail above, the disclosed technology can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium (storage medium), etc. Specifically, the technology may be applied to a system composed of multiple devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to an apparatus composed of a single device.

[0107] Needless to say, the object of the present invention can be achieved by the following: That is, a recording medium (or storage medium) on which is recorded a program code (computer program) of software that realizes the functions of the above-mentioned embodiments is supplied to a system or device. The storage medium is, of course, a computer-readable storage medium. Then, a computer (or a CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium realizes the functions of the above-mentioned embodiments, and the recording medium on which the program code is recorded constitutes the present invention.

[0108] The above disclosure includes the following configurations, methods, and programs.

[0109] [Configuration 1] an OCT motion contrast image acquisition unit that acquires a first OCT motion contrast image generated using a set of OCT data for a first time interval and a second OCT motion contrast image generated using a set of OCT data for a second time interval different from the first time interval; an OCT motion contrast synthetic image output unit that outputs an image quality improved OCT motion contrast synthetic image by inputting an image based on the first OCT motion contrast image and the second OCT motion contrast image to a trained model, The trained model is an image processing device that includes: input data that is an image based on a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval; and output data that is an OCT motion contrast synthetic image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and is a trained model trained using training data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1 <N1もしくは / およびM2<N2 (where M1, M2, N1, and N2 are all natural numbers)

[0110] [Configuration 2] The image processing device of configuration 1, wherein an image based on the first OCT motion contrast image and the second OCT motion contrast image is a first OCT motion contrast synthetic image obtained by synthesizing the first OCT motion contrast image and the second OCT motion contrast image.

[0111] [Configuration 3] 2. The image processing device of configuration 1, wherein an image based on the first OCT motion contrast image and the second OCT motion contrast image is the first OCT motion contrast image and the second OCT motion contrast image.

[0112] [Configuration 4] 2. The image processing device according to configuration 1, wherein the second time interval is longer than the first time interval, and the learning data further satisfies the following formula (4) and / or (5): (4)N1 / M1 <N2 / M2 (5)N1-M1 <N2-M2 (wherein M1, M2, N1, and N2 are the same as M1, M2, N1, and N2 in formulas (1) to (3), respectively.)

[0113] [Configuration 5] Further comprising a display unit and a display control unit that controls display on the display unit, 5. The image processing device according to any one of configurations 1 to 4, wherein the display control unit displays the image quality improved OCT motion contrast composite image on the display unit.

[0114] [Configuration 6] Further comprising a display unit and a display control unit that controls display on the display unit, 3. The image processing device according to configuration 2, wherein the display control unit displays the first OCT motion contrast composite image and the image quality improved OCT motion contrast composite image on the display unit.

[0115] [Configuration 7] 7. The image processing device according to configuration 6, wherein the display control unit switches between displaying the first OCT motion contrast composite image and the image quality improved OCT motion contrast composite image on the display unit.

[0116] [Configuration 8] The apparatus further includes an operation receiving unit that receives an operation from an operator, 8. The image processing device according to configuration 7, wherein the operation receiving unit receives an operation related to the switching.

[0117] [Configuration 9] 7. The image processing device according to configuration 6, wherein the display control unit displays the first OCT motion contrast composite image and the image quality improved OCT motion contrast composite image side by side on the display unit.

[0118] [Configuration 10] The image processing device of configuration 6, wherein the display control unit superimposes a two-dimensional index indicating an analysis region on each of the first OCT motion contrast synthetic image and the improved image quality OCT motion contrast synthetic image, and displays analysis results for each of the analysis regions.

[0119] [Configuration 11] 11. The image processing device of configuration 10, wherein each of the first OCT motion contrast composite image and the image quality improved OCT motion contrast composite image is a plurality of OCT motion contrast composite images of examinations performed on different dates and times.

[0120] [Configuration 12] 9. The image processing device of claim 8, wherein when the display unit displays the first OCT motion contrast composite image and time series data of analysis results regarding an analysis region based on a two-dimensional index superimposed on the first OCT motion contrast composite image, and the operation receiving unit receives an operation to switch from the first OCT motion contrast composite image to the high-image-quality OCT motion contrast composite image, the display control unit switches the time series data displayed on the display unit from the time series data of analysis results regarding an analysis region based on a two-dimensional index superimposed on the first OCT motion contrast composite image to the time series data of analysis results regarding an analysis region based on a two-dimensional index superimposed on the high-image-quality OCT motion contrast composite image.

[0121] [Configuration 13] an OCT motion contrast image acquisition unit that acquires a first OCT motion contrast image generated using M1 sets of OCT data at a first time interval, and a second OCT motion contrast image generated using M2 sets of OCT data at a second time interval different from the first time interval; an output unit that outputs a first OCT motion contrast image with improved image quality by inputting the first OCT motion contrast image to a first trained model, and outputs a second OCT motion contrast image with improved image quality by inputting the second OCT motion contrast image to a second trained model; a composite image generating unit that generates a composite image of a first OCT motion contrast image with improved image quality and a second OCT motion contrast image with improved image quality; An image processing device comprising: the first trained model is a trained model trained with training data including input data which is a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval, and output data which is a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval; An image processing device, wherein the second trained model is a trained model trained using training data including input data which is a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and output data which is a sixth OCT motion contrast image generated using N1 sets of OCT data at the first time interval. (Note that M1, M2, N1, and N2 are all natural numbers, and M1 <N1かつM2<N2)

[0122] [Configuration 14] 14. The image processing device according to any one of configurations 1 to 13, wherein the first to sixth OCT motion contrast images and the OCT motion contrast composite image are OCT front motion contrast images.

[0123] [Configuration 15] 14. The image processing device according to any one of configurations 1 to 13, wherein the first to sixth OCT motion contrast images and the OCT motion contrast composite image are three-dimensional OCT motion contrast images.

[0124] [Configuration 16] acquiring an OCT motion contrast image, the OCT motion contrast image being generated using a set of OCT data for a first time interval, and a second OCT motion contrast image being generated using a set of OCT data for a second time interval different from the first time interval; and an output step of an OCT motion contrast synthetic image, which outputs an OCT motion contrast synthetic image having a high image quality by inputting an image based on the first OCT motion contrast image and the second OCT motion contrast image to a trained model, The trained model is an image processing program including: input data that is an image based on a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval; and output data that is an OCT motion contrast synthetic image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, the image processing program being a trained model trained using training data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1 <N1もしくは / およびM2<N2 (where M1, M2, N1, and N2 are all natural numbers)

[0125] [Configuration 17] an OCT motion contrast image acquisition step of acquiring a first OCT motion contrast image generated using M1 sets of OCT data for a first time interval and a second OCT motion contrast image generated using M2 sets of OCT data for a second time interval different from the first time interval; an output step of inputting the first OCT motion contrast image to a first trained model to output a first OCT motion contrast image with improved image quality, and inputting the second OCT motion contrast image to a second trained model to output a second OCT motion contrast image with improved image quality; a composite image generating step of generating a composite image of the enhanced first OCT motion contrast image and the enhanced second OCT motion contrast image; An image processing program comprising: the first trained model is a trained model trained using training data including input data that is a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval, and output data that is a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval; An image processing program in which the second trained model is a trained model trained using training data including input data that is a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and output data that is a sixth OCT motion contrast image generated using N1 sets of OCT data at the first time interval. (Note that M1, M2, N1, and N2 are all natural numbers, and M1 <N1かつM2<N2)

[0126] [Configuration 18] a fourth OCT motion contrast image generated using M2 sets of OCT data at a second time interval different from the first time interval; and output data that is an OCT motion contrast synthetic image obtained by synthesizing a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval, and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, the learning device being characterized by learning using learning data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1 <N1もしくは / およびM2<N2 (where M1, M2, N1, and N2 are all natural numbers) [Explanation of symbols]

[0127] 1 OCT imaging section 10. Shooting optical system 20 spectrometer 101 Storage section 102 Personal Computers 103 Display section 104 Operation section 105 Control section 110 Optical Head 111 Stage Section 112 Base 113 Chin rest 401 Display area 402 Display area 411 Display area 412 Display area 413 Intelligent Denoise Button 420 display area 430 indicators 431 indicators 701 Display area 702 Display area

Claims

1. an OCT motion contrast image acquisition unit that acquires a first OCT motion contrast image generated using a set of OCT data for a first time interval and a second OCT motion contrast image generated using a set of OCT data for a second time interval different from the first time interval; and an OCT motion contrast synthetic image output unit that outputs an OCT motion contrast synthetic image with improved image quality by inputting an image based on the first OCT motion contrast image and the second OCT motion contrast image to a trained model, The trained model is and output data that is an OCT motion contrast composite image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and the input data is an image based on the third OCT motion contrast image generated using M1 sets of OCT data at the first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and the output data is an OCT motion contrast composite image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and is a trained model trained using training data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1<N1 or / and M2<N2 (where M1, M2, N1, and N2 are all natural numbers)

2. 2. The image processing device according to claim 1, wherein an image based on the first OCT motion contrast image and the second OCT motion contrast image is a first OCT motion contrast composite image obtained by combining the first OCT motion contrast image and the second OCT motion contrast image.

3. The image processing device according to claim 1 , wherein an image based on the first OCT motion contrast image and the second OCT motion contrast image is the first OCT motion contrast image and the second OCT motion contrast image.

4. 2 . The image processing device according to claim 1 , wherein the second time interval is longer than the first time interval, and the learning data further satisfies the following formula (4) or / and (5): (4) N1 / M1<N2 / M2 (5) N1-M1<N2-M2 (Note that M1, M2, N1, and N2 are the same as M1, M2, N1, and N2 in formulas (1) to (3).)

5. Further comprising a display unit and a display control unit that controls display on the display unit, The image processing device according to claim 1 , wherein the display control unit displays the image-quality-enhanced OCT motion contrast composite image on the display unit.

6. Further comprising a display unit and a display control unit that controls display on the display unit, The image processing device according to claim 2 , wherein the display control unit displays the first OCT motion contrast composite image and the image quality improved OCT motion contrast composite image on the display unit.

7. The image processing device according to claim 6 , wherein the display control unit switches between displaying the first OCT motion contrast composite image and the image quality improved OCT motion contrast composite image on the display unit.

8. The apparatus further includes an operation receiving unit that receives an operation from an operator, The image processing apparatus according to claim 7 , wherein the operation receiving unit receives an operation relating to the switching.

9. The image processing device according to claim 6 , wherein the display control unit displays the first OCT motion contrast composite image and the image quality improved OCT motion contrast composite image in parallel on the display unit.

10. The image processing device described in claim 6, characterized in that the display control unit superimposes a two-dimensional index indicating an analysis region on each of the first OCT motion contrast synthetic image and the improved image quality OCT motion contrast synthetic image, and displays analysis results for each of the analysis regions.

11. The image processing apparatus according to claim 10 , wherein each of the first OCT motion contrast synthetic image and the image quality improved OCT motion contrast synthetic image is a plurality of OCT motion contrast synthetic images obtained from examinations performed on different dates and times.

12. 8. The image processing device of claim 7, wherein when the display unit displays the first OCT motion contrast synthetic image and time series data of analysis results regarding an analysis region based on a two-dimensional index superimposed on the first OCT motion contrast synthetic image, and the operation receiving unit receives an operation to switch from the first OCT motion contrast synthetic image to the high-image-quality OCT motion contrast synthetic image, the display control unit switches the time series data displayed on the display unit from the time series data of analysis results regarding an analysis region based on a two-dimensional index superimposed on the first OCT motion contrast synthetic image to the time series data of analysis results regarding an analysis region based on a two-dimensional index superimposed on the high-image-quality OCT motion contrast synthetic image.

13. an OCT motion contrast image acquisition unit that acquires a first OCT motion contrast image generated using M1 sets of OCT data at a first time interval and a second OCT motion contrast image generated using M2 sets of OCT data at a second time interval different from the first time interval; an output unit that outputs a first OCT motion contrast image with improved image quality by inputting the first OCT motion contrast image to a first trained model, and outputs a second OCT motion contrast image with improved image quality by inputting the second OCT motion contrast image to a second trained model; a composite image generating unit that generates a composite image of a first OCT motion contrast image with improved image quality and a second OCT motion contrast image with improved image quality; An image processing device comprising: The first trained model is a trained model trained with training data including input data that is a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval, and output data that is a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval; An image processing device characterized in that the second trained model is a trained model trained using training data including input data which is a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and output data which is a sixth OCT motion contrast image generated using N1 sets of OCT data at the first time interval. (Note that M1, M2, N1, and N2 are all natural numbers, and M1<N1 and M2<N2)

14. 14. The image processing apparatus according to claim 1, wherein the first to sixth OCT motion contrast images and the OCT motion contrast composite image are OCT front motion contrast images.

15. 14. The image processing apparatus according to claim 1, wherein the first to sixth OCT motion contrast images and the OCT motion contrast composite image are three-dimensional OCT motion contrast images.

16. acquiring an OCT motion contrast image, the OCT motion contrast image being generated using a set of OCT data for a first time interval, and a second OCT motion contrast image being generated using a set of OCT data for a second time interval different from the first time interval; and an output step of an OCT motion contrast synthetic image, which outputs an OCT motion contrast synthetic image with improved image quality by inputting an image based on the first OCT motion contrast image and the second OCT motion contrast image to a trained model, The trained model is and output data that is an OCT motion contrast composite image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and the input data is an image based on the third OCT motion contrast image generated using M1 sets of OCT data at the first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and the output data is an OCT motion contrast composite image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and the image processing program is characterized in that the image processing program is a trained model trained using training data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1<N1 or / and M2<N2 (where M1, M2, N1, and N2 are all natural numbers)

17. an OCT motion contrast image acquisition step of acquiring a first OCT motion contrast image generated using M1 sets of OCT data for a first time interval and a second OCT motion contrast image generated using M2 sets of OCT data for a second time interval different from the first time interval; an output step of inputting the first OCT motion contrast image to a first trained model to output a first OCT motion contrast image with improved image quality, and inputting the second OCT motion contrast image to a second trained model to output a second OCT motion contrast image with improved image quality; a composite image generating step of generating a composite image of the first OCT motion contrast image with improved image quality and the second OCT motion contrast image with improved image quality; An image processing program comprising: The first trained model is a trained model trained using training data including input data that is a third OCT motion contrast image generated using M1 sets of OCT data at the first time interval, and output data that is a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval; An image processing program characterized in that the second trained model is a trained model trained using training data including input data which is a fourth OCT motion contrast image generated using M2 sets of OCT data at the second time interval, and output data which is a sixth OCT motion contrast image generated using N1 sets of OCT data at the first time interval. (Note that M1, M2, N1, and N2 are all natural numbers, and M1<N1 and M2<N2)

18. A learning device comprising: input data that is an image based on a third OCT motion contrast image generated using M1 sets of OCT data at a first time interval and a fourth OCT motion contrast image generated using M2 sets of OCT data at a second time interval different from the first time interval; and output data that is an OCT motion contrast composite image that combines a fifth OCT motion contrast image generated using N1 sets of OCT data at the first time interval and a sixth OCT motion contrast image generated using N2 sets of OCT data at the second time interval, and characterized in that the learning device learns using learning data that satisfies the following formulas (1) to (3). (1) M1≦N1 (2) M2≦N2 (3) M1<N1 or / and M2<N2 (where M1, M2, N1, and N2 are all natural numbers)

Citation Information

Patent Citations

  • Oct signal processing device and oct signal processing program

    WO2017119437A1