Image processing method, image processing device, program, and recording medium

Abstract

To provide a new technology to generate an image in which double refraction-derived artifacts are reduced, from an image acquired by the OCT modality having no polarization separation detection function.SOLUTION: An image acquisition unit of a device related to one aspect of an embodiment acquires an OCT image as a sample. A processing unit executes processing using a machine learning model constructed by machine learning using training data including an aggregate of pairs of a first image including double refraction information on an object and a second image not including double refraction information on the object, and constructed so as to output a denoised image in which double refraction-derived artifacts in the OCT image are reduced in response to the input of the OCT image. Further, the processing unit inputs the OCT image as a sample acquired by the image acquisition unit to the machine learning model, and on the basis of the input of the OCT image, stores or provides the denoised image output from the machine learning model.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to a method, an apparatus, a program, and a recording medium for processing optical coherence tomography (OCT) images. [Background technology]

[0002] In recent years, optical coherence tomography (OCT), which uses a light beam from a laser or other source to generate images that represent the surface and internal morphology of a sample, has been attracting attention. Unlike X-ray computed tomography (CT), OCT is not invasive to living organisms, and is therefore expected to be particularly applicable in the fields of medicine and biology. For example, in the field of ophthalmology, devices that generate images of the fundus and cornea have been put to practical use.

[0003] Various artifacts occur in OCT images. One of these is an artifact resulting from the birefringence of the sample. Birefringence is an optical property of a material, with a refractive index that depends on the polarization and propagation direction of light. A light beam passing through a birefringent sample is split into two beams depending on its polarization state. Artifacts resulting from birefringence (birefringence-related artifacts) occur not only in OCT images generated by OCT modalities capable of detecting polarized light, but also in standard OCT intensity images. Note that OCT modalities capable of detecting polarized light (polarization-sensitive OCT) include those described in Patent Documents 1 to 4, for example.

[0004] A known technique for removing birefringence-induced artifacts is to separate an OCT signal into two polarization components, detect them, and generate an OCT intensity signal by combining the intensities of the detected two polarization component signals. This technique is realized, for example, as described in Non-Patent Document 1, using a detection module (polarization separation detection function) that combines a polarizing beam splitter and two photodetectors. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 4344829 specification [Patent Document 2] Patent No. 6256879 specification [Patent Document 3] Patent No. 6346410 specification [Patent Document 4] Patent No. 6579718 specification [Non-patent literature]

[0006] [Non-Patent Document 1] Shuichi Makita, Toshihiro Mino, Tastuo Yamaguchi, Mashiro Miura, Shinnosuke Azuma, and Yoshiaki Yasuno, “Clinical prototype of pigment and flow imaging optical coherence tomography for posterior eye investigation”, Biomedical Optics Express, Vol. 9, Issue 9, pp. 4372-4389 (2018) Summary of the Invention [Problem to be solved by the invention]

[0007] One of the objectives of the present disclosure is to provide a new technique for generating images with reduced birefringence artifacts from images acquired using an OCT modality that does not have polarization separation detection capabilities. [Means for solving the problem]

[0008] One aspect of the embodiment is a method for processing optical coherence tomography (OCT) images, comprising: preparing a machine learning model constructed by machine learning using training data including a set of pairs of a first image containing birefringence information of an object and a second image not containing birefringence information of the object; the machine learning model constructed to receive an OCT image as input and output a denoised image in which birefringence-induced artifacts in the OCT image are reduced; acquiring an OCT image of a sample; inputting the OCT image of the sample to the machine learning model; and storing or providing the denoised image output from the machine learning model based on the input of the OCT image of the sample.

[0009] Another aspect of the embodiment is an apparatus for processing optical coherence tomography (OCT) images, comprising: an image acquisition unit that acquires OCT images of a sample; and a processing unit that performs processing using a machine learning model constructed by machine learning using training data including a set of pairs of a first image including birefringence information of an object and a second image not including birefringence information of the object, the machine learning model being constructed to receive an OCT image as input and output a denoised image in which birefringence-induced artifacts in the OCT image are reduced, wherein the processing unit inputs the OCT image of the sample acquired by the image acquisition unit to the machine learning model, and stores or provides the denoised image output from the machine learning model based on the input of the OCT image of the sample.

[0010] Yet another aspect of the embodiment is a program for processing optical coherence tomography (OCT) images, the program causing a computer to execute the following steps: acquiring an OCT image of a sample; inputting the OCT image of the sample into a machine learning model constructed by machine learning using training data including a set of pairs of a first image including birefringence information of an object and a second image not including birefringence information of the object, the machine learning model being constructed to receive an OCT image as input and output a denoised image in which birefringence-induced artifacts in the OCT image are reduced; and storing or providing the denoised image output from the machine learning model based on the input of the OCT image of the sample.

[0011] Yet another aspect of the embodiment is a computer-readable non-transitory recording medium having a program recorded thereon for processing optical coherence tomography (OCT) images, the program comprising: The program causes a computer to perform the following steps: acquiring an OCT image of a sample; inputting the OCT image of the sample into a machine learning model constructed by machine learning using training data including a set of pairs of a first image containing birefringence information of an object and a second image not containing birefringence information of the object, the machine learning model being constructed to receive an OCT image as input and output a denoised image in which birefringence-induced artifacts in the OCT image have been reduced; and storing or providing the denoised image output from the machine learning model based on the input of the OCT image of the sample. [Effects of the Invention]

[0012] According to the embodiment, it is possible to provide a new technique for generating an image in which birefringence-induced artifacts are reduced from an image acquired by an OCT modality that does not have a polarization separation detection function. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram illustrating an example of a configuration of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 2] 1 is a schematic diagram illustrating an example of a configuration of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 3] 1 is a schematic diagram illustrating an example of a configuration of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 4] 1 is a schematic diagram illustrating an example of a configuration of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 5] 1 is a schematic diagram illustrating an example of a configuration of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 6] 10 is a flowchart illustrating an example of an operation of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 7] 10 is a flowchart illustrating an example of an operation of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 8] 10 is a flowchart illustrating an example of an operation of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. [Figure 9] 10 is a flowchart illustrating an example of an operation of an image processing device (ophthalmologic apparatus) according to an exemplary aspect of an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] An image processing method, an image processing device, a program, and a recording medium according to some exemplary aspects of the embodiments will be described in detail with reference to the drawings. In the present disclosure, some application examples will be described, particularly in the field of ophthalmology, but the embodiments are not limited thereto and can be applied to any field in which OCT can be used and in which birefringence-induced artifacts occur or are likely to occur.

[0015] Any matter described in the documents cited in this disclosure or any matter related to other known technologies can be combined with the embodiments. In addition, in this disclosure, unless otherwise specified, no distinction is made between "image data" and "images," which are visualized information based on the image data.

[0016] The image processing device according to the embodiment has a function of acquiring an OCT image of a sample. Any configuration may be employed to achieve this function. In some exemplary embodiments, the image processing device is configured to acquire the OCT image of the sample by applying an OCT scan to the sample to collect data and then process the collected data to construct the OCT image. Any type of OCT (OCT modality) may be used for the OCT scan, and may be, for example, Fourier-domain OCT (swept-source OCT or spectral-domain OCT) or time-domain OCT.

[0017] In some exemplary embodiments, the image processing device is configured to externally acquire an OCT image of the sample. Such an image processing device may include, for example, a communication device for receiving an OCT image stored in a storage device via a communication line, a communication device for receiving an OCT image acquired by the OCT device via a communication line, or a data reader for reading an OCT image recorded on a recording medium.

[0018] Furthermore, the image processing device according to the embodiment has a function of generating an image in which birefringence artifacts have been reduced from the acquired OCT image. An image in which birefringence artifacts have been reduced by the processing according to the present disclosure is called a denoised image.

[0019] The image processing function for generating a denoised image is implemented using a pre-created machine learning model. A machine learning system according to an embodiment is constructed by machine learning using training data (teacher data, learning data) including a set of pairs of a first image including birefringence information of an object and a second image not including birefringence information of the object. The machine learning system receives an OCT image as an input and outputs a denoised image in which birefringence-related artifacts in the OCT image have been reduced. In some aspects of the embodiment, the first image including birefringence information of the object is an image including information (e.g., birefringence-related artifacts) derived from the birefringence of the object, and the second image not including birefringence information of the object is an image not including information derived from the birefringence of the object.

[0020] The denoised image generated by the image processing device according to the embodiment is stored and / or provided. In some exemplary embodiments, the denoised image is saved in a storage device located inside or outside the image processing device. The storage device inside the image processing device may be, for example, a hard disk drive, a solid state drive, etc. The storage device outside the image processing device may be, for example, a database system, a data management system, etc. An example of such a system is a picture archiving and communication system (PACS), which is a typical image management system in the medical field.

[0021] In some exemplary embodiments, the denoised image generated by the image processing device is provided to a computer that is internal or external to the image processing device. The computer has a function of processing the denoised image to generate new information. Examples of the computer's functions include a function of analyzing the denoised image to generate analytical information, a function of processing the denoised image to generate visualized information, a function of combining the denoised image with other information to generate new information, a function of generating training data for machine learning based on the denoised image, a function of performing machine learning using training data that includes the denoised image and / or information based on the denoised image, a function of performing inference using the denoised image and / or information based thereon, and a function that at least partially combines two or more of these functions.

[0022] At least a portion of the functionality of the elements of the present disclosure is implemented using circuitry or processing circuitry. The circuitry or processing circuitry may be a general-purpose processor, a special-purpose processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), a field programmable gate array (FPGA)), or a combination of these devices configured and / or programmed to perform at least a portion of the disclosed functionality. The term "circuitry," "unit," "means," or the like refers to hardware that performs at least a portion of the disclosed functions or that is programmed to perform at least a portion of the disclosed functions. The hardware may be the hardware described in this disclosure or may be known hardware that is programmed and / or configured to perform at least a portion of the functions described in this disclosure. In the case of a processor, where the hardware can be considered a type of circuitry, the term "circuitry," "unit," "means," or the like refers to a combination of hardware and software, where the software is used to configure the hardware and / or the processor.

[0023] <Configuration of image processing device (ophthalmic device)> 1 is an exemplary embodiment of an image processing device according to an embodiment, and is configured to acquire an OCT image of a sample by applying an OCT scan to the sample to collect data and then process the collected data to construct the OCT image. In this embodiment, the sample is a living human eye (examined eye).

[0024] The ophthalmic device 1 is a multifunction device that combines an OCT device and a fundus camera, and has the function of applying an OCT scan to the subject's eye E (fundus Ef and anterior segment Ea) and the function of taking digital photographs of the subject's eye E (fundus Ef and anterior segment Ea).

[0025] The ophthalmic apparatus 1 includes a fundus camera unit 2, an OCT unit 100, and an arithmetic and control unit 200. The fundus camera unit 2 is provided with various elements (optical system, mechanism, etc.) for acquiring a front image of the subject's eye. The OCT unit 100 is provided with some of the various elements (optical system, mechanism, etc.) for OCT scanning. Other elements for OCT scanning are provided in the fundus camera unit 2. The arithmetic and control unit 200 includes one or more processors and one or more storage devices configured to execute various processes (arithmetic, control, etc.). Furthermore, the ophthalmic apparatus 1 includes a chin rest, a forehead rest, etc.

[0026] The attachment 400 includes a lens group for switching the region where the OCT scan is applied between the posterior segment (fundus Ef) and the anterior segment Ea. The attachment 400 may be, for example, the optical unit disclosed in Japanese Patent Application Laid-Open No. 2015-160103. The attachment 400 can be arranged between the objective lens 22 and the subject's eye E. The attachment 400 may be retracted from the optical path to apply an OCT scan to the fundus Ef, and may be arranged in the optical path to apply an OCT scan to the anterior segment Ea. Conversely, the attachment 400 may be retracted from the optical path to apply an OCT scan to the anterior segment Ea, and may be arranged in the optical path to apply an OCT scan to the fundus Ef. The attachment 400 is moved manually or automatically. The element for switching the region where the OCT scan is applied is not limited to such an attachment and may be, for example, a configuration including one or more lenses movable along the optical path.

[0027] <Fundus camera unit 2> The fundus camera unit 2 is provided with elements (optical system, mechanism, etc.) for taking digital photographs of the subject's eye E. The acquired digital photographs are front images such as observed images and photographed images. The observed images are acquired by video capture using near-infrared light, for example, and are used for alignment, focusing, tracking, etc. The photographed images are still images captured using flash light in the visible or infrared range, for example, and are used for diagnosis, analysis, etc.

[0028] The fundus camera unit 2 includes an illumination optical system 10 and an imaging optical system 30. The illumination optical system 10 irradiates illumination light onto the subject's eye E. The imaging optical system 30 detects return light of the illumination light from the subject's eye E. The measurement light from the OCT unit 100 is guided to the subject's eye E through an optical path within the fundus camera unit 2, and the return light is guided to the OCT unit 100 through the same optical path.

[0029] Light (observation illumination light) output from an observation light source 11 of an illumination optical system 10 is reflected by a concave mirror 12, passes through a condenser lens 13, and passes through a visible light cut filter 14 to become near-infrared light. The observation illumination light is then focused near an imaging light source 15, reflected by a mirror 16, and passes through a relay lens system 17, a relay lens 18, an aperture 19, and a relay lens system 20. The observation illumination light is then reflected by the peripheral portion (the area around the hole) of a perforated mirror 21, passes through a dichroic mirror 46, and is refracted by an objective lens 22 (and optical elements in the attachment 400) to illuminate the subject's eye E. The return light of the observation illumination light from the subject's eye E is refracted by (the optical elements in the attachment 400 and) the objective lens 22, passes through the dichroic mirror 46, passes through the hole formed in the central region of the aperture mirror 21, passes through the dichroic mirror 55, passes through the photographing focusing lens 31, and is reflected by the mirror 32. Furthermore, this return light passes through the half mirror 33A, is reflected by the dichroic mirror 33, and is imaged on the light-receiving surface of the image sensor 35 by the imaging lens 34. The image sensor 35 detects the return light at a predetermined frame rate. The focus (focal position) of the photographing optical system 30 is typically adjusted to match the fundus Ef or the anterior segment Ea.

[0030] Light output from the imaging light source 15 (imaging illumination light) travels along the same path as the observation illumination light and is irradiated onto the subject's eye E. Return light of the imaging illumination light from the subject's eye E travels along the same path as the return light of the observation illumination light and is guided to the dichroic mirror 33, passes through the dichroic mirror 33, is reflected by a mirror 36, and is imaged by an imaging lens 37 on the light-receiving surface of an image sensor 38.

[0031] The liquid crystal display (LCD) 39 displays a fixation target (fixation target image). A portion of the light beam output from the LCD 39 is reflected by the half mirror 33A, reflected by the mirror 32, passes through the photographing focusing lens 31 and the dichroic mirror 55, and passes through the hole in the aperture mirror 21. The light beam that passes through the hole in the aperture mirror 21 passes through the dichroic mirror 46, is refracted by the objective lens 22, and is projected onto the fundus Ef. By changing the display position of the fixation target image, the fixation position of the subject's eye E determined by the fixation target can be changed. This makes it possible to guide the gaze of the subject's eye E in a desired direction.

[0032] The alignment optical system 50 generates an alignment index used to align the optical system with the subject's eye E. Alignment light output from a light-emitting diode (LED) 51 passes through an aperture 52, an aperture 53, and a relay lens 54, is reflected by a dichroic mirror 55, passes through the hole in the aperture mirror 21, transmits through the dichroic mirror 46, and is projected onto the subject's eye E via the objective lens 22. The return light of the alignment light from the subject's eye E is guided to the image sensor 35 via the same path as the return light of the observation illumination light. Manual alignment or automatic alignment can be performed based on the received light image (alignment index image).

[0033] The alignment method is not limited to the method using an alignment index. For example, as described in Japanese Patent Application Laid-Open No. 2013-248376, some exemplary embodiments of the ophthalmologic apparatus may be configured to photograph the anterior segment of the eye from different directions to obtain two or more anterior segment images, analyze these anterior segment images to determine the three-dimensional position of the subject's eye, and move the optical system based on the three-dimensional position (stereo alignment).

[0034] The focusing optical system 60 generates a split index used for focus adjustment of the subject's eye E. The focusing optical system 60 moves along the optical path (illumination optical path) of the illumination optical system 10 in conjunction with the movement of the photographing focusing lens 31 along the optical path (photography optical path) of the photographing optical system 30. To perform focus adjustment, a reflecting rod 67 is inserted into the illumination optical path and positioned at an angle. Focusing light output from the LED 61 passes through a relay lens 62, is split into two beams by a split index plate 63, passes through a two-hole diaphragm 64, is reflected by a mirror 65, and is temporarily imaged and reflected on the reflecting surface of the reflecting rod 67 by a condenser lens 66. The focusing light then passes through a relay lens 20, is reflected by a perforated mirror 21, passes through a dichroic mirror 46, and is projected onto the subject's eye E via an objective lens 22. The returning light of the focusing light from the subject's eye E is guided to the image sensor 35 via the same path as the returning light of the alignment light. Manual focusing or autofocusing can be performed based on the received light image (split target image).

[0035] Diopter correction lenses 70 and 71 are selectively inserted in the photographing optical path between the aperture mirror 21 and the dichroic mirror 55. The diopter correction lens 70 is a plus lens (convex lens) for correcting strong hyperopia. The diopter correction lens 71 is a minus lens (concave lens) for correcting strong myopia.

[0036] The dichroic mirror 46 couples the OCT optical path (measurement arm) to the digital photography optical path (illumination optical path and imaging optical path). The dichroic mirror 46 reflects light in the wavelength band for OCT scanning and transmits light in the wavelength band for digital photography. The measurement arm is provided with, in order from the OCT unit 100 side, a collimator lens unit 40, a retroreflector 41, a dispersion compensation member 42, an OCT focusing lens 43, an optical scanner 44, and a relay lens 45.

[0037] The retroreflector 41 is movable in the directions indicated by the arrows in Fig. 1 (the incident and exit directions of the measurement light LS). This changes the length of the measurement arm. Changing the measurement arm length is used, for example, to correct the optical path length according to the axial length of the eye, the shape of the cornea, or the shape of the fundus, or to adjust the interference state.

[0038] The dispersion compensation member 42, together with a dispersion compensation member 113 (described later) disposed in the reference arm, acts to match the dispersion characteristics of the measurement light LS with the dispersion characteristics of the reference light LR.

[0039] The OCT focusing lens 43 is movable along the direction indicated by the arrow in Fig. 1 (the optical axis of the measurement arm) to adjust the focus of the measurement arm. This changes the focus state (focal position, focal length) of the measurement arm. The ophthalmologic apparatus 1 may be capable of cooperatively controlling the movement of the imaging focusing lens 31, the movement of the focus optical system 60, and the movement of the OCT focusing lens 43.

[0040] The optical scanner 44 is disposed substantially at a position optically conjugate with the pupil of the subject's eye E. The optical scanner 44 is a deflector for changing the traveling direction (propagation direction) of the measurement light LS guided by the measurement arm. The optical scanner 44 is, for example, a two-dimensional deflector including a deflector for scanning in the x direction (x-scanner) and a deflector for scanning in the y direction (y-scanner). The type of deflector may be any type, and may be, for example, a galvanometer scanner.

[0041] The optical scanner 44 is positioned substantially optically conjugate to the pupil of the subject's eye E when the attachment 400 is retracted from the measurement arm for posterior segment OCT, and is positioned substantially optically conjugate to a position near the anterior segment Ea (e.g., a position between the anterior segment Ea and the attachment 400) when the attachment 400 is inserted into the measurement arm for anterior segment OCT.

[0042] <OCTユニット100> The exemplary OCT unit 100 shown in FIG. 2 is provided with an optical system and mechanisms for applying swept-source OCT. This optical system includes an interference optical system. This interference optical system splits light from a wavelength-tunable light source (swept-wavelength light source) into measurement light and reference light, superimposes the measurement light returned from the subject's eye E with the reference light that has passed through the reference light path, and generates interference light, which is then detected. An electrical signal (detection signal) generated by interference light detection includes a signal (interference signal) representing the spectrum of the interference light, and is sent to the arithmetic and control unit 200 (image construction unit 220).

[0043] Light source unit 101 includes, for example, a near-infrared wavelength-tunable laser that changes the wavelength of output light at high speed. Light L0 output from light source unit 101 is guided by optical fiber 102 to polarization controller 103, where its polarization state is adjusted, and then guided by optical fiber 104 to fiber coupler 105, where it is split into measurement light LS and reference light LR. The optical path of measurement light LS is called a measurement arm, and the optical path of reference light LR is called a reference arm.

[0044] The reference light LR is guided by an optical fiber 110 to a collimator 111, where it is converted into a parallel beam, and then guided to a retroreflector 114 via an optical path length correction element 112 and a dispersion compensation element 113. The optical path length correction element 112 is an optical element for matching the optical path length of the reference light LR with that of the measurement light LS. The dispersion compensation element 113, together with a dispersion compensation element 42 arranged in the measurement arm, acts to match the dispersion characteristics between the reference light LR and the measurement light LS. The retroreflector 114 is movable along the optical path of the reference light LR incident thereon, thereby changing the length of the reference arm. Changing the reference arm length is used, for example, to correct the optical path length according to the axial length of the eye, the shape of the cornea or the fundus, or to adjust the interference state.

[0045] The reference light LR that has passed through the retroreflector 114 passes through a dispersion compensation member 113 and an optical path length correction member 112, is converted from a parallel beam into a convergent beam by a collimator 116, and enters an optical fiber 117. The reference light LR that has entered the optical fiber 117 is guided to a polarization controller 118, where its polarization state is adjusted. The polarization controller 118 is used, for example, to optimize the interference intensity between the measurement light LS and the reference light LR. The reference light LR that has passed through the polarization controller 118 is guided through an optical fiber 119 to an attenuator 120, where its light amount is adjusted, and is then guided through an optical fiber 121 to a fiber coupler 122.

[0046] On the other hand, the measurement light LS generated by the fiber coupler 105 is guided by the optical fiber 127 to the collimator lens unit 40 and converted into a parallel beam, and then passes through the retroreflector 41, the dispersion compensation member 42, the OCT focusing lens 43, the optical scanner 44, the relay lens 45, the dichroic mirror 46, and the objective lens 22 (and the attachment 400) before being projected onto the subject's eye E. The measurement light LS incident on the subject's eye E is scattered and reflected at various depth positions in the subject's eye E. Return light (backscattered light, reflected light, etc.) of the measurement light LS from the subject's eye E travels in the opposite direction along the same path as the outward path and is guided to the fiber coupler 105, and is guided to the fiber coupler 122 via the optical fiber 128.

[0047] The fiber coupler 122 generates interference light by superimposing the measurement light LS (return light from the subject's eye E) from the optical fiber 128 and the reference light LR from the optical fiber 121. The fiber coupler 122 splits the generated interference light into two lights at a predetermined split ratio (for example, 1:1) to generate a pair of interference light LC. The pair of interference light LC is guided to a detector 125 via optical fibers 123 and 124, respectively.

[0048] The detector 125 includes, for example, a balanced photodiode. The balanced photodiode includes a pair of photodetectors that respectively detect a pair of interference lights LC, and outputs the difference between a pair of electrical signals generated by the pair of photodetectors. The output difference signal (detection signal) is sent to a data acquisition system (DAQ, DAS) 130.

[0049] A clock KC is supplied to the data collection system 130 from the light source unit 101. The clock KC is generated in the light source unit 101 in synchronization with the output timing of each wavelength swept within a predetermined wavelength range by the wavelength-tunable light source. The light source unit 101, for example, branches light L0 of each output wavelength to generate two branched lights, applies an optical delay to one of the branched lights, combines the delayed branched light with the other branched light, detects the resulting combined light, and generates a clock KC based on the detection result. The data collection system 130 samples the detection signal input from the detector 125 based on the clock KC input from the light source unit 101. The sampling result is sent to the arithmetic and control unit 200.

[0050] In this embodiment, both an element for changing the measurement arm length (such as retroreflector 41) and an element for changing the reference arm length (such as retroreflector 114, reference mirror, etc.) are provided, but in some exemplary embodiments, only one of these two elements may be provided.

[0051] In this embodiment, an element (polarization controller 118) for changing the polarization state of the reference light LR is provided. In some exemplary embodiments, an element (polarization controller) for changing the polarization state of the measurement light LS may be provided instead of an element for changing the polarization state of the reference light LR. In some exemplary embodiments, both an element for changing the polarization state of the reference light LR and an element for changing the polarization state of the measurement light LS may be provided.

[0052] Swept-source OCT, as used in the OCT unit 100 of FIG. 2, is a technique that splits light from a tunable light source into measurement light and reference light, superimposes return light from the test object on the reference light to generate interference light, detects this interference light with a photodetector, and constructs an image by applying Fourier transform or the like to the detection data collected in response to wavelength sweeps and measurement light scans. In contrast, spectral-domain OCT is a technique that splits light from a low-coherence light source (broadband light source) into measurement light and reference light, superimposes return light from the test object on the reference light to generate interference light, detects the spectral distribution of this interference light with a spectroscope, and constructs an image by applying Fourier transform or the like to the detected spectral distribution. Simply put, swept-source OCT is an OCT technique that acquires the spectral distribution of interference light in a time-division manner, while spectral-domain OCT is an OCT technique that acquires the spectral distribution of interference light in a space-division manner. It will be apparent to those skilled in the art that the OCT technique applicable to the embodiments is not limited to swept-source OCT.

[0053] <Control system / processing system> 3 shows an example of the configuration of the control system and processing system of the ophthalmic apparatus 1. The control unit 210, image construction unit 220, and data processing unit 230 are provided in, for example, an arithmetic control unit 200. Although not shown, the ophthalmic apparatus 1 may also include a communication device and a drive device (reader / writer).

[0054] <Control unit 210> The control unit 210 executes various types of control. The control unit 210 includes a main control unit 211 and a storage unit 212. The main control unit 211 includes a processor and controls the elements of the ophthalmologic apparatus 1 (elements shown in FIGS. 1 to 5). The main control unit 211 is realized by cooperation between hardware including the processor and control software. The storage unit 212 includes a storage device such as a hard disk drive or a solid state drive, and stores data.

[0055] The photographing focusing driver 31A moves the photographing focusing lens 31 provided in the photographing optical path and the focus optical system 60 provided in the illumination optical path under the control of the main controller 211. The retroreflector (RR) driver 41A moves the retroreflector 41 provided in the measurement arm under the control of the main controller 211. The OCT focusing driver 43A moves the OCT focusing lens 43 provided in the measurement arm under the control of the main controller 211. The retroreflector (RR) driver 114A moves the retroreflector 114 provided in the reference arm under the control of the main controller 211. The movement mechanism 150 moves the optical system of the ophthalmologic apparatus 1 three-dimensionally (in the x, y, and z directions). The insertion / removal mechanism 400A inserts the attachment 400 into the optical path and retracts it from the optical path.

[0056] <Image construction unit 220> The image construction unit 220 constructs OCT image data of the subject's eye E based on signals (sampling data) input from the data collection system 130. The constructed OCT image data is one or more A-scan image data, for example, B-scan image data (two-dimensional cross-sectional image data) made up of a plurality of A-scan image data. The image construction unit 220 is realized by cooperation between hardware including a processor and image construction software.

[0057] To construct OCT image data from the sampling data, the image forming unit 220 performs signal processing on the spectral distribution based on the sampling data for each A-line, similar to conventional swept-source OCT, to generate a reflection intensity profile for each A-line (A-line profile), visualizes each A-line profile to generate multiple A-scan image data, and arranges these A-scan image data according to a scan pattern (arrangement of multiple scan points). The signal processing for generating the A-line profile includes noise reduction, filtering, fast Fourier transform (FFT), etc. If another OCT method is used, a known OCT image data construction process appropriate for that method is performed.

[0058] The image construction unit 220 may be configured to construct three-dimensional image data representing a three-dimensional region (volume) of the subject's eye E. The three-dimensional image data is image data in which pixel positions are defined by a three-dimensional coordinate system, and examples thereof include stack data and volume data. The stack data is image data obtained by arranging multiple cross-sectional images obtained along multiple scan lines in accordance with the positional relationship of these scan lines. The volume data is image data in which pixels are three-dimensionally arranged voxels constructed by applying, for example, interpolation processing or voxelization processing to the stack data, and is also called voxel data.

[0059] The image constructor 220 can generate new OCT image data from the constructed OCT image data. In some exemplary embodiments, the image constructor 220 can apply rendering to the three-dimensional image data. Examples of rendering include volume rendering, surface rendering, maximum intensity projection (MIP), minimum intensity projection (MIP), and multiplanar reconstruction (MPR).

[0060] In some exemplary embodiments, the image constructor 220 may be configured to construct an OCT en face image from the three-dimensional image data. For example, the image constructor 220 may construct projection data by projecting the three-dimensional image data in the z direction (A-line direction, depth direction). The image constructor 220 may also construct projection data from partial data (e.g., slabs) of the three-dimensional image data. This partial data may be automatically specified using, for example, image segmentation (also simply referred to as segmentation) or manually specified by a user. This segmentation method may be any method and may include, for example, image processing such as edge detection and / or segmentation using machine learning. This segmentation may be performed, for example, by the image constructor 220 or the data processor 230.

[0061] The ophthalmic apparatus 1 may be capable of performing OCT motion contrast imaging. OCT motion contrast imaging is an imaging technique that extracts the movement of fluid or the like present in the eye (see, for example, JP 2015-515894 A). OCT motion contrast imaging is used, for example, in OCT angiography (OCTA) for visualizing blood vessels.

[0062] <Data processing unit 230> The data processing unit 230 is configured to apply specific data processing to the image of the subject's eye E. The data processing unit 230 is realized, for example, by cooperation between hardware including a processor and data processing software.

[0063] The data processing unit 230 of this embodiment has a function of executing processing using a model (machine learning model, mathematical model, inference model) constructed using machine learning.

[0064] The training data used in this machine learning includes a set of pairs of a first image containing birefringence information of an object and a second image not containing birefringence information of the object. In other words, the training data used in this machine learning includes multiple pairs of OCT images, one containing birefringence information and the other not containing birefringence information, acquired of the same object.

[0065] In this disclosure, a pair of a first image and a second image acquired of one object is called an image pair, and a set of image pairs included in training data is called a set of image pairs.

[0066] In this embodiment, the object to which OCT scans are applied to collect images included in the training data may be a living human eye, or more specifically, a predetermined structure of a living human eye (e.g., an ocular region, an ocular tissue, an intraocular prosthesis, etc.). The ocular region or ocular tissue may be any region or tissue of the eye, such as the fundus (posterior segment), anterior segment, eyelid, optic nerve head, lamina cribrosa, macula, retina, retinal sub-tissues (e.g., the internal limiting membrane, nerve fiber layer, ganglion cell layer, inner plexiform layer, inner nuclear layer, outer plexiform layer, outer nuclear layer, external limiting membrane, photoreceptor layer, and retinal pigment epithelium), Bruch's membrane, choroid, choroid-scleral junction (CSI), sclera, vitreous body, cornea, corneal sub-tissues (e.g., the corneal epithelium, Bowman's membrane, lamina propria, Dua's layer, Descemet's membrane, and corneal endothelium), conjunctiva, Schlemm's canal, trabecular meshwork, iris, lens, Zonule of Zinn, and ciliary body. An intraocular prosthesis can be any object implanted in the eye, examples of which include an intraocular lens (IOL), an intraocular contact lens (ICL), and a minimally invasive glaucoma surgery (MIGS) device.

[0067] Although the image pair sets included in the training data may be acquired from a single object, in some example embodiments, image pair sets acquired from multiple objects may be used. For example, the image pair sets may include one or more OCT image pairs acquired from a first object, one or more OCT image pairs acquired from a second object, ..., one or more OCT image pairs acquired from a Kth object (K is an integer greater than or equal to 2).

[0068] In some exemplary embodiments, the image pair set may be a set consisting only of images (raw images) acquired from an object by OCT scanning. In addition, in some exemplary embodiments, the image pair set may include images (processed images) obtained by processing the raw images.

[0069] Some examples of methods for creating training data will be described later.

[0070] The machine learning model of this embodiment is configured to receive an OCT image obtained by applying an OCT scan to a sample (subject's eye E) as input, and to output a denoised image in which birefringence-induced artifacts in this OCT image have been reduced.

[0071] Some examples of the configuration of the data processing unit 230 of this embodiment and some examples of the processing executed by the data processing unit 230 will be described with reference to FIGS.

[0072] A data processing unit 230A shown in Fig. 4 is one example of the configuration of the data processing unit 230 in Fig. 3. The data processing unit 230A includes a denoised image generating unit 231. The denoised image generating unit 231 is configured to generate, based on an OCT image of the subject's eye E, a denoised image in which birefringence-derived artifacts in the OCT image are reduced.

[0073] The denoised image generation unit 231 generates a denoised image using an inference model 2311. The inference model 2311 includes a neural network 2312 trained by the above-described machine learning so as to receive an input of an OCT image of the subject's eye E and output a denoised image.

[0074] 4, the inference model 2311 may be arranged inside the denoised image generation unit 231 (data processing unit 230), or may be arranged in a location of the ophthalmic apparatus 1 other than the data processing unit 230 (for example, in the memory unit 212), or may be arranged outside the ophthalmic apparatus 1. As an example of the latter, the inference model 2311 may be arranged inside a computer or storage device accessible by the ophthalmic apparatus 1.

[0075] The device that constructs the inference model 2311 (inference model construction device) may be provided in the ophthalmic device 1, or in a peripheral device (such as a computer) of the ophthalmic device 1, or may be another computer.

[0076] The model construction unit 300 shown in FIG. 5 is an example of an inference model construction device, and includes a learning processing unit 310 and a neural network 320.

[0077] The neural network 320 typically includes a convolutional neural network (CNN). Reference numeral 330 in Figure 5 indicates an example of the structure of this convolutional neural network.

[0078] An image is input to the input layer of the convolutional neural network 330. After the input layer, multiple pairs of convolutional layers and pooling layers are arranged. In the example shown in Figure 5, three pairs of convolutional layers and pooling layers are provided, but the number of pairs may be any number.

[0079] The convolution layer performs convolution operations to extract features (such as contours) from an image. A convolution operation is a multiplication and accumulation operation of a filter function (weighting coefficients, filter kernel) of the same dimension as the input image on the input image. The convolution layer applies the convolution operation to multiple parts of the input image. More specifically, the convolution layer multiplies the value of each pixel in the partial image to which the filter function has been applied by the value (weight) of the filter function corresponding to that pixel to calculate the product, and then calculates the sum of the products across multiple pixels in this partial image. The resulting sum-of-products value is assigned to the corresponding pixel in the output image. By performing the multiplication and accumulation operation while shifting the location (partial image) to which the filter function is applied, the convolution operation result for the entire input image is obtained. This convolution operation generates multiple images in which various features have been extracted using multiple weighting coefficients. In other words, multiple filtered images, such as smoothed images and edge images, are obtained. The multiple images generated by the convolution layer are called feature maps.

[0080] The pooling layer compresses (e.g., thins out data) the feature map generated by the immediately preceding convolutional layer. More specifically, the pooling layer calculates statistical values ​​of predetermined neighboring pixels of a pixel of interest in the feature map at predetermined pixel intervals, and outputs an image with dimensions smaller than the input feature map. The statistical values ​​applied to the pooling operation are, for example, maximum values ​​(max pooling) or average values ​​(average pooling). The pixel interval applied to the pooling operation is called the stride.

[0081] A convolutional neural network can extract many features from an input image by processing it using multiple pairs of convolutional layers and pooling layers.

[0082] A fully connected layer is provided after the last pair of convolutional and pooling layers. In the example shown in Figure 5, two fully connected layers are provided, but any number of fully connected layers may be used. In the fully connected layer, features compressed by a combination of convolution and pooling are used to perform processes such as image classification, image segmentation, and regression. After the last fully connected layer, an output layer is provided to provide output results.

[0083] In some exemplary embodiments, the convolutional neural network may not include a fully connected layer (e.g., a fully convolutional network (FCN)) and / or may include a support vector machine, a recurrent neural network (RNN), or the like. Furthermore, the machine learning performed on the neural network 320 may include transfer learning. That is, the neural network 320 may include a neural network that has already been trained using other training data (training images) and whose parameters have been adjusted. Furthermore, the model construction unit 300 (the learning processing unit 310) may be configured to be able to apply fine tuning to the trained neural network (the neural network 320). The neural network 320 may be constructed using a known open-source neural network architecture.

[0084] The learning processing unit 310 applies machine learning using training data to the neural network 320. When the neural network 320 includes a convolutional neural network, the parameters adjusted by the learning processing unit 310 include, for example, filter coefficients of the convolutional layer and connection weights and offsets of the fully connected layer.

[0085] The types of images included in the training data are not limited to OCT images. For example, the training data may include images acquired by ophthalmic modalities other than OCT (such as fundus cameras, slit lamp microscopes, SLOs, and surgical microscopes), images acquired by diagnostic imaging modalities of any medical department (such as ultrasound diagnostic devices, X-ray diagnostic devices, X-ray CT devices, and magnetic resonance imaging (MRI) devices), images created by processing actual eye images (processed image data), computer-generated images, simulated images, and any other images. Furthermore, the number of data items included in the training data may be increased using techniques such as data augmentation.

[0086] A training method (machine learning method) for constructing the neural network 2312 is, for example, supervised learning, but is not limited to this. In some exemplary embodiments, in the machine learning for constructing the neural network 2312, any known method such as unsupervised learning, reinforcement learning, semi-supervised learning, transduction, or multi-task learning can be used in addition to or instead of supervised learning.

[0087] In some exemplary embodiments, supervised learning is performed using training data generated by annotation that assigns metadata (labels, tags) to input images. For example, this annotation assigns labels indicating birefringence artifacts to images included in the training data. Identification of birefringence artifacts in an image may be performed by, for example, at least one of a doctor, an engineer, a computer, and other inference models. The image to be annotated in this example, i.e., the image to which annotation information indicating birefringence artifacts is added, is an image containing birefringence information of an object (the first image described above).

[0088] The learning processing unit 310 can construct the neural network 2312 by applying supervised learning using such training data to the neural network 320. This supervised learning is performed, for example, for each image pair included in the image pair set (a pair of a first image including birefringence information of an object and a second image not including birefringence information of the object) so that a second image is obtained as an output corresponding to the input of the first image. The inference model 2311 including the neural network 2312 constructed according to this example functions to receive an input of an OCT image of the subject's eye E and output a denoised image in which birefringence-related artifacts in the OCT image have been reduced. In some exemplary embodiments, the OCT image input to the inference model 2311 is an image acquired by an OCT modality without polarization separation detection function (e.g., a normal OCT intensity image), and the corresponding image output from the inference model 2311 is a denoised image in which birefringence-related artifacts in the input image have been reduced (a denoised image of a normal OCT intensity image).

[0089] When supervised learning is performed using training data including a set of image pairs in which annotation information indicating birefringence artifacts is attached to a first image, the supervised learning is performed, for example, for each image pair included in the set of image pairs (a pair of a first image including birefringence information of an object and annotated with annotation information, and a second image not including the birefringence information of the object) so that the birefringence artifacts indicated by the annotation information attached to the first image are reduced and a second image is obtained as an output corresponding to the input of the first image. The inference model 2311 including the neural network 2312 constructed according to this example functions to receive an input of an OCT image of the subject's eye E (e.g., a normal OCT intensity image) and output a denoised image in which birefringence artifacts in the OCT image have been reduced. According to this example, the annotation information indicates the location (area, range) of the birefringence artifacts, thereby improving the efficiency and quality (accuracy, precision, reproducibility, etc.) of machine learning.

[0090] To avoid concentrating processing on specific units of the neural network 2312, the learning processing unit 310 may randomly select and disable some units of the neural network 320 and perform learning using the remaining units (dropout).

[0091] The techniques used to build the inference model are not limited to the examples shown here. In some exemplary embodiments, any known technique can be used to build the inference model, such as a support vector machine, a Bayesian classifier, boosting, k-means, kernel density estimation, principal component analysis, independent component analysis, self-organizing maps, random forests, or generative adversarial networks (GANs).

[0092] The information that the data processing unit 230 can generate from an OCT image is not limited to a denoised image. For example, the data processing unit 230 may have any of the following functions: a function to detect birefringence artifacts from an OCT image; a function to generate attribute information of birefringence artifacts; a function to generate a discrimination result (discrimination information) between an image of the true structure of a sample (subject's eye E) and birefringence artifacts; and a function to generate image segmentation information related to birefringence artifacts. Each function is realized using a machine learning algorithm and / or a non-machine learning algorithm. That is, each function may be realized at least in part using a machine learning model, or at least in part without using a machine learning model.

[0093] The attribute information of the birefringence artifact is information that represents any property (characteristic) of the birefringence artifact in the OCT image, and may be information about any parameter of the birefringence artifact, such as the position, size, shape, strength, influence, etc. When a machine learning model is used, the training data used to construct this machine learning model includes annotation information generated by annotations related to the attributes of the birefringence artifact in the first image containing birefringence information of the object (human living eye), and the machine learning model trained using this training data is configured to receive an OCT image as input and output attribute information.

[0094] The discrimination information is information obtained by processing to discriminate (identify, distinguish) between an image derived from the true structure of a sample (a structure-derived image; an image of an ocular tissue, an image of an ocular site, an image of an intraocular prosthesis, etc.) and a birefringence-derived artifact. The discrimination information is, for example, information about an arbitrary position (an arbitrary pixel, an arbitrary group of pixels, an arbitrary image region, etc.) in an OCT image, and includes information indicating whether the position corresponds to a structure-derived image or a birefringence-derived artifact. In some exemplary embodiments, the discrimination information may be information (discrimination list information, discrimination table information, discrimination map information, etc.) in which an identifier indicating that each pixel in an OCT image of an eye is a pixel of a structure-derived image or a pixel of an image of a birefringence-derived artifact is assigned to each pixel. The identifier indicating that the pixel is a structure-derived image may include an identifier indicating that the pixel is an image of an ocular tissue (site) and an identifier indicating that the pixel is an image of an artificial object implanted in the eye. Furthermore, the identifier indicating that the image is of an ocular tissue (site) may include an identifier for each tissue (site). Furthermore, the identifier indicating that the image is an artifact may include an identifier for each type of artifact. Furthermore, an identifier different from both the identifier indicating that the pixel is a structure-related image pixel and the identifier indicating that the pixel is an image of a birefringence artifact may be used. Examples include an identifier indicating a pixel for which discrimination failed, and an identifier indicating the quality (certainty, reliability, accuracy, etc.) of the discrimination result. When a machine learning model is used, the training data used to build the machine learning model includes annotation information generated by annotations related to discrimination between a structure-related image and a birefringence artifact in a first image containing birefringence information of an object (living human eye), and the machine learning model trained using this training data is configured to receive an OCT image as input and output discrimination information.

[0095] The image segmentation information includes information obtained by image segmentation of the OCT image. Image segmentation is a process of dividing an image into multiple segments (multiple regions, multiple pixel groups). Any technique may be used for image segmentation in this embodiment, such as semantic segmentation, instance segmentation, or panoptic segmentation. Furthermore, any known segmentation method may be used or combined, such as thresholding, clustering, dual clustering, histograms, edge detection, region-growing, partial differential equations, calculus of variations, graph partitioning, or a watershed algorithm. When a machine learning model is used, the training data used to build the machine learning model includes annotation information generated by annotations related to image segmentation of a first image containing birefringence information of an object (a living human eye). The machine learning model trained using this training data is configured to receive an OCT image as input and output image segmentation information.

[0096] Although various exemplary embodiments of the data processing unit 230 have been described above, the embodiments of the data processing unit 230 are not limited to these exemplary embodiments, and the embodiments of the ophthalmic device 1 including the data processing unit 230 are not limited to embodiments that use these exemplary embodiments.

[0097] <User Interface 240> The user interface 240 includes a display unit 241 and an operation unit 242. The display unit 241 includes the display device 3. The operation unit 242 includes various operation devices and input devices. The user interface 240 may include a device that combines a display function and an operation function, such as a touch panel. It is also possible to construct an embodiment that does not include at least a part of the user interface 240. For example, the display device may be an external device connected to the ophthalmologic apparatus 1.

[0098] <Operation of ophthalmic device> Several examples of the operation of the ophthalmologic apparatus 1 will be described.

[0099] <First operation example> A first example of the operation of the ophthalmologic apparatus 1 will be described with reference to Fig. 6. This example provides an example of the operation of denoising image generation (birefringence-derived artifact reduction) using a machine learning algorithm.

[0100] (S1: Obtaining data on the living human eye using polarization-sensitive OCT) First, training data is prepared by acquiring data from living human eyes using polarization-sensitive OCT. In a typical implementation, data from a large number of living human eyes is collected and stored for building a machine learning model.

[0101] Polarization-sensitive OCT is an OCT modality capable of detecting polarized light and is implemented by an OCT device equipped with a polarization diversity receiver. The polarization diversity receiver may be, for example, one of those described in Patent Documents 1 to 4 and Non-Patent Document 1, but is not limited thereto. In this example, data on a living human eye is acquired using a polarization-sensitive OCT such as the devices described in Patent Documents 1 to 3, but the embodiment is not limited thereto. For example, data on a living human eye may be acquired using an OCT modality that simply incorporates a polarization diversity receiver into a regular OCT device, such as the device described in Non-Patent Document 1.

[0102] (S2: Generate polarization-sensitive and polarization-insensitive images) Next, a polarization-sensitive image and a polarization-insensitive image are generated from the OCT data of the living human eye acquired in step S1. For example, a pair of a polarization-sensitive image and a polarization-insensitive image is generated from one piece of OCT data. That is, one image pair is generated from one piece of OCT data. In a typical embodiment, a large number of image pairs are generated from a large number of OCT data to form an image pair set.

[0103] For a method for generating a polarization-sensitive image, see, for example, (Equation 21) in Patent Document 3. For a method for generating a polarization-insensitive image, see, for example, Equation (5) in Non-Patent Document 1. The polarization-sensitive image is an example of a first image that includes birefringence information of the living human eye. The polarization-insensitive image is an example of a second image that does not include birefringence information of the living human eye.

[0104] (S3: Annotate) Next, the polarization-sensitive image generated in step S2 is annotated by identifying birefringence artifacts in the polarization-sensitive image, generating annotation information indicating the identified birefringence artifacts, and attaching the generated annotation information to the polarization-sensitive image.

[0105] (S4: Create training data) Next, training data is created that includes the image pairs generated in step S2 and the annotation information attached to the polarization-sensitive images in step S3. In other words, training data is created that includes a set of pairs (image pair set) of polarization-sensitive images and polarization-insensitive images with annotation information attached in step S3.

[0106] The annotation information may be added to the polarization-sensitive images in all image pairs, or may be added to the polarization-sensitive images in only some image pairs.

[0107] The types of images included in the training data are not limited to polarization-sensitive and polarization-insensitive images. For example, as described above, the training data may include, in addition to the set of image pairs, eye images acquired by a modality other than OCT, images created by processing eye images, images generated by computer graphics, images generated by data augmentation, and simulated images.

[0108] (S5: Build a machine learning model) Next, a machine learning model is constructed by applying machine learning using the training data acquired in step S4 to a neural network. This machine learning model receives an OCT image of the eye as input and outputs a denoised image in which birefringence-induced artifacts in the OCT image have been reduced.

[0109] (S6: Provide the machine learning model to the ophthalmic device) Next, the machine learning model acquired in step S5 is provided to the ophthalmologic apparatus 1. This machine learning model is used as the inference model 2311 in FIG.

[0110] The above steps S1 to S6 are an example of preparations for causing the ophthalmic apparatus 1 to generate a denoised image. An example of generating a denoised image by the ophthalmic apparatus 1 will be described below. It is assumed that scan preparation operations such as input of a patient ID, alignment, and focus adjustment have already been performed.

[0111] Furthermore, an operation mode of the ophthalmic apparatus 1 for generating a denoised image may be specified. In this operation mode, the ophthalmic apparatus 1 applies, for example, an OCT scan to the subject's eye E for generating a normal OCT intensity image.

[0112] (S7: Acquire an OCT image of the test eye) The ophthalmologic apparatus 1 applies an OCT scan to the subject's eye E to construct an OCT image (a normal OCT intensity image).

[0113] (S8: Inputting OCT images into a machine learning model) The ophthalmologic apparatus 1 inputs the OCT image of the subject's eye E acquired in step S7 into the inference model 2311.

[0114] (S9: Generate a denoised image) The inference model 2311 generates a denoised image from the OCT image input in step S8.

[0115] (S10: Store / provide denoised image) The ophthalmologic apparatus 1 stores or provides the denoised image acquired in step S9.

[0116] Storing the denoised image may be, for example, storing the denoised image in a storage device located inside the ophthalmic device 1, and / or storing the denoised image in a storage device located outside the ophthalmic device 1.

[0117] Providing the denoised image may be, for example, one or more of providing the denoised image to a computer provided inside the ophthalmic device 1, providing the denoised image to a computer provided outside the ophthalmic device 1, providing the denoised image to a display device provided inside the ophthalmic device 1, and providing the denoised image to a display device provided outside the ophthalmic device 1.

[0118] This is the end of the first operation example (end).

[0119] <Second operation example> A second example of the operation of the ophthalmologic apparatus 1 will be described with reference to Fig. 7. Like the first example of operation, this example of operation provides an example of denoising image generation (birefringence-derived artifact reduction) using a machine learning algorithm, but differs from the first example of operation in that annotation is not performed. Unless otherwise specified, detailed descriptions of steps and processes similar to those in the first example of operation will be omitted.

[0120] First, data on a living human eye is acquired using polarization-sensitive OCT (S11), and a polarization-sensitive image and a polarization-insensitive image (image pair) are generated from the acquired OCT data of the living human eye (S12). Next, in this operational example, training data is created (S13) without annotation as in step S3 of the first operational example. The training data of this operational example includes multiple image pairs (image pair set) acquired for multiple living human eyes in step S12. As with the first operational example, the types of images included in the training data of this operational example are not limited to polarization-sensitive images and polarization-insensitive images.

[0121] Next, a machine learning model is constructed by applying machine learning using the training data acquired in step S13 to a neural network (S14), and this machine learning model is provided to the ophthalmic apparatus 1 (S15). This machine learning model functions to receive an input of an OCT image of the eye and output a denoised image in which birefringence-induced artifacts in the OCT image are reduced, and is used as the inference model 2311 in FIG.

[0122] The above steps S11 to S15 are an example of preparations for causing the ophthalmic apparatus 1 to generate a denoised image. An example of denoised image generation by the ophthalmic apparatus 1 will be described below. It is assumed that scan preparation operations such as input of a patient ID, alignment, and focus adjustment have already been performed. In addition, an operation mode of the ophthalmic apparatus 1 for generating a denoised image is specified.

[0123] The ophthalmic apparatus 1 applies an OCT scan to the subject's eye E to construct an OCT image (S16), and inputs the acquired OCT image to the inference model 2311 (S17). The inference model 2311 generates a denoised image from the input OCT image (S18). Then, the ophthalmic apparatus 1 stores or provides the generated denoised image (S19).

[0124] <Third operation example> A third example of the operation of the ophthalmologic apparatus 1 will be described with reference to Fig. 8. Like the first and second operation examples, this operation example provides an example of denoising image generation (birefringence-derived artifact reduction) using a machine learning algorithm, but differs from the first operation example in that annotation is not performed and in the method of acquiring images included in the training data. Unless otherwise specified, detailed explanations of steps and processes similar to those in the first operation example will be omitted.

[0125] First, data of a living human eye is acquired using polarization-sensitive OCT (S21), and a polarization-sensitive image is generated from the acquired OCT data of the living human eye (S22). In the first and second operation examples, both polarization-sensitive and polarization-insensitive images are generated from the OCT data acquired using the polarization-sensitive OCT, but in this embodiment, only a polarization-sensitive image is generated from the OCT data acquired using the polarization-sensitive OCT.

[0126] Furthermore, data of a living human eye is acquired using polarization-insensitive OCT (S23), and an image (polarization-insensitive image) is generated from this OCT data (S24). Polarization-insensitive OCT is an OCT modality that is implemented by an OCT device that does not have a polarization diversity receiver.

[0127] The order of steps S21 to S24 is not limited to this. For example, steps S23 and S24 may be performed before steps S21 and S22. Furthermore, steps S22 and S24 may be performed after steps S21 and S23.

[0128] The polarization-sensitive OCT of step S21 and the polarization-insensitive OCT of step S23 are applied to the same living human eye, thereby obtaining a pair of polarization-sensitive and polarization-insensitive images (image pairs) of the living human eye. In a typical embodiment, the polarization-sensitive OCT of step S21 and the polarization-insensitive OCT of step S23 are applied to a number of living human eyes, and a number of image pairs are collected.

[0129] Next, training data is created (S25). The training data in this operational example includes a plurality of image pairs (a set of image pairs) acquired for a plurality of living human eyes in steps S21 to S24. As in the first operational example, the types of images included in the training data in this operational example are not limited to polarization-sensitive images and polarization-insensitive images.

[0130] Next, a machine learning model is constructed by applying machine learning using the training data acquired in step S25 to a neural network (S26), and this machine learning model is provided to the ophthalmic apparatus 1 (S27). This machine learning model functions to receive an input of an OCT image of the eye and output a denoised image in which birefringence-induced artifacts in the OCT image have been reduced, and is used as the inference model 2311 in FIG.

[0131] The above steps S21 to S27 are an example of preparations for causing the ophthalmic apparatus 1 to generate a denoised image. An example of denoised image generation by the ophthalmic apparatus 1 will be described below. It is assumed that scan preparation operations such as input of a patient ID, alignment, and focus adjustment have already been performed. In addition, an operation mode of the ophthalmic apparatus 1 for generating a denoised image is specified.

[0132] The ophthalmic apparatus 1 applies an OCT scan to the subject's eye E to construct an OCT image (S28), and inputs the acquired OCT image to the inference model 2311 (S29). The inference model 2311 generates a denoised image from the input OCT image (S30). The ophthalmic apparatus 1 then stores or provides the generated denoised image (S31).

[0133] <Fourth operation example> A fourth example of the operation of the ophthalmologic apparatus 1 will be described with reference to FIG. 9. Like the first to third operation examples, this operation example provides an example of denoising image generation (birefringence-derived artifact reduction) using a machine learning algorithm, but differs from the first operation example in the method of acquiring images included in the training data. Furthermore, the difference from the third operation example is that annotation is performed. Unless otherwise specified, detailed descriptions of steps and processes similar to those in the first operation example and / or those similar to those in the third operation example will be omitted.

[0134] First, data of a living human eye is acquired using polarization-sensitive OCT (S41), a polarization-sensitive image is generated from the acquired OCT data of the living human eye (S42), and annotation is performed on the generated polarization-sensitive image (S43).

[0135] Furthermore, data of a living human eye is acquired using polarization-insensitive OCT (S44), and an image (polarization-insensitive image) is generated from this OCT data (S45).

[0136] The order of steps S41 to S45 is not limited to this. For example, steps S44 and S45 may be performed before steps S21 to S23. Alternatively, steps S41 and S44 may be performed first, followed by steps S42 and S45, and then step S43.

[0137] The polarization-sensitive OCT of step S41 and the polarization-insensitive OCT of step S44 are applied to the same living human eye, thereby obtaining a pair of polarization-sensitive and polarization-insensitive images (image pairs) of the living human eye. In a typical embodiment, the polarization-sensitive OCT of step S41 and the polarization-insensitive OCT of step S44 are applied to multiple living human eyes, and multiple image pairs are collected.

[0138] Next, training data is created (S46). The training data in this operational example includes a plurality of image pairs (image pair set) acquired for a plurality of living human eyes in steps S41, S42, S44, and S45, and annotation information attached to the polarization-sensitive images in step S43. In other words, the training data in this operational example includes a set of pairs (image pair set) of polarization-sensitive images and polarization-insensitive images attached with annotation information in step S43. As with the first operational example, the types of images included in the training data in this operational example are not limited to polarization-sensitive images and polarization-insensitive images.

[0139] Next, a machine learning model is constructed by applying machine learning using the training data acquired in step S46 to a neural network (S47), and this machine learning model is provided to the ophthalmic apparatus 1 (S48). This machine learning model receives an input of an OCT image of the eye and functions to output a denoised image in which birefringence-induced artifacts in the OCT image are reduced, and is used as the inference model 2311 in FIG.

[0140] The above steps S41 to S48 are an example of preparations for causing the ophthalmic apparatus 1 to generate a denoised image. An example of denoised image generation by the ophthalmic apparatus 1 will be described below. It is assumed that scan preparation operations such as input of a patient ID, alignment, and focus adjustment have already been performed. In addition, an operation mode of the ophthalmic apparatus 1 for generating a denoised image is specified.

[0141] The ophthalmic apparatus 1 applies an OCT scan to the subject's eye E to construct an OCT image (S49), and inputs the acquired OCT image to the inference model 2311 (S50). The inference model 2311 generates a denoised image from the input OCT image (S51). The ophthalmic apparatus 1 then stores or provides the generated denoised image (S52).

[0142] <Effects> Some advantages of this embodiment will be described.

[0143] According to this aspect, first, training data including a set of pairs of a first image including birefringence information of an object (eye) and a second image not including birefringence information of the object is prepared, and a machine learning model (inference model 2311) can be constructed by machine learning using this training data. This machine learning model functions to receive an OCT image as input and output a denoised image in which birefringence-induced artifacts in the OCT image have been reduced. Furthermore, according to this aspect, an OCT image of a sample (subject's eye E) is acquired, and this OCT image is input to the machine learning model (inference model 2311), and the denoised image output from the machine learning model can be stored or provided.

[0144] Thus, this embodiment makes it possible to reduce birefringence-induced artifacts in images acquired with an OCT modality (ophthalmic device 1) that does not have polarization separation detection capabilities by using a novel machine learning model trained to denoise images from OCT images of a sample, in other words, by using a novel machine learning model trained to perform birefringence-induced artifact reduction on OCT images of a sample.

[0145] In other words, this embodiment has a novel feature of employing a machine learning model generated by machine learning using training data including a set of pairs of a first image containing birefringence information of an object and a second image not containing birefringence information of the object, thereby enabling the generation of denoised images with reduced birefringence-related artifacts from images acquired with an OCT modality (ophthalmic device 1) that does not have polarization separation detection functionality.

[0146] The various matters described in this embodiment provide various examples for improving the function of reducing birefringence-induced artifacts.

[0147] <Programs and recording media> A program for causing a computer to execute the OCT image processing method realized by the image processing device (ophthalmologic apparatus 1) according to the above embodiment can be configured. Any of the features described in the above embodiment can be combined with this program.

[0148] It is also possible to create a computer-readable non-transitory recording medium on which such a program is recorded. Any of the features described in the above aspects can be combined with this program. The non-transitory recording medium may be in any form, including, for example, a magnetic disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

[0149] The embodiments and aspects thereof described in this disclosure are merely examples, and those who intend to implement the invention according to this disclosure may make any modifications (omissions, substitutions, additions, etc.) within the scope of the gist of the invention. [Explanation of symbols]

[0150] 1 Ophthalmology equipment 230 Data Processing Unit 231 Denoising Image Generation Unit 2311 Inference Model

Claims

1. 1. A method for processing an optical coherence tomography (OCT) image, comprising: a machine learning model constructed by machine learning using training data including a set of pairs of a first image including birefringence information of an object and a second image not including birefringence information of the object, the machine learning model being constructed to receive an input of an OCT image and output a denoised image in which birefringence-induced artifacts in the OCT image have been reduced; Obtaining an OCT image of the sample; inputting the OCT image of the sample into the machine learning model; storing or providing a denoised image output from the machine learning model based on an input of the OCT image of the sample; method.

2. the first image and the second image are generated based on the same data acquired from the object by an OCT device equipped with a polarization diversity receiver; The first image is accompanied by annotation information indicating birefringence-induced artifacts.

10. The method of claim 1.

3. the first image and the second image are generated based on the same data acquired from the object by an OCT device equipped with a polarization diversity receiver; 10. The method of claim 1.

4. the first image is generated based on first data acquired from the object by a first OCT device having a polarization diversity receiver; the second image is generated based on second data acquired from the object by a second OCT device that does not include a polarization diversity receiver; 10. The method of claim 1.

5. The first image is accompanied by annotation information indicating birefringence-induced artifacts.

10. The method of claim 1.

6. 1. An apparatus for processing optical coherence tomography (OCT) images, comprising: an image acquisition unit that acquires an OCT image of the sample; a processing unit that executes processing using a machine learning model constructed by machine learning using training data including a set of pairs of a first image including birefringence information of an object and a second image not including birefringence information of the object, the machine learning model being constructed so as to receive an input of an OCT image and output a denoised image in which birefringence-derived artifacts in the OCT image have been reduced; Including, The processing unit inputs the OCT image of the sample acquired by the image acquisition unit into the machine learning model, and stores or provides a denoised image output from the machine learning model based on the input of the OCT image of the sample. Device.

7. 1. A program for processing optical coherence tomography (OCT) images, comprising: On the computer, acquiring an OCT image of the sample; inputting an OCT image of the sample into a machine learning model constructed by machine learning using training data including a set of pairs of a first image including birefringence information of an object and a second image not including birefringence information of the object, the machine learning model being constructed to receive an OCT image and output a denoised image in which birefringence-induced artifacts in the OCT image are reduced; storing or providing a denoised image output from the machine learning model based on an input of the OCT image of the sample; A program that executes.

8. A computer-readable non-transitory recording medium having a program recorded thereon for processing optical coherence tomography (OCT) images, The program is configured to: acquiring an OCT image of the sample; inputting an OCT image of the sample into a machine learning model constructed by machine learning using training data including a set of pairs of a first image including birefringence information of an object and a second image not including birefringence information of the object, the machine learning model being constructed to receive an OCT image and output a denoised image in which birefringence-induced artifacts in the OCT image are reduced; storing or providing a denoised image output from the machine learning model based on an input of the OCT image of the sample; A recording medium for executing the above.

Images