Image processing device, image processing method and program

Abstract

To provide an image processing device which can three-dimensionally obtain a phase differential image of a sample in a non-destructive manner.SOLUTION: An image processing device includes a processing unit which generates a phase differential image of a sample on the basis of at least two front OCT images obtained by imaging the sample with OCT.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] BACKGROUND ART Optical coherence tomography (OCT) apparatuses (hereinafter, for convenience of explanation, also referred to as OCT apparatuses) that perform optical coherence tomography (OCT) are known. In the OCT device, an OCT image (hereinafter, for convenience of explanation, also referred to as an OCT image) is acquired. Also known is an optical coherence microscope (OCM) that uses OCT.

[0003] OCT is a technology that uses the coherence of light to obtain cross-sectional images of samples (mainly living organisms). OCT can obtain images that show not only the surface of a sample but also its internal structure with high spatial resolution. Traditionally, OCT has been put to practical use in retinal diagnosis in ophthalmology. It is also used to visualize and quantify the characteristics of tissue structures and microfiber structures (e.g., orientation, size statistics) below optical resolution using cultured tissues, in vitro (in a test tube), or ex vivo (outside the living body) samples.

[0004] When a sample is imaged from multiple directions using OCT, images are obtained with different signal intensity patterns for each imaging direction, which are thought to be due to the microstructure of the tissue that makes up the sample.

[0005] Incidentally, various studies have been conducted on OCT (see, for example, Patent Documents 1 and 2, and Non-Patent Document 1). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication No. 2021 / 095826 [Patent Document 2] International Publication No. 2021 / 095852 [Non-patent literature]

[0007] [Non-Patent Document 1] Oida et al., “Computational multi-directional optical coherence tomography for visualizing the microstructural directionality of the tissue”, Biomedical Opt. Express Vol.12, p.3851-3864, July 2021 Summary of the Invention [Problem to be solved by the invention]

[0008] However, in conventional phase imaging using OCM, the probe light is transmitted entirely through the sample, reflected by a mirror placed outside the sample, and then transmitted through the sample again (i.e., double-transmitted light) and the phase of this light is measured. For this reason, it has been difficult to evaluate thick samples, which is the greatest advantage of OCM, and it has also not been possible to perform three-dimensional volumetric phase imaging. For these reasons, phase measurements using conventional OCMs could only achieve roughly the same functionality as regular phase microscopes, and had only a very small advantage over regular phase microscopes.

[0009] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide an image processing device, an image processing method, and a program that can obtain three-dimensional phase differential images of a sample non-destructively. [Means for solving the problem]

[0010] As an example configuration,An image obtained by scattering light from the same region of a sample when the sample is irradiated with light from multiple directions, or a pseudo image equivalent thereto. The image processing device includes a processing unit that generates a phase differential image of the sample based on at least two frontal OCT images.

[0011] As an example configuration, An image obtained by scattering light from the same region of a sample when the sample is irradiated with light from multiple directions, or a pseudo image equivalent thereto. An image processing method for generating a phase differential image of the sample based on at least two frontal OCT images.

[0012] In one configuration example, the computer: An image obtained by scattering light from the same region of a sample when the sample is irradiated with light from multiple directions, or a pseudo image equivalent thereto. A program that generates a phase differential image of the sample based on at least two en face OCT images. [Effects of the Invention]

[0013] According to the image processing device, image processing method, and program of the present invention, a three-dimensional phase differential image of a sample can be obtained non-destructively. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a configuration diagram illustrating an example of an optical coherence tomography apparatus according to an embodiment. [Figure 2] 1 is a block diagram showing an example of the functional configuration of an image processing apparatus according to an embodiment (first embodiment). [Figure 3] FIG. 2 is an explanatory diagram showing an example of image processing according to an embodiment (first embodiment). [Figure 4] 10A to 10I are diagrams showing example images. [Figure 5] FIG. 10 is a block diagram showing an example of the functional configuration of an image processing apparatus according to an embodiment (second embodiment). [Figure 6] FIG. 10 is an explanatory diagram showing an example of image processing according to an embodiment (second embodiment). [Figure 7] 10A to 10O are diagrams showing example images. [Figure 8] FIG. 10 is an explanatory diagram showing another example of image processing according to an embodiment (second embodiment). [Figure 9] 10A to 10O are diagrams showing example images. [Figure 10] FIG. 1 is a configuration diagram illustrating an example of an optical system of a differential interference microscope. [Figure 11] 10(A) and 10(B) are diagrams showing examples of shear amount and phase difference. [Figure 12] FIG. 10 is a diagram illustrating an example of a defocus amount. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] In the following embodiments, phase measurement is achieved using scattered light from a sample (e.g., a living body). In the following embodiments, the sample is a scattering medium. Scattering occurs at any depth inside the sample. Furthermore, by utilizing the low-coherence interference characteristics of OCT (or OCM), it is possible to extract only scattered light from a specific depth. This enables phase imaging at any depth in the sample, and by repeating the process in the depth direction, three-dimensional microscopic phase imaging can be achieved.

[0017] Here, in this embodiment, a reflection-type OCT is used. The complex signal of OCT contains information on scattering intensity and phase. The phase of the raw OCT data is random, and for example, the phase includes components due to scattered light over a depth direction of several mm. In general, the phase is determined by the depth at which the scatterer exists (a minute displacement on the order of wavelength) and does not contain meaningful information about the sample. Due to the randomness of the scattering distribution, meaningful three-dimensional phase measurement in the scattering mode is not possible. In an embodiment, meaningful phase information is extracted from such OCT signals.

[0018] Specifically, in this embodiment, the refractive index distribution of a sample is acquired and the spatial derivative of the refractive index distribution of the sample is visualized. That is, the refractive index characterizes biological tissues or cells, as well as light scattering properties. Although it is difficult to directly measure the refractive index of biological tissues, the refractive index affects the phase of the OCT signal. Therefore, the refractive index can be evaluated indirectly through the phase of the OCT signal.

[0019] Furthermore, because OCT is a reflective type, it does not require slicing the living body, and OCT can perform non-destructive measurements from the surface of the sample to a depth of about 1 to 2 mm. The phase differential image obtained in the embodiment is similar to an image (DIC image) obtained by a differential interference contrast microscope (DIC). In the embodiment, it is possible to obtain a three-dimensional phase differential image. It should be noted that such a technique can be applied to, for example, all three-dimensional en face surfaces (for convenience of explanation, also referred to as the front surface).

[0020] In the first embodiment, an example will be described in which a phase difference is detected by performing numerical processing using two different masks (processing by a digital filter that performs numerical spatial frequency filtering) in order to realize phase imaging. In the second embodiment, an example will be described in which a phase difference is detected by performing numerical processing using a numerical image shift (a spatial image shift) to realize phase imaging. In the third embodiment, an example will be described in which a phase difference is detected using a physical configuration to achieve phase imaging.

[0021] (First embodiment) A first embodiment will be described.

[0022] [Optical coherence tomography] FIG. 1 is a configuration diagram showing an example of an optical coherence tomography device 1 according to an embodiment. The optical coherence tomography device 1 constitutes an observation system for observing the state of a sample using OCT. The optical coherence tomography device 1 is a device that irradiates a sample Sm with light, acquires the interference light generated by the interference between the scattered light from the sample Sm and the reference light reflected by the reference mirror 40, and generates an image showing the surface and internal state of the sample Sm from the acquired interference light. The object to be observed as the sample Sm may be, for example, a living human or animal body, or a non-living body. Living bodies may include the fundus of the eye, blood vessels, teeth, subcutaneous tissue, etc. Non-living bodies may include artificial structures such as electronic components or machine parts, natural structures such as stone or minerals, or substances without a specific shape.

[0023] The optical coherence tomography device 1 includes a light source 10, a beam splitter 20, collimators 30a, 30b, 50a, and 50b, a reference mirror 40, galvanometer mirrors 60a and 60b, a spectrometer 70, and an image processing device 100. Among these components, the beam splitter 20, the collimators 30a, 30b, 50a, and 50b, the reference mirror 40, the galvanometer mirrors 60a and 60b, and the spectrometer 70 form an optical system called an interferometer. The interferometer illustrated in FIG. 1 is a Michelson interferometer equipped with an optical fiber F. More specifically, the light source 10, the spectrometer 70, the collimators 30a, and the collimators 50a are each connected to the beam splitter 20 via the optical fiber F. The optical fiber F has a transmission band that includes the wavelength band of the light emitted from the light source 10.

[0024] The optical coherence tomography device 1 is a Fourier-domain OCT (FD-OCT). The optical coherence tomography device 1 may employ any of the methods classified as FD-OCT, such as spectral-domain OCT (SD-OCT) or swept-source OCT.

[0025] The light source 10 is a wavelength swept light source such as an ultrashort pulse laser or a superluminescent diode (SLD). The light source 10 irradiates probe light having, for example, a near-infrared wavelength (e.g., 800 to 1000 nm) and low coherence. The light irradiated from the light source 10 is guided inside the optical fiber F and enters the beam splitter 20.

[0026] The beam splitter 20 splits the incident light into light (hereinafter also referred to as reference light) that is guided toward the collimator 30a and light (hereinafter also referred to as measurement light) that is guided toward the collimator 50a. The beam splitter 20 is, for example, a cube beam splitter.

[0027] The collimator 30a converts the reference light guided from the beam splitter 20 into parallel light, and emits the parallel light toward the collimator 30b. The collimator 30b collects the parallel light incident from the collimator 30a and emits the collected reference light toward the reference mirror 40. The collimator 30b receives the reference light reflected by the reference mirror 40, converts it into parallel light, and emits the converted parallel light toward the collimator 30a. The collimator 30a collects the parallel light incident from the collimator 30b and guides it toward the beam splitter 20.

[0028] On the other hand, the collimator 50a converts the measurement light guided from the beam splitter 20 into parallel light and emits the parallel light toward the galvanometer mirror 60a. The parallel light incident from the collimator 50a is reflected by the surfaces of the galvanometer mirrors 60a and 60b, respectively, and emitted toward the collimator 50b. The collimator 50b collects the parallel light incident from the collimator 50a via the galvanometer mirrors 60a and 60b and irradiates the sample Sm with the collected measurement light. The measurement light irradiated to the sample Sm is scattered by a scattering surface of the sample Sm and enters the collimator 50b. The scattering surface is not limited to, for example, the boundary surface between the sample Sm and the surrounding environment (e.g., the atmosphere), but can also be a boundary surface separating materials or tissues with different refractive indices within the sample Sm. Hereinafter, the light scattered by the scattering surface of the sample Sm and incident on the collimator 50b is referred to as scattered light.

[0029] The collimator 50b emits the incident scattered light toward the galvanometer mirror 60b. The scattered light is reflected by the surfaces of the galvanometer mirrors 60b and 60a, respectively, and is emitted toward the collimator 50a. The collimator 50a collects the parallel light incident from the collimator 50b via the galvanometer mirrors 60a and 60b, and guides the collected scattered light toward the beam splitter 20. The beam splitter 20 guides the reference light reflected by the reference mirror 40 and the scattered light scattered by the sample Sm to the spectroscope 70 via an optical fiber F.

[0030] The spectrometer 70 includes a diffraction grating and a light receiving element therein. The diffraction grating separates the reference light and scattered light guided from the beam splitter 20. The separated reference light and scattered light interfere with each other to become interference light. The light receiving element is disposed on an imaging surface onto which the interference light is irradiated. The light receiving element detects the irradiated interference light and generates a signal (hereinafter also referred to as a detection signal) based on the detected interference light. The light receiving element outputs the generated detection signal to the image processing device 100.

[0031] The image processing device 100 acquires an OCT signal representing the state of the sample Sm from the detection signal input from the spectrometer 70. The image processing device 100 scans the observation points and sequentially accumulates the detection signals acquired at each observation point to generate an OCT signal within a predetermined observation region. The image processing device 100 converts the generated OCT signal (in this embodiment, a complex signal or its phase component) into the spatial frequency domain and estimates the electric field of scattered light at a predetermined plane corresponding to the reciprocal space of the sample Sm as the predetermined plane electric field. The image processing device 100 applies a mask indicating the transmission characteristic distribution of the electric field at the predetermined plane to electric field data indicating the estimated electric field to obtain mask electric field data. The image processing device 100 converts the obtained mask electric field data into the spatial domain to generate a mask OCT signal. The image processing device 100 generates a mask OCT image based on the generated mask OCT signal. In this embodiment, for the sake of convenience, the predetermined plane is a pupil plane, and in this case, the electric field on the predetermined plane is a pupil plane electric field.

[0032] The image processing device 100 may apply M masks (in this embodiment, M is 2) with different pass characteristic distributions to the electric field data to generate M mask electric field data, and convert the generated M mask electric field data into the spatial domain to generate M mask images.

[0033] Next, an example of the functional configuration of the image processing device 100 according to this embodiment will be described. FIG. 2 is a block diagram showing an example of the functional configuration of the image processing device 100 according to an embodiment (first embodiment). The image processing device 100 includes a control unit 110 and a storage unit 190. Some or all of the functions of the control unit 110 are realized as a computer including, for example, a processor such as a CPU (Central Processing Unit). The processor reads a program stored in advance in the storage unit 190 and performs processing instructed by commands written in the read program to realize its functions. In this embodiment, performing processing instructed by commands written in the program may be referred to as executing a program, program execution, etc. Some or all of the control unit 110 is not limited to general-purpose hardware such as a processor, and may be configured to include dedicated hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0034] The control unit 110 includes an optical system control unit 120 , a detection signal acquisition unit 130 , an electric field estimation unit 140 , a mask unit 150 , a conversion unit 160 , an image synthesis unit 170 , and an output processing unit 180 . The control unit 110 may be regarded as a processing unit that executes various processes.

[0035] The optical system control unit 120 drives a driving mechanism that changes the positions of the galvanometer mirrors 60a and 60b, and scans the observation point on the sample Sm. The observation point on the sample Sm is scanned in a direction intersecting the depth direction of the sample Sm (for example, a direction parallel to the front surface of the sample Sm).

[0036] The detection signal acquisition unit 130 sequentially acquires detection signals from the spectrometer 70. Based on the acquired detection signals, the detection signal acquisition unit 130 acquires signal values indicating the intensity distribution of scattered light in the depth direction of the sample Sm for each sample point arranged at a predetermined interval. The depth direction of the sample Sm corresponds to the incident direction of the measurement light. The detection signal acquisition unit 130 repeats the process of acquiring the intensity distribution of scattered light for each observation point changed by scanning. The acquired intensity distribution of scattered light is based on the distribution of the refractive index of the sample Sm in the depth direction. This allows the detection signal acquisition unit 130 to acquire data indicating the state of the sample Sm within an observable three-dimensional region (hereinafter referred to as the observable region) as a three-dimensional OCT signal. The detection signal acquisition unit 130 stores the three-dimensional OCT signal in the memory unit 190.

[0037] The detection signal acquisition unit 130 extracts, from the acquired three-dimensional OCT signal, a portion that represents the intensity distribution of scattered light in a two-dimensional plane to be observed (hereinafter also referred to as the observation target plane) as a two-dimensional OCT signal (hereinafter sometimes simply referred to as the OCT signal). The observation target plane is, for example, the front surface of the sample Sm. The front surface is a plane perpendicular to the depth direction of the sample Sm, and is also called the en face surface. The detection signal acquisition unit 130 stores the extracted OCT signal in the memory unit 190.

[0038] The detection signal acquiring unit 130 may select, for example, a two-dimensional plane of the depth indicated by the control signal input from the output processing unit 180 as the plane to be observed. Furthermore, it is desirable that the interval between adjacent sample points in the observable area be equal to or less than the spatial resolution of the optical system.

[0039] The detection signal acquisition unit 130 includes a focus adjustment unit 131 and a phase stabilization unit 132 . The focus adjustment unit 131 has a function of adjusting the focus of the OCT signal by digital processing. For example, the focus adjustment unit 131 has a function of refocusing the OCT signal and a function of defocusing the OCT signal. For example, after capturing an image, the focus adjustment unit 131 performs defocusing by performing digital processing to generate an OCT image in which the focus of the captured OCT image (captured OCT image) is pseudo-shifted.

[0040] The phase stabilization unit 132 has a function of performing phase stabilization processing on the OCT signal by digital processing. Note that if a configuration that does not perform phase stabilization processing is adopted, the phase stabilization unit 132 may not be provided. In this embodiment, the phase stabilization process may be, for example, a process for compensating for bulk phase errors in a three-dimensional image (see, for example, Patent Document 2).

[0041] In this embodiment, the detection signal acquisition section 130 performs the following processes on the complex OCT signal or the signal from which the phase component is extracted. Therefore, in the following processing, the OCT signal represents a complex OCT signal or a signal obtained by extracting its phase component. Here, extracting the phase component of the complex OCT signal means setting the amplitude to 1 while leaving the phase of the complex OCT signal unchanged.

[0042] The electric field estimation unit 140 reads out the OCT signals stored in the storage unit 190 and performs a two-dimensional Fourier transform on the read spatial domain OCT signals to generate spatial frequency domain data. The spatial frequency domain data obtained by the transform corresponds to data (hereinafter, electric field data) indicating the electric field at the pupil plane (hereinafter, pupil plane electric field). In other words, by performing a two-dimensional Fourier transform on the OCT signals related to the observation plane, the electric field of scattered light virtually projected onto the pupil plane is estimated. The electric field estimation unit 140 stores the generated spatial frequency domain data in the storage unit 190 as electric field data. Strictly speaking, such an electric field does not exist, but is a virtual electric field pattern corresponding to the image pattern.

[0043] Here, FIG. 3 is an explanatory diagram showing an example of image processing according to an embodiment (first embodiment). As illustrated in FIG. 3, the pupil plane Pp corresponds to the back-focal plane corresponding to the reciprocal space of the sample Sm. In other words, the pupil plane Pp is a plane located on the opposite side of the sample Sm from the objective lens Ol, and is a plane onto which scattered light diffusing from the sample Sm and collimated by the objective lens Ol is virtually projected. The entire bandwidth of spatial frequencies converted into the spatial frequency domain corresponds to the observation region on the pupil plane Pp where the electric field is to be observed. The intersection point between the optical axis of the objective lens Ol and the pupil plane Pp corresponds to the origin of the spatial frequency. The spatial frequency in the spatial frequency domain corresponds to a sample point within the observation region. The sample Sm is placed at a position where it intersects with the optical axis of the objective lens Ol.

[0044] Returning to FIG. 2 , the masking unit 150 reads the electric field data stored in the memory unit 190 and applies a mask to the read electric field data to generate masked electric field data. The mask is numerical data indicating the distribution of the pass characteristics of the electric field of scattered light in the spatial frequency domain. The mask indicates the pass characteristics of the electric field for each frequency sample point (frequency bin) corresponding to each spatial frequency as a mask value. For example, the mask value for each frequency sample point takes one of two values, 1 or 0, where 1 and 0 indicate whether or not the electric field passes. Therefore, the distribution of frequency sample points with a mask value of 1 indicates a pass band in the spatial frequency domain through which the electric field passes in the reciprocal space of the sample Sm. From another perspective, the pass band indicates a virtual aperture on the pupil plane through which scattered light passes. On the other hand, the distribution of frequency sample points with a mask value of 0 indicates a stop band in the spatial frequency domain through which the electric field does not pass in the reciprocal space of the sample Sm. In another sense, the stopband represents an imaginary shield on the pupil plane that does not allow scattered light to pass through.

[0045] When applying a mask, the mask unit 150 multiplies the electric field value for each frequency sample point indicated by the electric field data by the mask value of that frequency sample. The mask unit 150 generates mask electric field data, which contains the multiplied value obtained by the multiplication as the mask electric field value for each frequency sample point. The mask electric field data is used to generate an OCT image that reflects spatial characteristics (e.g., directivity) corresponding to the transmission characteristic distribution. The mask unit 150 stores the generated mask electric field data in the memory unit 190.

[0046] Here, if the shape and arrangement of the mask changes, the differentiation direction for the OCT signal changes. The mask may have any shape, for example, a semicircular shape or a crescent shape. The mask allows the intensity of the OCT signal at oblique incidence to be observed, and the phase is reflected in this intensity.

[0047] The transform unit 160 reads out the mask electric field data stored in the storage unit 190, performs a two-dimensional inverse Fourier transform on the read out mask electric field data in the spatial frequency domain, and generates a mask OCT signal in the spatial domain. The transform unit 160 stores the generated mask OCT signal in the storage unit 190. When M (M is an integer of 2 or more) masks with different electric field transmission characteristic distributions in the spatial frequency domain are set, the masking unit 150 may apply each of the M masks to the read electric field data to generate M masked electric field data. In this case, the conversion unit 160 converts each of the M masked electric field data in the spatial frequency domain into a masked OCT signal in the spatial domain.

[0048] The image synthesis unit 170 reads out the mask OCT signal stored in the storage unit 190 and converts the signal value of each sample point on the observation target plane indicated by the read mask OCT signal into a luminance value for each pixel using a predetermined conversion function. The converted luminance value is within a range that can be expressed by the bit depth of each pixel. The image synthesis unit 170 generates mask image data having the converted luminance values for each sample point. The generated mask image data represents a mask image that reflects spatial characteristics corresponding to the pass characteristic distribution.

[0049] In this embodiment, the image synthesis unit 170 calculates the result of subtraction of two mask image data obtained using two different masks. Here, the image synthesis unit 170 acquires image data obtained by subtracting one mask image data from the other mask image data. Furthermore, the image synthesis unit 170 may also acquire image data obtained by subtracting one mask image data from the other mask image data. The image synthesis unit 170 outputs the generated image data (image data resulting from the subtraction) to a display unit (not shown) as output image data in response to a control signal input from the output processing unit 180.

[0050] In addition, for each mask used to mask the pupil plane electric field, the image synthesis unit 170 may output the generated mask image data to a display unit (not shown) as output image data in response to a control signal input from the output processing unit 180. Furthermore, the image synthesis unit 170 may store the output image data in the storage unit 190 in response to a control signal input from the output processing unit 180 .

[0051] The output processing unit 180 controls the generation or output of output image data representing an OCT image based on an operation signal input from an operation input unit (not shown). The operation input unit may be configured to include components that accept user operations and generate operation signals in response to the accepted operations, such as buttons, knobs, dials, a mouse, a joystick, etc. The operation input unit may be an input interface that receives operation signals wirelessly or via a wired connection from another device (for example, a portable device such as a remote controller).

[0052] The operation signal specifies, as parameters, whether or not the OCT image is to be displayed or stored, the observation plane to be used as the observation region, the spatial frequency characteristics of the mask, etc. The output processing unit 180 may configure a user interface for displaying the OCT image by displaying, on the display unit, parameters that can be set by operation, parameter settings, and a setting screen that guides the settable parameters. For example, when an operation signal indicating whether or not an OCT image needs to be displayed is input, the output processing unit 180 outputs a control signal indicating whether or not the image needs to be displayed to the image synthesis unit 170. When the control signal indicating that the image needs to be displayed is input from the output processing unit 180, the image synthesis unit 170 outputs the output image data to the display unit, and when the control signal indicating that the image needs not to be displayed is input from the output processing unit 180, the image synthesis unit 170 does not output the output image data to the display unit.

[0053] When an operation signal indicating a plane to be observed is input, the output processing unit 180 outputs a control signal indicating the plane to be observed to the detection signal acquiring unit 130. The detection signal acquiring unit 130 outputs, as an OCT signal, a portion of the three-dimensional OCT signal that relates to the plane to be observed indicated by the control signal input from the output processing unit 180. The plane to be observed is defined using parameters such as the depth from the surface of the sample Sm, the observation direction, and the area of the observation region. When an operation signal indicating the spatial frequency characteristics of a mask is input, output processing unit 180 outputs a control signal indicating the spatial frequency characteristics to mask unit 150. Output processing unit 180 may set the spatial frequency characteristics of each of a plurality of masks based on the input operation signal, and output a control signal indicating the set spatial frequency characteristics to mask unit 150. Mask unit 150 sets a mask having the spatial frequency characteristics indicated by the control signal input from output processing unit 180.

[0054] Furthermore, when an operation signal indicating a defocus amount is input, output processing unit 180 outputs a control signal indicating the defocus amount to focus adjustment unit 131. Focus adjustment unit 131 sets the defocus amount indicated by the control signal input from output processing unit 180.

[0055] The storage unit 190 stores the above programs as well as various data used in the processes executed by the control unit 110 and various data acquired by the control unit 110. The storage unit 190 includes a non-volatile (non-transitory) storage medium such as a ROM (Read Only Memory), a flash memory, or an HDD (Hard Disk Drive). The storage unit 190 includes a volatile storage medium such as a RAM (Random Access Memory) or a register.

[0056] [Example of image processing] Next, an example of image processing according to this embodiment will be described. FIG. 3 is an explanatory diagram showing an example of image processing according to an embodiment (first embodiment). The image processing illustrated in Fig. 3 is an example of application of OCT images to directional imaging. In directional imaging, a mask having directionality in the spatial frequency domain (hereinafter also referred to as a directional mask) is used. The directional mask has a mask value that depends on a spatial frequency corresponding to the azimuth from the origin in the two-dimensional spatial frequency domain (hereinafter also referred to as an azimuthal spatial frequency).

[0057] (Step S01) The electric field estimation unit 140 performs a two-dimensional Fourier transform on the OCT signal representing the spatial-domain front OCT image Oi01 acquired by the detection signal acquisition unit 130, to generate pupil plane electric field data in the spatial frequency domain representing the pupil plane electric field Pe01. The front OCT image Oi01 is an OCT image in which the observation plane is the front.

[0058] (Step S02) The masking unit 150 applies a directional mask Dm01, as an example of a virtual mask, to the pupil plane electric field data generated by the electric field estimating unit 140 to generate mask electric field data indicating the mask electric field Ie01. One of the two directional masks, directional mask Dm01, has a spatial frequency (hereinafter also referred to as radial spatial frequency) corresponding to the distance (radius) from the origin within a predetermined band, and has a pass band that is a half-peripheral band from -π / 2 through 0 to π / 2 as an azimuthal spatial frequency, and has other bands as a stop band.

[0059] The other of the two directional masks has a passband that is a half-periphery band on the opposite side of the one directional mask, and the other bands are stopbands.

[0060] (Step S03) The transform unit 160 performs a two-dimensional inverse Fourier transform on the masked electric field data generated by the mask unit 150 to generate a masked OCT signal in the spatial domain. The image synthesis unit 170 converts the signal value of each sample point indicated by the mask OCT signal into a pixel value for each pixel, and generates mask image data having the pixel values obtained by the conversion.

[0061] In this embodiment, mask image data is generated for each of the two directional masks. Then, the image synthesis unit 170 generates image data that is the difference (subtraction result) between the mask image data generated for each of the two directional masks. The image synthesis unit 170 outputs the generated image data to a display unit (not shown) as output image data.

[0062] As described above, by applying the directional mask Dm01 to the pupil plane electric field Pe01, the pupil plane electric field in a spatial frequency band within a range of π with the azimuthal spatial frequency centered at 0° is passed, and the pupil plane electric field in other bands is blocked. As a result, the resulting mask electric field Ie01 produces an OCT image with directionality whose main direction is the azimuthal spatial frequency of 0.

[0063] Furthermore, by making the spatial frequency characteristics of the mask variable in response to an operation, the user can arbitrarily adjust the directionality of the OCT image using the acquired OCT signal without driving or adjusting the optical system. For example, the output processing unit 180 can set one or both of an azimuthal spatial frequency band and a radial spatial frequency band as a passband for passing the electric field based on an operation signal input from an operation input unit (not shown). The masking unit 150 sets the mask value at frequency sample points within the passband set by the output processing unit 180 to 1 and the mask values at other frequency sample points to 0, and generates a directional mask indicating the set mask value.

[0064] The Fourier transform may be, for example, a fast Fourier transform (FFT), and the inverse Fourier transform may be, for example, an inverse fast Fourier transform (IFFT).

[0065] [Example of an OCT image and an example of a difference image between two directional OCT images] Specific examples of images according to this embodiment are shown with reference to FIGS. 4(A) to 4(I). In this embodiment, the structure of the differential interference contrast image is obtained by using two masks with high-frequency apertures. For example, an image is created by post-capture data processing, as if a mask were placed in front of the objective lens of a differential interference contrast microscope. When a mask with a semicircular opening is applied to a complex OCT signal or an OCT signal (a phase-only signal created from the complex OCT signal), the resulting differential interference image contains only half the information of the OCT signal. Therefore, in this embodiment, the difference between the differential interference images of two masks with openings facing in opposite directions is obtained to obtain an image containing the entire OCT signal. Here, half of the OCT signal information represents either the positive or negative part of the differentiation. Specifically, one mask can only be used when the differentiation result is positive or negative. In a mask when the differentiation result is positive, the value of the negative part is zero (and may contain small amounts of noise). In a mask when the differentiation result is negative, the absolute value of the negative value is obtained for the negative part, and the rest (positive part) is zero (and may contain small amounts of noise).

[0066] Figures 4(A) to 4(I) are diagrams showing example images. Note that the images shown in Figures 4(A) to 4(I) are examples for the purpose of explanation, and are not necessarily shown with strict accuracy. Figure 4(A) shows an OCT image of the head of an 8-day-old zebrafish. The OCT image is an in-focus image. The scan area is 1 mm × 1 mm, and the number of pixels is 512 × 512.

[0067] FIG. 4(B) is a phase interference image obtained by applying a predetermined semicircular mask to the OCT image of FIG. 4(A). FIG. 4(C) is a phase interference image obtained by applying a semicircular mask (symmetrical semicircular mask) opposite to that in FIG. 4(B) to the OCT image in FIG. 4(A). These semicircular masks may be of any cycle (cycles / mm).

[0068] FIG. 4(D) is an image obtained by subtracting the phase interference image of FIG. 4(C) from the phase interference image of FIG. 4(B). FIG. 4(E) is an image obtained by subtracting the phase interference image of FIG. 4(B) from the phase interference image of FIG. 4(C).

[0069] FIG. 4(F) is a phase interference image obtained by applying a predetermined semicircular mask to an OCT image defocused by a defocus amount of 100 μm instead of the OCT image of FIG. 4(A). Figure 4(G) is a phase interference image obtained by applying a semicircular mask (symmetrical semicircular mask) opposite to that shown in Figure 4(F) to an OCT image defocused by 100 μm instead of the OCT image shown in Figure 4(A). Note that a calculated refocus may be applied before the calculated defocus is performed.

[0070] FIG. 4(H) is an image obtained by subtracting the phase interference image of FIG. 4(G) from the phase interference image of FIG. 4(F). FIG. 4(I) is an image obtained by subtracting the phase interference image of FIG. 4(F) from the phase interference image of FIG. 4(G).

[0071] Here, the image after subtraction was almost the same as the image before subtraction. The differential direction of the image obtained from the undefocused signal was opposite to the differential direction of the image obtained from the defocused signal. Intensity corresponds to phase. For example, a local average of the intensities may be performed on the difference image. In the examples of FIGS. 4(F) to 4(I), the structure appears strongly in the differential image due to defocusing. On the other hand, in the examples of FIGS. 4(B) to 4(E), the structure appears weakly because the measurement itself is defocused.

[0072] As shown in Figure 4, differential interference contrast images were calculated by changing the defocus amount, defocus direction, and mask direction. The amount of defocusing was estimated to be related to the amount of shear. The defocus direction and mask direction were assumed to be related to the contrast direction.

[0073] [Differential interference microscope, shear amount, defocus amount] Here, a differential interference microscope, the shear amount, and the defocus amount will be explained. FIG. 10 is a diagram showing an example of the optical system of the differential interference microscope 300. As shown in FIG. The differential interference microscope 300 includes, as its optical system, an analyzer 310, a DIC prism 320, a condenser lens 330, an objective lens 340, a DIC prism 350, and an analyzer 360, in that order from the side closest to the light source to the side furthest from the light source. Furthermore, a sample 410 is placed between the condenser lens 330 and the objective lens 340 .

[0074] Differential interference contrast (DIC) is a technique that measures the difference in optical path length caused by the refractive index and thickness of the sample 410. The front-stage DIC prism 320 splits the light into two beams, and each beam is directed to a different part of the sample 410. The rear-stage DIC prism 350 combines the two beams. If there is a phase difference between the two beams, they will interfere with each other, and contrast will appear where the phase of the sample 410 changes.

[0075] 11(A) and 11(B) are diagrams showing examples of the shear amount and the phase difference. Each of FIGS. 11(A) and 11(B) shows a sample 410, a shear amount, and a phase difference. In the example of FIGS. 11(A) and 11(B), the surface of the sample 410 is inclined with respect to the direction of light irradiation. Compared with the example of FIG. 11(A), the example of FIG. 11(B) has a larger shear amount and a larger phase difference.

[0076] FIG. 12 is a diagram showing an example of the defocus amount. FIG. 12 shows the objective lens 340, the sample 410, and the focal plane 420 when in focus. FIG. 12 also shows a schematic representation of the surface position when the defocus amount is 100 μm, the surface position when the defocus amount is 150 μm, and the surface position when the defocus amount is −100 μm.

[0077] For example, extracting the phase component from an en face OCT image and applying spatial differentiation can produce an image similar to a DIC image. The shear amount of a DIC image can also be changed by applying a calculated defocus.

[0078] [Regarding the first embodiment] As described above, the image processing device 100 according to this embodiment can nondestructively and noninvasively acquire a phase differential image (e.g., an image equivalent to a differential interference microscope image) at any depth of tissue using an OCT signal. The image processing device 100 according to this embodiment can nondestructively acquire a three-dimensional phase differential image of a sample. In this embodiment, after capturing an OCT image, a spatial filter (differential filter), which is a digital filter, is used to obtain the difference (phase differential image) of the OCT image as if it had been irradiated from multiple directions, without actually controlling the direction of light irradiation.

[0079] In this embodiment, the contrast of the phase differential image can be improved (clarified) by defocusing. Note that the defocusing may be achieved physically in advance during imaging, or may be achieved by digital processing after imaging, for example.

[0080] In this embodiment, a case has been described in which a differential phase image is obtained by subtracting two images, but any number of images may be used to obtain a differential phase image.

[0081] (Second embodiment) A second embodiment will be described.

[0082] [Optical coherence tomography] In the second embodiment, a case will be described in which the optical coherence tomography device 1 shown in FIG. 1 according to the first embodiment is used. However, in this embodiment, an image processing device 600 shown in FIG. 5 is used instead of the image processing device 100 shown in FIG. In this embodiment, for convenience of explanation, the same reference numerals as those used for the respective units (except for the image processing device 100) shown in FIG. 1 will be used.

[0083] The image processing device 600 acquires an OCT signal representing the state of the sample Sm from the detection signal input from the spectrometer 70. The image processing device 600 scans the observation points and sequentially accumulates the detection signals acquired at each observation point to generate an OCT signal within a predetermined observation region. The image processing device 600 multiplies (complex multiplication) the complex conjugate of the generated OCT signal by an OCT signal obtained by spatially shifting the OCT signal, and extracts phase information from the result of the multiplication. Note that local phase averaging may be performed before extracting the phase information.

[0084] Next, an example of the functional configuration of the image processing device 600 according to this embodiment will be described. FIG. 5 is a block diagram showing an example of the functional configuration of an image processing device 600 according to an embodiment (second embodiment). The image processing device 600 includes a control unit 610 and a storage unit 690. Some or all of the functions of the control unit 610 are realized as a computer including, for example, a processor such as a CPU. The processor reads a program stored in advance in the storage unit 690 and performs processing instructed by instructions written in the read program to realize its functions. In this embodiment, performing processing instructed by instructions written in the program may be referred to as "executing a program" or "running a program." Some or all of the control unit 610 is not limited to general-purpose hardware such as a processor, and may also be configured to include dedicated hardware such as an LSI or ASIC.

[0085] The control unit 610 includes an optical system control unit 620 , a detection signal acquisition unit 630 , a shift unit 640 , a complex conjugate unit 650 , a multiplication unit 660 , a phase extraction unit 670 , and an output processing unit 680 . The control unit 610 may be regarded as a processing unit that executes various processes. Here, the processing in the shift unit 640, the complex conjugate unit 650, the multiplication unit 660, and the phase extraction unit 670 is, for example, digital processing.

[0086] The optical system control unit 620 has, for example, the same functions as the optical system control unit 120 shown in FIG.

[0087] The detection signal acquisition unit 630 sequentially acquires detection signals from the spectrometer 70. Based on the acquired detection signals, the detection signal acquisition unit 630 acquires signal values indicating the intensity distribution of scattered light in the depth direction of the sample Sm for each sample point arranged at a predetermined interval. The depth direction of the sample Sm corresponds to the direction of incidence of the measurement light. The detection signal acquisition unit 630 repeats the process of acquiring the intensity distribution of scattered light for each observation point changed by scanning. The acquired intensity distribution of scattered light is based on the distribution of the refractive index of the sample Sm in the depth direction. This allows the detection signal acquisition unit 630 to acquire data indicating the state of the sample Sm within an observable three-dimensional region (hereinafter referred to as the observable region) as a three-dimensional OCT signal. The detection signal acquisition unit 630 stores the three-dimensional OCT signal in the memory unit 690.

[0088] The detection signal acquisition unit 630 extracts, from the acquired three-dimensional OCT signal, a portion that represents the intensity distribution of scattered light in a two-dimensional plane to be observed (hereinafter also referred to as the observation target plane) as a two-dimensional OCT signal (hereinafter sometimes simply referred to as the OCT signal). The observation target plane is, for example, the front surface of the sample Sm. The front surface is a plane perpendicular to the depth direction of the sample Sm, and is also called the en face surface. The detection signal acquisition unit 130 stores the extracted OCT signal in the memory unit 690. The detection signal acquisition unit 630 may select, for example, a two-dimensional plane of the depth indicated by the control signal input from the output processing unit 680 as the plane to be observed. Furthermore, it is desirable that the interval between adjacent sample points in the observable area be equal to or less than the spatial resolution of the optical system.

[0089] The detection signal acquisition unit 630 includes a focus adjustment unit 631 and a phase stabilization unit 632 . The focus adjustment unit 631 has a function of adjusting the focus of the OCT signal by digital processing. For example, the focus adjustment unit 631 has a function of refocusing the OCT signal and a function of defocusing the OCT signal. For example, after capturing an image, the focus adjustment unit 631 performs defocusing by performing digital processing to generate an OCT image in which the focus of the captured OCT image (captured OCT image) is pseudo-shifted. The phase stabilization unit 632 has a function of performing phase stabilization processing on the OCT signal by digital processing. Note that if a configuration that does not perform phase stabilization processing is adopted, the phase stabilization unit 632 does not need to be provided.

[0090] The shifter 640 spatially shifts the OCT signal. In this embodiment, the shift direction is the horizontal direction of the en face OCT image, which may be, for example, a direction parallel to one pair of sides of a rectangular (e.g., square) en face OCT image, or a direction parallel to another pair of sides. In this embodiment, a square front OCT image is assumed, with the direction parallel to one pair of sides being the direction parallel to the x-axis and the direction parallel to the other pair of sides being the direction parallel to the y-axis. In other words, an xy coordinate system, which is a two-dimensional Cartesian coordinate system, is set for the front OCT image.

[0091] The complex conjugate unit 650 obtains the complex conjugate of the OCT signal. The multiplication unit 660 multiplies (in this embodiment, complex multiplication) the complex conjugate of the OCT signal acquired by the complex conjugation unit 650 by the shifted OCT signal acquired by the shift unit 640.

[0092] The phase extraction section 670 detects phase information from the image resulting from the multiplication by the multiplication section 660, and obtains an image containing the phase information (differential phase image). In this embodiment, the phase extraction unit 670 acquires an image (differential phase image) that includes information on the lateral phase difference between the original OCT image and the shifted OCT image.

[0093] Here, the phase extraction unit 670 includes a local phase averaging unit 671 . The local phase averaging unit 671 has a function of performing local phase averaging on an image. In this embodiment, the phase extraction section 670 may extract phase information after the local phase averaging section 671 performs local phase averaging on the image resulting from the multiplication by the multiplication section 660 . If a configuration in which local phase averaging is not performed is adopted, the phase extraction section 670 does not need to be provided.

[0094] The phase extraction unit 670 outputs image data including the extracted phase information (phase differential image data) as output image data to a display unit (not shown) in response to a control signal input from the output processing unit 680. The phase extraction section 670 may store the output image data in the storage section 690 in response to a control signal input from the output processing section 680 .

[0095] The output processing unit 680 controls the generation or output of output image data representing an OCT image based on an operation signal input from an operation input unit (not shown). The operation input unit may be configured to include components that accept user operations and generate operation signals in response to the accepted operations, such as buttons, knobs, dials, a mouse, a joystick, etc. The operation input unit may be an input interface that receives operation signals wirelessly or via a wired connection from another device (for example, a portable device such as a remote controller).

[0096] The operation signal specifies parameters such as whether or not to display or store the OCT image, the observation plane to be used as the observation region, the characteristics of the spatial filter, etc. The output processing unit 680 may configure a user interface for displaying the OCT image by displaying on the display unit parameters that can be set by operation, parameter settings, and a setting screen that guides the settable parameters. For example, when an operation signal indicating whether or not an OCT image needs to be displayed is input, the output processing unit 680 outputs a control signal indicating whether or not the image needs to be displayed to the phase extraction unit 670. When a control signal indicating that the image needs to be displayed is input from the output processing unit 680, the phase extraction unit 670 outputs the output image data to the display unit, and when a control signal indicating that the image should not be displayed is input from the output processing unit 680, the phase extraction unit 670 does not output the output image data to the display unit.

[0097] When an operation signal indicating a plane to be observed is input, the output processing unit 680 outputs a control signal indicating the plane to be observed to the detection signal acquiring unit 630. The detection signal acquiring unit 630 outputs, as an OCT signal, a portion of the three-dimensional OCT signal that relates to the plane to be observed indicated by the control signal input from the output processing unit 680. The plane to be observed is defined using parameters such as the depth from the surface of the sample Sm, the observation direction, and the area of the observation region. When an operation signal indicating the characteristics of a spatial filter is input, output processing unit 680 outputs a control signal indicating the characteristics of the spatial filter to shift unit 640. Shift unit 640 sets a spatial filter having the characteristics of the spatial filter indicated by the control signal input from output processing unit 680.

[0098] Furthermore, when an operation signal indicating a defocus amount is input, output processing unit 680 outputs a control signal indicating the defocus amount to focus adjustment unit 631. Focus adjustment unit 631 sets the defocus amount indicated by the control signal input from output processing unit 680.

[0099] Storage unit 690 stores various data used in the processes executed by control unit 610 and various data acquired by control unit 610, in addition to the above programs. The storage unit 690 includes a non-volatile (non-transitory) storage medium such as a ROM, a flash memory, or an HDD. The storage unit 690 includes a volatile storage medium such as a RAM or a register.

[0100] [Example of image processing] Next, an example of image processing according to this embodiment will be described. FIG. 6 is an explanatory diagram showing an example of image processing according to an embodiment (second embodiment). It should be noted that the images shown in FIG. 6 are examples for the purpose of explanation and are not necessarily shown with strict accuracy.

[0101] (Step S21) The shifting unit 640 uses a spatial filter to shift the OCT signal indicating the front OCT image 1011 in the spatial domain acquired by the detection signal acquiring unit 630, and generates data for a shifted image 1012. The front OCT image 1011 is an OCT image in which the observation plane is the front.

[0102] (Step S22) The complex conjugate unit 650 generates a complex conjugate image 1013 of the front OCT image 1011 .

[0103] (Step S23) The multiplication unit 660 multiplies the shifted image 1012 by the complex conjugate image 1013 to generate the image 1014 of the multiplication result.

[0104] (Step S24) The phase extraction unit 670 extracts the phase component from the image 1014 resulting from the multiplication, and generates an image having the phase component (differential phase image 1015). The phase extraction unit 670 outputs the data of the generated differential phase image 1015 to a display unit (not shown) as output image data.

[0105] Here, examples of signals for each image are shown. The signal of the front OCT image 1011 is expressed by equation (1). In equation (1), r represents intensity, e represents natural logarithm, i represents imaginary number, and φ(x, y) represents phase. Furthermore, x and y represent coordinates on the xy plane in a two-dimensional Cartesian coordinate system. It should be noted that x and y are values that increase or decrease by one pixel each, for example.

[0106] [Number 1] re iφ(x,y) (1)

[0107] The signal of the image 1012 after the shift is expressed by equation (2): In equation (2), r' represents intensity, j represents the amount of shift in the x direction, and k represents the amount of shift in the y direction.

[0108] [Number 2] r'e iφ(x+j,y+k) (2)

[0109] The signal of the complex conjugate image 1013 of the front OCT image 1011 is expressed by equation (3).

[0110] [Number 3] re -iφ(x,y) (3)

[0111] The signal of the image 1014 resulting from the multiplication is expressed by equation (4).

[0112] [Number 4] rr'e i{φ(x+j,y+k)-φ(x,y)} (4)

[0113] The signal of the differential phase image 1015 is expressed by equation (5).

[0114] [Number 5] φ(x+j,y+k)-φ(x,y) (5)

[0115] [Image example] Specific examples of images according to this embodiment (images equivalent to the differential phase image 1015 shown in FIG. 6) are shown with reference to FIGS. 7(A) to 7(O). The defocus amount and shift amount differ for each image. In this embodiment, after capturing an OCT image, refocusing (digital refocusing) and defocusing (digital defocusing) are performed by digital processing, and then a slice image of the front OCT image is obtained by digital image processing.

[0116] In this example, the original image is a frontal OCT image of the head of an 8-day-old zebrafish in vivo. The scan area of this OCT image is 1mm x 1mm, and the number of pixels is 512 x 512. It should be noted that the images shown in FIGS. 7(A) to 7(O) are examples for the purpose of explanation, and are not necessarily shown with strict accuracy.

[0117] Each of the images shown in FIGS. 7(A) to 7(E) is an image obtained when a shift amount of one pixel is used in the horizontal direction (for example, the direction parallel to the x-axis). Each of the images shown in FIGS. 7(F) to 7(J) is an image obtained when a shift amount of one pixel is used in the vertical direction (eg, the direction parallel to the y-axis). Each image shown in Figures 7(K) to 7(O) is an image obtained when a shift amount of one pixel is used in each of the horizontal direction (e.g., the direction parallel to the x-axis) and the vertical direction (e.g., the direction parallel to the y-axis).

[0118] The images shown in Figures 7(A), 7(F), and 7(K) are images obtained when a defocus amount of -200 µm was used. The images shown in Figures 7(B), 7(G), and 7(L) are images obtained when a defocus amount of -100 µm was used. The images shown in Figures 7(C), 7(H), and 7(M) are images obtained when a defocus amount of 0 μm (ie, in-focus) was used. The images shown in Figures 7(D), 7(I), and 7(N) are images obtained when a defocus amount of 100 µm was used. The images shown in Figures 7(E), 7(J), and 7(O) are images obtained when a defocus amount of 200 µm was used.

[0119] In the examples of Figures 7(A), 7(F), 7(K), 7(E), 7(J), and 7(O), the structure is strongly visible due to defocusing. On the other hand, in the examples of FIGS. 7(C), 7(H), and 7(M), there is no defocus and the structure is not very apparent. In the examples of Figures 7(B), 7(G), 7(L), 7(D), 7(I), and 7(N), structures appear somewhere between these two.

[0120] As shown in FIGS. 7(A) to 7(O), the defocus amount and shift amount were changed and differential interference contrast (DIC) images were calculated. A phase differential image in the shift direction is obtained, and when the defocus direction is reversed, the differential direction is reversed. Although the contrast is low when the image is completely in focus, a phase differential image with high contrast is obtained by defocusing, and the contrast is enhanced, making it easier to distinguish between structured and noise areas.

[0121] In this way, three-dimensional phase-differential imaging in backscatter mode by OCT is realized. In this embodiment, complex operations are performed on the complex signals of the frontal OCT image to shift the frontal OCT image in a predetermined direction on the plane (in this embodiment, one or both of a direction parallel to the x-axis and a direction parallel to the y-axis) to directly obtain a phase difference, thereby obtaining an image similar to a conventional DIC image.

[0122] [Other examples of image processing] Next, an example of image processing according to this embodiment will be described. FIG. 8 is an explanatory diagram showing another example of image processing according to an embodiment (second embodiment). It should be noted that the images shown in FIG. 8 are examples for the purpose of explanation and are not necessarily shown with strict accuracy.

[0123] In the example of FIG. 8, the processes of (Step S21) to (Step S23) are the same as the processes shown in FIG. 6, and the same step numbers are assigned. The front OCT image 1201, the shifted image 1202, the complex conjugate image 1203, and the multiplication result image 1204 shown in Figure 8 are similar to the front OCT image 1011, the shifted image 1012, the complex conjugate image 1013, and the multiplication result image 1014 shown in Figure 6, respectively.

[0124] In the example of FIG. 8, after the process of (Step S23), the processes of (Step S31) and (Step S32) are performed.

[0125] (Step S31) The phase extraction unit 670 performs local phase averaging on the image 1204 resulting from the multiplication. In this embodiment, the local phase averaging process is a process of averaging (complex averaging) the values of a plurality of pixels within a small kernel. In this example, a 3 pixel by 3 pixel square is used as the kernel, but a kernel of any size, such as a 5 pixel by 5 pixel square, may also be used. The phase extraction unit 670 generates an image (average image 1205) by averaging the values included in each kernel for the image 1204 of the multiplication result.

[0126] (Step S32) The phase extraction unit 670 extracts a phase component from the average image 1205 and generates an image having the phase component (differential phase image 1206). The phase extraction unit 670 outputs the data of the generated differential phase image 1206 to a display unit (not shown) as output image data.

[0127] [Image example] Specific examples of images according to this embodiment (images equivalent to the differential phase image 1206 shown in FIG. 8) are shown with reference to FIGS. 9(A) to 9(O). The defocus amount and shift amount differ for each image. In this embodiment, after capturing an OCT image, refocusing (digital refocusing) and defocusing (digital defocusing) are performed by digital processing, and then a slice image of the front OCT image is obtained by digital image processing.

[0128] In this example, the original image is a frontal OCT image of the head of an 8-day-old zebrafish in vivo. The scan area of this OCT image is 1mm x 1mm, and the number of pixels is 512 x 512. It should be noted that the images shown in FIGS. 9(A) to 9(O) are examples for the purpose of explanation, and are not necessarily shown with strict accuracy.

[0129] Each of the images shown in FIGS. 9(A) to 9(E) is an image obtained when a shift amount of one pixel is used in the horizontal direction (for example, the direction parallel to the x-axis). Each of the images shown in FIGS. 9(F) to 9(J) is an image obtained when a shift amount of one pixel is used in the vertical direction (eg, the direction parallel to the y-axis). Each image shown in Figures 9(K) to 9(O) is an image obtained when a shift amount of one pixel is used in each of the horizontal direction (e.g., the direction parallel to the x-axis) and the vertical direction (e.g., the direction parallel to the y-axis).

[0130] The images shown in Figures 9(A), 9(F), and 9(K) are images obtained when a defocus amount of -200 µm was used. The images shown in Figures 9(B), 9(G), and 9(L) are images obtained when a defocus amount of -100 µm was used. The images shown in Figures 9(C), 9(H), and 9(M) are images obtained when a defocus amount of 0 μm (ie, in-focus) was used. The images shown in Figures 9(D), 9(I), and 9(N) are images obtained when a defocus amount of 100 µm was used. The images shown in Figures 7(E), 7(J), and 7(O) are images obtained when a defocus amount of 200 µm was used.

[0131] In the examples of Figures 9(A), 9(F), 9(K), 9(E), 9(J), and 9(O), the structure is strongly visible due to defocusing. On the other hand, in the examples of FIGS. 9(C), 9(H), and 9(M), there is no defocus and the structure is not very apparent. In the examples of Figures 9(B), 9(G), 9(L), 9(D), 9(I), and 9(N), structures appear somewhere between these two.

[0132] As shown in FIGS. 9(A) to 9(O), the defocus amount and shift amount were changed and differential interference contrast (DIC) images were calculated. A phase differential image in the shift direction is obtained, and when the defocus direction is reversed, the differential direction is reversed. Although the contrast is low when the image is completely in focus, a phase differential image with high contrast is obtained by defocusing, and the contrast is enhanced, making it easier to distinguish between structured and noise areas.

[0133] [Regarding the second embodiment] As described above, the image processing device 600 according to this embodiment can nondestructively and noninvasively acquire a phase differential image (e.g., an image equivalent to a differential interference contrast microscope image) at any depth of tissue using OCT signals. The image processing device 600 according to this embodiment can nondestructively acquire a three-dimensional phase differential image of a sample. In this embodiment, after capturing an OCT image, spatial shifting and multiplication, etc., can be used to obtain the difference (phase differential image) of the OCT image as if it had been irradiated from multiple directions, without actually controlling the direction of light irradiation.

[0134] In this embodiment, the contrast of the phase differential image can be improved (clarified) by defocusing. Note that the defocusing may be achieved physically in advance during imaging, or may be achieved by digital processing after imaging, for example. In this embodiment, only noise is obtained from a completely in-focus signal. In addition, in this embodiment, the contrast of the phase differential image can be improved by local phase averaging.

[0135] In this embodiment, a case has been described in which a differential phase image is obtained by subtracting two images, but any number of images may be used to obtain a differential phase image.

[0136] (Third embodiment) A third embodiment will be described.

[0137] In this embodiment, a case will be described in which two OCT images for obtaining a differential phase image are captured by physical arrangement or control. For example, if light is irradiated onto the same sample from different locations and the scattering from the same region of the sample is measured, two OCT images are obtained, each with a different light irradiation position. It should be noted that irradiation of light from a plurality of directions may be achieved by, for example, controlling the physical irradiation angle.

[0138] Furthermore, refocusing and defocusing may also be achieved by a physical configuration. For example, the focal position may be automatically changed using a tunable lens that changes the focal position. For the acquisition of scattered light from the same area of the sample, the illuminated area can be enlarged by defocusing the focal point of the sample. In this way, defocusing may be achieved by adjusting the optical system so that the specimen is out of focus.

[0139] In this embodiment, the method for generating a differential phase image from a front OCT image may be, for example, the same method as in the first embodiment or the same method as in the second embodiment.

[0140] [Regarding the third embodiment] As described above, the image processing device according to this embodiment can use OCT signals to non-destructively and non-invasively acquire a phase differential image (e.g., an image equivalent to a differential interference contrast microscope image) at any depth in tissue. In this embodiment, by actually controlling the light irradiation direction when capturing an OCT image, it is possible to obtain the difference (phase differential image) between OCT images irradiated from a plurality of directions.

[0141] (Regarding the above embodiments) As described above, the image processing device according to the embodiment acquires a three-dimensional OCT signal (original signal), performs phase stabilization processing (numerical processing) on the original signal, and then performs predetermined numerical processing on the phase-stabilized three-dimensional OCT signal. As the predetermined numerical processing, for example, numerical processing A or numerical processing B is performed.

[0142] (Numerical Processing A: Example of the First Embodiment) Numerical processing is performed to apply multiple (two types) spectral limiting filters to the frequency spectrum of the complex OCT signal of the frontal OCT image, and then image subtraction of the two OCT intensity signals resulting from the application is performed. Here, in the first embodiment, the spectral limiting filter is realized using a mask.

[0143] (Numerical Processing B: Example of the Second Embodiment) The phase difference between the OCT signal that has been numerically shifted laterally relative to the front and the original OCT signal is calculated by numerical processing.

[0144] (Use of physical configuration: Example of the third embodiment) Instead of numerical processing, two OCT signals may be acquired by physical arrangement or control. For example, illumination light from multiple directions may be used to acquire two OCT images based on scattered light from at least a portion of the same region of the target object (sample).

[0145] Each of the above embodiments can obtain an image that is almost equivalent to that obtained by a conventional, widely used phase contrast microscope (DIC). However, each of the embodiments has the following advantages over DIC. That is, while conventional DIC imaging requires tissue to be sectioned, the embodiment does not require tissue sectioning and allows imaging with reflected illumination, making it possible to achieve completely non-destructive phase imaging. Furthermore, in conventional DIC imaging, the tissue needs to be sliced, so only phase imaging at one depth of the tissue can be performed in one measurement. However, in the embodiment, phase imaging at any depth position up to a depth of several mm can be achieved in one measurement (in the embodiment, volumetric phase contrast imaging).

[0146] Furthermore, while conventional DIC imaging has not been able to perform phase imaging of deep tissue areas, the embodiment makes it possible to perform phase imaging of deep tissue areas. Furthermore, while conventional DIC imaging involves destructive phase imaging, the embodiment allows for non-destructive and non-invasive phase imaging.

[0147] As described above, in the above embodiment, the state or activity of minute tissues or cells can be visualized by phase imaging based on OCT images. The technology according to the embodiment is useful for improving the efficiency of drug development by, for example, evaluating minute changes in the optical properties of cultured tissues for drug efficacy testing. Techniques according to embodiments may be useful, for example, for enabling longitudinal experimental protocols through non-invasive measurement of minute changes in the optical properties of animal tumor models and reducing the costs of animal testing. The technology according to the embodiment is useful for, for example, improving the efficiency and accuracy of diagnosis of fundus diseases (particularly diseases related to irregularities in photoreceptor cells). The technology according to the embodiment enables technological replacement in fields where phase-contrast microscopes are applied, for example.

[0148] The technology according to the embodiment enables, for example, add-on functions for diagnosing fundus tissue using fundus OCT, image diagnosis in ophthalmology (especially deep fundus diseases), quality control of cultured tissues for regenerative medicine, quality control of organoids, animal experiments, research into the efficacy of drugs when measuring drug efficacy using cultured tissue, and efficient drug development using animal experiments.

[0149] <Configuration example> As an example configuration, the image processing device (the image processing device 100, 600 according to the first to third embodiments) includes a processing unit (in the examples of FIGS. 2 and 5, a processing unit having the processing function realized by the control unit 110, 610) that generates a phase differential image of a sample based on at least two front OCT images of the sample (sample Sm in the example of FIG. 1) captured by OCT.

[0150] As an example configuration, in an image processing device (image processing devices 100, 600 according to the first to third embodiments), at least two front OCT images are images obtained by acquiring scattered light from the same region of a sample when light is irradiated onto the sample from multiple directions, or equivalent pseudo images (essentially such images).

[0151] As an example configuration, in an image processing device (the image processing devices 100 and 600 according to the first and second embodiments), the processing unit generates at least two front OCT images based on one OCT image of a sample.

[0152] In one configuration example, in an image processing device (image processing device 100 according to the first embodiment), irradiation of light from multiple directions is achieved using digital filters (masks in the example of FIG. 3). The processing unit applies two different digital filters (e.g., semicircular masks that complement each other) to one OCT image, thereby generating two frontal OCT images. As one configuration example, in an image processing device (image processing device 100 according to the first embodiment), a processing unit generates a differential phase image by taking the difference between two front OCT images.

[0153] In one configuration example, in an image processing device (image processing device 600 according to the second embodiment), irradiation of light from multiple directions is achieved by shifting one OCT image (in the examples of FIGS. 6 and 8, shifting in two directions parallel to the front). The processing unit generates two front OCT images by shifting one OCT image by different amounts (including when the shift amount is 0). As an example configuration, in an image processing device (image processing device 600 according to the second embodiment), a processing unit multiplies the complex conjugate of one of two frontal OCT images by the other, and extracts a phase component based on the result of the multiplication to generate a phase differential image (for example, the examples in FIGS. 6 and 8). As an example configuration, in an image processing device (image processing device 600 according to the second embodiment), a processing unit performs local phase averaging on the result of the multiplication, and generates a phase differential image by extracting phase components from the result of the local phase averaging (for example, the example of Figure 8).

[0154] As an example configuration, in an image processing device (the image processing devices 100 and 600 according to the first and second embodiments), one OCT image is obtained by defocusing the focal point of the sample, and corresponds to an image shifted from the focal point. As an example of a configuration, in the image processing device (the image processing device 100, 600 according to the first and second embodiments), defocusing is physically performed by adjusting the optical system so as to shift the focus of the sample when photographing. Such a configuration may be used. As an example configuration, in an image processing device (the image processing devices 100 and 600 according to the first and second embodiments), after capturing an image, the processing unit performs defocusing by performing digital processing to generate a single OCT image in which the focus of the captured OCT image is pseudo-shifted.

[0155] As an example configuration, in an image processing device (image processing device 600 according to the third embodiment), at least two front OCT images are images obtained by controlling the physical irradiation angle of the light and acquiring scattered light from the same region of the sample when light is irradiated onto the sample from multiple directions.

[0156] As an example of configuration, in the image processing device (the image processing devices 100 and 600 according to the first to third embodiments), the processing unit generates a three-dimensional phase differential image.

[0157] It is also possible to provide a method for processing performed in the image processing device. In one configuration example, the image processing method generates a phase differential image of the sample based on at least two front OCT images of the sample taken by OCT.

[0158] It is also possible to provide a program (a computer-readable program) that executes the processing performed by the computer that constitutes the image processing device. In one configuration example, the program causes a computer to generate a phase differential image of the sample based on at least two front OCT images of the sample taken by OCT.

[0159] A program for implementing the functions of any of the components of any of the above-described devices may be recorded on a computer-readable recording medium and then loaded into a computer system for execution. The term "computer system" as used herein includes hardware such as an operating system (OS) or peripheral devices. The term "computer-readable recording medium" also refers to portable media such as flexible disks, optical magnetic disks, ROMs, and compact disc (CD)-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when the program is transmitted over a network such as the Internet or a communication line such as a telephone line.

[0160] The above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network such as the Internet or a communication line such as a telephone line. The above program may also be one that realizes part of the above-mentioned functions. Furthermore, the above program may be a so-called differential file that can realize the above-mentioned functions in combination with a program already recorded in the computer system. The differential file may also be called a differential program.

[0161] The functions of any of the components in any of the above-described devices may be implemented by a processor. For example, each process in the embodiments may be implemented by a processor operating based on information such as a program and a computer-readable recording medium storing information such as the program. Here, the functions of each unit of the processor may be implemented by, for example, individual hardware, or may be implemented by integrated hardware. For example, the processor may include hardware, and the hardware may include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the processor may be configured using one or more circuit devices mounted on a circuit board, or one or both of one or more circuit elements. An integrated circuit (IC) or the like may be used as the circuit device, and a resistor or a capacitor may be used as the circuit element.

[0162] Here, the processor may be, for example, a CPU. However, the processor is not limited to a CPU, and various types of processors such as a GPU (Graphics Processing Unit) or a DSP (Digital Signal Processor) may be used. The processor may also be, for example, a hardware circuit such as an ASIC (Application Specific Integrated Circuit). The processor may also be, for example, composed of multiple CPUs, or may be, for example, composed of a hardware circuit such as a multiple ASIC. The processor may also be, for example, composed of a combination of multiple CPUs and a hardware circuit such as a multiple ASIC. The processor may also include, for example, one or more of an amplifier circuit or a filter circuit that processes analog signals.

[0163] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0164] 1...optical coherence tomography, 10...light source, 20...beam splitter, 30a, 30b, 50a, 50b...collimator, 40...reference mirror, 60a, 60b...galvanometer mirror, 70...spectroscope, 100, 600...image processing device, 110, 610...controller, 120, 620...optical system control unit, 130, 630...detection signal acquisition unit, 131, 631...focus adjustment unit, 132, 632...phase stabilization unit, 140...electric field estimation unit, 150...mask unit, 160...conversion unit, 170...image synthesis unit, 180, 680...output processing unit, 190, 690...storage unit, 3 00...differential interference microscope, 310, 360...analyzer, 320, 350...DIC prism, 330...condenser lens, 340, Ol...objective lens, 410, Sm...sample, 420...focal plane, 640...shift unit, 650...complex conjugate unit, 660...multiplication unit, 670...phase extraction unit, 1011, 1201, Oi01...front OCT image, 1012-1014, 1202-1204...images, 1015, 1206...phase differential image, 1205...average image, Pe01...pupil plane electric field, Dm01...directional mask, Ie01...mask electric field, Pp...pupil plane

Claims

1. A processing unit that generates a phase differential image of the sample based on at least two frontal OCT images that are images obtained by capturing scattered light from the same region of the sample when light is irradiated onto the sample from multiple directions or pseudo images equivalent thereto, Image processing device.

2. The processing unit generates at least two front OCT images based on one OCT image of the sample. The image processing device according to claim 1 .

3. The irradiation of light from multiple directions is realized using digital filters; the processing unit applies two different digital filters to one OCT image to generate two front OCT images. The image processing device according to claim 2 .

4. the processing unit generates the phase differential image by subtracting the two front OCT images. The image processing device according to claim 3 .

5. the irradiation of light from multiple directions is achieved by shifting one of the OCT images; The processing unit generates two front OCT images by shifting the one OCT image by different amounts. The image processing device according to claim 2 .

6. the processing unit multiplies a complex conjugate of one of the two front OCT images by the other, and extracts a phase component based on a result of the multiplication, thereby generating the phase differential image. The image processing device according to claim 5 .

7. the processing unit performs local phase averaging on the result of the multiplication, and extracts a phase component from the result of the local phase averaging to generate the phase differential image. The image processing device according to claim 6 .

8. One of the OCT images corresponds to an image obtained by defocusing the focal point of the sample and shifted from the focal point. The image processing device according to any one of claims 2 to 7.

9. The defocusing is physically performed by adjusting the optical system so as to deviate from the focus of the sample during imaging. The image processing device according to claim 8 .

10. The processing unit performs the defocusing by performing digital processing to generate one OCT image in which the focus of the captured OCT image is pseudo-defocused after the capture. The image processing device according to claim 8 .

11. The at least two front OCT images are images obtained by controlling the physical irradiation angle of the light and acquiring scattered light from the same region of the sample when the light is irradiated onto the sample from multiple directions. The image processing device according to claim 1 .

12. the processing unit generates the three-dimensional differential phase image. The image processing device according to any one of claims 1 to 11.

13. A phase differential image of a sample is generated based on at least two frontal OCT images, which are images obtained by capturing scattered light from the same region of the sample when light is irradiated onto the sample from multiple directions, or equivalent pseudo-images. Image processing methods.

14. On the computer, generating a phase differential image of the sample based on at least two front OCT images, which are images acquired by scattering light from the same region of the sample when the sample is irradiated with light from multiple directions or pseudo images equivalent thereto; program.

Images