Image processing device, image processing method, and program

Abstract

According to the present invention, an electric field estimation unit converts an optical coherence tomographic signal representing the state of a sample into a spatial frequency domain and generates electric field data indicating the electric field on a pupil surface corresponding to the reciprocal space of the sample, a mask unit acts on the electric field data with a mask indicating the distribution of the transmission characteristics of the electric field on the pupil surface and generates mask electric field data, and a conversion unit converts the mask electric field data into a spatial region to generate a mask optical coherence tomographic signal. The present embodiment may be any of an image processing apparatus, an image processing method, and a program.

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. This application claims priority to Japanese Patent Application No. 2019-206436, filed in Japan on November 14, 2019, the contents of which are incorporated herein by reference. [Background technology]

[0002] Optical coherence tomography (OCT) is a technology that uses the coherence of light to obtain cross-sectional images of a sample (mainly a living organism). OCT can obtain images that show not only the surface of a sample, but also its internal structure with high spatial resolution. Traditionally, OCT has been put to practical use in retinal diagnosis in ophthalmology. It is also used to visualize and quantify the characteristics of tissue structures and microfiber structures (e.g., directionality, statistical properties of size) below the optical resolution using cultured tissues, ex vivo samples, and other tissues to be observed.

[0003] When a sample is photographed from multiple directions using OCT, images with different signal intensity patterns are obtained for each photographing direction. The change in the signal intensity pattern is thought to be due to the microstructure of the tissue that constitutes the sample. Therefore, as shown in Non-Patent Document 1, a method has been attempted to indirectly visualize the characteristics of the tissue microstructure by performing multi-directional OCT measurement. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Wartak et al., “Multi-directional optical coherence tomography for retinal imaging,” Biomedical Optics Express Vol.8, No.12, p.5560-5578, December 1, 2017. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method described in Non-Patent Document 1 uses hardware to realize multi-directional observation, which makes the device configuration complicated and difficult to realize economically. In addition, the measurement time tends to be long because measurements are performed multiple times. Therefore, it is not necessarily practical to apply it to living samples. In addition, the number of measurements is limited, which restricts the measurement target, making it unsuitable for quantitative observation that requires many measurements.

[0006] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to provide an image processing device, an image processing method, and a program that can eliminate or relax constraints related to observation, such as the observation direction, measurement time, number of measurements, and measurement target, and can more easily adjust spatial conditions related to sample observation. [Means for solving the problem]

[0007] (1) The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention comprises an electric field estimation unit that converts an optical coherence tomography signal representing a state of a sample into a spatial frequency domain to generate electric field data indicating the electric field on a pupil plane corresponding to a reciprocal space of the sample, a mask unit that applies a mask indicating a transmission characteristic distribution of the electric field on the pupil plane to the electric field data to generate masked electric field data, and a conversion unit that converts the masked electric field data into a spatial domain to generate a masked optical coherence tomography signal. a synthesis unit that synthesizes mask images based on M (M is an integer equal to or greater than 2) masked optical coherence tomography signals in different display modes to generate a synthetic image, wherein the mask unit applies M masks with different transmission characteristic distributions to the electric field data to obtain M masked electric field data, and the conversion unit converts the M masked electric field data into a spatial domain to generate the M masked optical coherence tomography signals. An image processing device.

[0008] (2) Another aspect of the present invention is the image processing device according to (1), in which the transmission characteristic distribution may indicate a spatial frequency band of an azimuth angle passing through the electric field on the pupil plane.

[0009] (3) Another aspect of the present invention is the image processing device of (1) or (2), in which the transmission characteristic distribution may indicate a spatial frequency band of a radius passing through the electric field on the pupil plane.

[0010] (4) Another aspect of the present invention is the image processing device of (1) or (2), in which the mask may indicate the presence or absence of passage of the electric field for each sample point corresponding to a spatial frequency.

[0011] (5) Another aspect of the present invention is the image processing device of (4), wherein the mask may have a mask value that changes monotonically between sample points within a predetermined range from the boundary between a passing region that passes through the electric field and a blocking region that does not pass through the electric field.

[0012] (6) Another aspect of the present invention is an image processing device according to any one of (1) to (5), which may include a synthesis unit that generates a mask image showing the structure of the sample based on the mask optical coherence tomography signal.

[0014] (7) Another aspect of the present invention is (1) In the image processing device, the M mask images may each be displayed in a different hue, as the display mode.

[0015] (8) Another aspect of the present invention is (7) In the image processing device, the display mode may be an achromatic color obtained by combining colors representing the M mask images, respectively.

[0016] (9)Another aspect of the present invention is a method for an image processing device, comprising: an electric field estimation step of converting an optical coherence tomography signal representing a state of a sample into a spatial frequency domain to generate electric field data indicative of an electric field on a pupil plane corresponding to a reciprocal space of the sample; a masking step of applying a mask indicative of a transmission characteristic distribution of the electric field on the pupil plane to the electric field data to generate masked electric field data; and a conversion step of converting the masked electric field data into a spatial domain to generate a masked optical coherence tomography signal. and a synthesis step of synthesizing mask images based on M masked optical coherence tomography signals (M is an integer equal to or greater than 2) in different display modes to generate a synthetic image, in which in the masking step, M masks having different transmission characteristic distributions are applied to the electric field data to obtain M masked electric field data, and in the converting step, the M masked electric field data are converted into a spatial domain to generate the M masked optical coherence tomography signals. An image processing method.

[0017] (10) Another aspect of the present invention is a computer of an image processing device that includes an electric field estimation step of converting an optical coherence tomography signal representing a state of a sample into a spatial frequency domain to generate electric field data indicating an electric field on a pupil plane corresponding to a reciprocal space of the sample, a masking step of applying a mask indicating a transmission characteristic distribution of the electric field on the pupil plane to the electric field data to generate masked electric field data, and a conversion step of converting the masked electric field data into a spatial domain to generate a masked optical coherence tomography signal. a synthesis process for generating a synthetic image by synthesizing mask images based on M (M is an integer equal to or greater than 2) mask optical coherence tomographic signals in different display modes; A program to execute In the masking step, M masks having different transmission characteristic distributions are applied to the electric field data to obtain M masked electric field data, and in the converting step, the M masked electric field data are converted into a spatial domain to generate the M masked optical coherence tomography signals. It is a program. Effect of the Invention

[0018] According to the present invention, it is possible to more easily adjust spatial conditions for observing a sample. For example, it is possible to virtually adjust the observation direction and observe from multiple directions without the need for providing or adjusting optical components for acquiring reflected light from each direction. [Brief description of the drawings]

[0019] [Figure 1] 1 is a configuration diagram illustrating an example of an optical coherence tomography device according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of a functional configuration of a signal processing device according to an embodiment of the present invention; [Diagram 3] FIG. 4 is an explanatory diagram showing an example of image processing according to the embodiment; [Figure 4] FIG. 11 is an explanatory diagram showing another example of image processing according to the embodiment. [Diagram 5]FIG. 11 is an explanatory diagram illustrating an example of a synthesis process according to the embodiment. [Figure 6A] FIG. 2 is a diagram showing a first example of a composite image according to the present embodiment. [Figure 6B] FIG. 11 is a diagram showing a second example of a composite image according to the embodiment. [Figure 6C] FIG. 11 is a diagram showing a third example of a composite image according to the embodiment. [Figure 6D] FIG. 11 is a diagram showing a fourth example of a composite image according to the embodiment. [Figure 6E] FIG. 11 is a diagram showing a fifth example of a composite image according to the present embodiment. [Figure 6F] FIG. 13 is a diagram showing a sixth example of a composite image according to the present embodiment. [Figure 7A] FIG. 2 is a diagram showing an example of a cross-sectional OCT image according to the embodiment. [Figure 7B] FIG. 2 is a diagram showing a first example of a composite image for a cross-sectional OCT image according to the present embodiment. [Figure 7C] FIG. 13 is a diagram showing a second example of a composite image for a cross-sectional OCT image according to the present embodiment. [Figure 8] 13A and 13B are diagrams illustrating other examples of the directional mask according to the embodiment. [Figure 9] FIG. 1 is a diagram showing a first example of the principal direction dependence of signal strength. [Figure 10] FIG. 13 is a diagram showing a second example of the principal direction dependence of signal strength. [Figure 11] FIG. 1 is a diagram showing an example of a sample model. [Figure 12A] FIG. 13 is a diagram showing a seventh example of a composite image according to the present embodiment. [Figure 12B] FIG. 13 is a diagram showing an eighth example of a composite image according to the present embodiment. [Figure 12C] FIG. 13 is a diagram showing a ninth example of a composite image according to the present embodiment. [Figure 12D] FIG. 17 is a diagram showing a tenth example of a composite image according to the present embodiment. [Figure 13] FIG. 13 is a diagram showing an example of an estimated tissue structure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of an optical coherence tomography device 1 according to this embodiment. The optical coherence tomography device 1 constitutes an observation system for observing the state of a sample using OCT. The optical coherence tomography device 1 is a device that irradiates light onto a sample Sm, acquires interference light generated by the interference between the light reflected from the sample Sm and the reference light reflected by a reference mirror 40 (described later), and generates an image showing the surface and internal condition of the sample Sm from the acquired interference light. The object to be observed as the sample Sm may be, for example, a living human or animal body, or a non-living body. The living body may be the fundus of the eye, blood vessels, teeth, subcutaneous tissue, etc. The non-living body may be an artificial structure such as an electronic part or a machine part, a natural structure such as stone or a mineral, or a substance without a specific shape.

[0021] The optical coherence tomography device 1 includes a light source 10, a beam splitter 20, collimators 30a, 30b, 50a, and 50b, a reference mirror 40, galvanometer mirrors 60a and 60b, a spectrometer 70, and an image processing device 100. Among these components, the beam splitter 20, the collimators 30a, 30b, 50a, and 50b, the reference mirror 40, the galvanometer mirrors 60a and 60b, and the spectrometer 70 constitute an optical system called an interferometer. The interferometer illustrated in FIG. 1 is a Michelson interferometer equipped with an optical fiber F. More specifically, the light source 10 and the spectrometer 70, the collimators 30a and 50a are each connected to the beam splitter using the optical fiber F. The optical fiber F has a transmission band including the wavelength band of the light irradiated from the light source 10.

[0022] The optical coherence tomography 1 employs Fourier-Domain OCT (FD-OCT). The optical coherence tomography 1 may employ any of the methods classified as FD-OCT, such as Spectral-Domain OCT (SD-OCT) and Swept-Source OCT (SS-OCT).

[0023] The light source 10 is a wavelength swept light source such as an ultrashort pulse laser or an SLD (Superluminescent Diode). The light source 10 irradiates probe light having, for example, a near-infrared wavelength (for example, 800 to 1000 nm) and low coherence. The light irradiated from the light source 10 is guided inside the optical fiber F and enters the beam splitter 20.

[0024] The beam splitter 20 splits the incident light into light guided toward the collimator 30a (hereinafter, reference light) and light guided toward the collimator 50a (hereinafter, measurement light). The beam splitter 20 is, for example, a cube beam splitter.

[0025] The collimator 30a converts the reference light guided from the beam splitter 20 into parallel light, and emits the parallel light toward the collimator 30b. The collimator 30b collects the parallel light incident from the collimator 30a, and emits the collected reference light toward the reference mirror 40. The collimator 30b receives the reference light reflected by the reference mirror 40, converts it into parallel light, and emits the converted parallel light toward the collimator 30a. The collimator 30a collects the parallel light incident from the collimator 30b and guides the light toward the beam splitter 20.

[0026] On the other hand, the collimator 50a converts the measurement light guided from the beam splitter 20 into parallel light and emits the converted parallel light toward the galvanometer mirror 60a. At the surfaces of the galvanometer mirrors 60a and 60b, the parallel light incident from the collimator 50a is reflected and emitted toward the collimator 50b. The collimator 50b collects the parallel light incident from the collimator 50a via the galvanometer mirrors 60a and 60b, and irradiates the collected measurement light onto the sample Sm. The measurement light irradiated onto the sample Sm is reflected by the reflecting surface of the sample Sm and enters the collimator 50b. The reflecting surface is not limited to, for example, the boundary surface between the sample Sm and the surrounding environment of the sample Sm (e.g., the atmosphere), but may be a boundary surface that divides materials or tissues with different refractive indices inside the sample Sm. Hereinafter, the light reflected by the reflecting surface of the sample Sm and incident onto the collimator 50b is called reflected light.

[0027] The collimator 50b outputs the incident reflected light toward the galvanometer mirror 60b. The light is reflected by the surfaces of the galvanometer mirrors 60b and 60a, respectively, and is output toward the collimator 50a. The collimator 50a collects the parallel light incident from the collimator 50a via the galvanometer mirrors 60a and 60b, and guides the collected reflected light toward the beam splitter 20. The beam splitter 20 guides the reference light reflected by the reference mirror 40 and the light reflected by the sample Sm to the spectroscope 70 via the optical fiber F.

[0028] The spectrometer 70 includes a diffraction grating and a light receiving element therein. The diffraction grating separates the reference light and the reflected light guided from the beam splitter 20. The separated reference light and the reflected light interfere with each other to become interference light. The light receiving element is disposed on an imaging surface onto which the interference light is irradiated. The light receiving element detects the irradiated interference light and generates a signal based on the detected interference light (hereinafter, detection signal). The light receiving element outputs the generated detection signal to the image processing device 100.

[0029] The image processing device 100 acquires an OCT signal representing the state of the sample Sm from the detection signal input from the spectrometer 70. The image processing device 100 sequentially changes the observation point in a predetermined order, and sequentially accumulates the detection signals acquired at each observation point or collectively for a plurality of observation points forming a predetermined unit, to generate an OCT signal within a predetermined observation region. The image processing device 100 converts the generated OCT signal into the spatial frequency domain and estimates the electric field of the reflected light on the pupil plane corresponding to the reciprocal space of the sample Sm as the pupil plane electric field. The image processing device 100 applies a mask indicating the transmission characteristic distribution of the electric field on the pupil plane to electric field data indicating the estimated electric field to obtain mask electric field data. The image processing device 100 converts the obtained mask electric field data into the spatial domain to generate a mask OCT signal. The image processing device 100 generates a mask OCT image based on the generated mask OCT signal.

[0030] The image processing device 100 may apply M masks (M is an integer equal to or greater than 2) having different transmission characteristic distributions to the electric field data to generate M mask electric field data, and convert the generated M mask electric field data into the spatial domain to generate M mask images. The image processing device 100 may generate a composite image by combining the generated M mask images with different display modes (e.g., colors).

[0031] Next, an example of the functional configuration of the image processing device 100 according to this embodiment will be described. FIG. 2 is a block diagram showing an example of the functional configuration of the image processing device 100 according to this embodiment. The image processing device 100 includes a control unit 110 and a storage unit 190. A part or all of the functions of the control unit 110 are realized as a computer including a processor such as a CPU (Central Processing Unit). The processor reads a program stored in advance in the storage unit 190, and performs processing instructed by commands written in the read program to fulfill its functions. In this application, performing processing instructed by commands written in the program may be referred to as executing a program, running a program, etc. A part or all of the control unit 110 is not limited to general-purpose hardware such as a processor, and may be configured to include dedicated hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0032] The control unit 110 includes an optical system control unit 120 , a detection signal acquisition unit 130 , an electric field estimation unit 140 , a mask unit 150 , a conversion unit 160 , an image synthesis unit 170 , and an output processing unit 180 .

[0033] The optical system control unit 120 drives a driving mechanism that changes the positions of the galvanometer mirrors 60a and 60b, and scans a sample point, which is an observation point of the sample Sm. The sample point of the sample Sm is scanned in the depth direction of the sample Sm and in a direction intersecting the depth direction (for example, a direction parallel to the front surface of the sample Sm). In the following description, the depth direction of the sample is sometimes called the z direction in a three-dimensional orthogonal coordinate system, and a first direction and a second direction mutually orthogonal to the z direction are sometimes called the x direction and the y direction, respectively. Signal acquisition in the depth direction of the sample is sometimes called an A-scan, and signal acquisition in a direction intersecting the depth direction (for example, the x direction or y direction) is sometimes called a B-scan.

[0034] The detection signal acquisition unit 130 sequentially acquires detection signals from the spectrometer 70. Based on the acquired detection signals, the detection signal acquisition unit 130 acquires signal values ​​indicating the intensity distribution of reflected light in the depth direction of the sample Sm for each sample point arranged at a predetermined interval in the three-dimensional space in which the sample Sm exists. The depth direction of the sample Sm corresponds to the incident direction (z direction) of the measurement light. The detection signal acquisition unit 130 repeats the process of acquiring the intensity distribution of reflected light for each observation point changed by signal acquisition along a plane (for example, an xy plane) intersecting the z direction. The acquired intensity distribution of reflected light is based on the distribution of the refractive index of the sample Sm in the depth direction. This allows the detection signal acquisition unit 130 to acquire data indicating the state of the sample Sm in a three-dimensional region that can be observed (hereinafter, an observable region) as a three-dimensional OCT signal (also called an OCT volume). The detection signal acquisition unit 130 stores the three-dimensional OCT signal in the storage unit 190. Note that the plane intersecting the z direction does not necessarily have to be orthogonal to the z direction, and may not be parallel to the z direction. In addition, in a two-dimensional plane intersecting the z direction, each sample point does not necessarily have to be arranged on each lattice point of an orthogonal lattice. Each sample point may be arranged, for example, on each lattice point of a diagonal lattice, or may be arranged non-periodically. In the above description, the term "scan" refers to signal acquisition involving a change in the target sample point or collective signal acquisition for multiple sample points, but may not necessarily involve scanning by driving a member constituting the optical system (e.g., the collimator 50b, the support base for the sample Sm) or a detector. In that case, the detection signal acquisition unit 130 is responsible for the function related to signal acquisition, and is omitted in the optical system control unit 120. For example, when performing an A-scan at a point on the xy plane using FD-OCT as the OCT method, the detection signal acquisition unit 130 detects interference light based on reflected light having different frequency components depending on the depth of the sample Sm as a detection signal. The detection signal acquisition unit 130 can perform a Fourier transform on the detection signal to calculate a conversion coefficient for each frequency, and determine the conversion coefficient of the depth associated with the frequency as the signal value at that depth. Furthermore, when scanning by driving in the x direction or y direction is not required, the optical system control unit 120 can be omitted.

[0035] The detection signal acquisition unit 130 extracts a portion of the acquired three-dimensional OCT signal that represents the intensity distribution of reflected light in a two-dimensional plane to be observed (hereinafter, the plane to be observed) as a two-dimensional OCT signal (hereinafter, sometimes simply referred to as an OCT signal). The plane to be observed is, for example, the front surface of the sample Sm. The front surface is a plane (i.e., an xy plane) perpendicular to the depth direction of the sample Sm, and is also called the en face surface. The detection signal acquisition unit 130 stores the extracted OCT signal in the storage unit 190. The detection signal acquiring section 130 may select, for example, a two-dimensional plane (for example, an xy plane) of the depth (that is, the z coordinate) indicated by the control signal input from the output processing section 180 as the observation target plane. In this embodiment, it is desirable that the interval between adjacent sample points in the observable area be equal to or less than the spatial resolution of the optical system.

[0036] The electric field estimation unit 140 reads out the OCT signal stored in the storage unit 190, performs a two-dimensional Fourier transform on the read spatial domain OCT signal, and generates spatial frequency domain data. The spatial frequency domain data obtained by the Fourier transform corresponds to data (hereinafter, electric field data) indicating the electric field at the pupil plane (hereinafter, pupil plane electric field). That is, the electric field of the reflected light projected on the pupil plane is estimated by performing a two-dimensional Fourier transform on the OCT signal related to the observation target plane. In addition, the conversion coefficient for each frequency sample point (frequency bin) converted into the spatial frequency domain is generally a complex number, and is represented by its absolute value and argument. Therefore, the amplitude intensity and phase of the electric field data are represented by the absolute value and argument of each conversion coefficient. The electric field estimation unit 140 stores the generated spatial frequency domain data in the storage unit 190 as electric field data.

[0037] As illustrated in FIG. 3, the pupil plane Pp corresponds to a back-focal plane corresponding to the reciprocal space of the sample Sm. In other words, the pupil plane Pp is a plane located on the opposite side of the objective lens Ol across the sample Sm, and is a plane onto which the reflected light that is diffused from the sample Sm and collimated by the objective lens Ol is virtually projected. The entire band of spatial frequencies converted into the spatial frequency domain corresponds to the observation region on the pupil plane where the electric field is to be observed. The intersection point where the optical axis of the objective lens Ol intersects with the pupil plane corresponds to the origin of the spatial frequency. The spatial frequency in the spatial frequency domain corresponds to a sample point within the observation region. The sample Sm is placed at a position where it intersects with the optical axis of the objective lens Ol.

[0038] Returning to FIG. 2, the mask unit 150 reads out the electric field data stored in the memory unit 190, and applies a mask to the read out electric field data to generate masked electric field data indicating a masked electric field. The mask is numerical data indicating the passing characteristic distribution of the electric field of the reflected light in the spatial frequency domain. The mask indicates a numerical value indicating the passing characteristic of the electric field for each frequency sample point corresponding to each spatial frequency as a mask value. For example, the mask value for each frequency sample point takes one of two values, 1 or 0, where 1 and 0 indicate the presence or absence of passing of the electric field, respectively. Therefore, the distribution of frequency sample points with a mask value of 1 indicates a pass band in the spatial frequency domain that passes through the electric field in the reciprocal space of the sample Sm. From another perspective, the pass band indicates a virtual aperture on the pupil plane that passes through the reflected light. On the other hand, the distribution of frequency sample points with a mask value of 0 indicates a stop band in the spatial frequency domain that does not pass through the electric field in the reciprocal space of the sample Sm. From another perspective, the stop band indicates a virtual shield on the pupil plane that does not pass through the reflected light.

[0039] When applying a mask, the mask unit 150 multiplies the electric field value for each frequency sample point indicated by the electric field data by the mask value of the frequency sample. The mask unit 150 generates mask electric field data, which has the multiplied value obtained by multiplication as a mask electric field value for each frequency sample point. The mask electric field data is used to generate an OCT image reflecting spatial characteristics (e.g., directivity) corresponding to the transmission characteristic distribution, as described later. The mask unit 150 stores the generated mask electric field data in the storage unit 190. Specific examples of masks will be described later.

[0040] The conversion unit 160 reads out the mask electric field data stored in the storage unit 190, performs a two-dimensional inverse Fourier transform on the read mask electric field data in the spatial frequency domain, and generates a mask OCT signal in the spatial domain. The conversion unit 160 stores the generated mask OCT signal in the spatial domain in the storage unit 190. In addition, when M masks each having a different electric field transmission characteristic distribution in the spatial frequency domain are set, the mask unit 150 may apply each of the M masks to the read electric field data to generate M mask electric field data. In this case, the conversion unit 160 converts each of the M mask electric field data in the spatial frequency domain into a mask OCT signal in the spatial domain.

[0041] The image synthesis unit 170 reads out the mask OCT signal stored in the storage unit 190, and converts the signal value of each sample point on the observation target plane indicated by the read mask OCT signal into a luminance value for each pixel using a predetermined conversion function. The converted luminance value is within a range that can be expressed by the bit depth of each pixel. The image synthesis unit 170 generates mask image data having the converted luminance value for each sample point. The generated mask image data indicates a mask image reflecting spatial characteristics corresponding to the pass characteristic distribution. When the number of masks used to mask the pupil plane electric field is one, the image synthesis unit 170 outputs the generated mask image data to a display unit (not shown) as output image data in response to a control signal input from the output processing unit 180.

[0042] When mask OCT signals corresponding to the M masks respectively are acquired, the image synthesis section 170 generates mask images having different display aspects (for example, colors) for the M mask OCT image signals. The image synthesis unit 170 synthesizes the generated M mask images to generate synthetic image data showing one synthetic image. For example, when generating a color synthetic image using the RGB color system, the image synthesis unit 170 determines the average pixel value of each pixel among the M mask images as the pixel value of the synthetic image. In the RGB color system, the color tone (chromaticity) is determined by the ratio of pixel values ​​of pixels representing the primary colors red, green, and blue for each pixel block in which shading is set. The image synthesis unit 170 outputs the generated synthetic image data to the display unit as output image data in response to a control signal input from the output processing unit 180. The image synthesis unit 170 may store the output image data in the storage unit 190 in response to a control signal input from the output processing unit 180 .

[0043] The output processing unit 180 controls the generation or output of output image data representing an OCT image based on an operation signal input from an operation input unit (not shown). The operation input unit may be configured to include a member that receives a user operation and generates an operation signal according to the received operation, such as a button, a knob, a dial, a mouse, a joystick, etc. The operation input unit may be an input interface that receives an operation signal wirelessly or wired from another device (for example, a portable device such as a remote controller).

[0044] The operation signal indicates, as parameters, for example, whether or not to display or store the OCT image, the observation target plane to be the observation target region, the spatial frequency characteristics of the mask, etc. The output processing unit 180 may display, on the display unit, parameters that can be set by operation, parameter settings, and a setting screen for guiding the settable parameters, thereby forming a user interface for displaying the OCT image. For example, when an operation signal indicating whether or not an OCT image needs to be displayed is input, the output processing unit 180 outputs a control signal indicating whether or not the image needs to be displayed to the image synthesis unit 170. When a control signal indicating a need for display is input from the output processing unit 180, the image synthesis unit 170 outputs output image data to the display unit, and when a control signal indicating a need for display is input from the output processing unit 180, the image synthesis unit 170 does not output the output image data to the display unit.

[0045] When an operation signal indicating the observation target plane is input, the output processing unit 180 outputs a control signal indicating the observation target plane to the detection signal acquiring unit 130. The detection signal acquiring unit 130 outputs, as an OCT signal, a portion of the three-dimensional OCT signal related to the observation target plane indicated by the control signal input from the output processing unit 180. The observation target plane is defined using parameters such as the depth from the surface of the sample Sm, the observation direction, and the area of ​​the observation region. When an operation signal indicating the spatial frequency characteristics of a mask is input, output processing unit 180 outputs a control signal indicating the spatial frequency characteristics to mask unit 150. Output processing unit 180 may set the spatial frequency characteristics of each of a plurality of masks based on the input operation signal, and output a control signal indicating the set spatial frequency characteristics to mask unit 150. Mask unit 150 sets a mask having the spatial frequency characteristics indicated by the control signal input from output processing unit 180.

[0046] The storage unit 190 stores various data used in the processes executed by the control unit 110 and various data acquired by the control unit 110, in addition to the above programs. The storage unit 190 includes a non-volatile (non-transient) storage medium such as a ROM (Read Only Memory), a flash memory, a HDD (Hard Disk Drive), etc. The storage unit 190 includes a volatile storage medium such as a RAM (Random Access Memory), a register, etc.

[0047] (Image Processing) Next, an example of image processing according to this embodiment will be described. Fig. 3 is an explanatory diagram showing an example of image processing according to this embodiment. The image processing illustrated in Fig. 3 is an example of application to directional imaging of an OCT image. In directional imaging, a mask having directionality in the spatial frequency domain (hereinafter, directional mask) is used. The directional mask has a mask value that is dependent on a spatial frequency corresponding to an azimuth from an origin in a two-dimensional spatial frequency domain (hereinafter, azimuth spatial frequency).

[0048] (Step S01) The electric field estimation unit 140 performs a two-dimensional Fourier transform on an OCT signal indicating a front OCT image Oi01 in the spatial domain acquired by the detection signal acquisition unit 130, and generates pupil plane electric field data in the spatial frequency domain indicating a pupil plane electric field Pe01. The front OCT image Oi01 is an OCT image in which the observation target plane is the front. (Step S02) The masking unit 150 applies a directional mask Dm01, as an example of a virtual mask, to the pupil plane electric field data generated by the electric field estimating unit 140 to generate mask electric field data indicating a mask electric field Ie01. The directional mask Dm01 has a passband in which the spatial frequency corresponding to the distance (radius) from the origin (hereinafter referred to as the radial spatial frequency) is within a predetermined band, and has an azimuth spatial frequency of half a band from -π / 2 through 0 to π / 2, and has other bands as the stopband.

[0049] (Step S03) The conversion unit 160 performs a two-dimensional inverse Fourier transform on the mask electric field data generated by the mask unit 150 to generate a mask OCT signal in the spatial domain. The image synthesis unit 170 converts the signal value of each sample point indicated by the mask OCT signal into a pixel value for each pixel, and generates mask image data having the pixel values ​​obtained by the conversion. The image synthesis unit 170 outputs the generated mask image data to a display unit (not shown) as output image data.

[0050] As described above, by applying the directional mask Dm01 to the pupil plane electric field Pe01, the pupil plane electric field in the spatial frequency band within the bandwidth π centered (main direction) at 0 (rad) in azimuth spatial frequency is passed, and the pupil plane electric field in the other bands is blocked. As a result, an OCT image having directivity with 0 (rad) in the main direction is obtained by the obtained mask electric field Ie01.

[0051] Furthermore, by making the spatial frequency characteristics of the mask variable according to the operation, the user can arbitrarily adjust the directivity of the OCT image using the acquired OCT signal without driving or adjusting the optical system. For example, the output processing unit 180 can set one or both of the azimuthal spatial frequency band and the radial spatial frequency band as a passband for passing the electric field based on an operation signal input from an operation input unit (not shown). The mask unit 150 sets the mask value at the frequency sample points within the passband set by the output processing unit 180 to 1 and the mask values ​​related to the other frequency sample points to 0, and generates a directivity mask indicating the set mask value.

[0052] 4 is an explanatory example showing another example of image processing according to the present embodiment. An application example to virtual multi-directional imaging will be described with reference to FIG. In multi-directional imaging, in step S02 (FIG. 3), the mask unit 150 generates mask electric field data using M directional masks each having a different directivity in the spatial frequency domain for a common pupil plane electric field. In the example shown in FIG. 4, three directional masks Dm01-Dm03 are used. The passbands of the directional masks Dm01-Dm03 are set to a period of 2π / 3 and do not overlap with each other in the spatial frequency domain.

[0053] In step S03 (FIG. 3), the conversion unit 160 performs a two-dimensional inverse Fourier transform on each of the generated M mask electric field data to generate a mask OCT signal in the spatial domain. The image synthesis unit 170 converts each of the generated M spatial domain mask OCT signals into a mask OCT image. The image synthesis unit 170 generates output image data indicating a synthesis image displayed in different colors for each of the M converted mask OCT images, and outputs the generated output image data to a display unit. Thus, a synthesis image in which OCT images with different directivities are multiplexed in different colors is displayed. Here, the output processing unit 180 may be capable of setting one or both of the azimuthal spatial frequency band and the radial spatial frequency band as a passband for passing the electric field for each of M masks (M is an integer of 2 or more) based on an operation signal input from an operation input unit (not shown). In this case, the mask unit 150 sets the mask value of the frequency sample points within the passband set by the output processing unit 180 to 1 for each mask, and the mask value of the other frequency sample points to 0.

[0054] Next, an example of a synthesis process for M mask OCT images (hereinafter, directional images) to which different directivities have been imparted will be described. FIG. 5 is an explanatory diagram showing an example of the synthesis process according to the present embodiment. (Step S11) The image synthesis unit 170 colors the M directional images with different colors, respectively, to generate colored image data representing the colored images. In the example shown in FIG. 5, the image synthesis unit 170 colors the two directional images Di11 and Di12 with different colors (e.g., red and green), respectively, to generate colored images Ci11 and Ci12. In FIG. 5, each color is represented by a diagonal line going up to the right and a diagonal line going down to the right. The directional images Di11 and Di12 are OCT images generated using a directivity mask having directivity 1 and 2, respectively. Directivity 1 indicates a passband whose main direction is an azimuth angle of 7 / 6π and whose bandwidth is π. Directivity 2 indicates a passband whose main direction is an azimuth angle of -5 / 6π and whose bandwidth is π.

[0055] (Step S12) The image synthesis unit 170 synthesizes the colored images indicated by the M pieces of colored image data to generate synthetic image data indicating a synthetic image. In the example shown in FIG. 5, the colors colored in the colored images Ci11 and Ci12 are mixed for each part. The synthetic image Si11 represents the mixed color obtained by mixing for each part. The diagonally shaded parts indicate the mixed color obtained by mixing the colors colored in the colored images Ci11 and Ci12. The parts with diagonal lines slanting upward to the right and the parts with diagonal lines slanting downward to the right indicate parts that are colored in the colored images Ci11 and Ci12, respectively, but are not colored in the colored images Ci12 and Ci11.

[0056] Therefore, a user who comes into contact with the displayed composite image can intuitively grasp the distribution of the state of the sample corresponding to the directivity in the observation area of ​​the sample by the color tone. By using the hue as the display mode, it is possible to express a finer difference in the distribution of directivity than when other types of display modes, for example, when distinguishing by a pattern such as a shading, are used. Hue generally means a color tone expressed by a component of a specific prominent wavelength in the image light, and is one of the three attributes of color. It is desirable that the hue of the color to be colored is as different as possible in the color space among the M mask OCT images. When M is 2, it is sufficient that the color of one mask OCT image is a complementary color to the color of the other mask OCT image. For example, when the color of one mask OCT image is red, the color of the other mask OCT image is green. When M is 3, it is sufficient that the colors of the three mask OCT images are the three primary colors of red, green, and blue, for example.

[0057] Also, the color obtained by simply mixing the colors of M mask OCT images among M may be set to be an achromatic color. An achromatic color is a color without saturation, that is, white, black, or various densities of gray. When M is 2, if the color of one mask OCT image is a complementary color to the color of the other mask OCT image, mixing the colors will result in an achromatic color. When M is 3, if the colors of the three mask OCT images are the three primary colors of red, green, and blue, mixing the colors will result in an achromatic color. This allows the difference in brightness between different directivities to be expressed by the colors of the composite image, so that a user who sees the composite image can easily recognize that a part where a particular color is prominent has a tissue structure that corresponds to the directivity associated with that color.

[0058] The image synthesis unit 170 may generate average image data having pixel values ​​that are average values ​​obtained by averaging pixel values ​​indicated by the M mask OCT image data. The image synthesis unit 170 then generates average colored image data indicating a new colored image (hereinafter, average colored image) that is colored with a color different from any of the M colored images for the average image data. The image synthesis unit 170 synthesizes the colored image represented by the M pieces of colored image data and the average colored image represented by the average colored image data to generate synthetic image data representing a synthetic image. For example, when M is 2, the image synthesis unit 170 sets the colors of the two colored images to red and green, respectively, and the color of the average colored image to blue. In this way, the areas where the conditions corresponding to each directivity are prominent due to the colors prominently displayed in the composite image are displayed as the average values ​​thereof for comparison.

[0059] The synthesis mode for generating the synthetic image is not limited to this. The synthesis mode refers to a combination of spatial frequency characteristics of masks used for generating each of the mask OCT images (for example, directional images) used for generating the synthetic image. 6A to 6F are diagrams showing examples of composite images for each synthesis mode according to this embodiment, in which a front OCT image Oi01 is used as a common OCT image in generating a composite image in Fig. 6A to Fig. 6F.

[0060] The low-pass image Li11 shown in FIG. 6A is a composite image generated using the composite mode 11. The composite mode 11 includes a first spatial frequency characteristic having a directivity with an azimuth angle of 7 / 6π as the main direction and a passband with a bandwidth of π, and a second spatial frequency characteristic having a directivity with an azimuth angle of -5 / 6π as the main direction and a passband with a bandwidth of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic each have a low radial spatial frequency range from frequency 1 to frequency 2 as the passband, and other radial spatial frequencies as the stopband. Frequency 1 is a predetermined radial spatial frequency that is 0 or sufficiently close to 0. Frequency 2 is a predetermined radial spatial frequency that is higher than frequency 1 and lower than the upper limit of the radial spatial frequency. The upper limit of the radial spatial frequency is determined based on the sampling point interval of the OCT signal according to the sampling theorem. The low-pass image Li11 is generated using the low-pass component of the pupil plane electric field, and is configured to include a pattern with a rougher and unclear contour.

[0061] The high-frequency image Hi11 shown in FIG. 6B is a composite image generated using the composite mode 12. The composite mode 12 is composed of a first spatial frequency characteristic having a directivity with an azimuth angle of 7 / 6π as the main direction and a passband with a bandwidth of π, and a second spatial frequency characteristic having a directivity with an azimuth angle of -5 / 6π as the main direction and a passband with a bandwidth of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic each have a high-frequency range of radial spatial frequencies from frequency 2 to frequency 3 as the passband, and the other radial spatial frequencies as the cutoff band. Frequency 3 is a predetermined low radial spatial frequency that is higher than frequency 2 and equal to or lower than the upper limit of the radial spatial frequency. The high-frequency image Hi11 is generated using the high-frequency components of the pupil plane electric field, and is configured to include a relatively fine pattern.

[0062] The composite image Si11 shown in FIG. 6C is a composite image generated using the composite mode 13. The composite mode 13 is composed of a first spatial frequency characteristic having a directivity with an azimuth angle of 7 / 6π as the main direction and a passband with a bandwidth of π, and a second spatial frequency characteristic having a directivity with an azimuth angle of -5 / 6π as the main direction and a passband with a bandwidth of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic are not band-limited in the radial spatial frequency domain. That is, the composite mode 13 is the same as the composite mode composed of directivities 1 and 2 illustrated in FIG. 5, and has both the passband of the composite mode 11 and the passband of the composite mode 12.

[0063] The low-pass image Li12 shown in FIG. 6D is a composite image generated using a composite mode 14. The composite mode 14 is composed of a first spatial frequency characteristic having a directivity with an azimuth angle of 5 / 6π as the main direction and a passband with a bandwidth of π, and a second spatial frequency characteristic having a directivity with an azimuth angle of -7 / 6π as the main direction and a passband with a bandwidth of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic each have a low-pass range of radial spatial frequencies from frequency 1 to frequency 2 as the passband, and other radial spatial frequencies as the stopband. The low-pass image Li12 is generated using a mask whose passband main direction is different from that of the low-pass image Li11, and therefore shows a different pattern from that of the low-pass image Li11. The low-pass image Li12 shows a relatively coarse pattern with unclear contours.

[0064] The high-pass image Hi12 shown in FIG. 6E is a composite image generated using the composite mode 15. The composite mode 15 is composed of a first spatial frequency characteristic having a directivity with a passband having an azimuth angle of 5 / 6π as the main direction and a bandwidth of π, and a second spatial frequency characteristic having a directivity with a passband having an azimuth angle of -7 / 6π as the main direction and a bandwidth of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic have a high passband having a radial spatial frequency of from frequency 2 to frequency 3 as the passband, and the other radial spatial frequencies as the cutoff band. The high-pass image Hi12 shows a pattern similar to the low-pass image Li12 having a common azimuth spatial frequency of the passband, rather than the high-pass image Hi11 having a common radial spatial frequency of the passband of the mask used for generation. The high-pass image Hi12 shows a finer pattern than the low-pass image Li12.

[0065] The composite image Si12 shown in FIG. 6F is a composite image generated using the composite mode 16. The composite mode 16 is composed of a first spatial frequency characteristic having a directivity with an azimuth angle of 7 / 6π as the main direction and a passband with a bandwidth of π, and a second spatial frequency characteristic having a directivity with an azimuth angle of -5 / 6π as the main direction and a passband with a bandwidth of π. However, the first spatial frequency characteristic and the second spatial frequency characteristic are not band-limited in the radial spatial frequency domain. That is, the composite mode 16 has both the passband of the composite mode 14 and the passband of the composite mode 15.

[0066] In the directivity patterns shown in Fig. 5 and Fig. 6A-6F, the passbands of the masks do not overlap in part or in whole, but this is not limiting. The passbands of at least two of the masks may overlap in part. Moreover, the entire passbands of the masks do not necessarily have to cover the entire spatial frequency band.

[0067] Although FIGS. 3 to 6F illustrate the case where the observation target plane is a front surface perpendicular to the depth direction of the sample Sm, the present invention is not limited to this as long as the plane crosses a part or the whole of the observation target region. Therefore, the output processing unit 180 may be configured to be able to set the direction of the observation target plane by a user's operation. More specifically, an operation signal indicating the direction of the observation target plane is input from an operation input unit (not shown), and a control signal indicating the direction of the observation target plane indicated by the operation signal is output to the detection signal acquisition unit 130. The detection signal acquisition unit 130 determines the observation target plane in the direction indicated by the control signal input from the output processing unit 180.

[0068] 7A to 7C show display examples in which the observation target plane is a cross section parallel to the depth direction of the sample Sm. 7A is an OCT image showing the state of the sample Sm in a cross section parallel to the depth direction. The cross-sectional OCT image Cs21 shows a different pattern from the front OCT image Oi01. A composite image Si21 illustrated in FIG. 7B is a composite image generated using a composite mode 13 for an OCT signal related to a cross-sectional OCT image Cs21. The diagonally shaded areas indicate areas commonly extracted using the masks having the first directivity and the second directivity constituting the composite mode 13. The areas shaded diagonally upward to the right and the areas shaded diagonally downward to the right indicate areas extracted by the masks having the first directivity and the second directivity, respectively, but not extracted by the masks having the second directivity and the first directivity. The uncolored areas indicate areas not extracted by either the masks having the first directivity or the masks having the second directivity. 7C is a composite image generated for an OCT signal related to the cross-sectional OCT image Cs21 by using a composite mode 16. The composite image Si22 shows a pattern different from that of the composite image Si21.

[0069] Next, another example of the directional mask will be described. The directional mask may have a narrower bandwidth of the azimuth spatial frequency. The synthesis mode of the directional mask Dm31 illustrated in FIG. 8 has two passbands. The bandwidth of the two passbands is π / 9. However, the main directions of the passbands are azimuth angles π / 2 and -π / 2, respectively, and are directed in opposite directions. By using a directional mask with a narrow bandwidth, it is possible to extract the azimuth spatial frequency components around the main direction as the target direction. Such a directional mask can be applied to the analysis of the structure of a sample.

[0070] The masking unit 150 rotates the main direction of the passband in the azimuth angle direction at intervals of a small angle (for example, 1° to 5°) and applies a directional mask to the pupil plane electric field data for each rotated main direction to generate masked electric field data. The conversion unit 160 performs a two-dimensional inverse Fourier transform on each generated masked electric field data to generate a masked OCT signal in the spatial domain. The control unit 110 may include an analysis unit (not shown) that analyzes the change characteristics of the intensity of the masked OCT signal for each rotated main direction or part or all of the masked image data based on the masked OCT signal.

[0071] 9 and 10 show the dependence of signal intensity on the main direction obtained by using bovine Achilles tendon and chicken breast muscle as samples, respectively. The vertical and horizontal axes indicate signal intensity and main direction, respectively. The total pixel intensity of the mask image data was used as the signal intensity. The pixel intensity corresponds to the sum of the signal values ​​of each pixel. 9 and 10 show that the signal intensity depends on the main direction and has a significant maximum value. In the examples shown in Figs. 9 and 10, the main directions with the maximum values ​​are 152° and 116°, respectively. The maximum value of the signal intensity is about four times the average value. This suggests that the tissue structure of the sample has high periodicity with respect to the main direction in which the signal intensity has the maximum value.

[0072] This is also supported by the following simulation. In the simulation, it was assumed that light rays from the z direction are incident on the sample model Sp shown in FIG. 11. The surface of the sample model Sp has a triangular wave whose height in the z direction does not depend on the y direction and has a constant period in the x direction. Then, OCT volumes were synthesized for each of the cases where the z coordinate of the sample model Sp is farther away (Δz>0) from the reference point (the focal point of the optical system, which corresponds to the focal point of the collimator 50b in FIG. 1) and close thereto (Δz<0). A directional mask was applied to pupil plane electric field data based on the OCT image signals of the front image and cross-sectional image obtained from each OCT volume to generate mask electric field data for each passband. However, as the directional mask, a synthesis mode 32 (FIG. 12A) was used that has a pass region in which the main direction is π and the bandwidth is π, and a pass region in which the main direction is 0 and the bandwidth is π. Then, a two-dimensional inverse Fourier transform was performed on the mask electric field data generated for each passband, and OCT image data showing one composite image was generated using the mask OCT signal in the spatial domain.

[0073] 12A and 12B show a composite image for a front image and a composite image for a cross-sectional image, respectively, generated on the assumption that Δz>0. Both Figs. 12A and 12B show a pattern according to the period of height in the x direction. However, in Fig. 12A, the brightness is constant in the y direction, and in the x direction, there are parts in the negative direction and the positive direction from the center of each period that are brighter than the surroundings. The negative direction and the positive direction from the center of each period correspond to passbands with π and 0 as the main directions, respectively. In Fig. 12B, there is a part in proportion to the height in the z direction on the surface of the sample model Sp that is brighter than the surroundings. Figs. 12C and 12D show a composite image for a front image and a composite image for a cross-sectional image, respectively, generated on the assumption that Δz<0. The patterns of the composite images shown in Figs. 12C and 12D are symmetrical to the patterns of the composite images shown in Figs. 12A and 12B, respectively.

[0074] Therefore, it is confirmed that the bovine Achilles tendon and chicken pectoral muscle used as samples in the examples of Figures 9 and 10 have a structure in which the height of the interface has high periodicity in one direction (x direction) intersecting with the height direction (z direction) and low dependency on the other direction (y direction) (see Figure 13). Furthermore, the images of the planes tilted in the negative direction of the period and the images of the planes tilted in the positive direction relative to the z direction are associated with the main directions of the individual passbands and are displayed in the composite image with different patterns.

[0075] In the above example, the number M of one directional mask is mainly one or two, but this is not limited. The number M of masks may be three or more, for example, 60, 120, 360, etc. As the number M of masks increases, the bandwidth of the passband related to each mask may become finer. The bandwidth may be fixed or variable not only for the azimuth spatial frequency but also for the radial spatial frequency or the pair of the azimuth spatial frequency and the radial spatial frequency, and may be further subdivided. The number of masks may be large enough that the individual passbands can be considered to be spatially continuous. The passbands of each of the M masks may not necessarily be distributed periodically, regularly, or comprehensively in the spatial frequency domain, but may be distributed randomly or intermittently.

[0076] In the above example, the mask value for each spatial frequency corresponding to each frequency sample point constituting the mask is either 0 or 1, but this is not limited to this. The mask value may be any real number. This allows the degree of passage of the pupil plane electric field for each spatial frequency to be set in more detail. For example, if the mask value in the passband is a value larger than the mask value in the stopband, the mask dependency of the masked OCT signal can be emphasized or alleviated rather than being limited to 1 and 0, respectively. In addition, by setting the mask value so that it changes monotonically between sample points corresponding to spatial frequencies within a predetermined range from the boundary between the passband and the stopband, the change in the mask value according to the spatial frequency around the boundary can be made more gradual. Therefore, it is possible to alleviate abnormal spatial changes in abnormal brightness of the masked OCT image caused by aliasing noise that may occur in the masked OCT signal.

[0077] The mask value for each spatial frequency may be a complex number, not limited to a real number. For example, the mask value may be a value obtained by further dividing the above mask value by a coefficient (complex number) indicating the phase and amplitude for each spatial frequency that represents the aberration occurring in the optical system and the measured sample. By using such a mask value, the aberration occurring in the optical system and the measured sample is compensated for. The generation of masked electric field data and therefore masked OCT signals using the mask described above can be realized by arithmetic of complex numbers in the spatial frequency domain, which is impossible to achieve using existing hardware such as optical components or image processing that handles real luminance or color signal values.

[0078] The control unit 110 may further include an analysis unit (not shown) that analyzes the spatial frequency characteristics of the OCT signal. The analysis unit may, for example, perform the processes of steps S01-S03 using a mask that has an azimuth spatial frequency band of a predetermined bandwidth as a pass band, calculate the power of the generated masked OCT signal, and repeat the process of changing the main direction of the pass band to be processed. In this way, the analysis unit can obtain the azimuth spatial frequency dependency of the power and identify the azimuth spatial frequency at which the power is maximum, maximal, minimum, or minimal. The analysis of the signal intensity illustrated in FIG. 9 and FIG. 10 using the directional mask illustrated in FIG. 8 corresponds to an example of the analysis of the azimuth spatial frequency. The analysis unit may calculate the power of the masked OCT signal for each passband determined by the radial spatial frequency band instead of or together with the azimuthal spatial frequency band, thereby obtaining the spatial frequency dependence of the power and identifying the spatial frequency at which the power is maximum, maximal, minimum, or minimal. The analysis unit may perform the above-mentioned power calculation process for each part of the observation plane indicated by the OCT signal, rather than for the entire observation plane. The analysis unit can obtain the spatial frequency dependence of the power for each part, and specify the spatial frequency at which the power is maximum, maximal, minimum, or minimal for each part. The analysis unit can also determine the range of power for each possible spatial frequency for each part, and specify the part where the spatial frequency dependence is more significant than a predetermined range. The output processing unit 180 may output at least one of the power calculated by the analysis unit, the identified spatial frequency, or information indicating the part to a display unit or other device in response to an operation signal input from the operation input unit.

[0079] As described above, the image processing device 100 according to the above-mentioned embodiment includes an electric field estimation unit 140 that converts an optical coherence tomography signal representing the state of the sample into a spatial frequency domain to generate electric field data indicating the electric field on a pupil plane corresponding to the reciprocal space of the sample. The image processing device 100 also includes a masking unit 150 that applies a mask indicating the transmission characteristic distribution of the electric field on the pupil plane to the electric field data to generate masked electric field data, and a conversion unit 160 that converts the masked electric field data into a spatial domain to generate a masked optical coherence tomography signal. With this configuration, by applying a mask to the electric field on the pupil plane, a masked optical tomographic signal reflecting the transmission characteristic distribution indicated by the mask can be acquired. Furthermore, the image synthesis unit 170 can acquire a masked optical tomographic image by converting the signal value of the acquired masked optical tomographic signal into a brightness value. Therefore, even if the optical system does not actually include various optical components, the transmission characteristic distribution of the electric field according to the spatial conditions related to the observation of the sample can be virtually adjusted by the action of the mask.

[0080] The mask transmission characteristic distribution may also indicate the azimuthal spatial frequency band passing through the electric field in the pupil plane. With this configuration, the spatial frequency band of the azimuth angle that is the pass band of the electric field is specified as the pass characteristic distribution of the mask, so that the observation of the sample with the directivity of the observation direction according to the pass band is realized. In addition, by acquiring a mask image showing the structure of the sample based on the mask, the constraints on the observation direction can be eliminated or relaxed, and the anisotropy of the fine structure of the tissue that constitutes the sample can be visualized.

[0081] The mask transmission characteristic distribution may also indicate a radial spatial frequency band passing through the electric field in the pupil plane. With this configuration, the radial spatial frequency band that is the pass band of the electric field is specified as the transmission characteristic distribution of the mask, so that observation of the sample with a spatial resolution according to the pass band is realized.

[0082] The mask may also indicate the presence or absence of passage of an electric field for each sample point corresponding to a spatial frequency. With this configuration, the spatial frequency band through which the pupil plane electric field passes is defined as digital data, so that calculations relating to the action of the mask can be performed simply.

[0083] The image processing device 100 according to the present embodiment may also include an image synthesis unit 170 that synthesizes mask images based on the M masked optical coherence tomography signals in different display modes to generate a synthetic image. However, the mask unit 150 applies M masks with different transmission characteristic distributions to the electric field data to obtain M masked electric field data. The conversion unit 160 converts the M masked electric field data into the spatial domain to generate M masked optical coherence tomography signals. With this configuration, a user who sees the composite image can intuitively distinguish between the individual mask images based on the display state of the composite image. By comparing the individual mask images, the user can intuitively grasp the dependency of the sample state on the transmission characteristic distribution (e.g., directivity).

[0084] As a display mode, the M mask images may each be represented by a different hue. With this configuration, the user can intuitively identify individual mask images based on the colors that appear in the composite image, and can easily grasp the areas that reveal the state of the sample corresponding to the transmission characteristic distribution (e.g., directivity).

[0085] Moreover, the color obtained by combining the colors representing the M mask images among the M mask images is an achromatic color. With this configuration, the user can identify areas where the state of the sample according to the pass characteristic distribution (e.g., directivity) corresponding to the color appearing in the composite image is more pronounced than other transmission characteristic distributions, and can recognize the state of the sample independent of the pass characteristic distribution from the shading.

[0086] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention.

[0087] For example, in the above description, the image processing device 100 is a part of the optical coherence tomography device 1, but the present invention is not limited to this. The image processing device 100 may be a single device independent of the optical coherence tomography device 1 and not equipped with an optical system. In this case, the optical system control unit 120 may be omitted from the control unit 110 of the image processing device 100. The detection signal acquisition unit 130 is not limited to an optical system, and may acquire detection signals or OCT signals from other devices such as a data storage device or a PC in a wired or wireless manner, for example, via a network.

[0088] Furthermore, the image processing device 100 may include the above-mentioned operation input unit and display unit, or one or both of them may be omitted. In the control unit 110 of the image processing device 100, one or both of the image synthesis unit 170 and the output processing unit 180 may be omitted. When the image synthesis unit 170 is omitted, the conversion unit 160 may output the generated mask OCT signal to another device, such as a data storage device, a PC, or another image processing device. The device as the output destination may have the same function as the image synthesis unit 170, that is, a function of generating output image data based on the mask OCT signal input from the image processing device 100 and displaying an image based on the generated output image data. The color system employed by the image synthesis unit 170 is not necessarily limited to the RGB color system, but may be another color system, for example, the YCbCr color system. The display mode of each of the M mask images is not limited to hue or gradation, and other methods such as patterns such as halftone dots, grids, and diagonal lines, and changes in luminance or hue over time such as blinking may be used.

[0089] In addition, when a part of the image processing device 100 in the above-mentioned embodiment, for example, all or a part of the control unit 110, is realized by a computer, a program for realizing this control function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read and executed by a computer system. In addition, the "computer system" referred to here is a computer system built into the image processing device 100, and includes hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, an optical magnetic disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" may also include a medium that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. In addition, the above-mentioned program may be a program for realizing a part of the above-mentioned function, and may further be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system.

[0090] Also, a part or all of the image processing device 100 in the above-mentioned embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the image processing device 100 may be individually made into a processor, or a part or all of the blocks may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. Also, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used. [Explanation of symbols]

[0091] 1...optical coherence tomography, 10...light source, 20...beam splitter, 30a, 30b, 50a, 50b...collimator, 40...reference mirror, 60a, 60b...galvanometer mirror, 70...spectroscope, 100...image processing device, 110...control unit, 120...optical system control unit, 130...detection signal acquisition unit, 140...electric field estimation unit, 150...mask unit, 160...conversion unit, 170...image synthesis unit, 180...output processing unit, 190...storage unit

Claims

1. an electric field estimating unit that converts an optical coherence tomography signal representing a state of a sample into a spatial frequency domain to generate electric field data representing an electric field in a pupil plane corresponding to a reciprocal space of the sample; a mask unit that applies a mask indicating a transmission characteristic distribution of the electric field on the pupil plane to the electric field data to generate masked electric field data; a conversion unit that converts the mask electric field data into a spatial domain to generate a mask optical coherence tomography signal; a synthesis unit that synthesizes mask images based on M (M is an integer equal to or greater than 2) mask optical coherence tomographic signals in different display modes to generate a synthetic image, the mask unit applies M masks having different transmission characteristic distributions to the electric field data to obtain M mask electric field data; The conversion unit converts each of the M mask electric field data into a spatial domain to generate the M mask optical coherence tomography signals. Image processing device.

2. The transmission characteristic distribution indicates a spatial frequency band of an azimuth angle passing through the electric field on the pupil plane. The image processing device according to claim 1 .

3. The transmission characteristic distribution indicates a spatial frequency band of a radius passing through the electric field on the pupil plane.

3. The image processing device according to claim 1 or 2.

4. The mask indicates the presence or absence of passage of the electric field for each sample point corresponding to a spatial frequency.

4. The image processing device according to claim 2.

5. The mask has a mask value that changes monotonically between sample points within a predetermined range from the boundary between a passing region that passes through the electric field and a blocking region that does not pass through the electric field. The image processing device according to claim 4.

6. a synthesis unit that generates a mask image showing a structure of the sample based on the mask optical coherence tomography signal; The image processing device according to claim 1 .

7. The display mode is such that the hue of the color representing each of the M mask images is different. The image processing device according to claim 1 .

8. The color obtained by synthesizing the colors representing the M mask images as the display mode is an achromatic color. The image processing device according to claim 7.

9. 1. A method in an image processing device, comprising: an electric field estimation process for converting an optical coherence tomography signal representing a state of a sample into a spatial frequency domain to generate electric field data indicative of an electric field in a pupil plane corresponding to a reciprocal space of the sample; a masking step of applying a mask indicating a transmission characteristic distribution of the electric field on the pupil plane to the electric field data to generate masked electric field data; a transforming step of transforming the mask electric field data into a spatial domain to generate a mask optical coherence tomography signal; a synthesis process of synthesizing mask images based on M (M is an integer equal to or greater than 2) mask optical coherence tomographic signals in different display modes to generate a synthetic image, In the masking step, M masks having different transmission characteristic distributions are applied to the electric field data to obtain M mask electric field data; In the converting step, the M mask electric field data are converted into a spatial domain to generate the M mask optical coherence tomography signals. Image processing methods.

10. The computer of the image processing device an electric field estimation step of converting an optical coherence tomography signal representative of a state of a sample into a spatial frequency domain to generate electric field data indicative of an electric field in a pupil plane corresponding to a reciprocal space of the sample; a masking step of applying a mask indicating a transmission characteristic distribution of the electric field on the pupil plane to the electric field data to generate masked electric field data; a transforming step of transforming the mask electric field data into a spatial domain to generate a masked optical coherence tomography signal; a synthesis process of synthesizing mask images based on M (M is an integer equal to or greater than 2) mask optical coherence tomographic signals in different display modes to generate a synthetic image; A program for executing In the masking step, M masks each having a different transmission characteristic distribution are applied to the electric field data to obtain M mask electric field data; In the transforming step, the M mask electric field data are transformed into the spatial domain to generate the M mask optical coherence tomography signals. program.

Images