Image processing device, image processing system, image processing method, and program

Abstract

To provide an image processing device capable of estimating depth information of tissues imaged by optical coherence tomography.SOLUTION: An image processing device is provided, comprising an image acquisition unit for acquiring a tomographic image captured by optical coherence tomography, a center emphasizing unit configured to generate a center-emphasized image that emphasizes the center of a tissue captured in the tomographic image using a template image including a shape of a given tissue, and an estimation unit for estimating depth information of the tissue based on the center-emphasized image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device, an image processing system, an image processing method, and a program. [Background technology]

[0002] Techniques for analyzing the three-dimensional structure of blood vessels based on tomographic images captured by optical coherence tomography (OCT) are known. For example, Non-Patent Document 1 discloses a technique for removing noise due to body movement by using region growing from the results of vascular enhancement of OCT tomographic images. Furthermore, for example, Non-Patent Document 2 discloses a technique for estimating the three-dimensional structure of blood vessels from ordinary vascular-enhanced OCT tomographic images without using a contrast agent, based on a trained model that is deep-learned from training data created using vascular-enhanced OCT tomographic images captured using a contrast agent as a reference. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Okamura, Chuuki, Okamoto, Takayuki, and Haneishi, Hideaki, "Extraction of 3D vascular structures from optical coherence tomography images," Proceedings of the 41st Annual Meeting of the Japan Society for Medical Imaging Technology, 2022 [Non-patent document 2] S. Stefan, et al., "Deep learning toolbox for automated enhancement, segmentation, and graphing of cortical optical coherence tomography microangiograms", Biomedical Optics Express, vol. 11, pp. 7325-7342, 2020. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the accuracy of estimating the depth information of blood vessels in conventional techniques can be improved. For example, in optical coherence tomography, artifacts occurring in the depth direction from blood vessels make it difficult to analyze the three-dimensional structure of blood vessels in detail.

[0005] An object of one aspect of the present disclosure is to provide an image processing device capable of estimating depth information of tissue imaged by optical coherence tomography. [Means for solving the problem]

[0006] An image processing device according to one aspect of the present disclosure includes an image acquisition unit that acquires a tomographic image captured by optical coherence tomography, a center emphasis unit that generates a center emphasis image that emphasizes the center of the tissue captured in the tomographic image using a template image including a predetermined tissue shape, and a depth estimation unit that estimates depth information of the tissue based on the center emphasis image. [Effects of the Invention]

[0007] According to one aspect of the present disclosure, depth information of tissue imaged by optical coherence tomography can be estimated. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example of the overall configuration of an image processing system. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a computer. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of the image processing apparatus. [Figure 4] FIG. 10 is a diagram illustrating an example of a template image. [Figure 5] 1 is a flowchart illustrating an example of an image processing method. [Figure 6] FIG. 10 is a diagram illustrating an example of a tomographic image. [Figure 7] FIG. 10 is a diagram illustrating an example of a planar image. [Figure 8]FIG. 10 is a diagram illustrating an example of a centerline image. [Figure 9] FIG. 10 is a diagram illustrating an example of an intensity profile. [Figure 10] FIG. 10 is a diagram illustrating an example of a depth profile. [Figure 11] FIG. 10 is a diagram showing an example of an estimation result according to the prior art. [Figure 12] FIG. 10 is a diagram illustrating an example of an estimation result according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0010] [Embodiment] One embodiment of the present invention is an image processing system that visualizes the three-dimensional structure of a predetermined tissue based on a tomographic image captured by optical coherence tomography (hereinafter also referred to as an "OCT image"). In this embodiment, the tomographic image may be a tomographic image captured by optical coherence tomography angiography (OCTA) (hereinafter also referred to as an "OCTA image").

[0011] The tissue to be analyzed may be, for example, microcirculation. Microcirculation includes blood vessels such as arterioles, capillaries, and venules. Activities important for maintaining life take place in the microcirculation. Examples of activities that take place in the microcirculation include the exchange of nutrients, metabolic waste products, and thermal energy. Visualizing the structure of the microcirculation is useful, for example, for understanding the state of a disease or elucidating its mechanism.

[0012] It is known that the three-dimensional structure of the microcirculation can be analyzed using optical coherence tomography (OCT). OCT is a technique for capturing cross-sectional images of a subject using the coherence of light. OCT has a high spatial resolution of several micrometers and can capture images non-invasively. OCT is widely used in medical fields such as ophthalmology, dermatology, and dentistry.

[0013] OCTA is a technology that enhances the contrast of blood vessel regions based on multiple OCT images taken at different times from the same location. OCTA captures multiple frames of OCT images at short time intervals and quantifies the changes in pixel values ​​between frames caused by the movement of red blood cells. OCTA makes it possible to image the three-dimensional structure of blood vessels, including microcirculation.

[0014] However, OCTA images may contain artifacts (hereinafter referred to as "tailing artifacts") that arise from blood vessels in the depth direction. Therefore, in analyses based on OCTA images, the influence of the tailing artifacts makes it difficult to analyze three-dimensional structures such as blood vessel diameters or overlapping blood vessels in detail.

[0015] In recent years, advances in deep learning have led to the proposal of a technology for estimating the three-dimensional structure of blood vessels based on trained deep learning models. However, building a deep learning model requires training data with correct labels. In particular, building a versatile and highly accurate deep learning model requires a large amount of diverse training data. Furthermore, the assignment of correct labels is often performed manually. Furthermore, as mentioned above, it is difficult to visually identify the position of blood vessels in OCTA images due to artifacts, making it difficult to create correct labels. Therefore, preparing training data is extremely difficult, and even if it were possible, it would require a huge amount of work.

[0016] This embodiment aims to estimate depth information of blood vessels captured in an OCTA image. To this end, this embodiment generates a center-weighted image that emphasizes the center of the blood vessel captured in the OCTA image using a template image including the shape of the blood vessel, and estimates depth information of the blood vessel based on the center-weighted image. In one aspect, this embodiment can suppress the influence of tailing artifacts, thereby enabling estimation of blood vessel depth information. In another aspect, this embodiment does not require training data, allowing efficient estimation of blood vessel depth information.

[0017] The tissue to be analyzed is not limited to blood vessels, and in this embodiment, any tissue that can be imaged by OCT can be used as the analysis target.

[0018] <Overall structure> The overall configuration of an image processing system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of an image processing system.

[0019] 1, the image processing system 1000 includes an imaging device 10 and an image processing device 20. The imaging device 10 and the image processing device 20 are connected to each other so as to be able to communicate data with each other via a communication network N such as a LAN (Local Area Network) or the Internet.

[0020] The imaging device 10 is an example of an optical device that captures a tomographic image of a subject. The subject may be any object having tissue to be analyzed. For example, the subject may be a body part such as an eyeball, skin, teeth, or gums. For example, the tissue to be analyzed may be tissue such as blood vessels or nerves. In this embodiment, the tissue to be analyzed is blood vessels.

[0021] The imaging device 10 outputs a tomographic image including a plurality of OCT images obtained by capturing a predetermined range of a subject at short time intervals. The imaging device 10 may generate an OCTA image based on a plurality of OCT images obtained by capturing a predetermined range of a subject at short intervals, and output a tomographic image including the plurality of OCTA images.

[0022] The imaging method of the imaging device 10 may be selected arbitrarily. For example, the imaging device 10 may perform imaging using the SD-OCT (Spectral Domain - OCT) method. For another example, the imaging device 10 may perform imaging using the SS-OCT (Swept Source - OCT) method.

[0023] The image processing device 20 is an example of an information processing device such as a personal computer, workstation, or server that generates image data showing the three-dimensional structure of blood vessels based on tomographic images captured by the imaging device 10. In this embodiment, the three-dimensional structure of blood vessels is a three-dimensional structure formed by connecting the center lines of the blood vessels. The image processing device 20 may acquire multiple OCT images output from the imaging device 10 and generate an OCTA image. If the imaging device 10 is capable of outputting an OCTA image, the image processing device 20 may acquire the OCTA image from the imaging device 10.

[0024] The image processing device 20 estimates the centerline of the blood vessel and depth information of the centerline of the blood vessel based on the OCTA image. The image processing device 20 generates image data showing the three-dimensional structure of the blood vessel based on the estimation result. The image processing device 20 may generate a three-dimensional image that allows the three-dimensional structure of the blood vessel to be observed from any viewpoint. The image processing device 20 may also generate one or more two-dimensional images that observe the three-dimensional structure of the blood vessel from a predetermined viewpoint.

[0025] The overall configuration of the image processing system 1000 shown in FIG. 1 is one example, and various system configuration examples are possible depending on the application and purpose. For example, the image processing system 1000 may include multiple imaging devices 10 and one or more image processing devices 20. For example, the image processing device 20 may be realized by multiple computers, or may be realized as a cloud computing service. For example, the image processing system 1000 may be realized by a standalone computer. The classification of devices such as the imaging device 10 and the image processing device 20 shown in FIG. 1 is one example.

[0026] <Hardware configuration> The image processing device 20 in this embodiment may be realized by, for example, a computer. Fig. 2 is a block diagram showing an example of the hardware configuration of a computer.

[0027] 2, the computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, a HDD (Hard Disk Drive) 504, an input device 505, a display device 506, a communication I / F (Interface) 507, and an external I / F 508. The CPU 501, the ROM 502, and the RAM 503 form a so-called computer. The hardware components of the computer 500 are connected to each other via a bus line 509. The input device 505 and the display device 506 may be connected to the external I / F 508 for use.

[0028] The CPU 501 is a computing device that reads programs and data from a storage device such as the ROM 502 or the HDD 504 onto the RAM 503 and executes the processes, thereby realizing the overall control and functions of the computer 500. The computer 500 may have a GPU (Graphics Processing Unit) in addition to or instead of the CPU 501.

[0029] The ROM 502 is an example of a non-volatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 502 functions as a main storage device that stores various programs, data, etc. required for the CPU 501 to execute various programs installed in the HDD 504. Specifically, the ROM 502 stores boot programs such as a Basic Input / Output System (BIOS) and an Extensible Firmware Interface (EFI) that are executed when the computer 500 starts up, as well as data such as OS (Operating System) settings and network settings.

[0030] The RAM 503 is an example of a volatile semiconductor memory (storage device) in which programs and data are erased when the power is turned off. The RAM 503 is, for example, a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 503 provides a working area in which various programs installed in the HDD 504 are expanded when executed by the CPU 501.

[0031] The HDD 504 is an example of a non-volatile storage device that stores programs and data. The programs and data stored in the HDD 504 include an OS, which is basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that the computer 500 may use a storage device that uses flash memory as a storage medium (e.g., an SSD (Solid State Drive)) instead of the HDD 504.

[0032] The input device 505 includes a touch panel, operation keys and buttons, a keyboard and mouse, a microphone for inputting sound data such as voice, and the like, which are used by the user to input various signals.

[0033] The display device 506 is configured with a display such as a liquid crystal display or organic EL (Electro-Luminescence) display for displaying a screen, a speaker for outputting sound data such as voice, and the like.

[0034] The communication I / F 507 is an interface that connects to a communication network and enables the computer 500 to perform data communication.

[0035] The external I / F 508 is an interface with external devices, such as a drive device 510.

[0036] The drive device 510 is a device for loading a recording medium 511. The recording medium 511 here includes media that record information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, or a magneto-optical disk. The recording medium 511 may also include semiconductor memories that record information electrically, such as ROMs and flash memories. This allows the computer 500 to read from and / or write to the recording medium 511 via the external I / F 508.

[0037] The various programs to be installed in the HDD 504 are installed, for example, by setting the distributed recording medium 511 in a drive device 510 connected to the external I / F 508 and reading out the various programs recorded on the recording medium 511 by the drive device 510. Alternatively, the various programs to be installed in the HDD 504 may be installed by being downloaded via the communication I / F 507 from a network different from the communication network.

[0038] <Functional configuration> The functional configuration of the image processing device 20 in this embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the functional configuration of the image processing device.

[0039] As shown in FIG. 3 , the image processing device 20 includes an image acquisition unit 201, a center line estimation unit 202, a path division unit 203, an intensity analysis unit 204, a template storage unit 205, a center emphasis unit 206, a depth estimation unit 207, and an image generation unit 208.

[0040] The image acquisition unit 201, the center line estimation unit 202, the path division unit 203, the intensity analysis unit 204, the center emphasis unit 206, the depth estimation unit 207, and the image generation unit 208 are realized, for example, by processing executed by the CPU 501 of a program loaded from the HDD 504 shown in FIG. 2 onto the RAM 503.

[0041] The template storage unit 205 is realized by, for example, the RAM 503 or the HDD 504 shown in FIG.

[0042] The image acquisition unit 201 acquires a tomographic image captured by the imaging device 10. For example, the image acquisition unit 201 may acquire the tomographic image by receiving the tomographic image output from the imaging device 10. The image acquisition unit 201 may acquire the tomographic image by reading the tomographic image from a portable storage medium on which the tomographic image captured by the imaging device 10 is stored.

[0043] When the acquired tomographic images are multiple OCT images, the image acquisition unit 201 generates an OCTA image based on those OCT images. As an example, the image acquisition unit 201 may generate the OCTA image according to the SV (Speckle Variance) method. The SV method is a technique for generating an image that shows the variance of pixel values ​​between OCT images based on multiple OCT images captured at short time intervals of the same cross section. Details of the SV method are disclosed in Reference 1 below.

[0044] [Reference 1] A. Marimpillai, et al., "Speckle variance detection of microvasculature using swept-source optical coherence tomography", Optics Letters, vol. 33, no. 13, pp. 1530-1532, 2008.

[0045] The centerline estimation unit 202 estimates the centerline of the blood vessel as seen from the surface of the subject based on the tomographic image acquired by the image acquisition unit 201. First, the centerline estimation unit 202 generates a planar image by projecting the tomographic image in the depth direction. As an example, the planar image may be a maximum value projection image by projecting the maximum value on the Z axis onto each pixel in the XY plane of the tomographic image. The X and Y axes are two orthogonal axes that are horizontal to the surface of the subject, and the Z axis is an axis perpendicular to the XY plane.

[0046] Next, the centerline estimation unit 202 estimates the centerline of the blood vessel by performing image analysis on the planar image. For example, the centerline estimation unit 202 may estimate the centerline by performing thinning processing on the planar image. For example, the centerline estimation unit 202 may perform graph cut on the planar image as preprocessing for the thinning processing. Graph cut is a semi-automatic segmentation method that sets an optimal label for each pixel of an image. Details regarding graph cut are disclosed in Reference 2 below.

[0047] [Reference 2] Y. Boykov, V. Kolmogorov, "An experimental comparison of min-cut / max-flow algorithms for energy minimization in vision", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, pp. 1124-1137, 2004.

[0048] The center line of the blood vessel may be manually input by the user. In this case, the center line estimation unit 202 may output a planar image to the display device 506 and acquire the center line input by the user's operation. The center line estimation unit 202 may perform preprocessing such as thinning processing on the center line input by the user's operation.

[0049] The path dividing unit 203 divides the center line estimated by the center line estimation unit 202 into multiple paths. As one example, the path dividing unit 203 may divide the path at a branch point where one center line branches into multiple center lines. As another example, the path dividing unit 203 may divide the path at an intersection where two center lines with different positional relationships in the depth direction intersect. After the division, each path becomes a single blood vessel that does not include branches.

[0050] The intensity analysis unit 204 analyzes pixel intensities of the tomographic image acquired by the image acquisition unit 201, and generates an image showing the distribution of pixel intensities (hereinafter also referred to as an "intensity distribution image"). The intensity analysis unit 204 may analyze pixel intensities for each cross section perpendicular to each path divided by the path division unit 203. As an example, the intensity analysis unit 204 may scan a predetermined circular kernel in the depth direction at the center of the cross section perpendicular to each path, and calculate the average value of pixel intensities within the circular kernel at each depth. The size of the circular kernel may be set arbitrarily depending on the thickness of the blood vessel to be analyzed.

[0051] As a preprocessing step for analyzing pixel intensities, the intensity analysis unit 204 may recognize the surface of the subject from the tomographic image and identify the entire tissue region including blood vessels. For example, the intensity analysis unit 204 may recognize the surface of the subject based on a trained machine learning model. For example, the machine learning model may be a region segmentation model that performs semantic segmentation. For example, the region segmentation model may be a convolutional neural network including a convolutional layer, or a fully convolutional network (FCN) consisting only of a convolutional layer.

[0052] The template storage unit 205 stores a template image in advance. The template image includes the shape of a blood vessel and may also include the shape of a tailing artifact occurring in the depth direction from the blood vessel. The template storage unit 205 may store a plurality of template images of different sizes. The template image may be a CG (Computer Graphics) model drawn to show typical shapes of blood vessels and tailing artifacts.

[0053] Fig. 4 is a diagram showing an example of a template image. As shown in Fig. 4, template image 400 includes a cylinder 401 with a diameter d and a height h, with the Z axis as the height direction, and a hemisphere 402 with a diameter d connected to the upper end of cylinder 401. Cylinder 401 is an example of a portion that represents the shape of a tailing artifact. Hemisphere 402 is an example of a portion that represents the shape of a blood vessel. The height h of cylinder 401 may be arbitrarily designed in consideration of estimation accuracy, etc.

[0054] The template storage unit 205 may store multiple template images 400 with different diameters d. The diameter d may be determined arbitrarily depending on the thickness of the blood vessel to be analyzed. The intervals between the multiple diameters d may be equal or may be different. As an example, the template storage unit 205 may store approximately six template images 400 with the diameters d set at equal intervals.

[0055] The center emphasis unit 206 uses the template image 400 read from the template storage unit 205 to generate an image (hereinafter also referred to as a "center emphasis image") in which the centers of blood vessels captured in the tomographic image acquired by the image acquisition unit 201 are emphasized. As an example, the center emphasis unit 206 may emphasize the centers of blood vessels by template matching. Specifically, the center emphasis unit 206 three-dimensionally scans the template image 400 in the tomographic image and calculates the similarity with the template image 400 at each coordinate of the tomographic image. As an example, the similarity may be zero-mean normalized cross-correlation (ZNCC).

[0056] When there are multiple template images, the center emphasis unit 206 may calculate the similarity of each coordinate using each template image 400 and integrate these similarities. The center emphasis unit 206 may integrate the similarities by calculating statistics of the similarities corresponding to each template image 400. The statistics may be, for example, the maximum value. Other examples of the statistics may include the sum, average, or median.

[0057] The depth estimation unit 207 estimates depth information of blood vessels captured in the tomographic image acquired by the image acquisition unit 201, based on the intensity distribution image generated by the intensity analysis unit 204 and the center-weighted image generated by the center-weighted unit 206. The depth estimation unit 207 may extract, from each of the intensity distribution image and the center-weighted image, images of cross sections of each path divided by the path division unit 203, and estimate depth information of blood vessels based on the extracted intensity distribution image and the center-weighted image. Hereinafter, the extracted intensity distribution image will be referred to as an "intensity profile." Furthermore, the extracted center-weighted image will be referred to as a "depth profile."

[0058] For example, the depth estimation unit 207 may estimate depth information by searching for a center line that minimizes a predetermined cost function using the intensity profile and depth profile extracted for each path. For example, the cost function may include the sum of pixel values ​​included in the center line in the intensity profile, the sum of pixel values ​​included in the center line in the depth profile, and the curvature of the center line. Specifically, the cost function may be a function E shown in Equation (1).

[0059]

number

[0060] where x is a set of pixels on the center line, S(x) is a spline curve of pixel set x, and E curv is the curvature of the spline curve S(x), and E octa is the sum of pixel values ​​of pixel set x in the intensity profile, and E vcw is the sum of pixel values ​​of pixel set x in the depth profile, and α, β, and γ are weighting coefficients.

[0061] The image generation unit 208 generates image data showing the three-dimensional structure of the blood vessel based on the center line estimated by the center line estimation unit 202 and the depth information estimated by the depth estimation unit 207. As an example, the image generation unit 208 may generate the image data by volume rendering.

[0062] <Image processing method> The image processing method executed by the image processing system 1000 in this embodiment will be described with reference to Figures 5 to 10. Figure 5 is a flowchart showing an example of the image processing method.

[0063] In step S1, the imaging device 10 captures OCT images of a predetermined number of cross sections within a predetermined range of the subject at short time intervals. The imaging device 10 outputs the captured OCT images. The OCT images output from the imaging device 10 are input to the image processing device 20.

[0064] In step S2, the image acquisition unit 201 of the image processing device 20 receives input of a plurality of OCT images output from the imaging device 10. Next, the image acquisition unit 201 generates a plurality of OCTA images based on the plurality of OCT images. Subsequently, the image acquisition unit 201 sends a tomographic image including the generated plurality of OCTA images to the center line estimation unit 202, the intensity analysis unit 204, and the center emphasis unit 206.

[0065] Fig. 6 is a diagram showing an example of a tomographic image. As shown in Fig. 6, the tomographic image 600 is three-dimensional data including a plurality of OCTA images 601 to 605 obtained by capturing an XZ cross section at equal intervals in the Y-axis direction. In Fig. 6, the tomographic image 600 is simplified to include five OCTA images 601 to 605, but the number of OCTA images included in the tomographic image 600 may be set according to the size of the subject. For example, the number of OCTA images included in the tomographic image 600 may be 1024.

[0066] Returning to FIG. 5, in step S3, the centerline estimation unit 202 of the image processing device 20 receives the tomographic image from the image acquisition unit 201. Next, the centerline estimation unit 202 generates a planar image by projecting the tomographic image in the depth direction. Subsequently, the centerline estimation unit 202 estimates the centerline of the blood vessel by performing image analysis on the planar image. The centerline estimation unit 202 sends the centerline image indicating the centerline of the blood vessel to the path division unit 203.

[0067] 7 is a diagram showing an example of a planar image 610. As shown in Fig. 7, the planar image 610 is a maximum value projection image in which the maximum value on the Z axis is projected onto each pixel on the XY plane of the tomographic image 600.

[0068] 8 is a diagram showing an example of a centerline image 620. As shown in FIG. 8, a centerline image 620 shows the centerlines of blood vessels projected onto the planar image 610.

[0069] Returning to FIG. 5, the explanation will be given. In step S4, the path dividing unit 203 of the image processing device 20 receives the centerline image from the centerline estimation unit 202. Next, the path dividing unit 203 divides the centerlines of the blood vessels shown in the centerline image at branching points or intersections. The path dividing unit 203 sends path information indicating each divided path to the intensity analysis unit 204. The path information may be, for example, information that associates, for each path, identification information that identifies the path and each coordinate from the start point to the end point of the path. Note that the coordinates indicated in the path information are two-dimensional coordinates on the XY plane.

[0070] In step S5, the intensity analysis unit 204 of the image processing device 20 receives a tomographic image from the image acquisition unit 201. The intensity analysis unit 204 also receives path information from the path division unit 203. Next, the intensity analysis unit 204 extracts cross sections orthogonal to each path indicated in the path information from the tomographic image. Subsequently, the intensity analysis unit 204 analyzes pixel intensities in the extracted cross sections. As a result, an intensity distribution image showing the distribution of pixel intensities in the tomographic image is generated. The intensity analysis unit 204 sends the intensity distribution image and path information to the depth estimation unit 207.

[0071] In step S6, the center emphasis unit 206 of the image processing device 20 receives the tomographic image from the image acquisition unit 201. Next, the center emphasis unit 206 reads out a plurality of template images from the template storage unit 205. Subsequently, the center emphasis unit 206 calculates the similarity between each of the read out plurality of template images and the template image at each coordinate by template matching. Then, the center emphasis unit 206 integrates the similarities calculated for each template image at each coordinate. As a result, a center emphasis image is generated in which the centers of the blood vessels captured in the tomographic image are emphasized. The center emphasis unit 206 sends the center emphasis image to the depth estimation unit 207.

[0072] In step S7, the depth estimation unit 207 of the image processing device 20 receives the intensity distribution image and path information from the intensity analysis unit 204. The depth estimation unit 207 also receives the center-emphasized image from the center emphasis unit 206.

[0073] Next, the depth estimation unit 207 extracts from the intensity distribution image an intensity profile indicating the relationship between depth and pixel intensity in a cross section perpendicular to the path indicated in the path information, and also extracts from the center-emphasized image a depth profile indicating the relationship between depth and similarity with the template image in a cross section perpendicular to the path indicated in the path information.

[0074] Next, the depth estimation unit 207 searches for a centerline that minimizes the cost function E shown in equation (1) for each path indicated in the path information, using the intensity profile and the depth profile. Next, the depth estimation unit 207 calculates depth information of the centerline of the blood vessel based on the searched centerline. The depth estimation unit 207 sends the calculated depth information of each path to the image generation unit 208.

[0075] FIG. 9 is a diagram showing an example of an intensity profile. The intensity profile is two-dimensional data showing the intensity of pixel values ​​in the VX plane. The intensity of pixel values ​​may be the average value of pixel values ​​within a circular kernel scanned in the Z-axis direction. The V-axis is the axis corresponding to the direction of the path. Therefore, the intensity profile is information showing the relationship between depth and pixel intensity in a cross section of a tomographic image that is perpendicular to the blood vessels.

[0076] FIG. 10 is a diagram showing an example of a depth profile. The depth profile is two-dimensional data indicating the degree of similarity with a template image in the VX plane. The template image shows the shape of blood vessels and the shape of tailing artifacts. Therefore, the depth profile is information indicating the relationship between depth and pixel intensity in a cross section of a tomographic image that is perpendicular to the blood vessels.

[0077] 5, in step S8, the image generation unit 208 of the image processing device 20 receives the centerline image from the centerline estimation unit 202. The image generation unit 208 also receives depth information from the depth estimation unit 207.

[0078] Next, the image generation unit 208 generates image data showing the three-dimensional structure of the blood vessels by volume rendering based on the centerline image and the depth information. The image generation unit 208 displays the three-dimensional structure of the blood vessels on the display device 506 of the image processing device 20 based on the generated image data.

[0079] <Estimation accuracy> The estimation accuracy by the image processing system 1000 in this embodiment will be described with reference to Fig. 11 and Fig. 12. Fig. 11 is a diagram showing an example of an estimation result by the conventional technology. Fig. 12 is a diagram showing an example of an estimation result by this embodiment.

[0080] In Figures 11 and 12, the dorsal dermis of a 10-week-old male mouse was imaged. Cross-sectional images were taken using the SD-OCT method. The image size was 1024 x 1024 pixels, and 1024 cross sections were captured across the entire imaging range. Five OCT images were taken at different times for each of the 1024 cross sections, resulting in a total of 5120 OCT images.

[0081] 11 and 12 are both views of a three-dimensional display by volume rendering viewed vertically from above. As shown in FIG. 11, in the conventional technology, blood vessels are visualized as structures elongated in the depth direction due to the influence of tailing artifacts that occur in the depth direction from the blood vessels. On the other hand, as shown in FIG. 12, in the embodiment, the depth of the centerline of the blood vessel can be estimated, and the three-dimensional structure of the blood vessel can be visualized. As shown in FIGS. 11 and 12, according to the embodiment, the influence of tailing artifacts can be suppressed and the three-dimensional structure of the blood vessel can be visualized.

[0082] <Effects of the embodiment> The image processing device 20 in this embodiment acquires a tomographic image captured by optical coherence tomography, generates a center-weighted image that emphasizes the center of the tissue captured in the tomographic image using a template image including a predetermined tissue shape, and estimates depth information of the tissue based on the center-weighted image. By using the template image, the image processing device 20 can emphasize the center of the tissue captured in the tomographic image. In one aspect, this embodiment makes it possible to estimate depth information of the tissue captured by optical coherence tomography. In another aspect, this embodiment does not require training data, making it possible to efficiently estimate depth information of the tissue.

[0083] The tomographic image may be captured by optical coherence tomography angiography. The tissue may be a blood vessel. According to this embodiment, depth information of the blood vessel can be estimated.

[0084] The image processing device 20 may generate an intensity distribution image showing the distribution of pixel intensities of the tomographic image, and estimate depth information based on the intensity distribution image and the center-weighted image. According to this embodiment, depth information can be estimated based on the results of analyzing the tomographic image from two perspectives.

[0085] The image processing device 20 may estimate depth information by searching for the centerline of the tissue based on a predetermined cost function. The cost function may include the sum of pixel values ​​included in the centerline in the intensity distribution image, the sum of pixel values ​​included in the centerline in the center-weighted image, and the curvature of the centerline. According to this embodiment, the pixel values ​​and curvature of the image can be taken into consideration.

[0086] The image processing device 20 may extract a center-weighted image and an intensity distribution image in which a cross section of tissue is captured from the center-weighted image and the intensity distribution image, respectively, and estimate depth information based on the extracted center-weighted image and intensity distribution image. According to this embodiment, depth information can be estimated along the path of the blood vessel.

[0087] The template image may include the shape of the tissue and the shape of artifacts occurring in the depth direction. According to this embodiment, it is possible to emphasize the center of the blood vessel while taking into consideration the influence of tailing artifacts.

[0088] The image processing device 20 may estimate the center line of the tissue captured in a planar image obtained by projecting the tomographic image in the depth direction, and generate image data showing the three-dimensional structure of the tissue based on the center line and depth information. According to this embodiment, it is possible to generate image data that allows the three-dimensional structure of a blood vessel to be observed from any viewpoint.

[0089] [supplement] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), and conventional circuit modules designed to perform each of the above-described functions.

[0090] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0091] 10. Imaging device 20 Image processing device 201: Image acquisition unit 202: Center line estimation part 203: Route division unit 204: Strength analysis department 205: Template storage section 206: Center emphasis 207: Depth estimation unit 208: Image generation unit 1000 Image Processing System

Claims

1. an image acquisition unit configured to acquire a tomographic image captured by optical coherence tomography; a center emphasis unit configured to generate a center emphasis image in which a center of the tissue imaged in the tomographic image is emphasized using a template image including a predetermined tissue shape; a depth estimation unit configured to estimate depth information of the tissue based on the center-weighted image; An image processing device comprising:

2. 2. The image processing device according to claim 1, the tomographic image is captured by optical coherence tomography angiography; The tissue is a blood vessel. Image processing device.

3. 3. The image processing device according to claim 1, an intensity analysis unit configured to generate an intensity distribution image showing a distribution of pixel intensities of the tomographic image; the depth estimation unit is configured to estimate the depth information based on the intensity distribution image and the center-emphasized image. Image processing device.

4. 4. The image processing device according to claim 3, the depth estimation unit is configured to estimate the depth information by searching for a centerline of the tissue based on a predetermined cost function. Image processing device.

5. 5. The image processing device according to claim 4, the cost function includes a sum of pixel values ​​included in the center line in the intensity distribution image, a sum of pixel values ​​included in the center line in the center-weighted image, and a curvature of the center line. Image processing device.

6. 3. The image processing device according to claim 1, the template image includes a shape of the tissue and a shape of an artifact occurring in the depth direction. Image processing device.

7. 3. The image processing device according to claim 1, the image acquisition unit is configured to acquire a three-dimensional image including a plurality of the tomographic images captured at a plurality of cross sections, a centerline estimation unit configured to estimate a centerline of the tissue captured in a planar image obtained by projecting the three-dimensional image in the depth direction; an image generator configured to generate image data showing a three-dimensional structure of the tissue based on the centerline and the depth information; The image processing device further comprises:

8. An image processing system including an imaging device and an image processing device, the imaging device is configured to capture a tomographic image of a subject including a predetermined tissue by optical coherence tomography; The image processing device includes: an image acquisition unit configured to acquire the tomographic image captured by the imaging device; a center emphasis unit configured to generate a center emphasis image in which a center of the tissue imaged in the tomographic image is emphasized using a template image including the shape of the tissue; a depth estimation unit configured to estimate depth information of the tissue based on the center-weighted image; An image processing device comprising:

9. The computer A step of acquiring a tomographic image captured by optical coherence tomography; a step of generating a center-weighted image in which the center of the tissue imaged in the tomographic image is emphasized using a template image including a predetermined tissue shape; estimating depth information of the tissue based on the center-weighted image; An image processing method that performs

10. On the computer, A step of acquiring a tomographic image captured by optical coherence tomography; a step of generating a center-weighted image in which the center of the tissue imaged in the tomographic image is emphasized using a template image including a predetermined tissue shape; estimating depth information of the tissue based on the center-weighted image; A program to execute.

Images