Medical image processing device and medical image processing program

Abstract

A control unit of this medical image processing device executes an image acquisition step (S1), an inclination reduction step (S2), and a medical data acquisition step (S4). In the image acquisition step, the control unit acquires a tomographic image including a layer of tissue. In the inclination reduction step, the control unit executes, on the tomographic image, an inclination reduction process of reducing inclination of the layer with respect to a principal direction. In the medical data acquisition step, the control unit acquires medical data by inputting an inclination-reduced image, which is the tomographic image having been subjected to the inclination reduction process in the inclination reduction step, into a mathematical model that has been trained by a machine learning algorithm and that carries out a process with respect to an input image to output medical data.

Description

Technical Field

[0006] , , ,

[0001] The present disclosure relates to a medical image processing apparatus that processes data of tomographic images of biological tissues, and a medical image processing program executed in the medical image processing apparatus.

Background Art

[0002] Techniques for obtaining medical data by inputting medical images into a mathematical model trained by a machine learning algorithm have been proposed. For example, the ophthalmic image processing apparatus described in Patent Document 1 obtains an image with higher image quality than the base image as medical data by inputting the base image into a mathematical model. In addition, techniques for obtaining, as medical data, analysis results regarding the boundaries of each layer of the tissue shown in a medical image are also known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004] When tomographic images of a specific tissue of a living body are taken in the same way, the layers of the medical image often appear in a state of extending along a specific direction (hereinafter referred to as the "main direction"). On the other hand, at a site where the degree of curvature of the layer is large, or at a site where a disease exists, etc., the inclination of the direction of the layer with respect to the main direction may become large. The inventor of the present invention has newly found that when the inclination of the direction of the layer with respect to the main direction becomes large, the accuracy of the medical data output by the mathematical model decreases.

[0005] A typical object of the present disclosure is to provide a medical image processing apparatus and a medical image processing program capable of obtaining medical data with higher accuracy using a mathematical model trained by a machine learning algorithm.

[0006] A medical image processing device provided in a typical embodiment of this disclosure is a medical image processing device that processes tomographic image data of living tissue, wherein the control unit of the medical image processing device includes an image acquisition step of acquiring a tomographic image in which layers of tissue are captured, a tilt reduction step of performing a tilt reduction process on the acquired tomographic image to reduce the tilt of the layers with respect to the principal direction, and a mathematical model that is trained by a machine learning algorithm and outputs medical data by processing the input image. This is a mathematical model in which, upon receiving an image as input, outputs high-resolution image data, which is an improved version of the input image, as medical data. The system then performs a medical data acquisition step, which involves inputting a tilt-reduced image, which is the tomographic image that has undergone tilt reduction processing in the tilt reduction step, to acquire medical data.

[0007] A medical image processing program provided in a typical embodiment of this disclosure is a medical image processing program executed by a medical image processing device that processes tomographic image data of living tissue, wherein the medical image processing program is executed by a control unit of the medical image processing device, and comprises: an image acquisition step of acquiring a tomographic image showing layers of tissue; a tilt reduction step of performing a tilt reduction process on the acquired tomographic image to reduce the tilt of the layers with respect to the principal direction; and a mathematical model trained by a machine learning algorithm and which outputs medical data by processing an input image. This is a mathematical model in which, upon receiving an image as input, outputs high-resolution image data, which is an improved version of the input image, as medical data. The medical image processing device is instructed to perform a medical data acquisition step, which involves inputting a tilt-reduced image, which is a tomographic image that has undergone tilt reduction processing in the tilt reduction step, to acquire medical data.

[0008] According to the medical image processing device and medical image processing program described herein, medical data is acquired with higher accuracy by using a mathematical model trained by a machine learning algorithm.

[0009] The medical image processing device illustrated in this disclosure processes tomographic image data of living tissue. The control unit of the medical image processing device performs an image acquisition step, a tilt reduction step, and a medical data acquisition step. In the image acquisition step, the control unit acquires a tomographic image in which layers of tissue are captured. In the tilt reduction step, the control unit performs a tilt reduction process on the tomographic image to reduce the tilt of the layers with respect to the principal direction. In the medical data acquisition step, the control unit acquires medical data by inputting the tilt-reduced image, which is a tomographic image that has undergone the tilt reduction process in the tilt reduction step, into a mathematical model that is trained by a machine learning algorithm and outputs medical data by processing an input image.

[0010] According to the technology disclosed herein, a tilt-reduced image, in which the tilt of the layers relative to the principal direction is reduced, is input to a mathematical model. As a result, compared to inputting the tomographic image directly into the mathematical model without tilt reduction processing, the decrease in the accuracy of medical data caused by the tilt of the layers relative to the principal direction is appropriately suppressed. Therefore, medical data is acquired with higher accuracy.

[0011] As mentioned above, when tomographic images of specific tissues in living organisms are taken using the same method, the layers of the medical images often appear extended along a specific direction (principal direction). Therefore, most layers, or most parts of layers, of the multiple medical images used to train the mathematical model will have a small slope relative to the principal direction. Consequently, it is thought that a mathematical model trained with multiple medical images will perform processing with high accuracy for layers with a small slope relative to the principal direction, while processing accuracy tends to decrease for layers with a large slope relative to the principal direction. For example, it is conceivable to improve the processing accuracy for layers with a large slope by adjusting the network structure (filter structure, etc.) of the mathematical model. However, in this case, adjusting the network structure may increase the number of parameters, potentially increasing the number of medical images required for training and the processing time. It is also conceivable to improve processing accuracy by including many medical images with a large slope relative to the principal direction among the multiple medical images used to train the mathematical model. However, preparing many medical images with a large slope relative to the principal direction is extremely time-consuming. In contrast, the technology exemplified in this disclosure allows for the acquisition of highly accurate medical data through simple processing without the need to reconstruct mathematical models.

[0012] The principal direction is the direction in which the tissue layers generally extend when a tomographic image of a specific tissue in a living organism is taken. This disclosure provides an example of obtaining medical data from a tomographic image of fundus tissue. In this case, the layers of fundus tissue shown in the tomographic image often extend in a direction perpendicular to the tissue depth direction (referred to as the Z direction in this disclosure) (referred to as the X direction in this disclosure). Therefore, the principal direction in this disclosure is the X direction. However, the principal direction can be set appropriately depending on the tissue from which the tomographic image is taken and the imaging method. Therefore, the principal direction is not limited to the X direction. For example, the principal direction in a three-dimensional tomographic image may be a direction perpendicular to the tissue depth direction (Z direction) (XY direction).

[0013] Various devices can be used to acquire (generate) tomographic images. For example, an OCT device that acquires tomographic images of tissue using the principle of optical coherence tomography can be used. In this case, the tomographic image may be a motion contrast image (e.g., an OCT angiography image) obtained by acquiring multiple OCT signals at different times from the same location in the retinal layer of the fundus. Alternatively, an MRI (magnetic resonance imaging) device or a CT (computed tomography) device may be used. The tomographic image acquired in the image acquisition step may be a two-dimensional tomographic image or a three-dimensional tomographic image.

[0014] The mathematical model may be trained with training data that includes tilt-reduced images in which the layer tilt with respect to the principal direction is reduced. In this case, the mathematical model can perform processing with higher accuracy on tomographic images in which the layer tilt has been reduced in the tilt reduction step.

[0015] The mathematical model may, upon receiving an image as input, output high-resolution image data as medical data, with the image quality of the input image improved. In this case, even images from layers with a large angle relative to the principal direction will have their image quality improved in the same way as images from layers with a small angle relative to the principal direction.

[0016] However, the mathematical model is not limited to a mathematical model that outputs high-resolution image data as medical data. For example, the mathematical model may perform analysis on at least one of a specific structure and disease visible in the tomographic image and output data showing the analysis results as medical data. If the tomographic image is an ophthalmic image of the eye under examination, for example, at least one of the analysis results of the fundus tissue layers of the eye under examination, the boundaries of the fundus tissue layers, the optic nerve head present in the fundus, the anterior segment tissue layers, the boundaries of the anterior segment tissue layers, and the disease site of the eye under examination may be output. The mathematical model may also perform automatic diagnostic processing on the tissue visible in the tomographic image and output data showing the automatic diagnostic results as medical data. Furthermore, the mathematical model may output confidence information as medical data, indicating the degree of confidence of the processing performed on the input medical image (e.g., analysis of structure or disease). "Confidence" may be the degree of certainty of the processing of the tomographic image by the mathematical model, or it may be the reciprocal of the degree of certainty (which can also be expressed as uncertainty).

[0017] The control unit may further perform a restoration step. In the restoration step, the control unit performs the reverse processing of the processing performed in the tilt reduction step on the medical data acquired in the medical data acquisition step, thereby restoring the arrangement of the medical data to the arrangement before the tilt reduction step was performed. In this case, the arrangement of the medical data is appropriately restored to the arrangement of the tissue that was actually photographed. Therefore, the effect of layer tilt is suppressed, and medical data with an appropriate arrangement is obtained.

[0018] Furthermore, if the medical data is high-resolution image data, the arrangement of the restored medical data may be the arrangement of tissues visible in the image. If the medical data is the result of structural analysis (for example, the result of analyzing layer boundaries, etc.), the arrangement of the restored medical data may be the arrangement of the analyzed structure.

[0019] However, if the arrangement of medical data is not important (for example, if it is sufficient to obtain values ​​such as confidence levels as medical data), the restoration step can be omitted.

[0020] The control unit may further perform an image region extraction step to extract the image region containing tissue from the tomographic image. The control unit may input the tomographic image, in which the layer inclination has been reduced and the image region has been extracted, into the mathematical model. In this case, the amount of computation required for processing by the mathematical model is appropriately reduced compared to when the tomographic image is input into the mathematical model without the image region being extracted.

[0021] Furthermore, if the control unit has extracted an image region and acquired medical data, it may perform a process to restore the regions other than the extracted image region to the acquired medical data. In this case, the size of the acquired medical data will be appropriately restored to the size of the tomographic image before the image region was extracted. It is also possible to input the tomographic image into the mathematical model without performing the image region extraction step.

[0022] In the tilt reduction step, the control unit may reduce the tilt of the layers by moving each of the multiple small regions in the tomographic image that extend in a direction intersecting the principal direction in the direction intersecting the principal direction and aligning them. In this case, the tilt of the layers is appropriately reduced by the parallel movement of each of the multiple small regions.

[0023] Multiple subregions that make up a tomographic image can be selected as appropriate. For example, when a tomographic image is acquired by an OCT device, the pixel rows in the tomographic image that are aligned with the optical axis of the OCT light may be called A-scan images. In this case, each of the multiple A-scan images that make up the tomographic image may be considered a subregion. Alternatively, each of the multiple pixel rows that intersect perpendicularly to the A-scan images may be considered a subregion. Each subregion may contain multiple pixel rows.

[0024] In addition, a specific method for aligning each of the plurality of small regions can also be appropriately selected. For example, the control unit may align the plurality of small regions such that the positions where the luminance is maximum coincide among each of the plurality of small regions extending in a direction intersecting the main scanning. Further, the control unit may detect a specific layer or a layer boundary (hereinafter simply referred to as "layer·boundary") shown in the tomographic image, and align the plurality of small regions such that the detected layer·boundary approaches linearly along the main direction. Further, the control unit may detect the amount of misalignment between adjacent small regions by the phase-limited correlation method or template matching or the like, and align the plurality of small regions such that the detected misalignment amount is eliminated.

[0025] However, in addition to or instead of the above methods, it is also possible to perform the tilt reduction process using other methods. For example, the control unit may reduce the tilt of the layer with respect to the main direction by performing image processing such as rotation and shear (skew) on the two-dimensional tomographic image. In this case, for example, an image processing method such as affine transformation may be adopted.

Brief Description of Drawings

[0026] [Figure 1] It is a block diagram showing a schematic configuration of the mathematical model construction device 1, the medical image processing device 21, and the medical image imaging devices 11A and 11B. [Figure 2] It is a diagram showing an example of input data and output data when data of a high-quality tomographic image is output as medical data to a mathematical model. [Figure 3] It is a flowchart of medical image processing executed by the medical image processing device 21. [Figure 4] It is a diagram showing an example of a tomographic image 50 taken by the medical image imaging device 11B. [Figure 5] It is a diagram showing a tilt reduction image 51 obtained by performing tilt reduction processing on the tomographic image 50 shown in FIG. 4. [Figure 6] It is a diagram showing an extraction image 52 obtained by extracting an image region from the tilt reduction image 51 shown in FIG. 5. [Figure 7] This figure shows a high-resolution image 60 obtained based on the extracted image 52 shown in Figure 6. [Figure 8] Figure 7 shows the restored image 61, which is the result of performing a restoration process on the high-resolution image 60 shown in Figure 7. [Figure 9] This is a comparative diagram illustrating the effects of applying the technology disclosed herein. [Modes for carrying out the invention]

[0027] Hereinafter, one typical embodiment of the present disclosure will be described with reference to the drawings. As shown in Figure 1, in this embodiment, a mathematical model construction device 1, a medical image processing device 21, and medical image acquisition devices 11A and 11B are used. The mathematical model construction device 1 constructs a mathematical model by training the mathematical model with a machine learning algorithm. The constructed mathematical model outputs medical data by processing the input image. The medical image processing device 21 acquires medical data based on tomographic images using the mathematical model. The medical image acquisition devices 11A and 11B capture tomographic images of living tissue (in this embodiment, fundus tissue of the eye under examination).

[0028] As an example, a personal computer (hereinafter referred to as "PC") is used in the mathematical model building device 1 of this embodiment. As will be described in detail later, the mathematical model building device 1 builds a mathematical model by training the mathematical model using images acquired from the medical image acquisition device 11A (hereinafter referred to as "input data") and medical data corresponding to the input data (hereinafter referred to as "output data"). However, the device that can function as the mathematical model building device 1 is not limited to a PC. For example, the medical image acquisition device 11A may function as the mathematical model building device 1. In addition, the control units of multiple devices (for example, the CPU of the PC and the CPU 13A of the medical image acquisition device 11A) may cooperate to build the mathematical model.

[0029] Furthermore, a PC is used as the medical image processing device 21 in this embodiment. However, the device that can function as the medical image processing device 21 is not limited to a PC. For example, a medical image acquisition device 11B or a server may function as the medical image processing device 21. When the medical image acquisition device (OCT device in this embodiment) 11B functions as the medical image processing device 21, the medical image acquisition device 11B can acquire medical data based on the acquired tomographic images while simultaneously acquiring tomographic images of biological tissue. Alternatively, a mobile terminal such as a tablet or smartphone may function as the medical image processing device 21. The control units of multiple devices (for example, the CPU of the PC and the CPU 13B of the medical image acquisition device 11B) may cooperate to perform various processes.

[0030] The mathematical model construction device 1 is described below. The mathematical model construction device 1 is installed, for example, at a manufacturer that provides a medical image processing device 21 or a medical image processing program to a user. The mathematical model construction device 1 includes a control unit 2 that performs various control processing and a communication interface 5. The control unit 2 includes a CPU 3, which is a controller that manages the control, and a storage device 4 that can store programs and data. The storage device 4 stores a mathematical model construction program for executing the mathematical model construction processing described later. The communication interface 5 connects the mathematical model construction device 1 to other devices (for example, a medical image acquisition device 11A and a medical image processing device 21, etc.).

[0031] The mathematical model building device 1 is connected to an operation unit 7 and a display device 8. The operation unit 7 is operated by the user to input various instructions to the mathematical model building device 1. The operation unit 7 can use at least one of the following: a keyboard, mouse, touch panel, etc. A microphone or the like may be used together with the operation unit 7, or in place of the operation unit 7, to input various instructions. The display device 8 displays various images. The display device 8 can use various devices capable of displaying images (for example, at least one of a monitor, display, projector, etc.). In this disclosure, "image" includes both still images and moving images.

[0032] The mathematical model building device 1 can acquire image data (hereinafter sometimes simply referred to as "images") from the medical image acquisition device 11A. The mathematical model building device 1 may acquire the image data from the medical image acquisition device 11A by at least one of the following: wired communication, wireless communication, or a removable storage medium (e.g., a USB memory stick).

[0033] The medical image processing device 21 will now be described. The medical image processing device 21 is installed, for example, in a facility that diagnoses or examines patients (for example, a hospital or a health checkup facility). The medical image processing device 21 is equipped with a control unit 22 that performs various control processing and a communication interface 25. The control unit 22 is equipped with a CPU 23, which is a controller that manages the control, and a storage device 24 that can store programs and data. The storage device 24 stores a medical image processing program for executing the medical image processing described later. The medical image processing program includes a program that realizes the mathematical model constructed by the mathematical model construction device 1. The communication interface 25 connects the medical image processing device 21 to other devices (for example, a medical image acquisition device 11B and the mathematical model construction device 1).

[0034] The medical image processing device 21 is connected to the operation unit 27 and the display device 28. Various devices can be used in the operation unit 27 and the display device 28, as with the operation unit 7 and the display device 8 described above.

[0035] The medical image acquisition device 11 (11A, 11B) comprises a control unit 12 (12A, 12B) that performs various control processing and a medical image acquisition unit 16 (16A, 16B). The control unit 12 comprises a CPU 13 (13A, 13B) which is a controller that manages the operation, and a storage device 14 (14A, 14B) that can store programs and data.

[0036] The medical image acquisition unit 16 is equipped with various components necessary for acquiring tomographic images of biological tissue (in this embodiment, ophthalmic images of the eye under examination). The medical image acquisition unit 16 in this embodiment includes an OCT light source, a branching optical element that splits the OCT light emitted from the OCT light source into measurement light and reference light, a scanning unit for scanning the measurement light, an optical system for irradiating the eye under examination with the measurement light, and a light-receiving element that receives the combined light of the light reflected by the tissue and the reference light.

[0037] The medical imaging device 11 can capture tomographic images (at least one of two-dimensional and three-dimensional tomographic images) of biological tissue (in this embodiment, the fundus of the eye under examination). Specifically, the CPU 13 scans OCT light (measurement light) along the scan line to capture two-dimensional tomographic images of the cross-sections intersecting the scan line. The two-dimensional tomographic image may be an averaged image generated by performing an averaged averaging process on multiple tomographic images of the same area. The CPU 13 can also capture three-dimensional tomographic images of tissue by scanning OCT light two-dimensionally.

[0038] (Mathematical model construction process) Referring to Figure 2, the mathematical model construction process performed by the mathematical model construction device 1 will be described. The mathematical model construction process is executed by the CPU 3 according to the mathematical model construction program stored in the storage device 4.

[0039] In the mathematical model building process, a mathematical model is trained using multiple training datasets to construct a mathematical model that outputs image-based medical data. The training dataset includes both input data and output data. The mathematical model can output various types of medical data. The type of training data used to train the mathematical model is determined by the type of medical data to be output by the mathematical model.

[0040] In this embodiment, we illustrate a case in which a tomographic image (e.g., a two-dimensional tomographic image) is input to a mathematical model as a base image, and a tomographic image with improved image quality (high-resolution image) is output to the mathematical model as medical data. In this embodiment, a two-dimensional tomographic image of the tissue of the eye under examination is used as input data, and a two-dimensional tomographic image of the same area with higher image quality than the input data is used as output data to train the mathematical model. The high-resolution image refers to at least one of the following: an image in which the noise of the input base image has been reduced, an image in which the resolution of the original image has been increased, or an image in which the visibility of the original image has been improved.

[0041] Figure 2 shows an example of training data (input and output data) when high-resolution tomographic image data is output to a mathematical model as medical data. In the example shown in Figure 2, CPU3 acquires a set 40 of multiple tomographic images 400A to 400X taken from the same part of the tissue. CPU3 uses a portion of the multiple tomographic images 400A to 400X within set 40 (a number smaller than the number used for the averaging of the output data described later) as input data. CPU3 also acquires an averaged image 41 of the multiple tomographic images 400A to 400X within set 40 as output data. When the mathematical model is trained with the input and output data exemplified in Figure 2, the tomographic images are input to the trained mathematical model as base images, and high-resolution image data with the effects of speckle noise suppressed is output as medical data.

[0042] However, it is also possible to modify the configuration of the mathematical model. For example, the mathematical model may perform analysis on at least one of the specific structures and diseases visible in the tomographic image and output data showing the analysis results as medical data. In this case, at least one of the analysis results of the fundus tissue layers of the eye under examination, the boundaries of the fundus tissue layers, the optic nerve head present in the fundus, the anterior segment tissue layers, the boundaries of the anterior segment tissue layers, and the diseased area of ​​the eye under examination may be output. The mathematical model may also perform automatic diagnostic processing on the tissues visible in the tomographic image and output data showing the automatic diagnostic results as medical data. Furthermore, the mathematical model may output confidence information indicating the degree of confidence of the processing performed on the input medical image (e.g., analysis of structures or diseases) as medical data. The form of the training data is appropriately selected according to the functions of the mathematical model to be constructed.

[0043] The mathematical model construction process will now be explained. CPU3 acquires at least a portion of the tomographic images captured by the medical imaging device 11A as input data. Next, CPU3 acquires output data corresponding to the input data. An example of the correspondence between input data and output data is as described above.

[0044] Next, CPU3 uses a machine learning algorithm to train a mathematical model with the training data. Commonly known machine learning algorithms include neural networks, random forests, boosting, and support vector machines (SVMs).

[0045] Neural networks are techniques that mimic the behavior of biological nerve cell networks. Examples of neural networks include feedforward neural networks, RBF networks (radiating basis function networks), spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Basian networks, etc.).

[0046] Random forests are a method for generating multiple decision trees by learning from randomly sampled training data. When using random forests, the system follows the branches of multiple decision trees that have been pre-trained as classifiers, and takes the average (or majority vote) of the results obtained from each decision tree.

[0047] Boosting is a technique for generating a strong classifier by combining multiple weak classifiers. It involves sequentially training simple, weak classifiers to construct a strong classifier.

[0048] SVM is a method for constructing a two-class pattern classifier using linear input elements. SVM learns the parameters of linear input elements based on a criterion (hyperplane separation theorem) that finds the margin-maximizing hyperplane that maximizes the distance to each data point from the training data.

[0049] A mathematical model refers to a data structure used to predict the relationship between input and output data. Mathematical models are built by being trained using training data. As mentioned earlier, training data consists of sets of input and output data. For example, training updates the correlation data (e.g., weights) between each input and output.

[0050] In this embodiment, a multilayer neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating the data to be predicted, and one or more hidden layers between the input and output layers. Each layer contains multiple nodes (also called units). Specifically, in this embodiment, a convolutional neural network (CNN), which is a type of multilayer neural network, is used. However, other machine learning algorithms may be used. For example, a generative adversarial network (GAN), which utilizes two competing neural networks, may be adopted as the machine learning algorithm.

[0051] The above process is repeated until the mathematical model is built. Once the mathematical model is built, the mathematical model building process ends. The program and data that realize the built mathematical model are incorporated into the medical image processing device 21.

[0052] Furthermore, the mathematical model used in this embodiment may be trained with training data that includes a slope reduction process (details of which will be described later) that reduces the slope of the layers with respect to the principal direction. In this case, the mathematical model can output medical data with higher accuracy by receiving tomographic images in which the slope of the layers with respect to the principal direction has been reduced.

[0053] (Medical image processing) Referring to Figures 3 to 9, an example of medical image processing performed by the medical image processing device 21 will be described. Figures 3 to 9 illustrate a case in which two-dimensional tomographic image data of the fundus is processed by a mathematical model to obtain two-dimensional tomographic image data with higher resolution than the image before processing. The medical image processing illustrated in Figure 3 is executed by the CPU 23 according to the medical image processing program stored in the storage device 24.

[0054] First, the CPU 23 acquires a tomographic image of the tissue of the eye being examined, which is captured by the medical imaging device (OCT device in this embodiment) 11B (S1). Figure 4 shows an example of a tomographic image 50 captured by the medical imaging device 11B. The two-dimensional tomographic image 50 captured by the OCT device is composed of multiple A-scan images. An A-scan image is a series of pixels extending in the direction along the optical axis of the OCT measurement light (i.e., the Z direction, which is the depth direction). In other words, a two-dimensional tomographic image 50 is constructed by arranging multiple A-scan images extending in the Z direction in the X direction (in this embodiment, the direction in which the OCT measurement light spot was scanned over the tissue) which intersects perpendicularly to the Z direction.

[0055] In this disclosure, the primary direction is defined as the direction in which the tissue layers generally extend when a tomographic image of a specific tissue in a living organism is taken. When a tomographic image of the fundus tissue of an eye examined is taken using the medical imaging device 11B exemplified in this embodiment, most of the fundus tissue layers shown in the captured tomographic image often extend in the X direction, which is perpendicular to the Z direction along the optical axis of the OCT measurement light. Therefore, the primary direction in this embodiment is defined as the X direction.

[0056] The tomographic image 50 shown in Figure 4 is an image of a fundus with a greater degree of curvature compared to a typical fundus. Therefore, in the tomographic image 50 shown in Figure 4, the inclination of the layers to the left of the center with respect to the principal direction (X direction) is relatively small, but the inclination of the layers to the right of the center with respect to the principal direction (X direction) is very large.

[0057] As will be explained in detail later with reference to Figure 9, it has been newly discovered that the accuracy of medical data output by the mathematical model decreases when the slope of the layers with respect to the principal direction becomes large. In the medical image processing of this embodiment, processing is performed to acquire medical data with higher accuracy regardless of the slope of the layers with respect to the principal direction. The details of the processing will be explained below.

[0058] Returning to the explanation of Figure 3, CPU 23 performs tilt reduction processing on the tomographic image 50 acquired in S1 (S2). Tilt reduction processing is a process that reduces the tilt of the layers in the tomographic image with respect to the principal direction (X direction).

[0059] Figure 5 shows the result of applying tilt reduction processing to the tomographic image 50 shown in Figure 4. As shown in Figure 5, in the tilt-reduced image 51 after the tilt reduction processing has been applied, the curvature of the layers that was present in the tomographic image 50 shown in Figure 4 is suppressed, and the layers are flattened in the X direction.

[0060] In detail, in the tilt reduction process (S2) of this embodiment, the CPU 23 moves each of the multiple small regions (multiple A-scan images in this embodiment) that extend in the Z-direction intersecting the main direction (X-direction) within the tomographic image 50 in the Z-direction, thereby aligning the Z-direction positions of the images contained in each small region. As a result, the tilt of the layers with respect to the main direction is appropriately reduced. The movement direction and amount of each A-scan image are stored in the storage device 24 for reference in the arrangement restoration process (S5) described later.

[0061] In step S2 of this embodiment, the CPU 23 detects a specific layer or layer boundary in the tomographic image 50 and performs image alignment in multiple sub-regions so that the detected layer or boundary approaches linearly along the principal direction (X direction). However, the specific method for aligning the images of each of the multiple sub-regions can be changed as appropriate. For example, the CPU 23 may move each of the multiple sub-regions in the Z direction so that the positions where the brightness is maximum in each of the multiple sub-regions coincide in the Z direction. Alternatively, the CPU 23 may detect the amount of misalignment between adjacent sub-regions using a phase-limited correlation method or template matching, and perform alignment of the multiple sub-regions so that the detected misalignment is eliminated.

[0062] Next, the CPU 23 performs an image region extraction process (S3) to extract the image region containing tissue from the tomographic image. In this embodiment, the image region extraction process (S3) is performed after the tilt reduction process (S2) is performed on the tomographic image. That is, the extracted image 52 shown in Figure 6 is an image from which the image region has been extracted from the tilt reduction image 51 shown in Figure 5. However, the CPU 23 may also perform the tilt reduction process after performing the image region extraction process on the tomographic image 50 acquired in S1. As shown in Figure 6, the amount of data in the extracted image 52 is smaller than the amount of data in the tomographic image before the image region extraction process is performed. As a result, the amount of computation required for processing by the mathematical model (details will be described later) is appropriately reduced.

[0063] Next, the CPU 23 inputs the tomographic image that has undergone tilt reduction processing (specifically, the extracted image 52, which is a tomographic image that has undergone both tilt reduction processing and image region extraction processing) into the mathematical model to acquire medical data (S4). As described above, the mathematical model exemplified in this embodiment processes the input tomographic image (base image) to improve the image quality of the input tomographic image and outputs high-resolution image data as medical data. The CPU 23 acquires the high-resolution image data output by the mathematical model. Figure 7 shows the high-resolution image 60 output by the mathematical model based on the extracted image 52 shown in Figure 6. The image quality of the high-resolution image 60 (see Figure 7) is improved compared to the image quality of the extracted image 52 (see Figure 6) before it was input into the mathematical model.

[0064] As mentioned above, the tomographic images input to the mathematical model in S4 undergo tilt reduction processing (S2). As a result, compared to inputting the tomographic images 50 directly to the mathematical model without tilt reduction processing, the reduction in the accuracy of the medical data (high-resolution image data in this embodiment) caused by the tilt of the layers with respect to the principal direction (X direction) is appropriately suppressed.

[0065] Next, the CPU 23 performs placement restoration processing and non-extracted area restoration processing (S5). In placement restoration processing, the CPU 23 performs the reverse processing of the processing performed in tilt reduction processing (S2) on the medical data (data of the high-resolution image 60) acquired in S4, thereby restoring the placement of the high-resolution image 60 to the placement before the tilt reduction processing was performed. In this embodiment, the CPU 23 restores the placement of each small region (A scan image) by moving each small region by the amount of movement in S2, in the opposite direction to the direction of movement performed on each small region (A scan image) in tilt reduction processing (S2). In addition, in non-extracted area restoration processing, the CPU 23 restores the area that was not extracted in the image region extraction processing (S3) to the high-resolution image 60 acquired in S4, thereby returning the size of the high-resolution image to the size before the image region was extracted. Figure 8 shows the restored image 61 in which placement restoration processing and non-extracted area restoration processing have been performed on the high-resolution image 60 shown in Figure 7.

[0066] Referring to Figure 9, the effects of applying the technology described herein will be explained. The top image in Figure 9 is the tomographic image 50 immediately after being captured by the medical imaging device 11B (i.e., the tomographic image 50 shown in Figure 4, before any high-resolution processing by the mathematical model has been performed). The middle image in Figure 9 is a high-resolution image (comparison image 99) obtained by inputting the tomographic image 50 directly into the mathematical model without performing tilt reduction processing (S2) on the tomographic image 50. The bottom image in Figure 9 is a high-resolution image obtained by performing processing S2 to S4 on the tomographic image 50 (i.e., the reconstructed image 61 shown in Figure 8).

[0067] As shown in Figure 9, both the comparison image 99 and the reconstructed image 61 show a similar level of image quality improvement compared to the topmost tomographic image 50 in the portion to the left of the center. This is because the layer tilt relative to the principal direction (X direction) is relatively small in the portion to the left of the center. In contrast, in the portion to the right of the center, the reconstructed image 61 shows even greater image quality improvement compared to the comparison image 99. From these results, it can be seen that by inputting an image that has undergone tilt reduction processing into a mathematical model, medical data can be acquired with higher accuracy regardless of the layer tilt relative to the principal direction.

[0068] The technologies disclosed in the above embodiments are merely examples. Therefore, it is possible to modify the technologies exemplified in the above embodiments. First, it is possible to perform only a part of the processes exemplified in the above embodiments. For example, in the medical image processing shown in Figure 3, it is possible to omit at least one of the image region extraction process (S3) and the restoration process (S5). Furthermore, the mathematical model is not limited to a mathematical model that outputs high-resolution image data as medical data.

[0069] Furthermore, the CPU 23 can extract two-dimensional tomographic images from three-dimensional tomographic images and acquire medical data (e.g., high-resolution image data) based on the extracted two-dimensional tomographic images using a mathematical model. In this case, the CPU 23 may perform a tilt reduction process on the extracted two-dimensional tomographic images and input them into the mathematical model. Alternatively, the CPU 23 may perform a tilt reduction process to reduce the tilt of the layers relative to the principal direction at the three-dimensional tomographic image stage, and then extract two-dimensional tomographic images from the three-dimensional tomographic images and input them into the mathematical model. In this case, it becomes unnecessary to perform the tilt reduction process each time a two-dimensional tomographic image is extracted. For example, when processing three-dimensional tomographic images of the fundus taken by an OCT device, the principal direction may be the XY direction perpendicular to the tissue depth direction (Z direction). Note that the method for extracting two-dimensional tomographic images from three-dimensional tomographic images can be arbitrarily selected. For example, when the three-dimensional tomography image is viewed from a direction along the optical axis of the imaging light (e.g., OCT light), the two-dimensional tomography image may be extracted in such a way that the positions where the two-dimensional tomography image was extracted form a circle or a cross shape.

[0070] The process of acquiring a tomographic image in S1 of Figure 3 is an example of an "image acquisition step". The tilt reduction process performed in S2 is an example of a "tilt reduction step". The process of acquiring medical data in S4 is an example of a "medical data acquisition step". The restoration process performed in S5 is an example of a "restoration step". The image region extraction process performed in S3 is an example of an "image region extraction step".

[0071] 21 Medical imaging processing equipment 23 CPU 24 Storage device 50 Tomographic Images 51 Tilt-reduced image 52 extracted images 60 high-resolution images 61 Restored Images

Claims

1. A medical image processing device that processes tomographic image data of living tissue, The control unit of the medical image processing device is Image acquisition step to obtain a tomographic image showing layers of tissue, A tilt reduction step is performed on the acquired tomographic image to reduce the tilt of the layer with respect to the principal direction, A mathematical model trained by a machine learning algorithm and outputting medical data by processing an input image, wherein when an image is input, the mathematical model outputs high-resolution image data with improved image quality as medical data, and a tilt-reduced image, which is the tomographic image that has undergone tilt reduction processing in the tilt reduction step, is input to acquire medical data in a medical data acquisition step, A medical image processing device characterized by performing the following actions.

2. A medical image processing apparatus according to claim 1, The control unit, A medical image processing apparatus characterized by further performing a restoration step, which restores the arrangement of the high-resolution image data to the arrangement before the tilt reduction step was performed, by performing the reverse process of the process performed in the tilt reduction step on the high-resolution image data acquired in the medical data acquisition step.

3. A medical image processing apparatus according to claim 1 or 2, The control unit, A further image region extraction step is performed to extract the image region in which tissue is visible from the aforementioned tomographic image. The medical image processing apparatus is characterized in that, in the medical data acquisition step, the tilt of the layer is reduced in the tilt reduction step, and the tomographic image extracted in the image region extraction step is input to the mathematical model.

4. A medical image processing apparatus according to any one of claims 1 to 3, The control unit is characterized in that, in the tilt reduction step, it reduces the tilt of the layer by moving each of the plurality of small regions that extend in a direction intersecting the main direction and constitute the tomographic image in a direction intersecting the main direction to perform alignment.

5. A medical image processing program executed by a medical image processing device that processes tomographic image data of living tissue, The medical image processing program is executed by the control unit of the medical image processing device, Image acquisition step to obtain a tomographic image showing layers of tissue, A tilt reduction step is performed on the acquired tomographic image to reduce the tilt of the layer with respect to the principal direction, A mathematical model trained by a machine learning algorithm and outputting medical data by processing an input image, wherein when an image is input, the mathematical model outputs high-resolution image data with improved image quality as medical data, and a tilt-reduced image, which is the tomographic image that has undergone tilt reduction processing in the tilt reduction step, is input to acquire medical data in a medical data acquisition step, A medical image processing program characterized by causing the medical image processing device to execute the above-mentioned medical image processing device.

Images