Oct image processing device and oct image processing program

Abstract

To provide an OCT image processing device and an OCT image processing program that can obtain higher quality medical information by utilizing a mathematical model learned by a machine learning algorithm.SOLUTION: A control unit acquires an OCT image captured by an OCT device (S1). The control unit performs normalization processing on the OCT image acquired in an image acquisition step (S2 to S9). The control unit acquires medical information output by a mathematical model by inputting a normalized image being the OCT image on which the normalization processing is performed into the mathematical model trained by a machine learning algorithm so as to output medical information by inputting the OCT image (S10). In the normalization processing, the control unit approximates at least one of the characteristics of the OCT image to statistical information of the characteristics of multiple training images, which are the OCT images used to train the mathematical model.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an OCT image processing device that processes data of an OCT image of biological tissue captured by an OCT device, and an OCT image processing program executed in the OCT image processing device. [Background technology]

[0002] A technology has been proposed for acquiring various medical information by processing images of biological tissues using a mathematical model trained by a machine learning algorithm. For example, an ophthalmic image processing device described in Patent Document 1 inputs an ophthalmic image into a mathematical model trained by a machine learning algorithm, and acquires a target image with higher image quality than the input ophthalmic image as medical information. In addition, an ophthalmic image processing device described in Patent Document 2 inputs an ophthalmic image into a mathematical model trained by a machine learning algorithm, and acquires a detection result of at least one of a specific boundary and a specific part of tissue in an ophthalmic image as medical information. In addition, a technology has also been proposed for acquiring medical information related to a disease of tissue in an ophthalmic image by using a mathematical model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6703319 [Patent Document 2] Patent No. 7180187 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional technology, when using a mathematical model trained by a machine learning algorithm, if an OCT image with characteristics different from the characteristics of the training image used to train the mathematical model is input to the mathematical model, high-quality medical information may not be obtained. For example, when obtaining an image with higher quality than the OCT image input to the mathematical model as medical information, if an OCT image with originally high quality is input to the mathematical model, there are cases where the quality of the obtained image is lower than that of the input image. Therefore, a technology that can obtain higher quality medical information by using a mathematical model trained by a machine learning algorithm is desired.

[0005] A typical object of the present disclosure is to provide an OCT image processing device and an OCT image processing program capable of obtaining higher quality medical information by utilizing a mathematical model learned by a machine learning algorithm. [Means for solving the problem]

[0006] An OCT image processing device provided by a typical embodiment of the present disclosure is an OCT image processing device that processes data of OCT images of biological tissue captured by an OCT device, and a control unit of the OCT image processing device executes an image acquisition step of acquiring an OCT image captured by the OCT device, a normalization step of performing a normalization process on the OCT image acquired in the image acquisition step, and a medical information acquisition step of inputting a normalized image, which is the OCT image on which the normalization process has been performed, into a mathematical model trained by a machine learning algorithm to output medical information by inputting an OCT image, thereby acquiring medical information to be output by the mathematical model, and the normalization process executed in the normalization step approximates at least one of the characteristics of the OCT image to statistical information of the characteristics of a plurality of training images, which are OCT images used to train the mathematical model.

[0007] An OCT image processing program provided by a typical embodiment of the present disclosure is an OCT image processing program executed by an OCT image processing device that processes data of OCT images of biological tissue captured by an OCT device, and the OCT image processing program is executed by a control unit of the OCT image processing device to cause the OCT image processing device to execute an image acquisition step of acquiring an OCT image captured by the OCT device, a normalization step of performing a normalization process on the OCT image acquired in the image acquisition step, and a medical information acquisition step of inputting a normalized image, which is the OCT image on which the normalization process has been performed, into a mathematical model trained by a machine learning algorithm to output medical information by inputting an OCT image, thereby acquiring medical information to be output by the mathematical model, and the normalization process executed in the normalization step approximates at least one of the characteristics of the OCT image to statistical information of characteristics of a plurality of training images, which are OCT images used to train the mathematical model.

[0008] According to the OCT image processing device and the OCT image processing program of the present disclosure, higher quality medical information is obtained by utilizing a mathematical model learned by a machine learning algorithm. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of a mathematical model construction device 1, an OCT image processing device 21, and OCT devices 11A and 11B. [Diagram 2] 13A and 13B are diagrams showing an example of input data and output data when high-quality OCT image data is output to a mathematical model as medical information. [Diagram 3] 13 is a flowchart of OCT image processing executed by the OCT image processing device 21. [Figure 4] 13 is an explanatory diagram for explaining an example of a method for adding a background region 61 to an OCT image 60 when the OCT image 60 is captured so that the image is positioned approximately at the center in the depth direction of the imaging range. FIG. [Diagram 5] FIG. 13 is an explanatory diagram for explaining an example of a method for adding a background region 61 to an OCT image 60 when the OCT image 60 is captured so that the surface layer position of the tissue remains substantially constant. [Figure 6] FIG. 13 is an explanatory diagram for explaining an example of a method for extracting a partial region in the depth direction from an OCT image 60. [Figure 7] 13 is an explanatory diagram for explaining an example of a method for restoring the arrangement of medical information to the arrangement before the layer inclination was reduced. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Summary> The OCT image processing device exemplified in the present disclosure processes data of an OCT image of a biological tissue captured by an OCT device. A control unit of the OCT image processing device executes an image acquisition step, a normalization step, and a medical information acquisition step. In the image acquisition step, the control unit acquires an OCT image captured by the OCT device. In the normalization step, the control unit executes a normalization process on the OCT image acquired in the image acquisition step. In the medical information acquisition step, the control unit acquires medical information output by the mathematical model by inputting a normalized image, which is an OCT image on which normalization process has been performed, into a mathematical model trained by a machine learning algorithm to output medical information by inputting an OCT image. In the normalization process executed in the normalization step, the control unit approximates at least one of the characteristics of the OCT image to statistical information of the characteristics of a plurality of training images, which are OCT images used to train the mathematical model.

[0011] It has been found that when using medical information output by a mathematical model by inputting an image into the mathematical model trained by a machine learning algorithm, if the characteristics of the image input into the mathematical model are different from the characteristics of the multiple training images used to train the mathematical model, the quality of the medical information output by the mathematical model is likely to deteriorate. In particular, the imaging range in the depth direction of tissues of an OCT image varies significantly depending on the device that captures the image. In addition, characteristics (e.g., SNR, etc.) may vary significantly depending on the imaging conditions, etc. Therefore, when inputting an OCT image into a mathematical model to obtain medical information, the quality of the medical information is particularly likely to deteriorate.

[0012] In contrast, according to the technology disclosed herein, a normalization process is performed to approximate the characteristics of an OCT image to statistical information of the characteristics of a plurality of training images used to train a mathematical model, and the normalized OCT image (normalized image) is input to the mathematical model. As a result, high-quality medical information is more likely to be appropriately output by the mathematical model.

[0013] The mathematical model may have various configurations. For example, the mathematical model may perform an analysis process on at least one of a specific structure and a disease shown in the OCT image, and output information indicating the analysis result as medical information. When the OCT image is an ophthalmologic image of the subject's eye, for example, at least one of the analysis results of the fundus tissue layer of the subject's eye, the boundary of the fundus tissue layer, the optic disc present in the fundus, the fovea, the tissue layer, the boundary of the anterior segment tissue layer, and the diseased part of the subject's eye may be output. In addition, the mathematical model may output data of a high-quality image obtained by improving the image quality (which may be the resolution) of the input OCT image by inputting the OCT image, as medical information. The mathematical model may perform an automatic diagnosis process on the tissue shown in the OCT image, and output data indicating the automatic diagnosis result as medical information. In addition, the mathematical model may output confidence information indicating the confidence of the process (e.g., structure or disease analysis process, etc.) performed on the input OCT image, as medical information. The "certainty" may be the degree of certainty of the processing of the OCT image by the mathematical model, or it may be the inverse of the degree of certainty (which may also be expressed as uncertainty).

[0014] In the normalization process, the control unit may refer to the additional information indicating the characteristics of the OCT image, which is added to the data of the OCT image acquired in the image acquisition step, and may approximate at least one of the characteristics of the OCT image indicated by the referred additional information to the statistical information of the characteristics of the multiple training images. In this case, the characteristics of the acquired OCT image are accurately grasped by the additional information. Therefore, the control unit can appropriately approximate the grasped characteristics of the OCT image to the statistical information of the characteristics of the training images.

[0015] Various types of information can be used as the incidental information. For example, incidental information conforming to an international standard (e.g., Digital Imaging and Communications in Medicine: DICOM, etc.) that defines a method of communication and storage of medical image data may be attached to each OCT image. In this case, the control unit may perform normalization processing by referring to the incidental information conforming to DICOM. The incidental information may indicate at least one of a plurality of characteristics such as the size of the OCT image (the size of the tissue in the depth direction), the number of pixels, and the resolution.

[0016] However, the control unit may also perform normalization processing without referring to the supplementary information of the OCT image. For example, the control unit may execute processing (e.g., analysis processing for at least one of a specific structure and a disease shown in the OCT image, etc.) on the OCT image acquired in the image acquisition step, and perform normalization processing based on the characteristics of the OCT image acquired by the processing.

[0017] In the normalization process, when the imaging range in the tissue depth direction of the OCT image acquired in the image acquisition step is narrower than the statistical information of the imaging range in the depth direction of the multiple training images, the control unit may add a background region in the depth direction of the OCT image to approximate the imaging range in the depth direction of the OCT image to the imaging range in the depth direction of the multiple training images. In this case, the image statistics (e.g., at least one of the average brightness and the standard deviation) of the acquired OCT image approach the image statistics of the multiple training images. Therefore, even if the imaging range in the depth direction differs between the acquired OCT image and the multiple training images, the normalized image is input to the mathematical model, making it easier for the mathematical model to appropriately output high-quality medical information.

[0018] A specific method for adding a background region in the depth direction of an OCT image can be appropriately selected. For example, regardless of the size (width) of the imaging range in the depth direction, an OCT image may be captured so that the image of the tomography is located approximately in the center of the imaging range in the depth direction. In this case, the control unit may add a background region to each of both ends in the depth direction of the OCT image. By adding a background region to each of both ends in the depth direction of the OCT image, the depth direction position of the image region in the normalized image to which a pair of background regions have been added becomes more likely to approximate the statistical depth direction position of the image region in the multiple training images. As a result, the quality of the acquired medical information is more likely to be improved. Note that a background region of the same size may be added to each of both ends in the depth direction of the OCT image. In this case, the depth direction position of the image region of the normalized image becomes more likely to approximate the position of the image region in the multiple training images with higher accuracy.

[0019] In addition, there are cases where an OCT image is captured with the surface layer position of the tissue being substantially constant, regardless of the size (width) of the imaging range in the depth direction. In this case, the control unit may add a background region only to the opposite side (deep side) of the tissue surface layer side of both ends of the OCT image in the depth direction. As a result, the depth direction position of the image region in the normalized image to which the background region has been added becomes more likely to approximate the statistical depth direction position of the image region in multiple training images.

[0020] In the normalization process, when a background region is added in the depth direction of the OCT image, the control unit may further execute an excluding step of excluding information of a region corresponding to the added background region from medical information acquired by inputting the normalized image, which is an image with the background region added, into the mathematical model. In this case, the region of the acquired medical information is returned to the region before the background region was added. Therefore, it becomes easier to obtain medical information of high quality and with an appropriate region.

[0021] However, when the area of ​​medical information is not important (for example, when it is sufficient to obtain a value such as a certainty factor as medical information), it is possible to omit the exclusion step.

[0022] In the normalization process, when the imaging range in the tissue depth direction of the OCT image acquired in the image acquisition step is wider than the statistical information of the imaging range in the depth direction of the multiple training images, the control unit may extract a part of the image region in the depth direction from the OCT image to approximate the imaging range in the depth direction of the OCT image to the imaging range in the depth direction of the multiple training images. In this case, the image statistics (e.g., at least one of the average brightness and the standard deviation) of the acquired OCT image approach the image statistics of the multiple training images. Therefore, even if the imaging range in the depth direction differs between the acquired OCT image and the multiple training images, the normalized image is input to the mathematical model, making it easier for the mathematical model to appropriately output high-quality medical information.

[0023] In the normalization process, when extracting a partial region in the depth direction from the OCT image, the control unit may extract at least a part of an image region in which an image of tissue appears in the OCT image. In this case, the image region of tissue appearing in the OCT image is extracted and input to a mathematical model, whereby medical information regarding the image region is appropriately obtained.

[0024] A specific method for extracting an image region from an OCT image can be appropriately selected. For example, assume that the pixel value of the region in which an image is captured is greater than the pixel value of the background region in which no image is captured. The control unit may calculate a composite pixel value (for example, a sum or average value of a plurality of pixel values) of a plurality of pixels included in the frame while moving a frame of the same size as the region to be extracted in the depth direction in the OCT image. The control unit may set the extraction position of the region to the position of the frame where the composite pixel value calculated within the frame is the highest. In this case, it becomes easier to appropriately extract as many parts of the image region as possible. Therefore, it becomes easier to output higher quality medical information by the mathematical model. Similarly, when the pixel value of the region in which an image is captured is smaller than the pixel value of the background region in which no image is captured, the control unit may set the extraction position of the region to the position of the frame where the composite pixel value (sum or average value, etc.) calculated within the frame is the lowest.

[0025] Furthermore, when extracting a portion of a region in the depth direction from the OCT image, the control unit may set the extraction position of the region in the OCT image to a position where the tissue image is located at the center in the depth direction. In this case, many regions of the tissue image region shown in the OCT image are easily extracted appropriately in the normalization process. Therefore, medical information of higher quality is easily obtained. Here, a case is assumed in which the pixel value of the region where the image is shown is greater than the pixel value of the background region where the image is not shown. The OCT image is configured by arranging a plurality of pixel rows extending in a direction intersecting the depth direction in the depth direction. For example, the control unit may calculate a composite value of pixel values ​​(for example, a sum or average value of a plurality of pixel values) for each of a plurality of pixel rows extending in a direction intersecting the depth direction. The control unit may match the center in the depth direction of the region to be extracted to the pixel row with the highest composite value of the calculated pixel values ​​among the plurality of pixel rows extending in a direction intersecting the depth direction. In this case, many regions of the tissue image region shown in the OCT image are easily extracted appropriately in the normalization process. Therefore, medical information of higher quality is easily obtained. Similarly, if the pixel values ​​of an area containing an image are smaller than the pixel values ​​of a background area containing no image, the control unit may align the depth-wise center of the area to be extracted with the pixel row having the lowest calculated composite value of pixel values ​​among multiple pixel rows extending in a direction intersecting the depth direction.

[0026] The OCT image may be an image in which a tissue layer is reflected. In the normalization process, when extracting a part of the region in the depth direction from the OCT image, the control unit may extract at least a part of the image region in a state in which the inclination of the tissue layer reflected in the OCT image is reduced. In this case, even if the tissue layer reflected in the OCT image is curved, many areas of the tissue image region reflected in the OCT image are easily appropriately extracted in the normalization process. Therefore, medical information of higher quality is easily obtained. In addition, in a case where information of a pixel located in a direction intersecting the depth direction with respect to the pixel of interest is used in the process using the mathematical model, the accuracy of the process using the mathematical model itself is easily improved by reducing the inclination of the layer.

[0027] A specific method for extracting an image region in a state where the inclination of the tissue layer reflected in the OCT image is reduced can be appropriately selected. For example, a case is assumed where the pixel value of the region where an image is reflected is greater than the pixel value of the background region where no image is reflected. The OCT image is configured by arranging a plurality of pixel rows extending in the depth direction (e.g., A-scan images extending in a direction along the optical axis of the OCT measurement light, etc.) in a direction intersecting the depth direction. The control unit may calculate a composite pixel value (e.g., a sum or average value of a plurality of pixel values, etc.) of a plurality of pixels included in the frame while moving a frame having the same width as the depth width of the region to be extracted in the depth direction on the pixel row extending in the depth direction. The control unit may set the region within the frame where the composite pixel value calculated within the frame is the highest as the region to be extracted from the pixel row of interest. The above process may be performed for each of the plurality of pixel rows arranged in a direction intersecting the depth direction, thereby extracting an image region from each of the plurality of pixel rows. In this case, the image regions extracted from each of the pixel rows are arranged in a direction intersecting the depth direction, so that at least a portion of the image region is extracted with a reduced tilt of the tissue layer in the OCT image. Similarly, when the pixel value of the region in which the image is captured is smaller than the pixel value of the background region in which no image is captured, the control unit may set the region in the frame in which the composite pixel value calculated within the frame is the lowest as the region to be extracted from the pixel row of interest.

[0028] The control unit may also perform image alignment in the depth direction between a plurality of pixel rows extending in the depth direction. The control unit may extract a partial region in the depth direction (e.g., a rectangular region, etc.) from the OCT image in which image alignment has been performed. Note that the above-mentioned various methods (e.g., a method of referring to a composite pixel value calculated within a frame, etc.) can be used as a method for extracting a partial region from the OCT image in which image alignment has been performed. Furthermore, regardless of whether or not at least a portion of the image region is extracted, it is also possible to perform a process of reducing the inclination of the tissue layer shown in the OCT image.

[0029] When at least a part of the image region is extracted in a state where the tilt of the tissue layer in the OCT image is reduced in the normalization process, the control unit may further execute a restoration step of restoring the arrangement of the medical information acquired by inputting the extracted normalized image into the mathematical model to the arrangement before the tilt of the tissue layer in the OCT image is reduced. In this case, the arrangement of the medical information is appropriately restored to match the arrangement of the tissue actually photographed. Therefore, it becomes easier to obtain high-quality medical information with an appropriate arrangement.

[0030] In addition, when the medical information is information such as a high-quality image, the arrangement of the restored medical information may be the arrangement of the tissues shown in the image. When the medical information is the analysis result of a structure (e.g., the analysis result of the boundary of layers, etc.), the arrangement of the restored medical information may be the arrangement of the analyzed structure. However, when the arrangement of the medical information is not important (e.g., when it is sufficient to obtain a value such as a confidence level as the medical information), it is also possible to omit the restoration step. Also, an image region may be extracted from an OCT image without reducing the inclination of the layers. In this case, there is no need to perform the restoration step.

[0031] The control unit may approximate other characteristics of the OCT image to statistical information of characteristics of a plurality of training images together with or instead of the imaging range in the depth direction of the OCT image. For example, in the normalization process, the control unit may approximate statistical information of at least one characteristic of the OCT image, such as the number of pixels, resolution, pixel value, etc., to statistical information of characteristics of a plurality of training images. Even in this case, high-quality medical information is more likely to be appropriately output by the mathematical model.

[0032] In the normalization process, the control unit may calculate statistical information of characteristics of at least a part of an image region in which an image of a tissue is captured in the OCT image captured in the image acquisition step, and approximate the calculated statistical information of characteristics of the image region to statistical information of characteristics of the image region in a plurality of training images. In this case, a normalization process more suitable for the OCT image is performed compared to the case where the entire statistical information of the acquired OCT image is approximated to the statistical information of the training images. Therefore, higher quality medical information is more likely to be appropriately output by the mathematical model.

[0033] Furthermore, in the normalization process, the control unit may calculate statistical information of characteristics of at least a part of the background region in which no image is captured in the OCT image captured in the image acquisition step, and approximate the calculated statistical information of the characteristics of the background region to the statistical information of the characteristics of the background region in the multiple training images. In this case, the characteristics of the background region as well as the characteristics of the image region are approximated to the statistical information of the training images. Therefore, higher quality medical information is more likely to be appropriately output by the mathematical model.

[0034] A specific method for calculating the statistical information of the characteristics of the image region can be appropriately selected. For example, the control unit may input the acquired OCT image to a mathematical model that is pre-trained to output an analysis result of the image region (which may be a background region) in the OCT image, thereby acquiring the analysis result of the image region output by the mathematical model. The control unit may calculate the statistical information of the characteristics of at least a part of the acquired image region. The control unit may also perform known image processing on the OCT image to detect the image region (which may be a background region) in the OCT image.

[0035] <Embodiment> (Device configuration) A typical embodiment of the present disclosure will be described below with reference to the drawings. As shown in FIG. 1, the OCT image processing system of this embodiment uses a mathematical model construction device 1, an OCT image processing device 21, and OCT devices 11A and 11B. The mathematical model construction device 1 constructs a mathematical model by training (learning) the mathematical model using a machine learning algorithm. A program for realizing the constructed mathematical model is stored in a storage device 24 of the OCT image processing device 21. The OCT image processing device 21 inputs a base image (OCT image) to the mathematical model as an input image, thereby acquiring medical information output by the mathematical model based on the base image. The OCT devices 11A and 11B capture OCT images, which are images of tissues of a living body. In this embodiment, the OCT devices 11A and 11B capture images of fundus tissues of a subject's eye as images of tissues of a living body. However, the tissues of a living body captured by the OCT devices 11A and 11B may be tissues different from the fundus tissues of the subject's eye (for example, the anterior segment of the subject's eye, or tissues other than the subject's eye).

[0036] As an example, a personal computer (hereinafter referred to as "PC") is used as the mathematical model construction device 1 of this embodiment. Although details will be described later, the mathematical model construction device 1 constructs a mathematical model by training (learning) the mathematical model using an OCT image acquired from the OCT device 11A (hereinafter referred to as "training OCT image") and medical information obtained from the training OCT image (in this embodiment, an image with improved image quality of the training OCT image). However, a device that can function as the mathematical model construction device 1 is not limited to a PC. For example, the OCT device 11A may function as the mathematical model construction device 1. Furthermore, control units of a plurality of devices (for example, a CPU of a PC and a CPU 13A of the OCT device 11A) may cooperate to construct a mathematical model.

[0037] In addition, in this embodiment, a CPU is used as an example of a controller that performs various processes. However, it goes without saying that a controller other than a CPU may be used for at least a part of the various devices. For example, a GPU may be used as a controller to speed up the processing.

[0038] The mathematical model construction device 1 will be described. The mathematical model construction device 1 is disposed, for example, at a manufacturer that provides an OCT image processing device 21 or an OCT image processing program to a user. The mathematical model construction device 1 includes a control unit 2 that performs various control processes, and a communication I / F 5. The control unit 2 includes a CPU 3 that is a controller that manages control, and a storage device 4 that can store programs, data, and the like. The storage device 4 stores a mathematical model construction program for executing the mathematical model construction process described below. In addition, the communication I / F 5 connects the mathematical model construction device 1 to other devices (for example, the OCT device 11A and the OCT image processing device 21, etc.).

[0039] The mathematical model construction device 1 is connected to an operation unit 7 and a display device 8. The operation unit 7 is operated by a user to input various instructions to the mathematical model construction device 1. For example, at least one of a keyboard, a mouse, a touch panel, etc. can be used for the operation unit 7. Note that a microphone or the like for inputting various instructions may be used together with or instead of the operation unit 7. The display device 8 displays various images. For the display device 8, various devices capable of displaying images (for example, at least one of a monitor, a display, a projector, etc.) can be used. Note that, in this disclosure, "image" includes both still images and moving images.

[0040] The mathematical model construction device 1 can acquire data of an OCT image (hereinafter, sometimes simply referred to as an "OCT image") from the OCT device 11A. The mathematical model construction device 1 may acquire the data of the OCT image from the OCT device 11A by, for example, at least one of wired communication, wireless communication, a removable storage medium (for example, a USB memory), and the like.

[0041] The OCT image processing device 21 will be described. The OCT image processing device 21 is disposed in, for example, a facility (for example, a hospital or a medical examination facility) where a diagnosis or examination is performed on a subject. The OCT image processing device 21 includes a control unit 22 that performs various control processes, and a communication I / F 25. The control unit 22 includes a CPU 23 that is a controller that manages control, and a storage device 24 that can store programs, data, and the like. The storage device 24 stores an OCT image processing program for executing OCT image processing (see FIG. 3) described later. The OCT image processing program includes a program for realizing the mathematical model constructed by the mathematical model construction device 1. The communication I / F 25 connects the OCT image processing device 21 to other devices (for example, the OCT device 11B and the mathematical model construction device 1, etc.).

[0042] The OCT image processing device 21 is connected to an operation unit 27 and a display device 28. As with the operation unit 7 and display device 8 described above, various devices can be used for the operation unit 27 and the display device 28.

[0043] The OCT image processing device 21 can acquire an OCT image from the OCT device 11B. The OCT image processing device 21 may acquire an OCT image from the OCT device 11B by at least one of wired communication, wireless communication, a removable storage medium (e.g., a USB memory), etc. The OCT image processing device 21 may also acquire a program or the like that realizes the mathematical model constructed by the mathematical model construction device 1 via communication or the like.

[0044] The OCT devices 11A and 11B will be described. As an example, in this embodiment, a case will be described in which an OCT device 11A that provides an OCT image to the mathematical model construction device 1 and an OCT device 11B that provides an OCT image to the OCT image processing device 21 are used. However, the number of OCT devices used is not limited to two. For example, the mathematical model construction device 1 and the OCT image processing device 21 may acquire OCT images from multiple OCT devices. Also, the mathematical model construction device 1 and the OCT image processing device 21 may acquire OCT images from one common OCT device.

[0045] The OCT device 11 (11A, 11B) includes a control unit 12 (12A, 12B) that performs various control processes, and an OCT section 16 (16A, 16B). The control unit 12 includes a CPU 13 (13A, 13B) that is a controller that manages control, and a storage device 14 (14A, 14B) that can store programs, data, and the like. When the OCT device 11 executes at least a part of the OCT image processing (see FIG. 3) described below, it goes without saying that at least a part of the OCT image processing program for executing the OCT image processing is stored in the storage device 14.

[0046] The OCT unit 16 includes various components necessary for capturing an OCT image of a living tissue (in this embodiment, the fundus tissue of the test eye). The OCT unit 16 in this embodiment includes an OCT light source, a branching optical element that branches the OCT light emitted from the OCT light source into measurement light and reference light, a scanning unit that scans the measurement light, an optical system that irradiates the test eye with the measurement light, a light receiving element that receives a composite light of the light reflected by the test eye tissue and the reference light, and the like.

[0047] The OCT device 11 can capture two-dimensional tomographic images and three-dimensional tomographic images of the fundus of the subject's eye. In particular, the CPU 13 scans the OCT light (measurement light) on a scan line to capture a two-dimensional tomographic image of a cross section passing through the scan line. The CPU 13 can also capture a three-dimensional tomographic image of tissue by two-dimensionally scanning the OCT light. For example, the CPU 13 acquires a plurality of two-dimensional tomographic images by scanning the measurement light on each of a plurality of scan lines at different positions within a two-dimensional area when the tissue is viewed from the front. Next, the CPU 13 acquires a three-dimensional tomographic image by combining the captured two-dimensional tomographic images.

[0048] Furthermore, the CPU 13 can also capture multiple OCT images of the same part by scanning the measurement light multiple times on the same part of the tissue (on the same scan line in this embodiment). The CPU 13 can acquire a summed image in which the influence of speckle noise is suppressed by performing an addition process on multiple OCT images of the same part. The image quality of the two-dimensional tomographic image can be improved by performing an addition process on multiple two-dimensional tomographic images of the same part. The addition process may be performed, for example, by averaging pixel values ​​of pixels at the same position among the multiple OCT images (i.e., an averaging process may be performed). The more images are subjected to the addition process, the easier it is to suppress the influence of speckle noise, but the longer the imaging time. The OCT device 11 performs a tracking process to make the scanning position of the OCT light follow the movement of the subject's eye while capturing multiple OCT images of the same part.

[0049] (Mathematical model construction process) 2, a description will be given of a mathematical model construction process executed by the mathematical model construction device 1. The mathematical model construction process is executed by the CPU 3 in accordance with a mathematical model construction program stored in the storage device 4.

[0050] In the mathematical model construction process, a mathematical model that outputs medical information based on an OCT image is constructed by training the mathematical model with multiple training data sets. The training data sets include input data (input training data) and output data (output training data). The mathematical model can output various types of medical information. The type of training data set used to train the mathematical model is determined according to the type of medical information to be output by the mathematical model.

[0051] In this embodiment, an example is given of an OCT image (e.g., a two-dimensional tomographic image) being input as a base image to a mathematical model, and an OCT image (high-quality image) with improved quality of the base image is output to the mathematical model as medical information. In this case, in this embodiment, a training image (two-dimensional tomographic image) which is an OCT image of the tissue of the subject's eye is used as input training data, and an OCT image of the same site with higher quality than the input training data is used as output training data to train the mathematical model. Note that the high-quality image indicates at least one of, for example, an image with reduced noise from the input base image, an image with increased resolution from the original image, and an image with improved visibility from the original image.

[0052] FIG. 2 shows an example of a training data set (input training data and output training data) when data of a high-quality OCT image is output to a mathematical model as medical information. In the example shown in FIG. 2, a CPU 3 acquires a set 40 of a plurality of OCT images 400A-400X captured at the same tissue site. The CPU 3 uses a portion of the plurality of OCT images 400A-400X in the set 40 (a number less than the number used for averaging the output training data described later) as input training data (data of training images). The CPU 3 also acquires an arithmetic average image 41 of the plurality of OCT images 400A-400X in the set 40 as output training data. When a mathematical model is trained using the input training data and output training data shown in FIG. 2, an OCT image is input to the trained mathematical model as a base image, and high-quality image data in which the influence of speckle noise is suppressed is output as medical information.

[0053] However, the configuration of the mathematical model may be changed. For example, the mathematical model may execute an analysis process on at least one of a specific structure and a disease shown in the OCT image, and output data indicating the analysis result as medical information. In this case, at least one of the analysis results may be output, such as a layer of a biological tissue (e.g., fundus tissue of a test eye), a boundary between layers of a biological tissue (e.g., fundus tissue), a specific structure of a tissue (e.g., blood vessels present in the fundus, optic disc, or macula), a disease site present in a biological tissue, etc. In addition, the mathematical model may execute an automatic diagnosis process on a tissue shown in the OCT image, and output data indicating the automatic diagnosis result as medical information. In addition, the mathematical model may output confidence information indicating the confidence of a process (e.g., analysis process of a structure or disease, etc.) executed on an input OCT image, as medical information. The mode of the training dataset is appropriately selected according to the function of the mathematical model to be constructed, etc.

[0054] The mathematical model construction process will be described. The CPU 3 acquires at least a part of the OCT image captured by the OCT device 11A as input training data. Next, the CPU 3 acquires output training data corresponding to the input training data. An example of the correspondence between the input training data and the output training data is as described above.

[0055] Next, the CPU 3 executes training of a mathematical model using the training data set by a machine learning algorithm, such as a neural network, a random forest, boosting, a support vector machine (SVM), etc.

[0056] Neural networks are a method to mimic the behavior of biological neural networks. Examples of neural networks include feedforward neural networks, radial basis function (RBF) networks, spiking neural networks, convolutional neural networks, recurrent neural networks (recurrent neural networks, feedback neural networks, etc.), and probabilistic neural networks (Boltzmann machines, Bayesian networks, etc.).

[0057] Random forest is a method to generate a large number of decision trees by learning based on randomly sampled training data. When using random forest, the branches of multiple decision trees that have been trained in advance as classifiers are traced, and the average (or majority vote) of the results obtained from each decision tree is taken.

[0058] Boosting is a method to generate a strong classifier by combining multiple weak classifiers. A strong classifier is constructed by sequentially training simple weak classifiers.

[0059] SVM is a method for constructing a two-class pattern classifier using linear input elements. For example, SVM learns the parameters of the linear input elements based on the criterion of finding the margin-maximizing hyperplane that maximizes the distance from each data point from the training data (hyperplane separation theorem).

[0060] A mathematical model refers to, for example, a data structure for predicting the relationship between input training data and output training data. A mathematical model is constructed by training using a training data set. As described above, a training data set is a set of input training data and output training data. For example, correlation data (e.g., weights) between each input and output is updated by training.

[0061] In this embodiment, a multi-layered neural network is used as the machine learning algorithm. The neural network includes an input layer for inputting data, an output layer for generating data to be predicted, and one or more hidden layers between the input layer and the output layer. A plurality of nodes (also called units) are arranged in each layer. In detail, in this embodiment, a convolutional neural network (CNN), which is a type of multi-layered neural network, is used. However, other machine learning algorithms may be used. For example, generative adversarial networks (GAN), which utilize two competing neural networks, may be adopted as the machine learning algorithm.

[0062] The above process is repeated until the construction of the mathematical model is completed. When the construction of the mathematical model is completed, a program and data for realizing the constructed mathematical model are incorporated into the OCT image processing device 21. In addition, statistical information on the characteristics of a plurality of training images (i.e., a plurality of OCT images used as input training data) used to train the constructed mathematical model is stored in the storage device 24 of the OCT image processing device 21. In this embodiment, statistical information on the shooting range in the depth direction of the plurality of training images is stored in the storage device 24 as statistical information on the characteristics of the plurality of training images. In this embodiment, statistical information on at least one of the characteristics of the plurality of training images, such as the number of pixels, resolution, pixel value, etc., is also stored in the storage device 24 as statistical information on the characteristics of the plurality of training images. However, it is also possible to change the statistical information on the characteristics of the training images stored in the storage device 24. For example, only statistical information on the shooting range in the depth direction of the training images may be stored in the storage device 24. In addition, instead of the statistical information on the shooting range in the depth direction, only statistical information on at least one of the characteristics, such as the number of pixels, resolution, pixel value, etc., may be stored in the storage device 24.

[0063] (OCT image processing) An example of OCT image processing executed by the OCT image processing device 21 will be described with reference to Figs. 3 to 7. In the OCT image processing, an OCT image is input to a mathematical model trained by a machine learning algorithm to acquire medical information. As described above, the mathematical model is trained in advance by a plurality of training data sets (a plurality of training images) so as to output medical information by inputting an OCT image. Here, in the OCT image processing of this embodiment, a normalization process is performed to approximate at least one of the characteristics of the OCT image to statistical information of the characteristics of the plurality of training images, and the normalized OCT image (normalized image) is input to the mathematical model. As a result, higher quality medical information is easily output appropriately by the mathematical model. The OCT image processing illustrated in Fig. 3 is executed by the CPU 23 according to an OCT image processing program stored in the storage device 24.

[0064] First, the CPU 23 acquires an OCT image 60 of tissue of a living body (in this embodiment, a tomographic image of the fundus tissue of the subject's eye) captured by the OCT device 11B (S1). An example of a two-dimensional OCT image 60 captured by the OCT device 11B is shown in FIG. 4 to FIG. 6. In this disclosure, the Z direction (the up-down direction in FIG. 4 to FIG. 6) along the optical axis of the OCT measuring light is the depth direction of the tissue. The two-dimensional OCT image 60 captured by the OCT device is composed of a plurality of A-scan images. The A-scan image is a pixel row extending in the direction along the optical axis of the OCT measuring light (i.e., the Z direction, which is the depth direction). In other words, the two-dimensional OCT image 60 is constructed by arranging a plurality of A-scan images extending in the Z direction in the X direction (in this embodiment, the direction in which the spot of the OCT measuring light is scanned on the tissue) perpendicularly intersecting the Z direction. In addition, when a tomographic image of the fundus tissue of the test eye is captured by the OCT device 11B illustrated in this embodiment, most of the layers of the fundus tissue shown in the captured tomographic image are often curved and extending in the X direction perpendicular to the Z direction along the optical axis of the OCT measurement light.

[0065] In this embodiment, additional information indicating the characteristics of the OCT image 60 is added to the data of each OCT image (for example, each data of a three-dimensional OCT image). In S1, the additional information added to the OCT image 60 is acquired together with the OCT image 60. In this embodiment, additional information conforming to an international standard (for example, Digital Imaging and Communications in Medicine: DICOM, etc.) that defines a communication method and a storage method of medical image data is added to each OCT image. The CPU 23 acquires the additional information conforming to DICOM together with the OCT image 60. The CPU 23 performs normalization processing of the OCT image 60, which will be exemplified in S2 to S9 described later, by referring to the additional information acquired in S1. The additional information indicates at least one of a plurality of characteristics such as the size of the OCT image (the size in the depth direction of the tissue), the number of pixels, the resolution, and the pixel value. As described above, statistical information on the characteristics of a plurality of training images used to train the mathematical model is stored in the storage device 24 of the OCT image processing device 21.

[0066] The CPU 23 executes a process of approximating the width of the imaging range in the depth direction of the tissue, among the multiple characteristics of the OCT image 60, to statistical information on the width of the imaging range in the depth direction of multiple training images used to train the mathematical model (S2 to S5). First, the CPU 23 refers to the supplementary information of the OCT image 60 and determines whether the imaging range in the depth direction of the OCT image 60 acquired in S1 is approximate to the statistical information on the imaging range in the depth direction of the training image (S2). In S2, for example, it may be determined whether the imaging range is approximated depending on whether the difference between the imaging range in the depth direction of the OCT image 60 acquired in S1 and the statistical information on the imaging range in the depth direction of the training image is less than a threshold value. If the imaging range in the depth direction is approximated (S2: YES), there is a high possibility that high-quality medical information will be acquired without adjusting the imaging range of the OCT image 60, and the process proceeds directly to S7.

[0067] When the depth of the imaging range of the OCT image 60 acquired in S1 is narrower than the statistical information of the depth of the imaging range of the training image (S3: YES), the CPU 23 adds a background region 61 (see FIG. 4 and FIG. 5) in the depth direction of the OCT image 60 acquired in S1 (S4). As a result, a normalized image 70 is generated in which the depth of the imaging range of the OCT image 60 acquired in S1 is approximated to the statistical information of the depth of the imaging range of the training image. By executing the process of S4, the image statistics (for example, at least one of the average brightness and the standard deviation) of the OCT image 60 acquired in S1 approaches the image statistics of the multiple training images. Therefore, even if the depth of the imaging range of the OCT image 60 acquired in S1 differs from the multiple training images used for training the mathematical model, the normalized image 70 is input to the mathematical model in S10 (described later), which makes it easier for the mathematical model to appropriately output high-quality medical information.

[0068] An example of detailed processing executed in S4 will be described. As shown in FIG. 4, there is a case where an OCT image 60 is captured such that an image of a tomography is located at approximately the center of the imaging range in the depth direction, regardless of the size (width) of the imaging range in the depth direction. In this case, the CPU 23 adds a background region 61 to each of both ends in the depth direction of the OCT image 60 (each of the upper and lower ends in FIG. 4). By adding the background region 61 to each of both ends in the depth direction of the OCT image 60, the depth direction position of the image region in the normalized image 70 to which the pair of background regions 61 are added is easily approximated to the statistical depth direction position of the image region in the multiple training images. As a result, the quality of medical information acquired in S10, which will be described later, is easily further improved. Note that a background region 61 of the same size may be added to each of both ends in the depth direction of the OCT image 60. In this case, the depth direction position of the image region in the normalized image 70 is easily approximated to the position of the image region in the multiple training images with higher accuracy.

[0069] Also, as shown in Fig. 5, there are cases where an OCT image 60 is captured with the surface layer position of the tissue being substantially constant, regardless of the extent (width) of the imaging range in the depth direction. In this case, the CPU 23 adds a background region 61 only to the deep layer side (lower part in Fig. 5), which is opposite to the surface layer side of the tissue, among both ends in the depth direction of the OCT image 60. As a result, the depth direction position of the image region in the normalized image 70 to which the background region 61 has been added, becomes more likely to approximate the statistical depth direction position of the image region in the multiple training images.

[0070] Returning to the description of FIG. 3, if the imaging range in the depth direction of the OCT image 60 acquired in S1 is wider than the statistical information of the imaging range in the depth direction of the training image (S3: NO), the CPU 23 extracts a part of the area in the depth direction from the OCT image 60 acquired in S1 (S5). As a result, a normalized image 70 is generated in which the imaging range in the depth direction of the OCT image 60 acquired in S1 is approximated to the statistical information of the imaging range in the depth direction of the training image. By executing the process of S5, the image statistics (for example, at least one of the average brightness and the standard deviation) of the OCT image 60 acquired in S1 approach the image statistics of a plurality of training images. Therefore, as in the case where the process of S4 is executed, the normalized image 70 is input to the mathematical model in S10 described later, which makes it easier for high-quality medical information to be appropriately output by the mathematical model.

[0071] An example of detailed processing executed in S5 will be described. As shown in Fig. 6, in S5, when extracting a partial region in the depth direction from the OCT image 60, the CPU 23 extracts at least a part of the image region in which the image of tissue appears in the OCT image 60. As a result, a normalized image 70 in which the image region of tissue appearing in the OCT image 60 has been appropriately extracted is input to a mathematical model in S10 described later. Therefore, medical information related to the image region is appropriately acquired.

[0072] As shown in FIG. 6, in S5, when extracting a part of the region in the depth direction from the OCT image 60, the CPU 23 extracts at least a part of the image region in a state where the inclination of the tissue layer shown in the OCT image 60 is reduced. Therefore, even if the tissue layer shown in the OCT image 60 is curved, many areas of the tissue image region shown in the OCT image 60 can be appropriately extracted in S5. Therefore, in S10 described later, medical information of higher quality can be easily obtained. Note that, in cases where information on pixels located in a direction intersecting the depth direction with respect to the pixel of interest (e.g., the X direction) is used for processing using a mathematical model, the accuracy of the processing using the mathematical model itself can be easily improved by reducing the inclination of the layer.

[0073] A method for extracting an image region in a state where the inclination of the layer is reduced can be appropriately selected. For example, assume that the pixel value of the region where an image is captured is greater than the pixel value of the background region where no image is captured. The OCT image 60 is configured by arranging a plurality of pixel rows extending in the depth direction (e.g., A-scan images extending in a direction along the optical axis of the OCT measurement light) in a direction intersecting the depth direction (e.g., X direction). The CPU 23 calculates a composite pixel value (e.g., a sum or average value of a plurality of pixel values) of a plurality of pixels included in the frame while moving a frame having the same width as the width in the depth direction of the region to be extracted in the depth direction on each pixel row extending in the depth direction. The CPU 23 sets the region within the frame where the composite pixel value calculated within the frame is the highest as the region to be extracted from the pixel row of interest. The above process is performed for each of the plurality of pixel rows arranged in a direction intersecting the depth direction, and an image region is extracted from each of the plurality of pixel rows. In this case, a plurality of image regions extracted from each of a plurality of pixel rows are arranged in a direction intersecting the depth direction, so that at least a portion of the image region is extracted with reduced tilt of the tissue layer depicted in the OCT image 60. Note that the positions at which the image regions are extracted from each of the plurality of pixel rows are stored in the storage device 24 for reference in the restoration process (S11) described later.

[0074] The CPU 23 may also perform image alignment in the depth direction (which may also be expressed as a tilt reduction process) between a plurality of pixel rows (e.g., a plurality of A-scan images) extending in the depth direction. The CPU 23 may extract a portion of a region (e.g., a rectangular region, etc.) in the depth direction from the OCT image 60 in which the image alignment has been performed. For example, the CPU 23 may move each of a plurality of small regions (e.g., a plurality of A-scan images, etc.) extending in the Z direction, which is the depth direction, in the OCT image 60 in the Z direction to align the Z-direction positions of the images included in each small region. As a result, the tilt of the layer is appropriately reduced. In detail, the CPU 23 may move each of the plurality of small regions in the Z direction so that the positions at which the luminance is maximum among the plurality of small regions coincide in the Z direction. The CPU 23 may also detect the amount of positional deviation between adjacent small regions by a phase-only correlation method, template matching, or the like, and perform positional deviation of the plurality of small regions so that the detected amount of deviation is eliminated. The movement direction and amount of movement of each small area are stored in the storage device 24 for reference in the restoration process (S11) described later.

[0075] The timing of executing the tilt reduction process is not limited to the timing of S5. For example, the CPU 23 may execute the tilt reduction process in any case where the width of the imaging range in the depth direction of the OCT image 60 acquired in S1 is wider, narrower, or similar to the statistical information of the width of the imaging range in the depth direction of the training image. In this case, for example, the tilt reduction process may be executed before the judgment in S2 described above or before the processing in S7 described later is executed.

[0076] Next, CPU 23 executes a process of approximating a characteristic, other than the width of the imaging range in the depth direction, among the multiple characteristics of OCT image 60, to statistical information of the characteristics of the multiple training images (S7 to S9). In this embodiment, in S7 to S9, a process of approximating statistical information of at least any of the characteristics of OCT image 60, such as the number of pixels, resolution, pixel value, etc., to statistical information of the characteristics of the multiple training images is performed.

[0077] First, the CPU 23 detects an image region in which an image of tissue appears in the OCT image 60 acquired in S1 (S7). For example, the CPU 23 may input the OCT image 60 acquired in S1 to a mathematical model that has been trained in advance to output a detection result (analysis result) of an image region in an OCT image, thereby acquiring a detection result of the image region output by the mathematical model. The CPU 23 may also detect the image region in the OCT image 60 by executing known image processing on the OCT image 60. Note that in this embodiment, all regions other than the image region in the OCT image 60 are background regions. Therefore, in the processing of S7, the background region is detected along with the image region.

[0078] The CPU 23 calculates (S8) statistical information on the characteristics of at least a part (all in this embodiment) of the image region of the OCT image 60 acquired in S1. Note that in S3 of this embodiment, statistical information on the characteristics of at least a part (all in this embodiment) of the background region in which no image is captured of the OCT image 60 is also calculated.

[0079] The CPU 23 approximates the statistical information of the image region characteristics in the OCT image 60 acquired in S1 to the statistical information of the image region characteristics in a plurality of training images used to train the mathematical model (S9). As a result, a normalization process more suitable for the OCT image 60 is performed compared to the case where the statistical information of the entire OCT image 60 acquired in S1 is calculated collectively and approximated to the statistical information of the training images. Therefore, in the process of S10 described later, higher quality medical information is more likely to be appropriately output by the mathematical model.

[0080] In addition, in S9 of the present embodiment, a process is also performed to approximate the statistical information of the characteristics of the background region in the OCT image 60 acquired in S1 to the statistical information of the characteristics of the background region in a plurality of training images used to train the mathematical model. As a result, the characteristics of the background region as well as the characteristics of the image region are approximated to the statistical information of the training images. Therefore, it becomes easier for higher quality medical information to be appropriately output by the mathematical model. However, in S9, only the statistical information of the characteristics of the image region may be processed.

[0081] The CPU 23 inputs a normalized image obtained by performing normalization processing (S2 to S9) on the OCT image 60 acquired in S1 to the mathematical model. As described above, the mathematical model is trained in advance according to a machine learning algorithm so as to output medical information when the OCT image (in this embodiment, the normalized image) is input. The CPU 23 acquires the mathematical model output by the mathematical model (S10). As a result, it becomes easier to acquire medical information of higher quality than when the OCT image 60 acquired in S1 is directly input to the mathematical model.

[0082] In addition, in the process of S5 described above, at least a part of the image area may be extracted in a state where the inclination of the tissue layer captured in the OCT image 60 is reduced. In this case, as shown in FIG. 7, the CPU 23 executes a restoration process to restore the arrangement of the medical information acquired by inputting the normalized image 70 extracted in S5 into a mathematical model to the arrangement before the inclination of the layer is reduced (S11). Therefore, the arrangement of the medical information is appropriately restored to match the arrangement of the actually captured tissue. Therefore, it becomes easier to obtain medical information that is high quality and appropriately arranged. Note that the medical information acquired in S10 of this embodiment is a high-quality image 80 (see FIG. 7) in which the image quality of the OCT image 60 acquired in S1 is improved. In this case, the arrangement of the medical information restored in S11 is the arrangement of the tissue captured in the high-quality image. In the example shown in FIG. 7, the arrangement of the tissue in the high-quality image 80 acquired in S10 is restored to generate a restored image 81. However, if the medical information acquired in S11 is an analysis result of a structure or disease (for example, an analysis result of a layer boundary, etc.), the arrangement of the medical information restored in S11 may be the arrangement of the analyzed structure or disease. As described above, the process of S11 may be performed by referring to the information stored when extracting a part of the region in S5.

[0083] Furthermore, in the process of S4 described above, a background region 61 (see FIGS. 4 and 5) may be added in the depth direction of the OCT image 60. In this case, the CPU 23 executes an exclusion process (S12) to exclude information on a region corresponding to the background region 61 added in S4 from medical information acquired by inputting the normalized image 70 to which the background region has been added in S4 into the mathematical model. As a result, the region of the acquired medical information is returned to the region before the background region 61 was added. This makes it easier to obtain high-quality medical information with an appropriate region.

[0084] The techniques disclosed in the above embodiments are merely examples. Therefore, it is possible to change the techniques exemplified in the above embodiments. First, it is possible to execute only a part of the processes exemplified in the above embodiments. For example, it is possible to omit one of the normalization process (S2 to S5) of the imaging range in the depth direction and the normalization process (S7 to S9) of the characteristic different from the imaging range in the depth direction in the OCT image processing shown in FIG. 3. Also, in the normalization process (S7 to S9) of the characteristic different from the imaging range in the depth direction, it is possible to process the statistical information of the characteristic in the entire OCT image 60 without processing the statistical information of the characteristic in the image region of the OCT image 60.

[0085] In addition, in S5 of the above embodiment, at least a part of the image region is extracted in a state where the inclination of the tissue layer shown in the OCT image 60 is reduced. However, in S5, the image region of the OCT image 60 may be extracted without reducing the inclination of the layer. In this case, a method for extracting the image region can be appropriately selected. For example, assume that the pixel value of the region where the image is shown is greater than the pixel value of the background region where no image is shown. The CPU 23 may calculate a composite pixel value (for example, a sum or average value of a plurality of pixel values) of a plurality of pixels included in the frame while moving a frame of the same size as the region to be extracted in the depth direction in the OCT image 60. The CPU 23 may set the extraction position of the region to the position of the frame where the composite pixel value calculated in the frame is the highest. In this case, it becomes easier to appropriately extract as much of the image region as possible. In addition, when extracting a portion of the region in the depth direction from the OCT image 60, the CPU 23 may set the extraction position of the region in the OCT image 60 to a position where the image of the tissue is located in the center in the depth direction. In this case, many areas of the tissue image area shown in the OCT image 60 are easily extracted appropriately in S5. In particular, the OCT image 60 is configured by arranging a plurality of pixel rows extending in a direction intersecting the depth direction (e.g., the X direction) in the depth direction. For example, the CPU 23 may calculate a composite value of pixel values ​​(e.g., a sum or average value of a plurality of pixel values) for each of the plurality of pixel rows extending in a direction intersecting the depth direction. The CPU 23 may match the center of the depth direction of the area to be extracted to the pixel row having the highest composite value of the calculated pixel values ​​among the plurality of pixel rows extending in a direction intersecting the depth direction. In this case, many areas of the tissue image area shown in the OCT image 60 are easily extracted appropriately in the process of S5.

[0086] The process of acquiring an OCT image in S1 of FIG. 3 is an example of an "image acquiring step." The normalization processes performed on the OCT image in S2 to S9 are an example of a "normalization step." The process of acquiring medical information in S10 is an example of a "medical information acquiring step." The restoration process performed in S11 is an example of a "restoration step." The exclusion process performed in S12 is an example of an "exclusion step."

[0087] 11(11A,11B) OCT device 21 OCT image processing device 23 CPU 24 Storage device 60 OCT images 61 Background area 70 Normalized Images

Claims

1. An OCT image processing device that processes data of an OCT image of a living body tissue captured by an OCT device, The control unit of the OCT image processing device includes: An image acquisition step of acquiring an OCT image captured by an OCT device; a normalization step of performing a normalization process on the OCT image acquired in the image acquisition step; a medical information acquisition step of inputting a normalized image, which is the OCT image that has been subjected to the normalization process, into a mathematical model trained by a machine learning algorithm to output medical information in response to the input of an OCT image, thereby acquiring medical information output by the mathematical model; Run An OCT image processing device characterized in that the normalization process performed in the normalization step approximates at least some of the characteristics of the OCT image to statistical information of the characteristics of a plurality of training images, which are OCT images used to train the mathematical model.

2. 2. The OCT image processing apparatus according to claim 1, In the normalization process, the control unit refers to additional information indicating characteristics of the OCT image that is attached to the data of the OCT image acquired in the image acquisition step, and approximates at least some of the characteristics of the OCT image indicated by the referenced additional information to statistical information of the characteristics of the multiple training images.

3. 3. The OCT image processing apparatus according to claim 1, In the normalization process, when the depth-direction shooting range of tissue of the OCT image acquired in the image acquisition step is narrower than the statistical information of the depth-direction shooting range of the multiple training images, the control unit adds a background region in the depth direction of the OCT image, thereby approximating the depth-direction shooting range of the OCT image to the depth-direction shooting range of the multiple training images.

4. 4. The OCT image processing apparatus according to claim 3, The control unit is The OCT image processing device further comprises, when the background region is added in the depth direction of the OCT image in the normalization process, an exclusion step of excluding information of a region corresponding to the added background region from medical information obtained by inputting the normalized image, which is an image to which the background region has been added, into the mathematical model.

5. 5. The OCT image processing apparatus according to claim 1, In the normalization process, when the depth-direction shooting range of tissue of the OCT image acquired in the image acquisition step is wider than the statistical information of the depth-direction shooting range of the multiple training images, the control unit extracts a portion of the depth-direction area from the OCT image, thereby approximating the depth-direction shooting range of the OCT image to the depth-direction shooting range of the multiple training images.

6. 6. The OCT image processing apparatus according to claim 5, An OCT image processing device characterized in that, in the normalization processing, when extracting a portion of a region in the depth direction from the OCT image, the control unit extracts at least a portion of an image region in which an image of tissue is captured within the OCT image.

7. 7. The OCT image processing apparatus according to claim 6, The OCT image is an image in which a tissue layer is captured, In the normalization process, the control unit extracts at least a portion of the image area while reducing the inclination of the tissue layer appearing in the OCT image when extracting a portion of the image area in the depth direction from the OCT image.

8. 8. The OCT image processing apparatus according to claim 7, The control unit is An OCT image processing device characterized in that, in the normalization process, when at least a portion of the image area is extracted with the tilt of the tissue layer shown in the OCT image reduced, a restoration step is further performed in which the arrangement of medical information obtained by inputting the extracted image, the normalized image, into the mathematical model is restored to the arrangement before the tilt of the tissue layer shown in the OCT image was reduced.

9. 9. The OCT image processing apparatus according to claim 1, In the normalization process, the control unit calculates statistical information of characteristics for at least a portion of the image area in the OCT image captured in the image acquisition step, in which an image of tissue is captured, and approximates the calculated statistical information of characteristics of the image area to statistical information of characteristics of the image area in the multiple training images.

10. An OCT image processing program executed by an OCT image processing device that processes data of an OCT image of a living body tissue captured by an OCT device, The OCT image processing program is executed by a control unit of the OCT image processing device, An image acquisition step of acquiring an OCT image captured by an OCT device; a normalization step of performing a normalization process on the OCT image acquired in the image acquisition step; a medical information acquisition step of inputting a normalized image, which is the OCT image that has been subjected to the normalization process, into a mathematical model trained by a machine learning algorithm to output medical information in response to the input of an OCT image, thereby acquiring medical information output by the mathematical model; The OCT image processing device executes the above. An OCT image processing program characterized in that the normalization process performed in the normalization step approximates at least some of the characteristics of the OCT image to statistical information of the characteristics of a plurality of training images, which are OCT images used to train the mathematical model.

Images