Medical image processing device, OCT device, and medical image processing program

The medical image processing device and program address image distortion by detecting and correcting positional deviations in three-dimensional images, enhancing accuracy and efficiency in medical imaging.

JP7595405B2Active Publication Date: 2024-12-06NIDEK CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2019066537
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2019-03-29
Publication Date
2024-12-06
Estimated Expiration
2039-03-29

AI Technical Summary

Technical Problem

Existing medical image capturing technologies face challenges in correcting image distortion caused by tissue movement during light scanning, requiring separate capture of correction images and increasing photographic time or man-hours.

Method used

A medical image processing device and program that detect and correct positional deviations in three-dimensional images by analyzing the motion and shape components of tissue movement, allowing for real-time correction without separate capture of correction images.

Benefits of technology

Effectively corrects image distortion due to tissue movement, reducing the need for additional imaging time and methods, and enabling accurate image correction even without prior correction images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007595405000001
    Figure 0007595405000001
  • Figure 0007595405000002
    Figure 0007595405000002
  • Figure 0007595405000003
    Figure 0007595405000003
Patent Text Reader

Abstract

To provide a medical image processing apparatus for appropriately correcting distortion of an image of a living tissue generated by scanning light.SOLUTION: A correction target image is generated such that a plurality of partial images acquired by continuously receiving light from a tissue of a living body are arranged while scanning light on the tissue. The control unit of the medical image processing apparatus executes a misalignment amount detection step, a component acquisition step, and a correction step. In the misalignment amount detection step, the control unit detects the misalignment amount between the plurality of partial images (S3). In the component acquisition step, the control unit acquires an estimation result of at least one of the component of a misalignment amount due to a movement of the tissue and the component of a misalignment amount due to the shape of the tissue (S4). In the correction step, the control unit corrects the position of each partial image according to the component of the misalignment amount (S7).SELECTED DRAWING: Figure 5
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a medical image processing device, an OCT device, and a medical image processing program executed in the medical image processing device, which processes image data of biological tissue generated by scanning with light. [Background technology]

[0002] There is known a medical imaging device that generates an image by scanning light on tissue of a living body while continuously receiving light from the tissue. For example, the imaging device described in Patent Document 1 repeatedly scans light to form data of two-dimensional images (partial images) of the tissue on multiple scan lines. The imaging device arranges data of the multiple two-dimensional images to generate data of a three-dimensional image of the tissue. In addition, for example, a scanning laser ophthalmoscope (SLO) and a rolling shutter imaging device are also known as medical imaging devices that generate images by scanning light. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-64220 A Summary of the Invention [Problem to be solved by the invention]

[0004] When generating image data of biological tissue by scanning light, the tissue may move during the light scanning, and the generated image may be distorted. As a method for correcting image distortion caused by tissue movement, for example, a method is considered in which at least a part of an image of the same tissue is captured as a correction image in a time shorter than the light scanning time required to generate the entire image to be corrected, and the image distortion is corrected using the correction image as a reference. Since the correction image is captured in a short time, image distortion caused by tissue movement is unlikely to occur. Therefore, by using the correction image as a reference, the distortion of the image to be corrected may be corrected.

[0005] However, in the above method, it is necessary to capture a correction image separately from the image to be corrected. Therefore, for example, it is necessary to improve the imaging device, devise an imaging method, extend the imaging time, increase the number of steps required for imaging, etc. In addition, if a correction image for the image to be corrected is not captured, it is not possible to correct the image after the fact. Therefore, in the above method, it is difficult to appropriately correct the image distortion caused by the movement of the tissue.

[0006] A typical object of the present disclosure is to provide a medical image processing device, an OCT device, and a medical image processing program for appropriately correcting distortion of an image of biological tissue generated by scanning with light. [Means for solving the problem]

[0007] A medical image processing apparatus according to an exemplary embodiment of the present disclosure is a medical image processing apparatus for processing data of an image of a living tissue, the image being: raw Body tissues of In a two-dimensional space On a scan line extending in a first direction, light Spot While scanning the tissue, light from the tissue is received continuously in time. The process of acquiring two-dimensional partial images extending in the first direction and the depth direction extending along the optical axis of the light is repeated by moving the position of the scan line in a second direction intersecting both the first direction and the depth direction, and the acquired data of the multiple partial images are arranged in the second direction. It is generated by It is a single 3D image. The control unit of the medical image processing device aligned in the second direction A positional deviation amount between the plurality of partial images at least in the depth direction of the tissue is , within the same three-dimensional imagea component acquisition step of acquiring, from the detected amount of misalignment, at least one of an estimated result of a movement component which is a component of the amount of misalignment caused by the movement of the tissue during the light scan and a shape component which is a component of the amount of misalignment caused by the shape of the tissue; aligned in the second direction within the same three-dimensional image A correction step is executed in which the position of each of the plurality of partial images is corrected based on the component of the acquired positional deviation amount.

[0008] The OCT device according to a typical embodiment of the present disclosure is an OCT device that captures an image of tissue by receiving interference light between a reference light and a reflected light of a measurement light irradiated on a biological tissue, and a control unit of the OCT device of In a two-dimensional space On a scan line extending in a first direction, Measurement light Spot While scanning, the interference light is received continuously in time. and repeating the process of acquiring a two-dimensional partial image extending in the first direction and a depth direction extending along the optical axis of the light by moving the position of the scan line in a second direction intersecting both the first direction and the depth direction; The acquired data of the plurality of partial images In the second direction By arranging Single A three-dimensional image is generated, and the same three-dimensional image is aligned in the second direction A positional deviation amount between the plurality of partial images at least in the depth direction of the tissue is , within the same three-dimensional image a component acquisition step of acquiring, from the detected positional deviation amount, at least one of an estimated result of a movement component, which is a component of the positional deviation amount caused by the movement of the tissue during the scanning of the measurement light, and a shape component, which is a component of the positional deviation amount caused by the shape of the tissue; aligned in the second direction within the same three-dimensional image A correction step is executed in which the position of each of the plurality of partial images is corrected based on the component of the acquired positional deviation amount.

[0009] A medical image processing program provided by an exemplary embodiment of the present disclosure is a medical image processing program executed by a medical image processing device that processes data of an image of a living tissue, the image being: raw Body tissues of In a two-dimensional space On a scan line extending in a first direction, light SpotWhile scanning the tissue, light from the tissue is received continuously in time. The process of acquiring two-dimensional partial images extending in the first direction and the depth direction extending along the optical axis of the light is repeated by moving the position of the scan line in a second direction intersecting both the first direction and the depth direction, and the acquired data of the multiple partial images are arranged in the second direction. It is generated by It is a single 3D image. The medical image processing program is executed by a control unit of the medical image processing device, thereby aligned in the second direction A positional deviation amount between the plurality of partial images at least in the depth direction of the tissue is , within the same three-dimensional image a component acquisition step of acquiring, from the detected amount of misalignment, at least one of an estimated result of a movement component which is a component of the amount of misalignment caused by the movement of the tissue during the light scan and a shape component which is a component of the amount of misalignment caused by the shape of the tissue; aligned in the second direction within the same three-dimensional image and a correction step of correcting the position of each of the plurality of partial images based on the acquired component of the amount of positional deviation.

[0010] According to the medical image processing device, OCT device, and medical image processing program of the present disclosure, distortion of an image of biological tissue generated by scanning light is appropriately corrected. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of a medical image generating system 100. [Diagram 2] 1 is an explanatory diagram for explaining a method in which the imaging device 1 captures a three-dimensional image of tissue 50 of a living body. [Diagram 3] 11 is an explanatory diagram for explaining a state in which a plurality of partial images 61 are arranged in the Y direction. [Figure 4] FIG. 13 is a diagram showing an example of a YZ tomographic image 62 extracted from a three-dimensional correction target image in which distortion has not been corrected. [Diagram 5] 11 is a flowchart of medical image processing executed by the medical image processing device 40. [Figure 6] FIG. 5 is a diagram showing a YZ tomographic image 63 extracted from the same position as in FIG. 4 after the alignment of a plurality of partial images 61 has been performed. [Figure 7]FIG. 13 is a diagram showing an example of a graph plotting the amount of movement of each of a plurality of partial images 61 when aligning the partial images 61. [Figure 8] 8 is a graph showing a function f(y) that approximates the amount of movement (amount of positional deviation) shown in FIG. 7. [Figure 9] 9 is a graph showing a primitive function F(y) of the function f(y) shown in FIG. 8. [Figure 10] FIG. 7 is a diagram showing a YZ tomographic image 64 extracted from the same positions as those in FIGS. 4 and 6 after the positions of a plurality of partial images 61 have been corrected based on the shape components. [Figure 11] 6 is a flowchart showing an example of a modified example of the medical image processing illustrated in FIG. 5. [Figure 12] 1A to 1C are diagrams illustrating an example of a method for correcting distortion of a two-dimensional correction target image 70. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <Summary> The medical image processing device exemplified in the present disclosure processes data of an image of a living tissue. The image to be processed (i.e., a correction target image to be corrected for distortion) is a two-dimensional image or a three-dimensional image, and is generated by arranging data of a plurality of partial images acquired by successively receiving light from the tissue in time while scanning light on the living tissue. The control unit of the medical image processing device executes a positional deviation amount detection step, a component acquisition step, and a correction step. In the positional deviation amount detection step, the control unit detects the positional deviation amount between the plurality of partial images constituting the image. In the component acquisition step, the control unit acquires, from the positional deviation amount detected in the positional deviation amount detection step, at least one of an estimated result of a component of the positional deviation amount caused by the movement of the tissue during the light scan (hereinafter referred to as a "movement component") and a component of the positional deviation amount caused by the shape of the tissue (hereinafter referred to as a "shape component"). In the correction step, the control unit corrects the position of each of the plurality of partial images based on the acquired positional deviation amount component.

[0013] The positional deviation amount detected in the positional deviation amount detection step includes a motion component, which is a component of the positional deviation amount caused by the movement of the tissue during the optical scan, and a shape component, which is a component of the positional deviation amount caused by the actual shape of the tissue. Therefore, the control unit can appropriately correct image distortion caused by tissue movement by acquiring an estimation result of at least one of the motion component and the shape component of the positional deviation amount and correcting the positions of the multiple partial images based on the estimated component.

[0014] According to the technology disclosed herein, there is no need to capture a correction image of the same tissue as the correction target image. Therefore, for example, there is no need to devise a shooting method or increase the shooting time. In addition, it is possible to correct a past correction target image for which no correction image has been captured after the fact. Therefore, image distortion caused by the movement of the tissue during scanning is appropriately corrected.

[0015] Of course, there may be cases where the movement component and the shape component cancel each other out, and the positional deviation value detected in the positional deviation detection step is 0. Therefore, when the positional deviation value detected in the positional deviation detection step is 0, both the movement component and the shape component are not always 0.

[0016] The control unit may acquire an estimated result of a shape component of the amount of misalignment in the component acquisition step. The control unit may correct the position of each of the partial images constituting the aligned image obtained by aligning the multiple partial images in accordance with the shape component in the correction step.

[0017] In each of the partial images constituting the aligned image, both the positional deviation caused by the movement of the tissue and the positional deviation caused by the shape of the tissue are eliminated. In other words, in the aligned image, the distortion caused by the movement of the tissue is eliminated, but the shape information of the tissue is also lost. Therefore, the control unit can appropriately correct the correction target image to an image in which the distortion is eliminated and which has shape information by correcting the positions of the partial images constituting the aligned image according to the shape components.

[0018] A specific method for correcting the position of the partial image according to the shape component can be appropriately selected. First, the amount of misalignment between the multiple partial images detected in the misalignment amount detection step may be the amount of misalignment of each partial image with respect to a partial image to be compared (for example, adjacent partial images, etc.). The partial image to be compared is another partial image whose position is compared to detect the amount of misalignment of one partial image. Similarly, the shape component of the estimated amount of misalignment may be the shape component of the amount of misalignment of each partial image with respect to the partial image to be compared. In this case, the control unit corrects the position of each partial image constituting the aligned image to a position shifted by the shape component with respect to the partial image to be compared in the correction step (if the value of the shape component is "0", the position of the partial image is set to a position without misalignment with respect to the partial image to be compared). As a result, the distortion of the correction target image caused by the movement of tissue is appropriately corrected.

[0019] The control unit may also accumulate shape components of the positional deviation amount to calculate a cumulative shape component. The shape component of the positional deviation amount corresponds to the inclination of the estimated tissue shape. Therefore, the calculated cumulative shape component indicates the estimated tissue shape. In this case, in the correction step, the control unit corrects the position of each partial image constituting the aligned image to a position shifted by the cumulative shape component from each aligned position (if the value of the cumulative shape component is "0", the position of the partial image is not changed). As a result, the distortion of the correction target image caused by the movement of the tissue is appropriately corrected.

[0020] The control unit may acquire an estimation result of a movement component of the positional deviation amount in the component acquisition step. The control unit may correct a position of each of the partial images constituting the correction target image to a position where the movement component is eliminated in the correction step.

[0021] As described above, the positional deviation amount detected in the positional deviation amount detection step includes a motion component and a shape component. Therefore, the control unit can appropriately correct the distortion of the correction target image caused by the movement of tissue by correcting the position of each partial image constituting the correction target image (the original image not subjected to processing such as alignment) to a position where the estimated motion component is eliminated.

[0022] A specific method for correcting the position of the partial image according to the motion component can be appropriately selected. For example, in the correction step, the control unit corrects the position of each partial image constituting the correction target image to a position shifted by an amount obtained by reversing the positive and negative of the motion component from the comparison target partial image (when the value of the motion component is "0", the position of the partial image is set to a position with no shift from the comparison target). As a result, the distortion of the correction target image caused by the motion of tissue is appropriately corrected.

[0023] The control unit may also accumulate the motion components of the positional deviation amounts to calculate the accumulated motion component. In the correction step, the control unit may correct the position of each partial image constituting the correction target image to a position shifted by an amount obtained by reversing the positive and negative of the accumulated motion component (if the value of the accumulated motion component is "0", the position of the partial image is not changed). As a result, the distortion of the correction target image caused by the motion of tissue is appropriately corrected.

[0024] In the misalignment amount detection step, the control unit may detect the misalignment amount between the multiple partial images based on the amount of movement of each partial image when aligning each of the multiple partial images. In this case, the misalignment amount between the multiple partial images is appropriately detected by utilizing a process for aligning the multiple partial images. Note that the control unit may detect the misalignment amount by virtually performing the alignment process without actually performing it.

[0025] A specific method for detecting the amount of misalignment based on the amount of movement in the alignment of the partial images can be appropriately selected. For example, the control unit may obtain the amount of movement of each of the partial images when aligning each of the multiple partial images, and detect the amount of misalignment by reversing the positive and negative values ​​of the obtained amount of movement.

[0026] In the positional deviation detection step, the control unit may detect the amount of positional deviation between the plurality of partial images based on the amount of movement of each partial image when adjusting the position of each partial image to a position where the similarity (e.g., correlation) between the partial images and other partial images (partial images to be compared as described above) is equal to or greater than a threshold. In this case, the similarity between the partial images is used to appropriately detect the amount of positional deviation between partial images including a movement component and a shape component.

[0027] In the positional deviation amount detection step, the control unit may detect the positional deviation amount between the plurality of partial images based on the movement amount of each partial image when a position of a specific tissue included in each of the plurality of partial images among tissues captured in the image is brought closer. In this case, the position of the plurality of partial images is (virtually) aligned so that the position of the specific tissue in each partial image is brought closer, whereby the positional deviation amount between the partial images including the movement component and the shape component is appropriately detected.

[0028] It is more preferable that the specific tissue used as the reference for alignment is a tissue commonly included in each of the multiple partial images. For example, when the partial images are tomographic images, the specific tissue may be at least one of a specific layer and a specific boundary included in the tomographic image. The number of specific tissues used as the reference for alignment is not limited to one. For example, alignment of the 1st to Lth (L is a natural number) partial images may be performed using a first tissue as a reference, and alignment of the Lmth to Nth (m is a natural number and N>L) partial images may be performed using a second tissue as a reference. In this case, even if the multiple partial images do not all contain a common tissue, the amount of misalignment between the partial images is appropriately detected.

[0029] In the component acquisition step, the control unit may acquire a function approximating the transition of the amount of misalignment between the multiple partial images as an estimated result of the shape component of the amount of misalignment. The amount of misalignment of a partial image in which a misalignment due to tissue movement occurs is likely to be a value that deviates (is far from) the amount of misalignment of a partial image in which a misalignment due to movement occurs (i.e., a partial image in which only a misalignment due to the actual shape of the tissue occurs). Therefore, the function approximating the transition of the amount of misalignment can be appropriately used as an estimated result of the shape component in which the movement component has been removed from the amount of misalignment.

[0030] The function approximating the transition of the positional deviation amount may be appropriately selected according to the expected shape of the actual tissue. As described above, the shape component of the positional deviation amount corresponds to the inclination of the shape of the estimated tissue. Therefore, the cumulative shape component obtained by accumulating the shape components of the positional deviation amount indicates the shape of the estimated tissue. Therefore, for example, when it is expected that the shape of the actual tissue can be approximated by a quadratic function, the function approximating the transition of the positional deviation amount may be a linear function. Also, when it is expected that the shape of the actual tissue can be approximated by a cubic function, the function approximating the transition of the positional deviation amount may be a quadratic function. In other words, the derivative of the function expected to be able to approximate the shape of the actual tissue may be adopted as the function approximating the transition of the positional deviation amount.

[0031] In the component acquisition step, the control unit may acquire an estimated result of the shape component of the positional deviation amount by removing a value whose difference from other positional deviation amounts is equal to or greater than a threshold value from the positional deviation amounts between the multiple partial images. In other words, the control unit may acquire, as an estimated result of the shape component, a positional deviation amount whose difference from other positional deviation amounts is less than a threshold value from among the positional deviation amounts between the multiple partial images. As described above, the positional deviation amount of a partial image in which a positional deviation due to tissue movement occurs is likely to be a value that is out of line with the positional deviation amount of a partial image in which a positional deviation due to movement does not occur. Therefore, the shape component of the positional deviation amount is appropriately acquired by removing a value whose difference from other positional deviation amounts is equal to or greater than a threshold value from the positional deviation amounts between the multiple partial images.

[0032] In addition, when acquiring a shape component by comparing the difference in the misalignment amount with a threshold, the relationship between the misalignment amount of the object to be determined whether to remove or not and the misalignment amount of the object to be compared (i.e., the above-mentioned "other misalignment amount") can be appropriately selected. For example, the misalignment amount of the object to be compared may be a misalignment amount adjacent to the misalignment amount of the object to be determined along the direction in which the partial images are arranged. In other words, the control unit may exclude the misalignment amount of the object to be determined when the difference between the misalignment amount of the object to be determined and the misalignment amount adjacent to this misalignment amount is equal to or greater than a threshold. In addition, the misalignment amount of the object to be compared may be a plurality of misalignment amounts in the vicinity of the misalignment amount of the object to be determined. In other words, the control unit may exclude the misalignment amount of the object to be determined when the difference between the average value of a plurality of misalignment amounts in the vicinity of the misalignment amount of the object to be determined and the misalignment amount of the object to be determined is equal to or greater than a threshold.

[0033] The control unit may execute an interpolation process for interpolating the removed value for the positional deviation amount obtained by removing the value whose difference is equal to or greater than a threshold value from the positional deviation amount in the component acquisition step. In this case, the correction target image is corrected without any blanks in the shape components, thereby further improving the accuracy of image correction.

[0034] It is also possible to change the method of acquiring the estimated result of the shape component of the misalignment amount. For example, the control unit may acquire the estimated result of the shape component by performing a process of removing outliers from the misalignment amount acquired in the misalignment amount detection step. In this case, the control unit may perform a process of removing outliers using, for example, a median filter.

[0035] In the component acquisition step, the control unit may acquire a function that approximates the transition of the amount of misalignment between the multiple partial images, and acquire the difference of the amount of misalignment from the value of the acquired function as an estimated result of the movement component of the amount of misalignment. In this case, the movement component included in the amount of misalignment is appropriately acquired by using the function. Note that the specific method of acquiring the function (e.g., a method of selecting a function according to the expected shape of actual tissue, etc.) may be the same as the method described above.

[0036] It is also possible to change the method of acquiring the estimated result of the movement component of the positional deviation amount. For example, the control unit may acquire a shape component using various methods (e.g., a method using a threshold value, a method using a median filter, etc.), and acquire the difference of the positional deviation amount from the acquired shape component as the estimated result of the movement component of the positional deviation amount.

[0037] The control unit may execute an accumulated shape component acquisition step. In the accumulated shape component acquisition step, the control unit accumulates the shape components acquired in the component acquisition step to acquire an accumulated shape component indicating the shape of the estimated tissue. As described above, the shape component of the positional deviation amount corresponds to the inclination of the shape of the estimated tissue. Therefore, the accumulated shape component obtained by accumulating the shape components can be appropriately used as information indicating the estimated shape of the tissue. For example, as described above, the accumulated shape component may be used in the correction step. Furthermore, the accumulated shape component may be used when a doctor diagnoses the tissue, etc.

[0038] The method of accumulating the shape components may be selected as appropriate. For example, when the control unit acquires a function approximating the transition of the positional deviation amount as an estimated result of the shape component, the control unit may acquire the accumulated shape component by integrating the acquired function to obtain a primitive function. In this case, the control unit can appropriately acquire the accumulated shape component while suppressing an increase in the amount of calculation processing. The control unit may also acquire the accumulated shape component by sequentially accumulating a plurality of positional deviation amounts.

[0039] The control unit may further execute a component storage step and a post-correction step. In the component storage step, the control unit stores at least one of the shape components and the cumulative shape components in the storage device. In the post-correction step, the control unit corrects the position of each of the multiple partial images based on the components related to the image to be corrected and the image of the same object, among the components stored in the storage device. In this case, when images of the same object are photographed multiple times, the control unit can execute the post-correction step using the stored components without executing the component acquisition step. Therefore, the image can be appropriately corrected by simple processing. For example, this is useful in cases where images of the same tissue are periodically photographed to observe the progress of the tissue. Also, even if it is difficult to acquire the components of the positional deviation of the image to be corrected (for example, when the tissue always moves during photography), the image to be corrected can be appropriately corrected if the components are stored in the storage device.

[0040] The partial image may be a two-dimensional image extending in the Z direction along the optical axis of the light and the X direction perpendicular to the Z direction. The correction target image may be a three-dimensional image generated by arranging a plurality of partial images in the Y direction intersecting both the Z direction and the X direction. In general, the time required for optical scanning tends to be longer for capturing a three-dimensional image than for capturing a two-dimensional image. The longer the time required for optical scanning, the more likely image distortion due to tissue movement will occur. However, according to the technology disclosed herein, even if a three-dimensional image is distorted due to tissue movement, the effect of distortion is appropriately suppressed.

[0041] Various devices can be used as the imaging device that captures (generates) the three-dimensional correction target image. For example, an OCT device that captures a tomographic image of tissue using the principle of optical coherence tomography can be used. For example, the imaging method using the OCT device can employ a method of scanning a spot of light (measurement light) two-dimensionally to obtain a three-dimensional tomographic image, or a method of scanning light extending in a one-dimensional direction to obtain a three-dimensional tomographic image (so-called line field OCT). Also, an MRI (magnetic resonance imaging) device, a CT (computed tomography) device, or the like may be used.

[0042] The correction target image may be a two-dimensional image. In this case, the imaging device for capturing the correction target image may be, for example, an OCT device for capturing a two-dimensional tomographic image, a scanning laser ophthalmoscope (SLO) for capturing a two-dimensional front image, or a rolling shutter imaging device. The imaging device may generate image data by receiving light reflected by tissue, or may generate image data by receiving light (e.g., fluorescence) emitted from tissue irradiated with light.

[0043] <Embodiment> A typical embodiment according to the present disclosure will be described below. In this embodiment, an example will be given of a case where an image of the fundus tissue of the subject's eye E photographed by an OCT device is used as the correction target image. However, the correction target image may be an image of tissue other than the fundus tissue. For example, the correction target image may be an image of tissue other than the fundus of the subject's eye E (e.g., the anterior segment of the eye, etc.), or an image of biological tissue other than the subject's eye E (e.g., the skin, digestive organs, or brain, etc.). In addition, as described above, the photographing device for photographing the correction target image is not limited to the OCT device.

[0044] A schematic configuration of a medical image acquisition system 100 of this embodiment will be described with reference to FIG. 1. The medical image acquisition system 100 of this embodiment includes an imaging device 1 and a medical image processing device 40. The imaging device 1 acquires (captures) a plurality of partial images by scanning light on tissue of a living body and receiving light from the tissue continuously in time. In this embodiment, the partial images are two-dimensional tomographic images. A correction target image is generated by arranging the plurality of partial images. In this embodiment, the correction target image is a three-dimensional tomographic image. The medical image processing device 40 processes the data of the image acquired by the imaging device 1 (for example, a process of correcting distortion of the correction target image, etc.).

[0045] The configuration of the imaging device 1 of this embodiment will be described. The imaging device (OCT device) 1 includes an OCT section 10 and a control unit 30. The OCT section 10 includes an OCT light source 11, a coupler (light splitter) 12, a measurement optical system 13, a reference optical system 20, and a light receiving element 22.

[0046] The OCT light source 11 emits light (OCT light) for acquiring image data. The coupler 12 splits the OCT light emitted from the OCT light source 11 into measurement light and reference light. The coupler 12 of this embodiment combines the measurement light reflected by tissue (the fundus of the patient's eye E in this embodiment) with the reference light generated by the reference optical system 20 to cause interference. That is, the coupler 12 of this embodiment serves both as a branching optical element that branches the OCT light into measurement light and reference light, and as a combining optical element that combines the reflected light of the measurement light with the reference light. It is also possible to change the configuration of at least one of the branching optical element and the combining optical element. For example, an element other than a coupler (for example, a circulator, a beam splitter, etc.) may be used.

[0047] The measurement optical system 13 guides the measurement light split by the coupler 12 to the subject, and returns the measurement light reflected by the tissue to the coupler 12. The measurement optical system 13 includes a scanning unit (scanner) 14, an irradiation optical system 16, and a focus adjustment unit 17. The scanning unit 14 can scan the measurement light in a two-dimensional direction intersecting the optical axis of the measurement light by being driven by a driving unit 15. In this embodiment, two galvanometer mirrors capable of deflecting the measurement light in different directions are used as the scanning unit 14. However, another device for deflecting light (for example, at least one of a polygon mirror, a resonant scanner, an acousto-optical element, etc.) may be used as the scanning unit 14. The irradiation optical system 16 is provided downstream of the optical path (i.e., on the subject side) from the scanning unit 14, and irradiates the tissue with the measurement light. The focus adjustment unit 17 adjusts the focus of the measurement light by moving an optical member (for example, a lens) included in the irradiation optical system 16 in a direction along the optical axis of the measurement light.

[0048] The reference optical system 20 generates a reference light and returns it to the coupler 12. The reference optical system 20 of this embodiment generates the reference light by reflecting the reference light split by the coupler 12 using a reflective optical system (e.g., a reference mirror). However, the configuration of the reference optical system 20 can also be changed. For example, the reference optical system 20 may transmit the light incident from the coupler 12 without reflecting it and return it to the coupler 12. The reference optical system 20 includes an optical path length difference adjustment unit 21 that changes the optical path length difference between the measurement light and the reference light. In this embodiment, the optical path length difference is changed by moving the reference mirror in the optical axis direction. The configuration for changing the optical path length difference may be provided in the optical path of the measurement optical system 13.

[0049] The light receiving element 22 detects an interference signal by receiving interference light between the measurement light and the reference light generated by the coupler 12. In this embodiment, the principle of Fourier domain OCT is adopted. In Fourier domain OCT, the spectral intensity of the interference light (spectral interference signal) is detected by the light receiving element 22, and a complex OCT signal is acquired by Fourier transforming the spectral intensity data. Examples of Fourier domain OCT that can be adopted include Spectral-domain OCT (SD-OCT) and Swept-source OCT (SS-OCT). It is also possible to adopt, for example, Time-domain OCT (TD-OCT).

[0050] In this embodiment, the measuring light spot is scanned in a two-dimensional region by the scanning unit 14 to obtain three-dimensional image data. However, it is also possible to change the principle of obtaining the three-dimensional image data. For example, the three-dimensional image data may be obtained by the principle of line field OCT (hereinafter referred to as "LF-OCT"). In LF-OCT, measuring light is simultaneously irradiated onto an irradiation line extending in a one-dimensional direction in tissue, and the reflected light of the measuring light and the interference light of the reference light are received by a one-dimensional light receiving element (e.g., a line sensor) or a two-dimensional light receiving element. The measuring light is scanned in a direction intersecting the irradiation line in a two-dimensional measurement region to obtain three-dimensional OCT data.

[0051] The control unit 30 is responsible for various controls of the photographing device 1. The control unit 30 includes a CPU 31, a RAM 32, a ROM 33, and a non-volatile memory (NVM) 34. The CPU 31 is a controller that performs various controls. The RAM 32 temporarily stores various information. The ROM 33 stores programs executed by the CPU 31, various initial values, etc. The NVM 34 is a non-transient storage medium that can retain the stored contents even if the power supply is cut off.

[0052] A monitor 37 and an operation unit 38 are connected to the control unit 30. The monitor 37 is an example of a display unit that displays various images. The operation unit 38 is operated by a user in order for the user to input various operation instructions to the photographing device 1. For the operation unit 38, various devices such as a mouse, a keyboard, a touch panel, a foot switch, etc. may be used. Note that various operation instructions may be input to the photographing device 1 by inputting sound into a microphone.

[0053] A schematic configuration of the medical image processing device 40 will be described. In this embodiment, a personal computer (hereinafter referred to as "PC") is used as the medical image processing device 40. However, a device other than a PC may be used as the medical image processing device. For example, the imaging device 1 itself may function as a medical image processing device that performs a process of correcting distortion of a correction target image. The medical image processing device 40 includes a CPU 41, a RAM 42, a ROM 43, and an NVM 44. A medical image processing program for executing medical image processing (see FIG. 5 and FIG. 11) described later may be stored in the NVM 44. In addition, a monitor 47 and an operation unit 48 are connected to the medical image processing device 40. The monitor 47 is an example of a display unit that displays various images. The operation unit 48 is operated by a user so that the user can input various operation instructions to the medical image processing device 40. As with the operation unit 38 of the imaging device 1, various devices such as a mouse, a keyboard, and a touch panel can be used for the operation unit 48. In addition, various operation instructions may be input to the medical image processing device 40 by inputting sound to the microphone 46.

[0054] The medical image processing device 40 can acquire various data (e.g., data of images captured by the imaging device 1, etc.) from the imaging device 1. The various data may be acquired by at least one of wired communication, wireless communication, and a removable storage device (e.g., a USB memory), etc.

[0055] With reference to Figs. 2 to 4, an example of a method for capturing an image to be corrected by the medical image processing device 40 of this embodiment and a configuration of the image to be corrected will be described. As shown in Fig. 2, the imaging device 1 of this embodiment scans light (measurement light) within a two-dimensional region 51 in a tissue 50 of a living body (in the example shown in Fig. 2, the fundus tissue). In detail, the imaging device 1 of this embodiment scans light on a scan line 52 extending in a predetermined direction within the region 51 to obtain (capture) a two-dimensional image extending in the Z direction along the optical axis of the light and the X direction perpendicular to the Z direction as a partial image 61 (see Fig. 3). In the example shown in Fig. 2, the Z direction is a direction perpendicular to the two-dimensional region 51 (depth direction), and the X direction is a direction in which the scan line 52 extends. Next, the imaging device 1 moves the position of the scan line 52 in the Y direction within the region 51 to repeatedly obtain the partial image 61. The Y direction is a direction that intersects both the Z direction and the X direction (perpendicularly in this embodiment). As a result, a plurality of partial images 61 (two-dimensional tomographic images) are acquired that pass through each of the plurality of scan lines 52 and extend in the depth direction of the tissue. Next, as shown in Fig. 3, the plurality of partial images 61 are arranged in the Y direction to generate a correction target image that is a three-dimensional tomographic image of the region 51.

[0056] Here, the tissue 50 may move while scanning with light, and the generated correction target image may be distorted. Particularly in this embodiment, the time required to move the scan line 52 in the Y direction is longer than the time required to scan with light on one scan line 52. Therefore, positional deviations due to the movement of the tissue 50 are likely to occur between the multiple partial images 61. If positional deviations occur between the multiple partial images 61, the correction target image 60 will be distorted.

[0057] 4 is a diagram showing an example of a two-dimensional tomographic image (hereinafter referred to as a "YZ tomographic image") 62 extending in the Y and Z directions, extracted from a three-dimensional correction target image in which distortion caused by the movement of the tissue 50 has not been corrected. In the example shown in FIG. 4, the tissue 50 moves while the scan line 52 is moved in the Y direction, resulting in a significant positional deviation, particularly in the Z direction, between the multiple partial images 61 (see FIG. 3). As a result, the layers of the tissue 50 are distorted in the YZ tomographic image 62 shown in FIG. 4. In the medical image processing described below, the distortion of the correction target image caused by the movement of the tissue 50 is appropriately corrected.

[0058] The medical image processing in this embodiment will be described with reference to Figs. 5 to 10. In this embodiment, the medical image processing device 40, which is a PC, acquires data of an image to be corrected from the photographing device 1 and corrects the acquired image to be corrected. However, as described above, another device may function as the medical image processing device. For example, the photographing device (OCT device in this embodiment) 1 itself may execute the medical image processing. Also, a plurality of control units (for example, the CPU 31 of the photographing device 1 and the CPU 41 of the medical image processing device 40) may cooperate to execute the medical image processing. In this embodiment, the CPU 41 of the medical image processing device 40 executes the medical image processing shown in Fig. 5 according to a medical image processing program stored in the NVM 44.

[0059] First, the CPU 41 acquires data of an image to be corrected (S1). For example, a user operates the operation unit 38 or the operation unit 48 (see FIG. 1) to select an image for which distortion is to be corrected from among a plurality of images. The CPU 41 acquires data of the image selected by the user as data of the image to be corrected. The CPU 41 also sets center coordinates in the image to be corrected. The position of the center coordinates can be set appropriately, but it is more desirable to set the center coordinates in a region that is likely to include tissue (for example, the center of the image).

[0060] Next, the CPU 41 judges whether or not at least one of the shape components of the positional deviation amount related to the correction target image acquired in S1 and the information of the accumulated shape components obtained by accumulating the shape components is stored in a storage device (e.g., NVM 44, etc.) (S2). Although details will be described later, the CPU 41 corrects the distortion of the correction target image based on the shape components of the positional deviation amount of the partial image 61 or the accumulated shape components. If the shape components or the accumulated shape components have already been stored for an image of the same subject as the correction target image (S2: YES), the process of acquiring the shape components and the accumulated shape components (S3 to S5) is omitted, and the process proceeds directly to S7 (details will be described later). As a result, the distortion of the correction target image is appropriately corrected by simple processing. For example, it is useful in cases where images of the same tissue 50 are periodically captured for follow-up observation. Even if it is difficult to acquire the component of the positional deviation amount of the correction target image, if at least one of the shape components and the accumulated shape components is stored in the storage device, the correction of the correction target image is appropriately performed. If neither the geometric components nor the accumulated geometric components related to the correction target image are stored (S2: NO), the CPU 41 executes a process of acquiring the geometric components and the accumulated geometric components (S3 to S5).

[0061] First, the CPU 41 detects the amount of misalignment between the multiple partial images 61 constituting the correction target image (S3). The position of the tissue 50 included in each of the multiple partial images 61 is shifted according to the actual shape of the tissue 50. For example, when the actual shape of the tissue 50 is inclined in the Y direction, the multiple partial images 61 are arranged in the Y direction, so that the position of the tissue 50 included in each of the multiple partial images 61 also gradually shifts according to the actual shape of the tissue 50. In addition, as a result of the tissue 50 moving during the optical scan, the position of the tissue 50 included in each of the multiple partial images 61 may shift. The amount of misalignment between the multiple partial images 61 detected in the process of S3 includes both a component of the amount of misalignment caused by the actual shape of the tissue 50 (hereinafter referred to as a "shape component") and a component of the amount of misalignment caused by the movement of the tissue 50 during the optical scan (hereinafter referred to as a "movement component").

[0062] In this embodiment, the CPU 41 detects the amount of misalignment between the multiple partial images 61 based on the amount of movement of each partial image 61 when aligning the multiple partial images 61 so that the positions of the tissues 50 included in each of the multiple partial images 61 approach each other (S3). In detail, the CPU 41 in this embodiment obtains the amount of movement of each partial image 61 relative to a comparison target image (an adjacent partial image 61 in this embodiment) when aligning the multiple partial images 61, and detects the amount of misalignment between the multiple partial images 61 by reversing the positive and negative values ​​of the obtained amount of movement.

[0063] FIG. 6 shows a YZ tomographic image 63 extracted from the same position as in FIG. 4 after the alignment of each of the multiple partial images 61 is actually performed for the same correction target image as in FIG. 4. As an example, the alignment in this embodiment is performed based on the partial image 61 of y=y0 (i.e., the partial image 61 passing through the center coordinate). By aligning the tissue 50 included in each of the multiple partial images 61, the distortion caused by the movement of the tissue 50 is reduced. As a result, the shape of the tissue 50 shown in the YZ tomographic image 63 shown in FIG. 6 is smoother than the shape of the tissue 50 shown in the YZ tomographic image 62 shown in FIG. 4. However, by aligning the tissue 50 included in each of the multiple partial images 61, information on the actual shape of the tissue 50 is also lost. As a result, in the YZ tomographic image 63 shown in FIG. 6, the shape of the tissue 50 is closer to being flatter than the actual shape, compared to the YZ tomographic image 62 shown in FIG. 4.

[0064] When aligning the partial images 61 in S3, the CPU 41 can use the similarity (e.g., correlation) between the partial images 61. In detail, the CPU 41 aligns the position of each partial image 61 to a position where the similarity between the partial images 61 and other partial images 61 to be compared (e.g., adjacent partial images 61, etc.) is equal to or greater than a threshold (in this embodiment, the position where the similarity is maximum). As a result, as shown in Fig. 6, both the positional shift of the partial image 61 caused by the movement of the tissue 50 and the positional shift of the partial image 61 caused by the actual shape of the tissue 50 are cancelled. Therefore, the amount of positional shift including the shape component and the movement component is appropriately detected based on the amount of movement when aligning the partial images.

[0065] Also, in S3, the CPU 41 may align each of the partial images 61 so that the position of a specific tissue among a plurality of tissues included in each of the plurality of partial images 61 approaches each other. In this case, it is more preferable that the specific tissue used as the reference for alignment is a tissue commonly included in each of the plurality of partial images 61. For example, as shown in FIG. 3, when the partial images 61 are two-dimensional tomographic images, the specific tissue may be at least one of a specific layer and a specific boundary included in the tomographic image. Also, the number of specific tissues used as the reference for alignment is not limited to one. That is, a plurality of tissues commonly included in each of the partial images 61 may be used as the reference for alignment. Also, the CPU 41 may align the 1st to Lth (L is a natural number) partial images 61 using a first tissue as a reference, and align the Lmth to Nth (m is a natural number and N>L) partial images using a second tissue as a reference, for example. In this case, even if there is no tissue that is commonly contained in all of the partial images 61, the partial images 61 can be appropriately aligned.

[0066] In S3 of the present embodiment, the CPU 41 aligns the positions of the multiple partial images 61 by adjusting the position of each partial image 61 with respect to adjacent partial images 61. However, the other partial images 61 to be compared when aligning the positions of each partial image 61 are not limited to the adjacent partial images 61. For example, the CPU 41 may align each partial image 61 by comparing partial images 61 that were scanned with light at similar times.

[0067] 6 shows an example of a YZ tomographic image 63 of an image (hereinafter, referred to as a "aligned image") in which the alignment of the partial images 61 has actually been performed. However, the CPU 41 may detect the amount of misalignment by virtually aligning the partial images 61 instead of actually aligning them.

[0068] Furthermore, in S3 of this embodiment, the partial images 61 are aligned by translating each of the partial images 61 in the Z direction. However, the direction in which each of the partial images 61 is moved in the alignment process in S3 is not limited to the Z direction. For example, the CPU 41 may move each of the partial images 61 in both the Z direction and the X direction. Furthermore, the CPU 41 may perform rotational movement of each of the partial images 61. Furthermore, the CPU 41 may perform at least one of transformation processes such as distortion correction, enlargement, and reduction on each of the partial images 61. The CPU 41 may use a method such as affine transformation for the processes such as movement and transformation.

[0069] FIG. 7 is a graph plotting the amount of movement of each partial image 61 relative to the comparison image when aligning the partial images 61 so that the positions of the tissues 50 included in each of the partial images 61 are closer to each other. The horizontal axis in FIG. 7 indicates the Y direction, and the vertical axis indicates the amount of movement of each partial image 61 relative to the comparison image. When aligning the partial images 61, the amount of movement of each partial image 61 relative to the comparison image (specifically, the value obtained by reversing the positive and negative values ​​of the amount of movement) corresponds to the amount of misalignment between the partial images 61 in the correction target image (i.e., the amount of misalignment of each partial image 16 relative to the comparison image). As described above, when the partial images 61 are aligned, the shape of the tissue 50 becomes flatter than the actual shape. Therefore, the value (i.e., the shape component) obtained by removing the component of the amount of misalignment (movement) caused by the movement of the tissue 50 from the amount of misalignment (movement) of the partial images 61 illustrated in FIG. 7 corresponds to the inclination of the shape of the tissue 50. Here, the amount of positional shift of the partial image 61 including a motion component is likely to be a value that deviates (is far from) the amount of positional shift of the partial image 61 not including a motion component (i.e., the amount of positional shift caused only by the shape of the actual tissue 50).

[0070] Returning to the explanation of FIG. 5, the CPU 41 acquires an estimated result of the shape component from the displacement amount detected in S3 (see FIG. 7) (S4). In detail, the CPU 41 of this embodiment acquires a function approximating the transition of the displacement amount between the multiple partial images 61 (see FIG. 7) as an estimated result of the shape component of the displacement amount. FIG. 8 is a graph showing a function f(y) approximating the displacement amount (movement amount) shown in FIG. 7. As described above, a displacement amount including a movement component is likely to be an outlier compared to a displacement amount not including a movement component. Therefore, by acquiring a function approximating the transition of the displacement amount, outlier values ​​are excluded, and an estimated result of the shape component of the displacement amount is appropriately acquired.

[0071] Next, the CPU 41 accumulates the shape components of the misalignment amount acquired in S4 to acquire the accumulated shape component (S5). In detail, the CPU 41 of this embodiment acquires the accumulated shape component by integrating a function f(y) (see FIG. 8) that approximates the transition of the misalignment amount to obtain a primitive function F(y) (see FIG. 9). As described above, the function f(y) (see FIG. 8) indicating the estimation result of the shape component of the misalignment amount represents the slope of the estimated shape of the tissue. Therefore, the primitive function F(y) (i.e., the accumulated shape component) obtained by integrating and accumulating the function f(y) represents the estimated shape of the tissue.

[0072] Here, in this embodiment, the derivative f(y) (see FIG. 8) approximating the transition of the positional deviation amount and the primitive function F(y) (see FIG. 9) acquired as the cumulative shape component are selected according to the expected shape of the actual tissue 50. In detail, in this embodiment, it is expected that the cross-sectional shape of the fundus tissue 50 captured in the correction target image can be approximated by a quadratic function. Therefore, in this embodiment, in order to make the primitive function F(y) a quadratic function, the derivative f(y) approximating the transition of the positional deviation amount is set to a linear function. However, the derivative f(y) is not limited to a linear function, and the primitive function F(y) is not limited to a quadratic function. For example, when it is expected that the shape of the tissue 50 can be approximated by a cubic function, the derivative f(y) may be a quadratic function. In other words, the derivative of a function that is expected to be able to approximate the shape of the actual tissue 50 may be adopted as a function approximating the transition of the positional deviation amount.

[0073] In addition, in this embodiment, the derivative f(y) is set to a linear function as a result of previously predicting that the cross-sectional shape of the fundus tissue 50 shown in the correction target image can be approximated by a quadratic function. However, the derivative f(y) may be appropriately changed according to the shape of the tissue 50. For example, a two-dimensional YZ tomographic image may be acquired by scanning light in the Y direction in which the multiple partial images 61 are arranged, and the derivative f(y) and the primitive function F(y) may be appropriately set based on the shape of the tissue 50 shown in the acquired YZ tomographic image. In this case, a function is appropriately set according to the shape of the tissue 50. In addition, the derivative f(y) and the primitive function F(y) may be set based on the shape of the tissue 50 shown in at least one of the partial images (XZ tomographic image in this embodiment) included in the correction target image.

[0074] Returning to the description of Fig. 5, the CPU 41 corrects the position of each of the plurality of partial images 61 based on the geometric component of the positional deviation amount (S7). In detail, in this embodiment, the CPU 41 corrects the positions of the plurality of partial images 61 constituting an aligned image (in this embodiment, a three-dimensional image from which the YZ tomographic image 63 shown in Fig. 6 is extracted) obtained by aligning the positions of the plurality of partial images 61 to positions shifted by the cumulative geometric component (see Fig. 9) from each aligned position.

[0075] FIG. 10 shows a YZ tomographic image 64 extracted from the same positions as those in FIG. 4 and FIG. 6 after the positions of the partial images 61 are corrected based on the shape components for the same correction target image as those in FIG. 4 and FIG. 6. As shown in FIG. 10, the positions of the partial images 61 are corrected based on the shape components, so that the distortion of the image caused by the movement of the tissue 50 is appropriately corrected while maintaining a shape close to the shape of the actual tissue 50. Note that the CPU 41 may perform a process of smoothing the deviation between the adjacent partial images 61 for the correction target image in which the positions of the partial images 61 have been corrected in S7. In this case, the corrected image becomes smoother.

[0076] In addition, in S7, it is also possible to correct the position of each of the multiple partial images 61 using a shape component instead of the cumulative shape component. In this case, the CPU 41 corrects the position of each partial image 61 constituting the registration image to a position shifted by the shape component with respect to the comparison target image. As a result, similar to the example shown in FIG. 10, the distortion of the correction target image caused by the movement of the tissue 50 is appropriately corrected. In addition, when the position of the partial image 61 is corrected using the shape component, it is also possible to omit the process of S5. Also, in cases where correction is performed using the shape component or the cumulative shape component stored in the storage device (S2: YES, S7), the registration image may be generated in S7 instead of S3.

[0077] Next, the CPU 41 stores at least one of the geometric components acquired in S4 and the cumulative geometric components acquired in S5 in a storage device (e.g., NVM 44) in association with the subject (S8). The stored parameters can be used later when correcting an image of the same subject.

[0078] Next, the CPU 41 executes at least one of an analysis process, a detection process, and a process for cutting out an arbitrary tomographic image on the image whose distortion has been corrected in S7 (S9). Since the distortion of the image to be processed has been appropriately corrected, the accuracy of various processes such as analysis is ensured. In addition, various processes can be adopted for the analysis process, such as a process for analyzing the distribution of the thickness of a specific layer. In addition, various processes can be adopted for the detection process, such as a process for detecting a specific layer or a specific boundary in the tissue 50. In addition, when cutting out an arbitrary image, the cut out image may be a two-dimensional image or a three-dimensional image.

[0079] The techniques disclosed in the above embodiments are merely examples. Therefore, the techniques exemplified in the above embodiments can be modified. FIG. 11 is a flowchart showing an example of a modified example of the medical image processing exemplified in FIG. 5. A part of the processing shown in FIG. 11 can adopt the same processing as the medical image processing exemplified in FIG. 5. Therefore, among the processing shown in FIG. 11, the processing that can be executed in the same manner as the processing exemplified in FIG. 5 is given the same step numbers as those given in FIG. 5, and the description thereof will be omitted or simplified.

[0080] In the modified example of medical image processing shown in FIG. 11, the CPU 41 corrects the position of each partial image 61 constituting the correction target image based on the movement component of the positional deviation amount. First, the CPU 41 acquires data of the correction target image (S1) and detects the positional deviation amount between the multiple partial images 61 (S3). Next, the CPU 41 acquires an estimation result of the movement component caused by the movement of the tissue 50 from the positional deviation amount detected in S3 (S14). In detail, the CPU 41 acquires a function (for example, see FIG. 8) that approximates the transition of the positional deviation amount detected in S3. The CPU 41 acquires the difference of the positional deviation amount with respect to the value of the acquired function as an estimation result of the movement component of the positional deviation amount. As a result, the movement component of the positional deviation amount is appropriately acquired by using the function. The movement component acquired in S14 is the positional deviation amount of each partial image 61 with respect to the comparison target image caused by the movement of the tissue 50.

[0081] However, it is also possible to change the method of acquiring the estimated result of the movement component in S14. For example, the CPU 41 may acquire the shape component using various methods (for example, a method using a threshold value, or a method using a median filter, which will be described later), and acquire the difference in the positional deviation amount from the acquired shape component as the movement component of the positional deviation amount.

[0082] Next, the CPU 41 accumulates the motion components acquired in S14 to acquire an accumulated motion component (S15). The accumulated motion component indicates the amount of positional deviation of each partial image 61 with respect to the origin, not the comparison image.

[0083] Next, the CPU 41 corrects the position of each of the multiple partial images 61 to a position where the movement component detected in S14 is eliminated (S16). In detail, the CPU 41 corrects the position of each partial image 61 constituting the correction target image (the original image not subjected to processing such as alignment) to a position shifted by an amount obtained by reversing the positive and negative of the movement component from the comparison target image. As a result, the estimated movement component is eliminated, and the distortion of the correction target image is appropriately corrected.

[0084] In addition, in S16, it is also possible to correct the position of each of the multiple partial images 61 using the accumulated motion component instead of the motion component. In this case, the CPU 41 corrects the position of each partial image 61 constituting the correction target image to a position shifted by an amount obtained by reversing the positive and negative of the accumulated motion component. As a result, the estimated motion component is eliminated. In addition, when correcting the position of the partial image 61 using the motion component, it is also possible to omit the process of S15.

[0085] Other modifications may be made to the above embodiment. For example, in S4 (see FIG. 5) of the above embodiment, the CPU 41 obtains an estimation result of the shape component of the positional deviation amount by obtaining a function f(y) that approximates the transition of the positional deviation amount between the multiple partial images 61. However, it is also possible to change the method of obtaining the estimation result of the shape component of the positional deviation amount. For example, the CPU 41 may obtain the estimation result of the shape component by removing, from the positional deviation amount between the multiple partial images 61, values ​​whose difference from other positional deviation amounts is equal to or greater than a threshold value.

[0086] In this case, the relationship between the misalignment amount of the object to be removed or not and the misalignment amount of the object to be compared can be appropriately selected. For example, the misalignment amount of the object to be compared may be a misalignment amount adjacent to the misalignment amount of the object to be compared along the direction in which the partial images 61 are arranged (Y direction in the above embodiment). Also, the misalignment amounts of the object to be compared may be a plurality of misalignment amounts in the vicinity of the misalignment amount of the object to be compared.

[0087] Furthermore, when comparing the difference between the amounts of positional deviation with a threshold value and removing the amounts of positional deviation whose difference is equal to or greater than the threshold value to obtain a geometric component, the CPU 41 may execute an interpolation process for the obtained geometric component to interpolate the removed values. In this case, the correction target image is corrected based on geometric components that have no blanks (empty values), improving the accuracy of the correction.

[0088] Furthermore, the CPU 41 may obtain an estimation result of the shape component of the misalignment amount by performing a process of removing outliers from the misalignment amount detected in S3. In this case, the process of removing outliers may be performed using, for example, a median filter or the like.

[0089] Also, in the above embodiment, a derivative f(y), which is a curve, is acquired as a shape component from the positional deviation amounts of a plurality of two-dimensional partial images 61, and a primitive function F(y), which is a curve, is acquired as a cumulative shape component from the acquired derivative f(y). However, the information of the cumulative shape component that can be used as a reference for correcting the partial images 61 may be information indicating a curved surface instead of a curve. For example, when correcting distortion of a three-dimensional correction target image, a cumulative shape component indicating a curved surface may be acquired based on the positional deviation amounts of a plurality of one-dimensional partial images (for example, one-dimensional partial images extending in the Z direction along the optical axis of light).

[0090] In the above embodiment, the correction target image is a three-dimensional image. However, a two-dimensional image may be used as the correction target image. FIG. 12 is a diagram showing an example of a method for correcting distortion of a two-dimensional correction target image 70. In the example shown in FIG. 12, the correction target image 70 is a two-dimensional XZ tomographic image captured by scanning light in the X direction. Even in the two-dimensional correction target image 70, if tissue moves during the light scan, distortion may occur in the image as shown in FIG. 12. The correction target image 70 shown in FIG. 12 is generated by arranging multiple partial images in the X direction. The partial images may be one-dimensional images extending in the Z direction (for example, one-dimensional images extending in the optical axis direction of the OCT measurement light (so-called A-scan images)) or two-dimensional images. The CPU 41 detects the amount of positional deviation between the multiple partial images arranged in the X direction (S3), and acquires at least one of a shape component and a movement component (S4, S14). After that, the CPU 41 corrects the positions of the multiple partial images based on the acquired components (S7, S16). As a result, the distortion of the two-dimensional image caused by the movement of the tissue is appropriately corrected. Note that the two-dimensional correction target does not have to be a tomographic image, but may be a two-dimensional front image extending in a direction intersecting with the optical axis of the light.

[0091] The process of detecting the amount of misalignment in S3 of FIG. 5 and FIG. 11 is an example of a "misalignment amount detection step." The process of S4 of FIG. 5 and S14 of FIG. 11, and the process of acquiring the estimated results of at least one of the shape components and the motion components are an example of a "component acquisition step." The process of correcting the positions of the multiple partial images 61 in S7 of FIG. 5 and S16 of FIG. 11 is an example of a "correction step." The process of acquiring the cumulative shape components in S5 of FIG. 5 is an example of a "cumulative shape component acquisition step." The process of storing at least one of the shape components and the cumulative shape components in a storage device in S8 of FIG. 5 is an example of a "component storage step." The process of correcting the positions of the multiple partial images in S2: YES and S7 of FIG. 5 is an example of a "post-correction step."

[0092] 1. Imaging device 14 Scanning section 22 Photodetector 40 Medical image processing device 41 CPU 44 NVM 50 organizations 61 Partial Image 100 Medical image generation system

Claims

1. A medical image processing device that processes data of an image of a biological tissue, The image is a single three-dimensional image is generated by arranging data of a plurality of partial images obtained by repeatedly scanning a single light spot on a scan line extending in a first direction within a two-dimensional region of tissue of a living body, continuously receiving light from the tissue in time, and acquiring two-dimensional partial images extending in the first direction and a depth direction extending along an optical axis of the light while moving a position of the scan line in a second direction intersecting both the first direction and the depth direction, and arranging the data of the plurality of partial images in the second direction; The control unit of the medical image processing device a positional deviation amount detection step of detecting, within the same three-dimensional image, a positional deviation amount in at least a depth direction of the tissue between the plurality of partial images arranged in the second direction within the same three-dimensional image; a component acquisition step of acquiring, from the detected positional deviation amount, at least one of an estimated result of a movement component, which is a component of the positional deviation amount caused by the movement of the tissue during the light scan, and a shape component, which is a component of the positional deviation amount caused by the shape of the tissue; a correction step of correcting the positions of each of the plurality of partial images arranged in the second direction within the same three-dimensional image based on the acquired component of the positional deviation amount; A medical image processing device comprising:

2. 2. The medical image processing apparatus according to claim 1, The control unit is In the component acquisition step, an estimation result of the shape component of the positional deviation amount is acquired, A medical image processing apparatus, characterized in that in the correcting step, the position of each of the partial images constituting an aligned image obtained by aligning the partial images is corrected in accordance with the shape components.

3. 2. The medical image processing apparatus according to claim 1, The control unit is In the component acquisition step, an estimation result of the movement component of the positional deviation amount is acquired; A medical image processing apparatus, comprising: a correction step of correcting a position of each of the partial images constituting the three-dimensional image to a position where the movement component is eliminated.

4. 4. The medical image processing apparatus according to claim 1, The control unit, in the position deviation detection step, A medical image processing apparatus comprising: a processor for detecting an amount of positional deviation between a plurality of partial images based on an amount of movement of each of the plurality of partial images when performing position alignment for each of the plurality of partial images.

5. 5. The medical image processing apparatus according to claim 4, The control unit, in the position deviation detection step, A medical image processing device characterized by detecting the amount of positional deviation between multiple partial images based on the amount of movement of each partial image when adjusting the position of each partial image to a position where the similarity between the partial images and other partial images is greater than or equal to a threshold.

6. An OCT apparatus that captures an image of tissue by receiving interference light between a reference light and a reflected light of a measurement light irradiated onto biological tissue, comprising: The control unit of the OCT apparatus includes: A process of scanning a single measurement light spot on a scan line extending in a first direction within a two-dimensional region of a living body tissue, continuously receiving the interference light in time, and acquiring two-dimensional partial images extending in the first direction and a depth direction extending along the optical axis of the light, is repeated by moving the position of the scan line in a second direction intersecting both the first direction and the depth direction, and generating a single three-dimensional image by arranging data of the acquired multiple partial images in the second direction, a positional deviation amount detection step of detecting, within the same three-dimensional image, a positional deviation amount in at least a depth direction of the tissue between the plurality of partial images arranged in the second direction within the same three-dimensional image; A component acquisition step of acquiring, from the detected positional deviation amount, at least one of an estimated result of a movement component which is a component of the positional deviation amount caused by the movement of the tissue during the scanning of the measurement light and a shape component which is a component of the positional deviation amount caused by the shape of the tissue; a correction step of correcting the positions of each of the plurality of partial images arranged in the second direction within the same three-dimensional image based on the acquired component of the positional deviation amount; An OCT apparatus comprising:

7. A medical image processing program executed by a medical image processing device for processing data of an image of a biological tissue, The image is a single three-dimensional image is generated by arranging data of a plurality of partial images obtained by repeatedly scanning a single light spot on a scan line extending in a first direction within a two-dimensional region of tissue of a living body, continuously receiving light from the tissue in time, and acquiring two-dimensional partial images extending in the first direction and a depth direction extending along an optical axis of the light while moving a position of the scan line in a second direction intersecting both the first direction and the depth direction, and arranging the data of the plurality of partial images in the second direction; The medical image processing program is executed by a control unit of the medical image processing device, a positional deviation amount detection step of detecting, within the same three-dimensional image, a positional deviation amount in at least a depth direction of the tissue between the plurality of partial images arranged in the second direction within the same three-dimensional image; a component acquisition step of acquiring, from the detected positional deviation amount, at least one of an estimated result of a motion component, which is a component of the positional deviation amount caused by the movement of the tissue during the light scan, and a shape component, which is a component of the positional deviation amount caused by the shape of the tissue; a correction step of correcting the positions of each of the plurality of partial images arranged in the second direction within the same three-dimensional image based on the acquired component of the positional deviation amount; A medical image processing program for causing the medical image processing device to execute the above-mentioned.

Citation Information

Patent Citations

  • Image processing apparatus, image processing method, program, and program storing medium

    JP2010110556A

  • Method and apparatus for motion correction and image enhancement for optical coherence tomography

    JP2013525035A

  • Optical coherence tomography apparatus

    JP2017064220A

  • Apparatus and method for processing three-dimensional image of anterior eye segment

    JP2017086772A