Medical image processing device and medical image processing program

Abstract

To provide a medical image processing device and a medical image processing program capable of allowing a user to easily and appropriately compare a frontal image and a tomographic image of the same biological tissue.SOLUTION: In an image acquisition step, a control unit acquires a frontal image and a tomographic image captured for the same biological tissue of the same subject. In an image display step, the control unit displays the frontal image and the tomographic image on a display unit. In a magnification synchronization change step, when input of a change instruction for the display magnification on the display unit is received for one of the frontal image and the tomographic image, the control unit changes both the display magnification of the image for which the change instruction for the display magnification is performed and the display magnification of the image for which the change instruction for the display magnification is not performed in the frontal image and the tomographic image in accordance with the input instruction.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present disclosure relates to a medical image processing apparatus that processes image data of a biological tissue, an OCT apparatus, and a medical image processing program executed in the medical image processing apparatus.

Background Art

[0002] Images of biological tissues are useful for assisting in the diagnosis and treatment of patients by users (such as medical staff). In recent years, techniques for capturing tomographic images that extend in the depth direction of biological tissues have also been actively utilized. For example, the ophthalmic imaging apparatus described in Patent Document 1 displays a tomographic image and a frontal image of the eye to be examined side by side on the same screen of a display monitor.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Medical staff often want to obtain useful information about a patient by comparing a frontal image and a tomographic image of the same biological tissue. However, in the conventional technology, the user can only individually change the display modes of the frontal image and the tomographic image, so complicated operations may be required to appropriately compare the frontal image and the tomographic image.

[0005] A typical object of the present disclosure is to provide a medical image processing apparatus and a medical image processing program that can easily and appropriately allow a user to compare a frontal image and a tomographic image of the same biological tissue.

Means for Solving the Problems

[0006] A medical image processing apparatus provided by a typical embodiment in the present disclosure is a medical image processing apparatus that processes image data of a biological tissue. A control unit of the medical image processing apparatus includes: an image acquisition step of acquiring a front image, which is a two-dimensional image of the same biological tissue of the same subject taken from a direction along the optical axis of imaging light, and a tomographic image, which is a two-dimensional image extending in the depth direction of the biological tissue; an image display step of displaying the front image and the tomographic image on a display unit; and a magnification synchronous change step of changing both the display magnification of the image for which a change instruction of the display magnification to the display unit is received among the front image and the tomographic image and the display magnification of the image for which no change instruction of the display magnification is received according to the input instruction when the input of the change instruction of the display magnification to the display unit is received for one of the front image and the tomographic image.

[0007] A medical image processing program provided by a typical embodiment in the present disclosure is a medical image processing program executed by a medical image processing apparatus that processes image data of a biological tissue. When the medical image processing program is executed by a control unit of the medical image processing apparatus, the medical image processing apparatus is caused to execute: an image acquisition step of acquiring a front image, which is a two-dimensional image of the same biological tissue of the same subject taken from a direction along the optical axis of imaging light, and a tomographic image, which is a two-dimensional image extending in the depth direction of the biological tissue; an image display step of displaying the front image and the tomographic image on a display unit; and a magnification synchronous change step of changing both the display magnification of the image for which a change instruction of the display magnification to the display unit is received among the front image and the tomographic image and the display magnification of the image for which no change instruction of the display magnification is received according to the input instruction when the input of the change instruction of the display magnification to the display unit is received for one of the front image and the tomographic image.

[0008] According to the medical image processing apparatus and the medical image processing program according to the present disclosure, a front image and a tomographic image of the same biological tissue can be easily and appropriately compared by a user.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0010] <Summary> The medical image processing apparatus exemplified in the present disclosure processes image data of a biological tissue. The control unit of the medical image processing apparatus executes an image acquisition step, an image display step, and a magnification synchronization change step. In the image acquisition step, the control unit acquires a frontal image and a tomographic image taken of the same biological tissue of the same subject. The frontal image is a two-dimensional image when the biological tissue is viewed in the direction along the optical axis of the imaging light. The tomographic image is a two-dimensional image that extends in the depth direction of the biological tissue. In the image display step, the control unit causes the frontal image and the tomographic image to be displayed on the display unit. In the magnification synchronization change step, when an input of an instruction to change the display magnification to the display unit is received for one of the frontal image and the tomographic image, the control unit changes (enlarges or reduces) both the display magnification of the image for which the magnification change instruction has been given and the display magnification of the image for which the magnification change instruction has not been given among the frontal image and the tomographic image according to the input instruction.

[0011] According to the technology according to the present disclosure, just by inputting an instruction to change the display magnification for one of the frontal image and the tomographic image displayed on the display unit to the medical image processing apparatus, the display magnifications of both the frontal image and the tomographic image are changed synchronously. Therefore, the user can easily and appropriately compare the two images as compared with the case of individually changing the magnifications of the frontal image and the tomographic image.

[0012] Note that in the magnification synchronization change step, the control unit may change the display magnifications of the frontal image and the tomographic image within the display frame while maintaining the size and shape of the display frame for displaying each of the frontal image and the tomographic image. In this case, even if the display magnifications of the frontal image and the tomographic image are changed, each image is displayed within the same display frame. Therefore, the user can appropriately check the images. However, the control unit may change the size or shape of at least one of the display frames of the frontal image and the tomographic image according to the display magnification of the image. For example, while changing the size of the display frame in the depth direction of the display frame of the tomographic image according to the display magnification, the size of the direction display frame in the direction intersecting the depth direction may be maintained.

[0013] The front image may be an angiographic front image taken by an OCT device. The OCT device can capture an image of a biological tissue by utilizing the principle of optical coherence tomography. The angiographic front image is a front image (Enface image in the present disclosure) generated from motion contrast data (OCT angiographic data). The motion contrast data is generated by processing a plurality of OCT signals acquired from the same position of the biological tissue at different times. Information on the movement in the tissue (e.g., the movement of blood flow in blood vessels in the biological tissue, etc.) appears in the motion contrast data. Therefore, by checking the angiographic front image displayed on the display unit, the user can appropriately grasp the state of the movement (e.g., blood flow, etc.) in the biological tissue from the front side of the tissue.

[0014] The tomographic image may be taken by an OCT device. As described above, the OCT device can capture an image of a biological tissue by utilizing the principle of optical coherence tomography. Information on the motion contrast data may be superimposed on the tomographic image. As described above, the motion contrast data is generated by processing a plurality of OCT signals acquired from the same position of the biological tissue at different times. Information on the movement in the tissue (e.g., the movement of blood flow in blood vessels in the biological tissue, etc.) appears in the motion contrast data. Therefore, by checking the tomographic image with the information on the motion contrast data superimposed thereon, the user can appropriately grasp the state of the movement (e.g., blood flow, etc.) in the biological tissue on the tomographic image.

[0015] Note that an angiography frontal image and a tomographic image (hereinafter referred to as "MC superimposed tomographic image") in which information of motion contrast data is superimposed may be both displayed on the display unit. In this case, the user can appropriately grasp information on the movement of blood flow or the like in the living tissue by both the angiography frontal image and the MC superimposed tomographic image. For example, it is also possible to confirm the state of the tissue at the site appearing as a blood vessel in the frontal image on the tomographic image. Further, according to the technology according to the present disclosure, when an instruction to change the display magnification for one of the angiography frontal image and the MC superimposed tomographic image is input, the display magnifications of both the angiography frontal image and the MC superimposed tomographic image are changed synchronously. Therefore, the user can more easily and appropriately grasp information on the movement of blood flow or the like in the living tissue by comparing the angiography frontal image and the MC superimposed tomographic image whose display magnifications are changed synchronously.

[0016] In the image acquisition step, the control unit may acquire both a frontal image and a tomographic image from data of the same three-dimensional image (for example, a three-dimensional image captured by an OCT device). In this case, since the relative positional relationship between the acquired frontal image and the tomographic image becomes clear, it becomes easier to appropriately grasp the state of the living tissue.

[0017] Note that the three-dimensional image may be configured by arranging a plurality of two-dimensional B-scan images in a direction intersecting the plane direction in which the B-scan images spread. The B-scan image is a two-dimensional tomographic image captured by scanning the spot of the measurement light emitted by the OCT device. The B-scan image is a two-dimensional image that spreads in the direction in which the spot of the measurement light is scanned and in the depth direction (Z direction) along the optical axis of the measurement light. In this case, the tomographic image displayed on the display unit may be at least any one of the plurality of B-scan images constituting the three-dimensional image. Further, the control unit may extract a two-dimensional tomographic image that spreads both in the depth direction and in a direction intersecting each B-scan image from the three-dimensional image and display it on the display unit. The number of tomographic images displayed together with the frontal image may be one or plural.

[0018] However, the method of capturing an image by the OCT device is not limited to the method of scanning the spot of the measurement light. For example, the irradiation optical system of the OCT device may simultaneously irradiate the measurement light onto a two-dimensional region on the tissue of the subject. In this case, the light receiving element may be a two-dimensional light receiving element that detects the interference signal in the two-dimensional region on the tissue. That is, the OCT device may capture a three-dimensional image based on the principle of so-called full-field OCT (FF-OCT). Further, the OCT device may simultaneously irradiate the measurement light onto an irradiation line extending in a one-dimensional direction in the tissue and scan the measurement light in a direction intersecting the irradiation line. In this case, the light receiving element may be a one-dimensional light receiving element (for example, a line sensor) or a two-dimensional light receiving element. That is, the OCT device may capture a three-dimensional image based on the principle of so-called line-field OCT (LF-OCT).

[0019] Further, the control unit may change the position of extracting the tomographic image to be displayed on the display unit among the regions of the three-dimensional image according to the position where the front image is enlarged and displayed. For example, the control unit can also change the extraction position of the tomographic image so that the extraction position of the tomographic image in the region of the three-dimensional image is maintained at a specific position (for example, the center position, etc.) within the display frame of the front image displayed on the display unit. In this case, since the relative positional relationship between the front image and the tomographic image changes appropriately, the state of the living tissue can be more appropriately grasped.

[0020] However, the front image and the tomographic image may be images captured separately. Also, the imaging device for capturing the front image and the imaging device for capturing the tomographic image may be different devices.

[0021] In addition, the frontal image displayed on the display unit is not limited to the angiography frontal image. For example, when an OCT device is used as the imaging device for capturing the frontal image, the frontal image may be an Enface image generated based on three-dimensional OCT data (which may not be motion contrast data). The data of the Enface image may be, for example, integrated image data in which luminance values are integrated in the depth direction (Z direction) at each position in the XY direction, the integrated value of spectral data at each position in the XY direction, luminance data at each position in the XY direction in a certain depth direction, luminance data at each position in the XY direction in any layer of the biological tissue (for example, the superficial layer of the retina, etc.), or the like. Further, the frontal image may be a thickness map showing the two-dimensional distribution of the thickness of a specific layer in the biological tissue. Further, the control unit may input a three-dimensional image into a mathematical model trained by a machine learning algorithm to obtain a probability distribution for identifying a specific structure (for example, at least one of a layer and a boundary, etc.) in the biological tissue shown in the three-dimensional image. The control unit may obtain, as the frontal image, a divergence map showing the two-dimensional distribution of the divergence degree of the obtained probability distribution with respect to the probability distribution when the structure to be identified is accurately identified. Further, the frontal image may be captured by a device different from the OCT device (for example, at least one of a fundus camera, a scanning laser ophthalmoscope (SLO), an infrared camera, etc.).

[0022] The tomographic image displayed on the display unit is not limited to the tomographic image captured by the OCT device. For example, a tomographic image captured by an MRI (magnetic resonance imaging) device, a CT (computed tomography) device, or the like may be displayed on the display unit.

[0023] In the magnification synchronization change step, the control unit may change both the display magnification of the front image and the display magnification of the tomographic image in a state where the display range in the display unit in the extension direction extending in a direction perpendicular to the depth direction among the tomographic images is synchronized with the display range in the display unit in the extension direction of the front image. In this case, even if the display magnifications of both the front image and the tomographic image are changed synchronously, the display range in the extension direction of the front image and the display range in the extension direction of the tomographic image match. Therefore, the user can more appropriately compare the front image and the tomographic image of the part of interest.

[0024] The control unit may further execute a layer / boundary acquisition step of acquiring the position of a specific layer or boundary among the biological tissues shown in the tomographic image displayed on the display unit. In the magnification synchronization change step, when an input of an instruction to change the display magnification with respect to the front image is received, the control unit maintains at least one of a specific layer and a boundary (hereinafter simply referred to as "layer / boundary") in the tomographic image at an average position in the depth direction at a target position in the depth direction (for example, a predetermined position, or a position calculated according to the display magnification), and may change the display magnification of the tomographic image. In this case, regardless of the display magnification of the tomographic image, a specific layer / boundary is likely to be shown in the vicinity of a predetermined position in the depth direction (for example, near the center, or near the position immediately before changing the display magnification). Therefore, even when the user changes the display magnifications of the front image and the tomographic image, the user can appropriately observe a specific layer / boundary on the tomographic image.

[0025] When changing the display magnification of the tomographic image, the target position for maintaining the position in the depth direction of a specific layer / boundary can be appropriately selected. For example, the control unit may change the display magnification of the tomographic image while maintaining the average position in the depth direction of a specific layer / boundary at the center in the depth direction within the display range of the image. Also, the control unit may change the display magnification of the tomographic image while maintaining the average position in the depth direction of a specific layer / boundary at the position immediately before changing the display magnification. Further, the control unit may identify the range in the depth direction of the layer / boundary (hereinafter referred to as "slab") for generating the aforementioned Enface image, and use a predetermined position (such as the center position, etc.) in the depth direction of the identified slab as the center when changing the display magnification, and change the display magnification of the tomographic image. The slab can be identified by the boundary on the superficial side and the boundary on the deep side of the range for generating the Enface image. The boundary for identifying the slab may be a layer of the living body actually shown in the image, or may be a position offset by a predetermined distance in the depth direction from the layer of the living body. By changing the display magnification of the tomographic image based on the slab, it becomes easier to appropriately observe a specific layer / boundary.

[0026] Also, in the magnification synchronization change step, the control unit may change only the display magnification in the extension direction intersecting the depth direction in the tomographic image, without changing the display magnification in the depth direction, and synchronize it with the display magnification of the frontal image. In this case, even if the tomographic image is enlarged, the possibility that at least one of the layer and boundary (hereinafter simply referred to as "layer / boundary") that the user pays attention to deviates from the display range of the tomographic image is reduced. Therefore, the user can more appropriately observe the layer / boundary of interest on the tomographic image. Also, in the synchronous magnification change step, the display magnifications of both images may be changed while maintaining both the aspect ratio of the frontal image and the aspect ratio of the tomographic image.

[0027] The control unit may acquire a front image, a first tomographic image, and a second tomographic image in the image acquisition step. The first tomographic image is a tomographic image that spreads in the depth direction and a first extension direction (such as the X direction, etc.), which is one of the directions perpendicular to the depth direction. The second tomographic image is a tomographic image that spreads in the depth direction and a second extension direction (such as the Y direction, etc.), which is one of the directions perpendicular to the depth direction. The first extension direction and the second extension direction are different from each other. The control unit may cause the front image, the first tomographic image, and the second tomographic image to be displayed on the display unit in the image display step. When an input of an instruction to change the display magnification for any one of the plurality of images displayed on the display unit is received in the magnification synchronization change step, the control unit may synchronously change the display magnification of the front image, the display magnification of the first tomographic image, and the display magnification of the second tomographic image. In this case, while the user checks all of the front image, the first tomographic image, and the second tomographic image, by simply inputting an instruction to change the display magnification for any one of the plurality of images, the display magnifications of the plurality of images are synchronously changed. Therefore, it is possible to easily and appropriately compare the plurality of images.

[0028] For example, the control unit may change the display magnifications of the first tomographic image and the second tomographic image in a state where the display ranges in the depth direction of the first tomographic image and the display ranges in the depth direction of the second tomographic image are synchronized. The control unit may synchronize the display range in the first extension direction of the first tomographic image and the display range in the first extension direction of the front image, and change the display magnifications of the first tomographic image and the front image. The control unit may synchronize the display range in the second extension direction of the second tomographic image and the display range in the second extension direction of the front image, and change the display magnifications of the second tomographic image and the front image. In this case, the display magnifications of each image are changed in a state where the display ranges of the front image, the first tomographic image, and the second tomographic image are appropriately synchronized.

[0029] The control unit may further execute a first reference position setting step and a second reference position setting step. In the first reference position setting step, the control unit sets a first reference position that serves as a reference for enlargement and reduction in the first extension direction of the first tomographic image (for example, the center of enlargement / reduction in the first extension direction, etc.). In the second reference position setting step, the control unit sets a second reference position that serves as a reference for enlargement and reduction in the second extension direction of the second tomographic image (for example, the center of enlargement / reduction in the second extension direction, etc.). In the magnification synchronization change step, the control unit may synchronously change the display magnifications of the first tomographic image and the second tomographic image, with the first reference position serving as the reference for enlargement and reduction (such as the center, etc.) of the first tomographic image and the second reference position serving as the reference for enlargement and reduction (such as the center, etc.) of the second tomographic image. In this case, based on the reference positions set for each of the first tomographic image and the second tomographic image, the display magnifications of the first tomographic image and the second tomographic image are synchronously changed. Therefore, the user can easily confirm an appropriate range of image areas based on the set reference positions in each of the first tomographic image and the second tomographic image. Note that the control unit may synchronously change the display magnifications of the front image, the first tomographic image, and the second tomographic image, with the reference for enlargement and reduction in the first extension direction of the front image being the first reference position and the reference for enlargement and reduction in the second extension direction being the second reference position. In this case, the display magnifications of the plurality of images are appropriately synchronously changed.

[0030] Specific methods for setting the first reference position and the second reference position can be appropriately selected. For example, in the image acquisition step, the control unit may acquire the front image, the first tomographic image, and the second tomographic image from the data of the same three-dimensional image. The control unit may automatically set the first reference position at the position in the first extension direction of the three-dimensional image where the second tomographic image is extracted. The control unit may automatically set the second reference position at the position in the second extension direction of the three-dimensional image where the first tomographic image is extracted. In this case, based on the position where one tomographic image is extracted, the display magnification of the other tomographic image is changed. Thus, it becomes easier to compare the plurality of images.

[0031] Further, the control unit may set at least one of the first reference position and the second reference position according to an instruction input by the user. In this case, the user can change the display magnification of at least one of the first tomographic image and the second tomographic image based on a desired position.

[0032] The control unit may further execute a reference position display step of displaying the first reference position and the second reference position on the front image displayed on the display unit. In this case, the user can confirm both the first reference position of the first tomographic image and the second reference position of the second tomographic image on the front image. Therefore, the user can input an instruction to change the display magnification of the image to the medical image processing apparatus after appropriately and easily grasping the first reference position and the second reference position.

[0033] For example, the control unit may display the first reference position and the second reference position by displaying a first reference line that passes through the first reference position and extends in the second extension direction and a second reference line that passes through the second reference position and extends in the first extension direction on the front image. In this case, the user can more appropriately grasp the positional relationship between the first reference position and the second reference position.

[0034] Further, in addition to the line on the front image, the control unit may display a reference line that passes through the first reference position and extends in the depth direction on the first tomographic image. The control unit may display a reference line that passes through the second reference position and extends in the depth direction on the second tomographic image. In this case, the user can grasp the first reference position and the second reference position in both the front image and the tomographic image.

[0035] The user may input an instruction to specify the first reference position by inputting an instruction to move at least one of the first reference line on the frontal image and the reference line on the first tomographic image. Similarly, the user may input an instruction to specify the second reference position by inputting an instruction to move at least one of the second reference line on the frontal image and the reference line on the second tomographic image. In this case, the user can easily and appropriately specify at least one of the first reference position and the second reference position.

[0036] However, it is also possible to change the method of displaying the first reference position and the second reference position. For example, the control unit may display only the intersection of a virtual first reference line passing through the first reference position and extending in the second extension direction and a virtual second reference line passing through the second reference position and extending in the first extension direction on the frontal image. In this case, the user can grasp both the first reference position and the second reference position only by grasping the position of the intersection displayed on the frontal image. Also, although details will be described later, the control unit may display a cursor that moves within the display area of the display unit in response to an operation instruction by the user. The control unit may set at least one of the first reference position and the second reference position based on the position of the cursor superimposed on the image. For example, when the cursor is displayed on the frontal image, the control unit may set the first reference position and the second reference position at the position of the cursor on the frontal image. When the cursor is displayed on the first tomographic image, the control unit may set the first reference position at the position of the cursor on the first tomographic image. When the cursor is displayed on the second tomographic image, the control unit may set the second reference position at the position of the cursor on the second tomographic image.

[0037] Also, a specific method for synchronously changing the display magnifications of the frontal image and the tomographic image can be appropriately selected. For example, the control unit may synchronously change the display magnifications of the frontal image and the tomographic image based on the shooting information of each of the frontal image and the tomographic image (for example, at least any one of the shooting angles of view of the frontal image and the tomographic image, the number of scan points, the resolution, the model of the imaging device, and the axial length of the eye when the biological tissue is the eye to be examined, etc.).

[0038] In the magnification synchronization change step, the control unit may receive an input of an instruction to specify a reference position that serves as a reference for enlargement and reduction within the display area of at least one of the front image and the tomographic image. The control unit may perform enlargement processing and reduction processing on at least one of the front image and the tomographic image centered on the specified reference position. In this case, the user can enlarge or reduce the image centered on the specified reference position by designating an appropriate position within the display range of at least one of the front image and the tomographic image as the reference position. Therefore, it becomes easier for the user to compare the front image and the tomographic image more appropriately.

[0039] Note that a specific method for allowing the user to specify the reference position can be appropriately selected. For example, the control unit may display a cursor that moves within the display area in response to an operation of a mouse or the like on the display unit. The control unit may use the position of the cursor at that time as the reference position within the display area of the front image or the tomographic image. Also, the reference position may be specified within the display area of the front image or the tomographic image by an operation on a touch panel or the like.

[0040] However, it is also possible to change the method of setting the reference position. For example, the control unit may automatically set the reference position at the center position of at least one of the front image and the tomographic image. In this case, the image is enlarged and reduced with the center of the image as the reference. Also, when a tomographic image extending in a first direction and a tomographic image extending in a second direction different from the first direction are both displayed, the control unit may automatically set the reference position at the position where the tomographic image extending in the first direction and the tomographic image extending in the second direction intersect. In this case, the reference position is set at an appropriate position based on the displayed tomographic image.

[0041] When an instruction to reset at least one of the display magnification and the display position of the image to the initial state is received, the control unit may further execute a display reset step of resetting at least one of the display magnifications of the frontal image and the tomographic image and the display position to be displayed within the display frame of the display unit to the state at the start of display. In this case, the user can return at least one of the display magnifications and the display positions of the frontal image and the tomographic image to the state at the start of display with a simple operation.

[0042] When an instruction to change the display magnification to the display unit for one of the frontal image and the tomographic image is received, the control unit may be able to execute a magnification independent change step of enlarging or reducing the display magnification of the image for which the magnification change instruction has been given among the frontal image and the tomographic image according to the input instruction. The control unit may execute either a magnification synchronous change step or a magnification independent change step according to the instruction input by the user. In this case, the user can appropriately distinguish between the magnification synchronous change step and the magnification independent change step according to various situations. Therefore, the user can compare the two images more appropriately.

[0043] A specific method for allowing the user to input an instruction to change the display magnification can be appropriately selected. For example, the user may input an instruction to change the display magnification to the medical image processing apparatus by performing at least any one of a mouse operation, a key operation, a touch panel operation, an operation of a button displayed on the display unit, and the like.

[0044] In the image acquisition step, the control unit may acquire a plurality of image sets each of which is a set of a frontal image and a tomographic image taken of the same biological tissue of the same subject. In the image display step, the control unit may display a plurality of image sets on the display unit. In the magnification synchronous change step, when an instruction to change the display magnification to the display unit for any of the plurality of frontal images and the plurality of tomographic images is received, the control unit may change the display magnification of each of the plurality of frontal images and the plurality of tomographic images included in the plurality of image sets according to the input instruction.

[0045] In this case, the user can appropriately grasp the state of the biological tissue by comparing a plurality of image sets taken of the same biological tissue of the same subject. Further, when an input of an instruction to change the display magnification to the display unit is received for any of the plurality of front images and the plurality of tomographic images, not only the display magnification of the image for which the change instruction has been made, but also the display magnifications of the other plurality of images (front images and tomographic images) displayed on the display unit are changed according to the input instruction. Therefore, the user can more easily and appropriately compare the plurality of images (front images and tomographic images) as compared with the case of individually changing the magnification of each of the plurality of images.

[0046] Note that the shooting times of each of the plurality of image sets may be different from each other. In this case, the user can more easily and appropriately observe the temporal changes (i.e., follow-up observation) of the biological tissue of the subject to be photographed.

[0047] The control unit may execute alignment of the plurality of front images included in the plurality of image sets in the image display step and cause the display unit to display the plurality of front images for which the alignment has been performed. In this case, the user can appropriately compare the biological tissues at the same position shown in each of the plurality of front images. Therefore, it becomes easier to more appropriately grasp the state of the biological tissue.

[0048] However, in the case where the positions of the biological tissues shown in each of the plurality of front images already coincide, etc., the alignment process of the plurality of front images may be omitted.

[0049] In the magnification synchronization change step, the control unit may set a reference position serving as a reference for enlargement and reduction at the same position within the display area of each of the plurality of front images, and perform enlargement processing and reduction processing on the plurality of front images centered on the set reference position. The control unit may set a reference position serving as a reference for enlargement and reduction at the same position within the display area of each of the plurality of tomographic images, and perform enlargement processing and reduction processing on the plurality of tomographic images centered on the set reference position. In this case, the display magnification of the plurality of front images is changed while the display positions of the plurality of front images are synchronized. Similarly, the display magnification of the plurality of tomographic images is changed while the display positions of the plurality of tomographic images are synchronized. Therefore, the user can more easily and appropriately compare the plurality of images (front images and tomographic images).

[0050] In the image display step, the control unit may cause the display unit to display a plurality of tomographic images whose imaging positions in the biological tissue are common to each other. In this case, the user can appropriately compare the plurality of tomographic images taken at the same position in the biological tissue. Therefore, it becomes easier to more appropriately grasp the state of the biological tissue.

[0051] For example, the control unit may acquire a plurality of image sets by obtaining a front image and a tomographic image from each of the plurality of three-dimensional images taken at the same position of the same biological tissue. In this case, the control unit may extract tomographic images from the same position of the plurality of three-dimensional images. In this case, the imaging positions in the biological tissue of the plurality of extracted tomographic images are common to each other. Note that the control unit may extract tomographic images from the same position of the plurality of three-dimensional images in a state where alignment of the plurality of three-dimensional images is performed at least when viewed from the front direction. In this case, the imaging positions in the biological tissue of the plurality of tomographic images are more likely to be approximated. Therefore, it becomes easier to more appropriately grasp the state of the biological tissue.

[0052] Further, when an instruction to change the extraction position from the three-dimensional image is input for any of the plurality of tomographic images, the control unit may change the extraction positions of the tomographic images in each of the plurality of three-dimensional images to the same position. In this case, the user can change the extraction positions of the plurality of tomographic images collectively by simply changing the extraction position of any one of the plurality of tomographic images.

[0053] Note that the technology according to the present disclosure can also be expressed as follows. A medical image processing method executed by a medical image processing apparatus that processes image data of a biological tissue, the method including: an image acquisition step of acquiring a frontal image that is a two-dimensional image of the same biological tissue of the same subject as viewed from a direction along the optical axis of imaging light, and a tomographic image that is a two-dimensional image extending in the depth direction of the biological tissue; an image display step of displaying the frontal image and the tomographic image on a display unit; and a magnification synchronous change step of changing both the display magnification of the image for which a change instruction of the display magnification to the display unit is received and the display magnification of the image for which no change instruction of the display magnification is received among the frontal image and the tomographic image in accordance with the input instruction.

[0054] <Embodiment> Hereinafter, one of the typical embodiments according to the present disclosure will be described. In this embodiment, a case of processing image data of the fundus tissue of the subject eye E photographed by an OCT apparatus will be exemplified. The OCT apparatus can photograph an image of a biological tissue using the principle of optical coherence tomography. However, the image to be processed by the technology of the present disclosure may be an image of a biological tissue other than the fundus tissue. For example, the image to be processed may be an image of a biological tissue (such as the anterior segment, etc.) other than the fundus of the subject eye E, or an image of a biological tissue (such as skin, digestive tract, or brain, etc.) other than the subject eye E. Further, as described above, the imaging apparatus that photographs the image to be processed is not limited to the OCT apparatus.

[0055] Referring to FIG. 1, the schematic configuration of the medical image processing system 100 according to this embodiment will be described. The medical image processing system 100 of this embodiment includes an imaging device 1 and a medical image processing device 40. The imaging device (OCT device in this embodiment) 1 captures an image of a living tissue (a three-dimensional image in this embodiment) by receiving light from the living tissue. The medical image processing device 40 executes processing of the image data captured by the imaging device 1 (for example, image display control processing to the monitor 47, etc.). A PC is used for the medical image processing device 40 of this embodiment. However, the device that can function as the medical image processing device 40 is not limited to a PC. For example, the imaging device (OCT device) 1 or a server, etc. may function as the medical image processing device 40. When the imaging device 1 functions as the medical image processing device 40, the imaging device 1 can appropriately process the captured image while capturing the image of the living tissue. Also, a mobile terminal such as a tablet terminal or a smartphone may function as the medical image processing device 40. The control units of a plurality of devices (for example, the CPU of the PC and the CPU 31 of the imaging device 1) may cooperate to perform various processes.

[0056] The configuration of the imaging device 1 of this embodiment will be described. The imaging device (OCT device) 1 includes an OCT unit 10 and a control unit 30. The OCT unit 10 includes an OCT light source 11, a coupler (optical splitter) 12, a measurement optical system 13, a reference optical system 20, and a light receiving element 22.

[0057] The OCT light source 11 emits light (OCT light) for acquiring image data. The coupler 12 splits the OCT light emitted from the OCT light source 11 into measurement light and reference light. Further, the coupler 12 in the present embodiment combines and interferes the measurement light reflected by the biological tissue (the fundus of the eye E to be examined in the present embodiment) and the reference light generated by the reference optical system 20. That is, the coupler 12 in the present embodiment also serves as a branching optical element that branches the OCT light into measurement light and reference light and a multiplexing optical element that multiplexes the reflected light of the measurement light and the reference light. Note that it is also possible to change at least one of the configurations of the branching optical element and the multiplexing optical element. For example, elements other than the coupler (for example, a circulator, a beam splitter, etc.) may be used.

[0058] The measurement optical system 13 guides the measurement light split by the coupler 12 to the subject and returns the measurement light reflected by the biological tissue to the coupler 12. The measurement optical system 13 includes a scanning unit (scanner) 14, an irradiation optical system 16, and a focus adjustment unit 17. The scanning unit 14 can scan (scan) the spot of the measurement light in a two-dimensional direction intersecting the optical axis of the measurement light by being driven by the driving unit 15. In the present embodiment, two galvanometer mirrors capable of deflecting the measurement light in different directions are used as the scanning unit 14. However, another device that deflects light (for example, at least any one of a polygon mirror, a resonant scanner, an acousto-optic element, etc.) may be used as the scanning unit 14. The irradiation optical system 16 is provided on the downstream side of the optical path (that is, the subject side) with respect to the scanning unit 14 and irradiates the biological tissue with the measurement light. The focus adjustment unit 17 adjusts the focus of the measurement light by moving an optical member (for example, a lens) included in the irradiation optical system 16 in the direction along the optical axis of the measurement light.

[0059] The reference optical system 20 generates reference light and returns it to the coupler 12. The reference optical system 20 of the present embodiment generates reference light by reflecting the reference light split by the coupler 12 with a reflection optical system (for example, a reference mirror). However, the configuration of the reference optical system 20 can also be changed. For example, the reference optical system 20 may transmit the light incident from the coupler 12 without reflecting it and return it to the coupler 12. The reference optical system 20 includes an optical path length adjustment unit 21 that changes the optical path length difference between the measurement light and the reference light. In the present embodiment, the optical path length difference is changed by moving the reference mirror in the optical axis direction. Note that the configuration for changing the optical path length difference may be provided in the optical path of the measurement optical system 13.

[0060] The light receiving element 22 detects an interference signal by receiving the interference light of the measurement light and the reference light generated by the coupler 12. In the present embodiment, the principle of Fourier domain OCT is adopted. In Fourier domain OCT, the spectral intensity of the interference light (spectral interference signal) is detected by the light receiving element 22, and a complex OCT signal is acquired by performing a Fourier transform on the spectral intensity data. As an example of Fourier domain OCT, Spectral-domain-OCT (SD-OCT), Swept-source-OCT (SS-OCT), etc. can be adopted. Also, for example, Time-domain-OCT (TD-OCT), etc. can also be adopted.

[0061] In addition, in this embodiment, the spot of the measurement light is scanned within a two-dimensional region by the scanning unit 14, thereby acquiring the data of the three-dimensional image. However, it is also possible to change the principle of acquiring the data of the three-dimensional image. For example, the data of the three-dimensional image may be acquired according to the principle of line-field OCT (hereinafter referred to as "LF-OCT"). In LF-OCT, the measurement light is simultaneously irradiated on the irradiation line extending in the one-dimensional direction in the tissue, and the interference light between the reflected light of the measurement light and the reference light is received by a one-dimensional light receiving element (for example, a line sensor) or a two-dimensional light receiving element. In the two-dimensional measurement region, the measurement light is scanned in the direction intersecting the irradiation line, thereby acquiring the three-dimensional OCT data. Further, the data of the three-dimensional image may be acquired according to the principle of full-field OCT. In full-field OCT, the measurement light is simultaneously irradiated on the two-dimensional region on the tissue of the subject, and the interference light is received by the two-dimensional light receiving element.

[0062] In addition, the imaging device 1 can acquire (generate) an Enface image, which is a two-dimensional front image when the tissue is viewed from the direction (front direction) along the optical axis of the measurement light (imaging light) based on the acquired three-dimensional OCT data. When the Enface image is acquired in real time, the acquired Enface image can also be used as an image (front observation image) for observing the living tissue during imaging from the front. The data of the Enface image may be, for example, integrated image data in which luminance values are integrated in the depth direction (Z direction) at each position in the XY direction, integrated values of spectral data at each position in the XY direction, luminance data at each position in the XY direction in a certain depth direction, luminance data at each position in the XY direction in any layer of the retina (for example, the superficial layer of the retina), and the like. Further, the imaging device 1 of the present embodiment can also generate an Enface image from the motion contrast data. The motion contrast data is data obtained by processing a plurality of OCT signals acquired at different times from the same position on the living tissue. Information on the movement of the living tissue (for example, the movement of blood flow in blood vessels in the living tissue) appears in the motion contrast data. In the present embodiment, an angiography front image (blood vessel image), which is an image showing the blood vessel positions included in a specific layer, is generated by generating an Enface image of the specific layer based on the motion contrast data.

[0063] The control unit 30 controls various operations of the imaging device 1. The control unit 30 includes a CPU 31, a RAM 32, a ROM 33, and a non-volatile memory (NVM) 34. The CPU 31 is a controller that performs various controls. The RAM 32 temporarily stores various information. The ROM 33 stores programs executed by the CPU 31 and various initial values and the like. The NVM 34 is a non-transitory storage medium that can retain the stored content even when the power supply is cut off. When the imaging device 1 functions as a medical image processing device, a medical image processing program for executing image display processing (see FIGS. 6, 7, 9, and 10), which will be described later, may be stored in the NVM 34 or the like.

[0064] A monitor 37 and an operation unit 38 are connected to the control unit 30. The monitor 37 is an example of a display unit that displays various images. The operation unit 38 is operated by a user to input various operation instructions to the imaging device 1. For the operation unit 38, various devices such as a mouse, a keyboard, a touch panel, and a foot switch can be used, for example. Note that various operation instructions may be input to the imaging device 1 by inputting sound to the microphone.

[0065] The schematic configuration of the medical image processing apparatus 40 will be described. The medical image processing apparatus 40 includes a CPU 41, a RAM 42, a ROM 43, and an NVM 44. A medical image processing program for executing image display processing (see FIGS. 6, 7, 9, and 10), which will be described later, may be stored in the NVM 44. Also, a monitor 47 and an operation unit 48 are connected to the medical image processing apparatus 40. The monitor 47 is an example of a display unit that displays various images. The operation unit 48 is operated by a user to input various operation instructions to the medical image processing apparatus 40. For the operation unit 48, various devices such as a mouse, a keyboard, and a touch panel can be used in the same way as the operation unit 38 of the imaging device 1. Also, various operation instructions may be input to the medical image processing apparatus 40 by inputting sound to the microphone.

[0066] The medical image processing apparatus 40 can acquire various data (for example, image data captured by the imaging device 1) from the imaging device 1. The various data may be acquired by at least any one of, for example, wired communication, wireless communication, and a removable storage device (for example, a USB memory).

[0067] (Image) Referring to FIGS. 2 to 5, an example of an image to be processed by the medical image processing apparatus 40 of the present embodiment will be described. As shown in FIG. 2, the imaging apparatus 1 of the present embodiment scans light (measurement light) within a two-dimensional imaging region 51 in a biological tissue 50 (in the example shown in FIG. 2, the fundus tissue). Specifically, the imaging apparatus 1 of the present embodiment scans light on a scan line 52 extending in a predetermined direction within the imaging region 51, thereby capturing (acquiring) a two-dimensional tomographic image (B-scan image) 61 (see FIG. 3) that spreads in the Z direction along the optical axis of the light and in the X direction perpendicular to the Z direction. In the example shown in FIG. 3, the Z direction is a direction perpendicular to the two-dimensional imaging region 51 (the depth direction of the biological tissue 50), and the X direction is the direction in which the scan line 52 extends. Next, the imaging apparatus 1 moves the position of the scan line 52 in the Y direction within the imaging region 51 and repeats the capture of the tomographic image 61. The Y direction is a direction that intersects both the Z direction and the X direction (perpendicularly intersects in the present embodiment). As a result, a plurality of tomographic images 61 that pass through each of the plurality of scan lines 52 and spread in the depth direction of the biological tissue 50 are acquired. Next, as shown in FIG. 3, the plurality of tomographic images 61 are arranged in the Y direction (a direction intersecting the image region of each tomographic image), thereby generating a three-dimensional image 60 (see FIG. 4) in the imaging region 51. That is, the image data captured by the imaging apparatus 1 of the present embodiment is three-dimensional image data that spreads in the Z direction, which is the depth direction of the biological tissue, and in the two-dimensional XY direction that intersects the Z direction.

[0068] The medical image processing apparatus 40 (specifically, the CPU 41) can acquire both the frontal image 70 and the tomographic image 80 (see FIG. 5) from the data of the same three-dimensional image 60. The medical image processing apparatus 40 can display the acquired frontal image 70 and tomographic image 80 on the monitor 47. In this case, since the relative positional relationship between the acquired frontal image 70 and tomographic image 80 becomes clear, the state of the biological tissue can be more appropriately grasped.

[0069] The medical image processing apparatus 40 can extract a two-dimensional tomographic image 80 from within the image region of the three-dimensional image 60 (see FIG. 4) and display the extracted tomographic image 80 on the monitor 47. In the example shown in FIG. 5, within the image region of the front image 70, an extraction line 75X indicating the extraction position of the tomographic image 80X extending in the XZ direction and an extraction line 75Y indicating the extraction position of the tomographic image 80Y extending in the YZ direction are displayed. The medical image processing apparatus 40 extracts and acquires a tomographic image 80X that passes through the extraction line 75X and extends in the Z direction (depth direction) from within the image region of the three-dimensional image 60. Further, the medical image processing apparatus 40 extracts and acquires a tomographic image 80Y that passes through the extraction line 75Y and extends in the Z direction from within the image region of the three-dimensional image 60.

[0070] Note that in the present embodiment, the tomographic image 80X corresponds to a first tomographic image that extends in the depth direction (Z direction) and a first extension direction (the X direction in the present embodiment), which is one of the directions perpendicular to the depth direction. Further, the tomographic image 80Y corresponds to a second tomographic image that extends in the depth direction (Z direction) and a second extension direction (Y direction) that intersects perpendicularly to the depth direction and is different from the first extension direction.

[0071] As described above, the three-dimensional image 60 of the present embodiment is configured by arranging a plurality of tomographic images (B-scan images) 61 extending in the XZ direction in the Y direction. Therefore, the tomographic image 80X extending in the XZ direction may coincide with at least any one of the plurality of tomographic images 61 that make up the three-dimensional image 60. Further, the tomographic image 80Y extending in the YZ direction extends in the Z direction (depth direction) and in a direction intersecting the B-scan image 61. In this case, the medical image processing apparatus 40 extracts and acquires a tomographic image 80Y extending in the YZ direction from each of the plurality of B-scan images 61 included in the three-dimensional image 60.

[0072] The medical image processing apparatus 40 can change the extraction position of the tomographic images 80(80X, 80Y) from the three-dimensional image 60 according to an instruction input by a user. As an example, in the present embodiment, when the user operates the operation unit 48 to change the position of at least one of the extraction lines 75X and 75Y displayed within the image area of the front image 70, an instruction to change the extraction position of the tomographic image 80 is input. The medical image processing apparatus 40 changes the extraction position of the tomographic image 80 to a position passing through the changed extraction lines 75X, 75Y within the image area of the three-dimensional image 60.

[0073] As will be described in detail later, in the present embodiment, the first reference position serving as a reference for enlargement and reduction in the X direction (first extension direction) of the tomographic image 80X (first tomographic image) can be set to the position where the tomographic image 80Y (second tomographic image) is extracted in the three-dimensional image (that is, the position of the extraction line 75Y). In this case, the extraction line 75Y displayed on the front image 70 indicates the first reference position. Also, the second reference position serving as a reference for enlargement and reduction in the Y direction (second extension direction) of the tomographic image 80Y (second tomographic image) can be set to the position where the tomographic image 80X (first tomographic image) is extracted in the three-dimensional image (that is, the position of the extraction line 75X). That is, the medical image processing apparatus 40 of the present embodiment can also display a first reference line indicating the first reference position (the extraction line 75Y in the example shown in FIG. 5) and a second reference line indicating the second reference position (the extraction line 75X in the example shown in FIG. 5) on the front image 70. However, the medical image processing apparatus 40 can also set the first reference position and the second reference position at positions different from the extraction lines 75X and 75Y. In this case, the medical image processing apparatus 40 may display the first reference position and the second reference position on the front image 75 separately from the extraction lines 75X and 75Y. Also, in the present embodiment, the medical image processing apparatus 40 also displays a first reference line indicating the first reference position on the tomographic image 80X (first tomographic image). Further, the medical image processing apparatus 41 also displays a second reference line indicating the second reference position on the tomographic image 80Y (second tomographic image). The user may input an instruction to the medical image processing apparatus 40 to specify the first reference position by inputting an instruction to move at least one of the first reference line on the front image 70 and the first reference line on the tomographic image 80X. Similarly, the user may input an instruction to the medical image processing apparatus 40 to specify the second reference position by inputting an instruction to move at least one of the second reference line on the front image 70 and the second reference line on the tomographic image 80Y.

[0074] The medical image processing apparatus 40 can acquire (generate) an Enface image, which is a two-dimensional frontal image 70 when viewing the tissue from the direction (front direction) along the optical axis of the measurement light (imaging light) based on the three-dimensional OCT data acquired by the imaging apparatus 1. The frontal image 70 illustrated in FIG. 5 is an example of the Enface image. The data of the Enface image may be, for example, integrated image data in which luminance values are integrated in the depth direction (Z direction) at each position in the XY direction, integrated values of spectral data at each position in the XY direction, luminance data at each position in the XY direction in a certain depth direction, luminance data at each position in the XY direction in any layer of the retina (for example, the superficial layer of the retina), and the like.

[0075] The medical image processing apparatus 40 can generate motion contrast data (which may be expressed as OCT angiography data) by processing the OCT signal acquired by the imaging apparatus 1. The motion contrast data is data obtained by processing a plurality of OCT signals acquired from the same position on the living tissue at different times. Information on the movement of the living tissue (for example, the movement of blood flow in blood vessels in the living tissue, etc.) appears in the motion contrast data.

[0076] The medical image processing apparatus 40 can acquire an angiography frontal image by generating an Enface image based on the motion contrast data. The frontal image 70 illustrated in FIG. 5 is an example of the angiography frontal image. By checking the angiography frontal image, the user can appropriately grasp the state of the movement (for example, blood flow, blood vessels, etc.) in the living tissue from the front side of the tissue. In the present embodiment, based on the motion contrast data, an Enface image of a specific layer in the fundus tissue is generated, and thus an angiography frontal image (blood vessel image), which is an image showing the blood vessel positions included in the specific layer, is generated.

[0077] As shown in FIG. 5, the medical image processing apparatus 40 of the present embodiment can superimpose the information of the motion contrast data on the tomographic image 80 (80X, 80Y). As described above, the motion contrast data represents information on the movement in the tissue (for example, the movement of blood flow in blood vessels in a living tissue, etc.). Therefore, the user can appropriately grasp the state of the movement (for example, blood flow, blood vessels, etc.) in the living tissue on the tomographic image 80 by checking the tomographic image 80 on which the information of the motion contrast data is superimposed.

[0078] (First Embodiment) With reference to FIGS. 6 to 8, the image display process executed by the medical image processing apparatus 40 of the first embodiment will be described. In the present embodiment, the medical image processing apparatus 40 which is an OCT acquires the data of the image of the living tissue from the imaging device 1 and executes the display process of the acquired image. However, as described above, other devices may function as the medical image processing apparatus. For example, the imaging device (OCT device in the present embodiment) 1 itself may execute the image display process. Also, a plurality of control units (for example, the CPU 31 of the imaging device 1 and the CPU 41 of the medical image processing apparatus 40) may cooperate to execute the image display process. In the present embodiment, the CPU 41 of the medical image processing apparatus 40 executes the image display process shown in FIG. 6 according to the medical image processing program stored in the NVM 44.

[0079] First, the CPU 41 acquires the three-dimensional image 60 (see FIG. 4) of the living tissue photographed by the imaging device 1 (S1). The CPU 41 acquires the front image 70 (see FIGS. 5 and 8) from the three-dimensional image 60 acquired in S1 (S2). As described above, in the present embodiment, the front image 70 is acquired by generating an Enface image based on the three-dimensional OCT data. More specifically, in the present embodiment, the angiography front image is acquired by generating an Enface image based on the motion contrast data.

[0080] However, in S2, the CPU 41 may acquire a frontal image different from the angiography frontal image. For example, the CPU 41 may acquire a frontal image by generating an Enface image based on three-dimensional OCT data different from the motion contrast data. The CPU 41 may acquire a thickness map showing a two-dimensional distribution of the thickness of a specific layer in the biological tissue as the frontal image. Further, the CPU 41 may input a three-dimensional image into a mathematical model trained by a machine learning algorithm to obtain a probability distribution for identifying a specific structure (such as at least one of a layer and a boundary, etc.) in the biological tissue shown in the three-dimensional image. The CPU 41 may acquire a divergence map showing a two-dimensional distribution of the divergence degree of the acquired probability distribution with respect to the probability distribution when the structure to be identified is accurately identified, as the frontal image. The CPU 41 may acquire data of a frontal image taken by a device different from the OCT device (such as at least one of a fundus camera, a scanning laser ophthalmoscope (SLO), an infrared camera, etc.).

[0081] Return to the description of FIG. 6. The CPU 41 sets the extraction position of the tomographic image 80 at the default position in the three-dimensional image 60 acquired in S1 (S3). The CPU 41 acquires the tomographic image 80 (see FIGS. 5 and 8) by extracting a two-dimensional image at the extraction position set in S3 from the three-dimensional image 60 (S4). As shown in FIG. 5, in the present embodiment, extraction lines 75X and 75Y are set on the frontal image 70, so that the extraction positions of the tomographic images 80X and 80Y are set. The CPU 41 extracts the tomographic images 80X and 80Y that pass through each of the extraction lines 75X and 75Y and extend in the depth direction (Z direction) from the three-dimensional image 60. Note that the default position at which the extraction position is set in S3 can be set as appropriate. As an example, in the present embodiment, a default extraction line 75X is set at the central position in the Y direction within the image range of the three-dimensional image 60. Also, a default extraction line 75Y is set at the central position in the X direction within the image range of the three-dimensional image 60.

[0082] However, the tomographic image obtained in S4 is not limited to the tomographic image captured by the OCT device. Tomographic images captured by an MRI (Magnetic Resonance Imaging) device, a CT (Computed Tomography) device, or the like may be obtained.

[0083] The CPU 41 causes the front image 70 acquired in S2 and the tomographic image 80 acquired in S4 to be displayed on the monitor 47 (S5). Next, a display mode change process (S6, see FIG. 7) is executed.

[0084] As shown in FIG. 7, when the display mode change process is started, the CPU 41 determines whether an instruction (trigger) to change the extraction position of the tomographic image 80 from the three-dimensional image 60 has been input (S11). If not input (S11: NO), the process directly proceeds to S15. As described above, in this embodiment, the user operates the operation unit 48 to change at least one of the positions of the extraction line 75X and the extraction line 75Y displayed within the image area of the front image 70, thereby inputting an instruction (trigger) to change the extraction position of the tomographic image 80. Also, a trigger to change the extraction position may be automatically input. For example, the CPU 41 may automatically output a trigger to change the extraction position to a position corresponding to the analysis result of the image. In this case, a specific method for changing the extraction position according to the analysis result of the image can be appropriately selected. For example, the extraction position of the tomographic image may be automatically changed to a position where it is determined that there is a high possibility of deteriorated blood flow based on the angiographic front image, or a position where the divergence degree indicated by the divergence degree map is equal to or greater than a threshold value. When an instruction to change the extraction position is input (S11: YES), the CPU 41 changes the extraction position of the tomographic image 80 to a position passing through the changed extraction lines 75X and 75Y within the image area of the three-dimensional image 60 (S12). The CPU 41 acquires the tomographic image 80 again from the changed new extraction position in the three-dimensional image 60 and causes it to be displayed on the monitor 47 (S13). Thereafter, the process proceeds to S15.

[0085] The CPU 41 determines whether an instruction to execute the magnification synchronization change process has been input by the user (S15). The medical image processing apparatus 40 of the present embodiment can execute a magnification synchronization change process (S18 to S29) and a magnification independent change process (S16). In the magnification synchronization change process, when a change instruction for the display magnification of one of the front image 70 and the tomographic image 80 displayed on the monitor 47 is input by the user, the display magnifications of both the front image 70 and the tomographic image 80 are changed synchronously. On the other hand, in the magnification independent change process, the display magnification of the image for which the change instruction for the display magnification has been given among the front image 70 and the tomographic image 80 displayed on the monitor 47 is changed independently. The user can input an instruction to specify which of the magnification synchronization change process and the magnification independent change process to execute by operating the operation unit 48. When an instruction to execute the magnification synchronization change process is input (S15: YES), the CPU 41 executes the magnification synchronization change process (S18 to S29) described below. On the other hand, when an instruction to execute the magnification independent change process is input (S15: NO), the CPU 41 executes the magnification independent change process according to the instruction input by the user (S16). The processes of S11 to S16 are repeated. As described above, the user can properly use the magnification synchronization change process and the magnification independent change process according to various situations.

[0086] Hereinafter, the magnification synchronization change process (S18 to S29) will be described in detail. In the present embodiment, the display magnifications of all of the front image 70, the tomographic image 80X extending in the XZ direction, and the tomographic image 80Y extending in the YZ direction are changed synchronously. However, hereinafter (in FIG. 8 and the like), for the sake of simplifying the explanation, there may be cases where only the case of synchronously changing the display magnification of the front image 70 and the display magnification of the tomographic image 80X extending in the XZ direction is exemplified. In order to synchronously change the display magnification of the front image 70 and the display magnification of the tomographic image 80Y extending in the YZ direction, the method described below may be executed by replacing "X direction" with "Y direction".

[0087] The CPU 41 determines whether an instruction to change the display magnification on the monitor 47 for the front image 70 displayed on the monitor 47 has been input (S18). If not (S18: NO), the process directly proceeds to S23. The method for allowing the user to input an instruction to change the display magnification for the front image 70 can be appropriately selected. As an example, in this embodiment, the CPU 41 causes a cursor that moves within the display area in response to the operation of the operation unit 48 (a mouse in this embodiment) to be displayed on the monitor 47. The user operates the operation unit 48 to move the position of the cursor displayed on the monitor 47 to a position (hereinafter referred to as the "reference position") that is the center of the change (enlargement and reduction) of the display magnification of the front image 70. Thereafter, the user performs an operation for enlarging or reducing the image (in this embodiment, a forward or reverse rotation operation of the mouse wheel). As a result, an instruction to change the display magnification for the front image 70 and an instruction to specify the reference position within the front image 70 are input to the medical image processing apparatus 40. Therefore, by designating an appropriate position as the reference position, the user can enlarge or reduce the image centered on the designated reference position.

[0088] When an instruction to change the display magnification for the front image 70 is input (S18: YES), the CPU 41 sets the reference position for changing the magnification of the front image 70 at the position within the front image 70 designated by the user (in this embodiment, the position of the cursor on the front image 70 when the rotation operation of the mouse wheel is performed) (S19). The CPU 41 executes the process of changing the display magnification (enlargement process or reduction process) of the front image 70 centered on the reference position set in S19 (S20).

[0089] In FIG. 8, the front image 70 and the tomographic image 80X before image enlargement are shown on the left side, and the front image 70 and the tomographic image 80X after image enlargement are shown on the right side. In the example shown in FIG. 8, an instruction to enlarge the front image 70 is input with the cursor placed slightly below and to the right of the center of the front image 70. Therefore, the CPU 41 sets a reference position based on the position of the cursor on the front image 70, and enlarges the front image 70 around the reference position within the display frame for the front image 70 on the monitor 47. As a result, the area within the rectangular frame 78 centered on the position of the cursor in the front image 70 before image enlargement (left side of FIG. 8) is enlarged and displayed within the display frame of the monitor 47.

[0090] Also, the CPU 41 executes a process (S19, S20) of changing the display magnification of the front image 70 and a process (S21) of changing the display magnification of the tomographic images 80X, 80Y. In S21 of the present embodiment, the CPU 41 synchronizes the display range on the monitor 47 in the extension direction (X direction in the example shown in FIG. 8) that extends perpendicular to the depth direction (Z direction) in the tomographic image 80X with the display range on the monitor 47 in the extension direction (X direction in the example shown in FIG. 8) of the front image 70, and changes the display magnification of the tomographic image 80X. In the example shown in FIG. 8, both the display magnification of the front image 70 and the display magnification of the tomographic image 80X are changed with the display range in the X direction of the tomographic image 80X and the display range in the X direction of the front image (both are the display ranges indicated by "ED" in the figure) being synchronized. As a result, even if the display magnifications of both the front image 70 and the tomographic image 80X are changed synchronously, the display range in the extension direction of the front image 70 and the display range in the extension direction of the tomographic image 80X match. Therefore, the user can more appropriately compare the front image 70 and the tomographic image 80X of the part of interest.

[0091] In addition, the method for determining the display position of the tomographic image 80X in the depth direction (Z direction) when changing the display magnification of the tomographic image 80 in S21 can be appropriately selected. For example, the CPU 41 may execute a layer / boundary acquisition process for acquiring the position of a specific layer or boundary (hereinafter simply referred to as "layer / boundary") among the biological tissues shown in the tomographic image 80X displayed on the monitor 47. In S21, the CPU 41 may change the display magnification of the tomographic image 80X while maintaining the average position of a specific layer / boundary in the tomographic image 80X in the depth direction (Z direction) at a target position in the depth direction (for example, a predetermined position, or a position calculated according to the display magnification, etc.). In this case, regardless of the display magnification of the tomographic image 80X, a specific layer / boundary is likely to be shown in the vicinity of a predetermined position in the depth direction. Therefore, even when the user changes the display magnifications of the front image 70 and the tomographic image 80X, the user can appropriately observe a specific layer / boundary on the tomographic image 80X. Note that a mathematical model trained by a machine learning algorithm may be used for the layer / boundary acquisition process. The position of the layer / boundary determined by the operator may be acquired as it is.

[0092] When changing the display magnification of the tomographic image 80, the target position for maintaining the position in the depth direction of a specific layer or boundary can be appropriately selected. For example, the CPU 41 may change the display magnification of the tomographic image while maintaining the average position in the depth direction of a specific layer or boundary at the center in the depth direction within the display range of the image. Also, the CPU 41 may change the display magnification of the tomographic image while maintaining the average position in the depth direction of a specific layer or boundary at the position immediately before changing the display magnification. Further, the CPU 41 specifies the range in the depth direction (hereinafter referred to as "slab") of the layer or boundary for generating the aforementioned Enface image, and uses a predetermined position (such as the center position, etc.) in the depth direction of the specified slab as the center when changing the display magnification, and may change the display magnification of the tomographic image. The slab can be specified by the boundary on the superficial side and the boundary on the deep side of the range for generating the Enface image. The boundary for specifying the slab may be a layer of the living body actually shown in the image, or may be a position offset by a predetermined distance in the depth direction from the layer of the living body. By changing the display magnification of the tomographic image 80 based on the slab, it becomes easier to appropriately observe a specific layer or boundary.

[0093] Also, in S21, the CPU 41 may change only the display magnification in the elongation direction (X direction in FIG. 8) intersecting the depth direction without changing the display magnification in the depth direction (Z direction) of the tomographic image 80X, in synchronization with the display magnification of the frontal image 70. In this case, even if the tomographic image 80X is enlarged, the possibility that at least one of the layer and boundary (hereinafter simply referred to as "layer·boundary") that the user pays attention to deviates from the display range of the tomographic image 80X is reduced. Therefore, the user can more appropriately observe the layer·boundary of interest on the tomographic image 80X. Also, the CPU 41 may change the display magnification of the tomographic image 80X so that the center position in the depth direction of the tomographic image 80X always coincides with the center position in the depth direction within the display frame of the tomographic image 80X. Also, in S21, the CPU 41 may change the display magnification of both images while maintaining both the aspect ratio of the frontal image 70 and the aspect ratio of the tomographic image 80X.

[0094] In addition, a specific method for synchronously changing the display magnifications of the frontal image 70 and the tomographic image 80X can also be appropriately selected. For example, the CPU 41 may synchronously change the display magnifications of the frontal image 70 and the tomographic image 80X based on the shooting information of each of the frontal image 70 and the tomographic image 80X (for example, at least any one of the shooting field angles of the frontal image 70 and the tomographic image 80X, the number of scan points, the resolution, the model of the imaging device 1, and the axial length of the eye when the biological tissue is the eye to be examined, etc.).

[0095] In this embodiment, when an instruction to change the display magnification of the frontal image 70 is input, not only the display magnification of the tomographic image 80X extending in the XZ direction but also the display magnification of the tomographic image 80Y extending in the YZ direction is synchronously changed. In order to synchronously change the display magnification of the frontal image 70 and the display magnification of the tomographic image 80Y extending in the YZ direction, the method of S21 may be executed by replacing "X direction" with "Y direction". Further, the CPU 41 may synchronously change the display magnifications of the tomographic image 80X and the tomographic image 80Y in a state where the display ranges in the depth direction of the tomographic image 80X and the display ranges in the depth direction of the tomographic image 80Y are made to coincide. In this case, since the display ranges in the depth direction of the plurality of tomographic images 80X and 80Y coincide, it becomes easier to appropriately grasp the state of the tissue.

[0096] Next, the CPU 41 determines whether an instruction to change the display magnification on the monitor 47 for the tomographic image 80 (either the tomographic image 80X or the tomographic image 80Y in this embodiment) displayed on the monitor 47 has been input (S23). If not (S23: NO), the process directly proceeds to S27. The method for allowing the user to input an instruction to change the display magnification for the tomographic image 80 can be appropriately selected. As an example, in this embodiment, the CPU 41 causes a cursor that moves within the display area in response to the operation of the operation unit 48 (a mouse in this embodiment) to be displayed on the monitor 47. The user operates the operation unit 48 to move the position of the cursor displayed on the monitor 47 to a position (hereinafter referred to as the "reference position") that is the center of the change (enlargement and reduction) of the display magnification of the tomographic image 80. Thereafter, the user performs an operation for enlarging or reducing the image (in this embodiment, a forward or reverse rotation operation of the mouse wheel). As a result, an instruction to change the display magnification for the tomographic image 80 and an instruction to specify the reference position within the tomographic image 80 are input to the medical image processing apparatus 40. Therefore, by designating an appropriate position as the reference position, the user can enlarge or reduce the image centered on the designated reference position.

[0097] When an instruction to change the display magnification for the tomographic image 80 is input (S23: YES), the CPU 41 sets the reference position for changing the magnification of the tomographic image 80 at the position within the tomographic image 80 designated by the user (in this embodiment, the position of the cursor on the tomographic image 80 when the rotation operation of the mouse wheel is performed) (S24). The CPU 41 executes the process of changing the display magnification (enlargement process or reduction process) of the tomographic image 80 centered on the reference position set in S24 (S20).

[0098] Also, the CPU 41 executes a process (S26) of changing the display magnification of the frontal image 70 together with the process (S24, S25) of changing the display magnification of the tomographic image 80. In S26 of the present embodiment, the CPU 41 changes the display magnification of the frontal image 70 in a state where the display range on the monitor 47 in the extension direction extending perpendicular to the depth direction (Z direction) among the tomographic images 80X or 80Y is synchronized with the display range on the monitor 47 in the extension direction of the frontal image 70. Here, when an instruction to change the display magnification for the tomographic image 80X is given, the extension direction is the X direction. When an instruction to change the display magnification for the tomographic image 80Y is given, the extension direction is the Y direction. Note that the tomographic image 80 may extend in a direction intersecting each of the X direction and the Y direction. Also in this case, the display range on the monitor 47 in the extension direction of the frontal image 70 is synchronized with the display range in the extension direction of the tomographic image 80. By performing the process of S26, the user can more appropriately compare the frontal image 70 of the site of interest with the tomographic image 80X.

[0099] Note that the method of determining the display position of the frontal image 70 in the intersection direction (for example, when the extension direction is the X direction, the intersection direction is the Y direction) intersecting the extension direction when changing the display magnification of the frontal image 70 in S26 can be appropriately selected. For example, the CPU 41 may change the display magnification of the frontal image 70 while maintaining the center position of the frontal image 70 in the intersection direction. Also, the CPU 41 may change the display magnification of the frontal image 70 while maintaining the position of the extraction line 75 (75X or 75Y) extending in the extension direction at a constant position within the display area of the frontal image 70 regardless of the display magnification. In this case, since the relative positional relationship between the frontal image 70 and the tomographic image 80 is appropriately maintained, the state of the biological tissue is more easily and appropriately grasped.

[0100] When an instruction to change the display magnification is input for one of the plurality of tomographic images 80X and 80Y, the CPU 41 also changes the display magnification of the other tomographic images synchronously. As an example, the CPU 41 changes the display magnification of the plurality of tomographic images 80 in a state where the display range in the depth direction of the tomographic image 80 for which the display magnification change instruction is given is synchronized with the display range in the depth direction of the other tomographic images 80. Further, the CPU 41 changes the display magnification of the other tomographic images 80 in a state where the display range in the extension direction (direction intersecting the depth direction) of the other tomographic images 80 is synchronized with the display range in the extension direction of the front image 70.

[0101] Further, the CPU 41 may set a first reference position that serves as a reference for enlargement and reduction in the X direction (first extension direction) of the tomographic image 80X (first tomographic image) (the center of enlargement and reduction in this embodiment). The CPU 41 may set a second reference position that serves as a reference for enlargement and reduction in the Y direction (second extension direction) of the tomographic image 80Y (second tomographic image) (the center of enlargement and reduction in this embodiment). The CPU 41 may synchronously change the display magnification of the tomographic image 80X and the tomographic image 80Y with the first reference position as the center of enlargement and reduction in the X direction of the tomographic image 80X and the second reference position as the center of enlargement and reduction in the Y direction of the tomographic image 80Y. The CPU 41 may synchronously change the display magnification of the front image 70, the tomographic image 80X, and the tomographic image 80Y with the center of enlargement and reduction in the X direction of the front image 70 as the first reference position and the center of enlargement and reduction in the Y direction as the second reference position. In this case, based on the set first reference position and second reference position, the display magnification of the plurality of images is appropriately changed synchronously.

[0102] A specific method for setting the first reference position and the second reference position can be appropriately selected. For example, as described above, the CPU 41 may automatically set the first reference position at the position in the X direction of the three-dimensional image 60 where the tomographic image 80Y is extracted (the position of the extraction line 75Y in this embodiment). The CPU 41 may automatically set the second reference position at the position in the Y direction of the three-dimensional image 60 where the tomographic image 80X is extracted (the position of the extraction line 75X in this embodiment). In this case, the display magnification of the other tomographic image is changed with reference to the position where one tomographic image is extracted. Therefore, it becomes easier to compare a plurality of images.

[0103] Further, the CPU 41 may set at least one of the first reference position and the second reference position according to an instruction input by the user. In this case, the user can change the display magnification of at least one of the first tomographic image and the second tomographic image with reference to a desired position. Specifically, the user may input an instruction to the medical image processing apparatus 40 to specify the first reference position by inputting an instruction to move at least one of the first reference line on the frontal image 70 and the first reference line on the tomographic image 80X. Similarly, the user may input an instruction to the medical image processing apparatus 40 to specify the second reference position by inputting an instruction to move at least one of the second reference line on the frontal image 70 and the second reference line on the tomographic image 80Y.

[0104] Next, the CPU 41 determines whether a reset instruction for returning the display magnification and the display position of the image to the initial display state of the image has been input (S27). If not input (S27: NO), the process directly proceeds to S29. When the reset instruction is input (S27: YES), the CPU 41 resets at least one of the display magnification of the frontal image 70 and the tomographic image 80 and the display position to be displayed within the display frame of the monitor 47 to the state at the start of image display (S28). Therefore, the user can return the display magnification and the display position of the frontal image 70 and the tomographic image 80 to the state at the start of display with a simple operation.

[0105] If an instruction to end the process is not input (S29: NO), the process returns to S11, and the processes of S11 to S29 are repeated. When the end instruction is input (S29: YES), the image display process ends.

[0106] According to the present embodiment, just by inputting an instruction to change the display magnification for one of the front image 70 and the tomographic image 80 displayed on the monitor 47 to the medical image processing apparatus 40, the display magnifications of both the front image 70 and the tomographic image 80 are changed synchronously. Therefore, the user can more easily and appropriately compare the two images as compared with the case of individually changing the magnifications of the front image 70 and the tomographic image 80.

[0107] Also, in the present embodiment, an angiography front image and a tomographic image (hereinafter referred to as "MC superimposed tomographic image") in which information of motion contrast data is superimposed are both displayed on the monitor 47. Therefore, the user can appropriately grasp information on the movement such as blood flow in the living tissue by both the angiography front image and the MC superimposed tomographic image. For example, it is also possible to confirm the state of the tissue at the site appearing as a blood vessel in the front image 70 on the tomographic image 80. Further, according to the present embodiment, just by inputting an instruction to change the display magnification for one of the angiography front image and the MC superimposed tomographic image, the display magnifications of both the angiography front image and the MC superimposed tomographic image are changed synchronously. Therefore, the user can more easily and appropriately grasp information on the movement such as blood flow in the living tissue by comparing the angiography front image and the MC superimposed tomographic image whose display magnifications are changed synchronously.

[0108] (Second Embodiment) Referring to FIGS. 9 to 11, the image display process executed by the medical image processing apparatus 40 of the second embodiment will be described. Note that, for some of the processes in the second embodiment, the same processes as those described in the first embodiment can be adopted. Therefore, hereinafter, when the same processes as those in the first embodiment can be adopted, the description will be omitted or simplified. The CPU 41 of the medical image processing apparatus 40 executes the image display process shown in FIG. 9 according to the medical image processing program stored in the NVM 44.

[0109] Note that, also in the second embodiment, all the display magnifications of the frontal image 70, the tomographic image 80X extending in the XZ direction, and the tomographic image 80Y extending in the YZ direction are changed synchronously. However, hereinafter (FIG. 11 and the like), for the sake of simplifying the description, there may be cases where only the case of synchronously changing the display magnification of the frontal image 70 and the display magnification of the tomographic image 80X extending in the XZ direction is exemplified, as in the first embodiment. Note that, in order to synchronously change the display magnification of the frontal image 70 and the display magnification of the tomographic image 80Y extending in the YZ direction, the method described below may be executed by replacing "X direction" with "Y direction".

[0110] First, the CPU 41 acquires the data of a plurality of three-dimensional images 60 (see FIG. 4) taken of the same biological tissue of the same subject (S31). Here, the imaging times of each of the plurality of three-dimensional images 60 acquired in S31 may be different from each other. In this case, by executing the processes described below, the user can more easily and appropriately observe the temporal changes (i.e., longitudinal observation) of the biological tissue of the subject to be imaged.

[0111] The CPU 41 acquires a front image 70 from each of the plurality of three-dimensional images 60 acquired in S31. In the example shown in FIG. 11, the first front image 70A is acquired from the first three-dimensional image 60A, the second front image 70B is acquired from the second three-dimensional image 60B, and the third front image 70C is acquired from the third three-dimensional image 60C. Similar to the first embodiment, in the second embodiment, a plurality of front images 70 are acquired by generating an Enface image based on each of the plurality of three-dimensional OCT data. More specifically, a plurality of angiography front images are acquired by generating an Enface image based on each of the plurality of motion contrast data. However, as described above, a front image different from the angiography front image may be acquired.

[0112] Next, the CPU 41 performs alignment processing on the plurality of three-dimensional images 60 acquired in S31 (S33). As an example, in the present embodiment, the CPU 41 performs alignment processing (registration processing) on the plurality of front images 70 acquired in S32 to align the positions when the plurality of three-dimensional images 60 are viewed from the front direction. As a result, the user can appropriately compare the biological tissues at the same position shown in each of the plurality of front images 70. Further, in the present embodiment, the plurality of three-dimensional images 60 are also aligned using the front image 70. Therefore, the positions of the biological tissues shown in each of the plurality of tomographic images 80 acquired in the process of S35 described later are also likely to coincide. Note that the alignment processing includes at least any one (all in the present embodiment) of rotation, translation, and scaling processing.

[0113] The CPU 41 sets the extraction position of the tomographic image 80 at the same position (default position) in each of the plurality of three-dimensional images 60 acquired in S31 (S34). The CPU 41 acquires a plurality of tomographic images 80 (see FIG. 11) by extracting two-dimensional images at the extraction positions set in S34 from each of the plurality of three-dimensional images 60 (S35). As a result, the imaging positions in the biological tissue of the plurality of extracted tomographic images 80 are common to each other. Also, as described above, in the present embodiment, the tomographic images 80 are extracted from the same positions of the plurality of three-dimensional images 60 in a state where the alignment of the plurality of three-dimensional images 60 is performed at least when viewed from the front direction. Therefore, the imaging positions in the biological tissue of the plurality of tomographic images 80 are more likely to be approximated. In the example shown in FIG. 11, the first tomographic image 80XA is extracted from the first three-dimensional image 60A, the second tomographic image 80XB is extracted from the second three-dimensional image 60B, and the third tomographic image 80XC is extracted from the third three-dimensional image 60C. The imaging positions in the biological tissue of each of the first tomographic image 80XA, the second tomographic image 80XB, and the third tomographic image 80XC are approximated. Also, although not shown, in the present embodiment, tomographic images 80 extending in the YZ direction are also extracted from the same positions of each of the three three-dimensional images 60A to 60C. Note that, similar to the first embodiment, the default position where the extraction position is set can be set as appropriate.

[0114] The CPU 41 causes the monitor 47 to display a plurality of image sets (sets of the front images 70 and the tomographic images 80) acquired from each of the plurality of three-dimensional images 60 (S36). In the example shown in FIG. 11, the first image set (the first front image 70A and the first tomographic image 80XA) acquired from the first three-dimensional image 60A, the second image set (the second front image 70B and the second tomographic image 80XB) acquired from the second three-dimensional image 60B, and the third image set (the third front image 70C and the third tomographic image 80XC) acquired from the third three-dimensional image 60C are displayed on the monitor 47. Next, the display mode change process (S37, see FIG. 10) is executed.

[0115] As shown in FIG. 10, when the display mode change process is started, the CPU 41 determines whether an instruction (trigger) to change the extraction position of the tomographic image 80 from each of the three-dimensional images 60A to 60C has been input (S41). As described above, the instruction (trigger) to change the extraction position of the tomographic image 80 may be input by the user or automatically input. ) If not input (S41: NO), the process directly proceeds to S45. In the second embodiment, the user operates the operation unit 48 to change the position of any of the extraction lines 75X and 75Y of the plurality of front images 70 displayed on the monitor 47, thereby inputting an instruction to change the extraction position from any of the three-dimensional images 60 of the plurality of tomographic images 80. When an instruction to change the extraction position of any of the tomographic images 80 is input, the CPU 41 changes the extraction positions from the three-dimensional images 60 for all of the displayed tomographic images 80 to the same position (S42). The CPU 41 re-acquires the tomographic image 80 from the new extraction position of each of the plurality of three-dimensional images 60 and causes it to be displayed on the monitor 47 (S43). In the process of S33 described above, the alignment process of the plurality of three-dimensional images 60 has been performed in advance. Therefore, by performing the process of S43, the user can change the extraction positions of the plurality of tomographic images 80 to the same position in a batch by simply changing the extraction position of any of the plurality of tomographic images 80.

[0116] The CPU 41 determines whether an instruction to execute the magnification synchronization change process has been input by the user (S45). The medical image processing apparatus 40 according to the second embodiment can execute a magnification synchronization change process (S48 to S59) and a magnification independent change process (S46). In the magnification synchronization change process, when a change instruction for the display magnification of any one of the plurality of frontal images 70 and the plurality of tomographic images 80 displayed on the monitor 47 is input by the user, the display magnifications of all the plurality of frontal images 70 and the plurality of tomographic images 80 being displayed are synchronously changed. On the other hand, in the magnification independent change process, the display magnification of the image for which the change instruction for the display magnification has been given among the plurality of frontal images 70 and the plurality of tomographic images 80 displayed on the monitor 47 is independently changed. The user can input an instruction for designating which of the magnification synchronization change process and the magnification independent change process to execute by operating the operation unit 48. When an instruction to execute the magnification synchronization change process is input (S45: YES), the CPU 41 executes the magnification synchronization change process (S48 to S59) described below. On the other hand, when an instruction to execute the magnification independent change process is input (S45: NO), the CPU 41 executes the magnification independent change process according to the instruction input by the user (S46). The processes of S41 to S46 are repeated. As described above, the user can appropriately use the magnification synchronization change process and the magnification independent change process according to various situations.

[0117] The magnification synchronization change process (S18 to S29) will be described in detail below. The CPU 41 determines whether an instruction to change the display magnification to any of the plurality of front images 70 displayed on the monitor 47 has been input (S48). If not (S48: NO), the process directly proceeds to S53. As an example, in the second embodiment, the user operates the operation unit 48 to move the position of the cursor displayed on the monitor 47 to a position that is the center of the change (enlargement and reduction) of the display magnification of any of the front images 70 (hereinafter referred to as the "reference position"). Thereafter, the user performs an operation to enlarge or reduce the image (in this embodiment, the operation of rotating the mouse wheel forward or backward). As a result, an instruction to change the display magnification for any of the front images 70 and an instruction to specify the reference position within the front image 70 are input to the medical image processing apparatus 40.

[0118] When an instruction to change the display magnification for any of the front images 70 is input (S48: YES), the CPU 41 sets the reference position at the position within the front image 70 specified by the user (in this embodiment, the position of the cursor on the front image 70 when the mouse wheel is rotated). Further, the CPU 41 also sets the reference position at the same position within the front images 70 among the plurality of front images 70 for which no instruction to change the display magnification has been input (S49). The CPU 41 synchronizes and executes the change process (enlargement process or reduction process) of the display magnification of each of the plurality of front images 70 with the reference position set in S49 as the center (S50). That is, the CPU 41 sets a reference position for enlargement and reduction at the same position within the image area of each of the plurality of front images 70, and executes the enlargement process and reduction process of the plurality of front images 70 with the set reference position as the center. As a result, the display magnifications of the plurality of front images 70 are changed while the display positions of the plurality of front images 70 are synchronized.

[0119] In FIG. 11, a plurality of front images 70 before image enlargement and a plurality of tomographic images 80X are shown on the upper side, and a plurality of front images 70 and a plurality of tomographic images 80X after image enlargement are shown on the lower side. In the example shown in FIG. 11, each of the plurality of front images 70 is enlarged centering on the same reference position within each image area.

[0120] Further, the CPU 41 executes a process (S49, S50) of changing the display magnification of the plurality of front images 70 and a process (S51) of synchronously changing the display magnification of the tomographic images 80X, 80Y. As a method of changing the display magnification of each tomographic image 80, a method similar to the method described in S21 of the first embodiment can be adopted. Therefore, this detailed description is omitted.

[0121] Next, the CPU 41 determines whether an instruction to change the display magnification to the monitor 47 has been input for a plurality of tomographic images 80 (in this embodiment, any one of the plurality of tomographic images 80X and the plurality of tomographic images 80Y) displayed on the monitor 47 (S53). If not input (S53: NO), the process directly proceeds to S57. As an example, in this embodiment, the user operates the operation unit 48 to move the position of the cursor displayed on the monitor 47 to a position (hereinafter referred to as the "reference position") that is the center of changing the display magnification (enlarging and reducing) of any of the plurality of tomographic images 80. Thereafter, the user performs an operation for enlarging or reducing the image (in this embodiment, an operation of rotating the mouse wheel forward or backward). As a result, an instruction to change the display magnification for any of the tomographic images 80 and an instruction to specify the reference position within the tomographic image 80 are input to the medical image processing apparatus 40.

[0122] When an instruction to change the display magnification for any of the tomographic images 80 is input (S53: YES), the CPU 41 sets a reference position at the position within the tomographic image 80 (tomographic image 80X or tomographic image 80Y) specified by the user (in this embodiment, the position of the cursor on the tomographic image 80 when the rotation operation of the mouse wheel is performed). Further, the CPU 41 sets a reference position also at the same position within the tomographic image 80X or tomographic image 80Y among the plurality of tomographic images 80X or the plurality of tomographic images 80Y extending in the same first extension direction (X direction or Y direction) as the extension direction of the tomographic image 80 for which the display magnification change instruction has been given and for which no display magnification change instruction has been input (S54). The CPU 41 changes the display magnifications of the plurality of tomographic images 80 (tomographic image 80X or tomographic image 80Y) extending in the first extension direction in synchronization with each other with the reference position set in S54 as the center (S55). That is, the CPU 41 sets a reference position serving as a reference for enlargement and reduction at the same position within the image area of each of the plurality of tomographic images 80 extending in the first extension direction, and executes an enlargement process and a reduction process of the plurality of tomographic images 80 extending in the first extension direction with the set reference position as the center. As a result, the display magnifications of the plurality of tomographic images 80 are changed in a state where the display positions of the plurality of tomographic images 80 are synchronized.

[0123] Further, the CPU 41 executes a process (S54, S55) of changing the display magnification of a plurality of tomographic images 80 extending in the first extension direction and a process (S56) of changing the display magnification of a plurality of front images 70. In S56 of the present embodiment, the display magnification of the plurality of front images 70 is changed in a state where the display range on the monitor 47 in the first extension direction (the direction extending perpendicular to the depth direction) of the tomographic image 80 and the display range on the monitor 47 in the first extension direction of the front image 70 are synchronized. Note that, in S56, the method of determining the display position of the front image in the second extension direction intersecting the first extension direction when changing the display magnification of the front image 70 can be appropriately selected. For example, the CPU 41 may change the display magnification of the plurality of front images 70 while maintaining the center position in the second extension direction of each front image 70. Further, the CPU 41 may change the display magnification of the plurality of front images 70 while maintaining the position of the extraction line 75 (75X or 75Y) extending in the first extension direction at a constant position within the display area of each front image 70 regardless of the display magnification. In this case, since the relative positional relationship between the front image 70 and the tomographic image 80 is appropriately maintained, the state of the biological tissue can be more appropriately grasped. Note that, in the present embodiment, when the first extension direction is the X direction, the second extension direction is the Y direction. When the first extension direction is the Y direction, the second extension direction is the X direction.

[0124] Note that when an instruction to change the display magnification is input for any of the plurality of tomographic images 80 extending in the first extension direction, the CPU 41 also synchronously changes the display magnification of the plurality of tomographic images extending in the second extension direction. As an example, the CPU 41 changes the display magnification of the plurality of tomographic images 80 extending in the second extension direction in a state where the display range in the depth direction of the plurality of tomographic images 80 extending in the first extension direction and the display range in the depth direction of the plurality of tomographic images 80 extending in the second extension direction are synchronized. Further, the CPU 41 changes the display magnification of the plurality of tomographic images 80 extending in the second extension direction in a state where the display range in the second extension direction of the tomographic image 80 extending in the second extension direction and the display range in the second extension direction of the front image 70 are synchronized.

[0125] Next, the CPU 41 determines whether a reset instruction to return the display magnification and display position of the image to the initial display state of the image has been input (S57). If not (S57: NO), the process directly proceeds to S59. When a reset instruction is input (S57: YES), the CPU 41 resets at least one of the display magnifications of the plurality of front images 70 and the plurality of tomographic images 80, and the display position to be displayed within the display frame of the monitor 47, to the state at the start of image display (S58). Therefore, the user can return the display magnifications and display positions of the plurality of front images 70 and the plurality of tomographic images 80 to the state at the start of display with a simple operation.

[0126] If an instruction to end the process has not been input (S59: NO), the process returns to S41, and the processes of S41 to S59 are repeated. When an end instruction is input (S59: YES), the image display process ends.

[0127] According to the second embodiment, the user can appropriately grasp the state of the biological tissue by comparing a plurality of image sets taken of the same biological tissue of the same subject. Further, when an input of an instruction to change the display magnification to the monitor 47 is received for any one of the plurality of front images 70 and the plurality of tomographic images 80, not only the display magnification of the image for which the display magnification change instruction has been given, but also the display magnifications of the other plurality of images (front images 70 and tomographic images 80) displayed on the monitor 47 are changed according to the input instruction. Therefore, the user can more easily and appropriately compare the plurality of images (front images 70 and tomographic images 80) compared to the case of individually changing the magnification of each of the plurality of images.

[0128] The technology disclosed in the above embodiments is merely an example. Therefore, it is also possible to change the technology exemplified in the above embodiments. First, it is also possible to execute only a part of the processes exemplified in the above embodiments. Further, the CPU 41 may change the position for extracting the tomographic image 80 to be displayed on the monitor 47 among the regions of the three-dimensional image 60 according to the position for enlargedly displaying the front image 70. For example, the CPU 41 can also change the extraction position of the tomographic image 80 so that the extraction position of the tomographic image 80 within the region of the three-dimensional image 60 is maintained at a specific position (for example, the center position, etc.) within the display frame of the front image 70 displayed on the monitor 47. In this case, since the relative positional relationship between the front image 70 and the tomographic image 80 changes appropriately, it becomes easier to appropriately grasp the state of the biological tissue.

[0129] The processes of acquiring images in S2, S4 in FIG. 6 and S32, S35 in FIG. 9 are examples of the "image acquisition step". The processes of displaying images in S5 in FIG. 6 and S36 in FIG. 9 are examples of the "image display step". The processes of changing the display magnification of the images in S18 to S26 in FIG. 7 and S48 to S56 in FIG. 10 are examples of the "magnification synchronous change step". The processes of resetting the display magnification and display position of the images in S28 in FIG. 7 and S58 in FIG. 10 are examples of the "display reset step". The processes of changing the display magnification of the images in S16 in FIG. 7 and S46 in FIG. 10 are examples of the "magnification independent change step".

[0130] 1 Imaging device 40 Medical image processing device 41 CPU 44 NVM 47 Monitor 48 Operation unit 60 Three-dimensional image 70 Front image 75(75X, 75Y) Extraction line 80(80X, 80Y) Tomographic image 100 Medical image processing system

Claims

1. A medical image processing apparatus for processing image data of a biological tissue, wherein a control unit of the medical image processing apparatus acquires a front image, which is a two-dimensional image of the same biological tissue of the same subject taken from a direction along the optical axis of the imaging light, and a tomographic image, which is a two-dimensional image extending in the depth direction of the biological tissue, in an image acquisition step; displays the front image and the tomographic image on a display unit in an image display step; when an input of an instruction to change the display magnification to the display unit is received for one of the front image and the tomographic image, a magnification synchronous change step of changing both the display magnification of the image for which the instruction to change the display magnification is made and the display magnification of the image for which the instruction to change the display magnification is not made among the front image and the tomographic image according to the input instruction; and executes the steps. A medical image processing apparatus characterized by the above.

2. The medical image processing apparatus according to claim 1, wherein the front image is an angiographic front image generated by processing a plurality of OCT signals acquired from the same position of a biological tissue at different times by an OCT apparatus. A medical image processing apparatus characterized by the above.

3. The medical image processing apparatus according to claim 1 or 2, wherein the tomographic image is taken by an OCT apparatus that takes an image of a biological tissue using the principle of optical coherence tomography, and information on motion contrast data generated by processing a plurality of OCT signals acquired from the same position of a biological tissue at different times by the OCT apparatus is superimposed on the tomographic image. A medical image processing apparatus characterized by the above.

4. The medical image processing apparatus according to any one of claims 1 to 3, In the magnification synchronization change step, the control unit changes both the display magnification of the frontal image and the display magnification of the tomographic image in a state where the display range on the display unit in the extension direction extending in a direction perpendicular to the depth direction among the tomographic images is synchronized with the display range on the display unit in the extension direction of the frontal image. A medical image processing apparatus characterized by this.

5. A medical image processing apparatus according to any one of claims 1 to 4, wherein the control unit in the image acquisition step, acquires the frontal image, a first tomographic image extending in a first extension direction which is one of the directions perpendicular to the depth direction, and a second tomographic image extending in a second extension direction which is perpendicular to the depth direction and different from the first extension direction, in the image display step, causes the frontal image, the first tomographic image, and the second tomographic image to be displayed on the display unit, and in the magnification synchronization change step, when an input of a change instruction for the display magnification of any of the plurality of displayed images is received, changes the display magnification of the frontal image, the display magnification of the first tomographic image, and the display magnification of the second tomographic image in synchronization. A medical image processing apparatus characterized by this.

6. A medical image processing apparatus according to claim 5, wherein the control unit further executes a first reference position setting step of setting a first reference position which is a reference for enlargement and reduction in the first extension direction of the first tomographic image, and a second reference position setting step of setting a second reference position which is a reference for enlargement and reduction in the second extension direction of the second tomographic image, and in the magnification synchronization change step, changes the display magnification of the first tomographic image and the second tomographic image in synchronization with the first reference position as a reference for enlargement and reduction of the first tomographic image and the second reference position as a reference for enlargement and reduction of the second tomographic image. A medical image processing apparatus characterized by this.

7. The medical image processing apparatus according to claim 6, wherein the control unit further executes a reference position display step of displaying the first reference position and the second reference position on the frontal image displayed on the display unit. The medical image processing apparatus is characterized by this.

8. The medical image processing apparatus according to any one of claims 1 to 7, wherein in the magnification synchronization change step, the control unit receives an input of an instruction for designating a reference position serving as a reference for enlargement and reduction within a display area of at least one of the frontal image and the tomographic image, and performs an enlargement process and a reduction process of at least one of the frontal image and the tomographic image centered on the designated reference position. The medical image processing apparatus is characterized by this.

9. The medical image processing apparatus according to any one of claims 1 to 8, wherein when an input of an instruction to return at least one of the display magnification and the display position of an image to an initial state is received, the control unit further executes a display reset step of resetting at least one of the display magnifications of the frontal image and the tomographic image and the display position to be displayed within the display frame of the display unit to the state at the start of display. The medical image processing apparatus is characterized by this.

10. The medical image processing apparatus according to any one of claims 1 to 9, wherein the control unit when an input of an instruction to change the display magnification to the display unit for one of the frontal image and the tomographic image is received, the control unit is capable of executing a magnification independent change step of enlarging or reducing the display magnification of the image for which the magnification change instruction has been given among the frontal image and the tomographic image according to the input instruction, and executes either the magnification synchronization change step or the magnification independent change step according to an instruction input by a user. The medical image processing apparatus is characterized by this.

11. The medical image processing apparatus according to any one of claims 1 to 10, wherein the control unit in the image acquisition step, acquires a plurality of image sets each being a set of the frontal image and the tomographic image taken of the same biological tissue of the same subject, in the image display step, causes the display unit to display the plurality of image sets, and in the magnification synchronization change step, when an input of an instruction to change the display magnification to the display unit is received for any one of the plurality of frontal images and the plurality of tomographic images, changes the display magnification of each of the plurality of frontal images and the plurality of tomographic images included in the plurality of image sets according to the input instruction. A medical image processing apparatus characterized by this.

12. The medical image processing apparatus according to claim 11, wherein the control unit in the image display step, performs alignment of the plurality of frontal images included in the plurality of image sets, and causes the display unit to display the plurality of aligned frontal images. A medical image processing apparatus characterized by this.

13. The medical image processing apparatus according to claim 11 or 12, wherein the control unit, in the magnification synchronization change step, sets a reference position serving as a reference for enlargement and reduction at the same position within the display area of each of the plurality of frontal images, and performs enlargement processing and reduction processing of the plurality of frontal images centered on the set reference position, sets a reference position serving as a reference for enlargement and reduction at the same position within the display area of each of the plurality of tomographic images, and performs enlargement processing and reduction processing of the plurality of tomographic images centered on the set reference position. A medical image processing apparatus characterized by this.

14. A medical image processing program executed by a medical image processing apparatus that processes image data of biological tissue, When the medical image processing program is executed by the control unit of the medical image processing apparatus, an image acquisition step of acquiring a front image, which is a two-dimensional image of the same biological tissue of the same subject as viewed from a direction along the optical axis of the imaging light, and a tomographic image, which is a two-dimensional image extending in the depth direction of the biological tissue; an image display step of causing the front image and the tomographic image to be displayed on a display unit; a magnification synchronous change step of changing both the display magnification of the image for which a change instruction of the display magnification to the display unit has been input and the display magnification of the image for which no change instruction of the display magnification has been input, among the front image and the tomographic image, in accordance with the input instruction when an input of a change instruction of the display magnification to the display unit is received for one of the front image and the tomographic image; A medical image processing program, characterized in that the medical image processing apparatus is caused to execute the program.

Images