Medical image processing apparatus, medical image processing method and medical image processing program

Abstract

To provide a medical image processing apparatus and the like which improve the operability when operating a displayed object in a configuration that can combine a 3D anatomical model with an affected portion image and display the combined image when performing a surgery while capturing the affected portion image of a subject with a medical optical imaging device.SOLUTION: There is provided a medical image processing apparatus (10A) which includes an image display control unit. The image display control unit is configured to perform processing of reading a 3D anatomical model that three-dimensionally expresses an anatomical structure of the subject and processing of changing the position of a rotational axis of the 3D anatomical model (Ax1→Ax2) in accordance with the position after the change when the image formation position of the 3D anatomical model is changed.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a medical image processing device, a medical image processing method, and a medical image processing program, and in particular to a medical image processing device, a medical image processing method, and a medical image processing program that improve operability when operating a displayed object in a configuration in which an image of an affected part of a subject is captured by a medical optical imaging device and a 3D anatomical model is synthesized and displayed when performing surgery. [Background technology]

[0002] Laparoscopic surgery is a technique in which an endoscope is inserted into the abdominal cavity of a patient, for example, from around the navel, and surgery is performed in the abdominal cavity based on the image displayed on a monitor. Specifically, a small incision is made in the skin, and forceps are inserted into the abdominal cavity to perform the surgery.

[0003] Patent Document 1 discloses an example of a surgical procedure in which a stereoscopic endoscope is used to display an image of the affected area on a 3D monitor while inserting forceps or the like into the abdomen of a subject to perform treatment. The number of 3D monitors may be one or more, and the observer wears 3D viewing glasses to observe the monitor. Patent Document 2 discloses a technique in which other medical images, such as ultrasound images, are superimposed on the volume of a virtual anatomical model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] WO2015 / 053043 issue [Patent Document 2] Special table number 2019-517291 Summary of the Invention [Problem to be solved by the invention]

[0005] In recent years, it has become common to perform surgery while referring to a 3D anatomical model of the subject during surgery, and surgery is sometimes performed while the 3D anatomical model is displayed on a monitor along with a three-dimensional image of the affected area.

[0006] Being able to compare the 3D anatomical model with images of the actual affected area of ​​the subject during surgery is desirable from the perspective of providing highly reliable surgery, but there is still room for improvement in terms of operability and ease of use when manipulating the 3D anatomical model during surgery.

[0007] Therefore, the object of the present invention is to provide a medical image processing device, a medical image processing method, and a medical image processing program that improve operability when operating a displayed object in a configuration in which a 3D anatomical model can be synthesized and displayed on an image of an affected area of ​​a subject when performing surgery while the affected area is imaged with a medical optical imaging device. [Means for solving the problem]

[0008] In order to solve the above problems, a medical image processing apparatus according to one aspect of the present invention is as follows: A medical image processing device including an image display control unit, The image display control unit is A process of reading a 3D anatomical model that represents the anatomical structures of the subject in three dimensions; When the position of the image of the 3D anatomical model (imaging position: in one example, a position in the depth direction in a three-dimensional space) is changed, a process is performed to change the position of the rotation axis of the 3D anatomical model according to the changed position. Medical imaging equipment.

[0009] (Terminology explanation) "Medical optical imaging device" refers to, for example, an endoscopic device used in laparoscopic surgery, or an imaging system with a high-definition camera used in craniotomy. The "control unit" may be, for example, a computer unit having a CPU (Central Processing Unit) for performing arithmetic processing, memory, an interface, etc., and implementing various functions by executing computer programs stored in the memory. As an example, the computer unit may be a so-called one-chip microcomputer having hardware such as a CPU, ROM, RAM, and an I / F, and on which a program is implemented. With regard to a "computer program", the subject of its implementation may be singular or plural. The computer program may be stored in a computer-readable recording medium. The recording medium storing the computer program may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a memory card, a CD-ROM, or other recording medium. The computer program stored in the recording medium can be implemented in a computer unit via an appropriate reader. Examples of the appropriate reader include a card reader when the recording medium is a memory card, and a CD drive when the recording medium is a CD-ROM. With regard to a "part (which can also be expressed as a "section", "unit", or "module", etc.)", what is expressed in this specification as, for example, "(name of function)" + "part" is something that can be realized as a function of a computer. Such a "part" may be provided in any device in the system. Also, it does not necessarily have to be provided in one device, and the corresponding function may be distributed and provided in two or more devices. Furthermore, only a specific one or more "parts" may be provided in an external device via a communication network (e.g. the Internet). Such a "part" may be various functions that a computer logically has. Effect of the Invention

[0010] According to the present invention, it is possible to provide a medical image processing device, etc., which is configured to be able to synthesize and display a 3D anatomical model onto an image of a subject's affected area when performing surgery while capturing the image with a medical optical imaging device, and which has improved operability when operating the displayed object. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating a schematic configuration of a medical image processing apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic diagram showing object relationships of a 3D anatomical model. [Figure 3A] 1 is an example of a system configuration including a medical image processing apparatus according to an embodiment of the present invention. [Figure 3B] 1 is another example of a system configuration including a medical image processing apparatus according to an embodiment of the present invention. [Figure 4] This is an example of an image in which an endoscopic image and a 3D anatomical model are combined and displayed. [Diagram 5] 1 is a flowchart for adjusting the display manner of a 3D anatomical model according to the characteristics of a medical optical imaging device. [Figure 6] FIG. 2 is a schematic diagram showing a state in which a test chart is imaged by an endoscope; [Figure 7] FIG. 2 is a schematic diagram showing the relationship between an endoscopic image and the imaging position of a 3D anatomical model. [Figure 8] FIG. 2 is a diagram for explaining the relationship between the imaging position of a 3D anatomical model and the rotation axis of the model. [Figure 9] 1 is an example of a graphical user interface. [Figure 10] 1 is a diagram showing an example of an image format output by a medical image processing apparatus according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] As shown in FIG. 1, the medical image processing system 1 of the present embodiment includes an image display control unit 11, a data storage unit 12, a display device 13, and an input device 14. The system may further include a data input / output interface 15, a medical information management device 21, a liquid medicine injector 22, and an imaging device 23. In one embodiment of the present invention, the area indicated by reference numeral 10 in FIG. 1 may be configured as a medical image processing device, and the medical image processing device 10 may be a computer device having the image display control unit 11, the data storage unit 12, the display device 13, the input device 14, and the data input / output interface 15. More specifically, the system may be a portable terminal (e.g., a tablet terminal) that is housed in a single housing and has a touch panel as the display device 13 and the input device 14.

[0014] In this example, the medical image processing apparatus 10 is connected to the medical information management apparatus 21 and the like, but may be connected to an endoscope apparatus or the like as described later.

[0015] The image display control unit 11 controls various image displays including medical three-dimensional images (also called "3D anatomical models" or "3D models" etc.), and the data storage unit 12 is a unit for storing various data. The display device 13 may be any display capable of displaying images created by the image display control unit 11, such as a liquid crystal display or an organic EL display. Note that in this embodiment, an example in which an image such as a 3D anatomical model is displayed on the display device 13 as a function of one medical image processing device 10 will be described, but the present invention is not necessarily limited thereto, and may be as follows. That is, the medical image processing device 10 may simply perform predetermined image processing and output image data, and another device (for example, a mixer (described in detail later)) may perform image synthesis.

[0016] The input device 14 may be any device, such as a keyboard or a mouse (including a 3D mouse), that can accept an input operation by an operator and input data to the image display control unit 11. A touch panel that combines a display and a touch screen can also be used as the display device 13 and the input device 14.

[0017] The input device 14 may further be combined with a non-contact input unit capable of performing input without contact. Non-contact input units can be divided into those using gesture recognition technology and those using voice recognition technology. An example of a non-contact input unit using gesture recognition technology is the "Leap Sensor" (manufactured by Leap Motion). The "Leap Sensor" is an input device capable of recognizing the movement of an operator's fingers without contact, and includes an infrared emitting unit and an infrared camera. The sensor has a function of capturing the reflected light when infrared light emitted from the infrared emitting unit hits the operator's hand with an infrared camera and performing image analysis. This makes it possible to detect the position, movement, shape, etc. of the operator's hand and fingers in a three-dimensional space in real time. Another example of a non-contact input unit using gesture recognition technology is the "RealSense" (manufactured by Intel). The "RealSense" is a modularization of a 3D camera consisting of an RGB camera and an infrared camera, an infrared sensor, etc. In addition to color information, depth information can be acquired, and the movement of the operator's fingers, etc. can be recognized in three dimensions. In this embodiment, both the Leap Sensor and the RealSense can be used. In either case, the operator's input actions include the same actions as those performed by an operator on a touch panel (e.g., tap, double tap, swipe, flick, pinch in, pinch out, etc.), as well as actions in the depth direction.

[0018] A voice recognition unit is an example of a non-contact input unit that uses voice recognition technology. The voice recognition unit may have a microphone that captures the voice generated by the operator, and a voice recognition device that recognizes the voice captured by the microphone and converts it into an operation signal. The voice recognition device may be installed anywhere, but it is preferable to install the microphone near the operator.

[0019] In this application, any input to the device can be made using various input means as described above without departing from the spirit of the present invention.

[0020] The data storage unit 12 may include at least one of a hard disk drive (HDD), a solid state drive (SSD), and various memories. The data storage unit 12 stores data of a 3D anatomical model created based on data obtained by imaging the subject's body. The data storage unit 12 may also store at least one program, graphical user interface data, tables, and the like, required for processing performed by the image display control unit 11. A part of the data stored in the data storage unit 12 may be acquired from the medical information management device 21 through the data input / output interface 15.

[0021] The medical information management device 21 may be one or more of PACS (picture archiving and communication systems), RIS (radiology information system), HIS (hospital information system), etc. In one example, the medical information management device 21 manages medical image data of a subject into which a medical fluid (e.g., a contrast agent) is injected by a medical fluid injector 22 and an image is captured by an imaging device 23.

[0022] Any injection device that automatically injects a liquid such as a contrast medium filled in a syringe, a liquid bag, or the like into a subject according to preset injection conditions can be used as the liquid injection device 22. As an example, the liquid injection device may be one that includes an injection head to which one or more syringes can be attached and a console electrically connected thereto.

[0023] The imaging device 23 may be any device capable of capturing medical images composed of image data, such as a CT (Computed Tomography) device, an MRI (Maganetic Resonance Imaging) device, an angiography device (angiography device), a PET (Positoron Emission Tomography) device, an ultrasonic diagnostic device, etc. The data stored in the data storage unit 12 may be directly acquired from the imaging device 23.

[0024] Regarding the volume data of an anatomical structure, this data may be based on a data set obtained by arranging a plurality of (e.g., 300) slice image data successively captured at regular intervals (e.g., 1 mm intervals) in a specific direction (e.g., body axis direction, left-right direction, front-back direction, a direction inclined to at least one of these directions, etc.) of the subject by the imaging device 23 in the body axis direction. The volume data includes a plurality of voxels, and a plurality of anatomical structures can be extracted by a predetermined process based on the voxel value of each voxel. For each voxel, transmittance and hue may be set for each extracted anatomical structure, and therefore each voxel may include coordinate information, transmittance information, and hue information. A visible 3D anatomical model can be created by performing a rendering process on the volume data.

[0025] The 3D anatomical model may be data including a plurality of independent objects (anatomical structures) 18a to 18c, as shown in a simplified example in Fig. 2. Here, as an example, an organ object 18a, a blood vessel object 18b, and a tumor part object 18c are included. Conventionally known methods can be used to determine the shape of each object and how to extract (segment) them. Conventionally known methods can also be used to set the color and transmittance of the objects.

[0026] The medical image processing device 10 may be configured as a system in which the display device 13 and part of the input device 14 are configured as units separate from the image display control unit 11. An example in which a plurality of display devices are provided will be described later with reference to another drawing.

[0027] The image display control unit 11 can be configured with a processor unit equipped with a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The image display control unit 11 executes various processes for controlling the display of images on the display device 13 according to inputs received by the input device 14. The processes performed by the image display control unit 11 may be realized by a computer program, or may be realized by hardware using a logic circuit, or may be a combination of these.

[0028] Specifically, the image display control unit 11 may perform the following processes: (a) Input determination process The image display control unit 11 recognizes an input received via the input device 14 based on the type of the input device 14 that received the input, a change in the input signal, and the like. (b) Display processing The image display control unit 11 performs processing for the display area of ​​the screen displayed on the display device 13 or the like, and processing for image display. This display processing includes processing for displaying a 3D anatomical model on the display device 13 or the like, and processing for moving, enlarging / reducing, rotating, etc. the displayed image. In addition, processing for combining the 3D anatomical model with an image of the affected area captured by an endoscope or the like, processing for changing the imaging position of the 3D anatomical model, etc. However, it is not essential to actually display the image on the display device 13, and it may be possible to simply output image data that has been subjected to the above-mentioned predetermined processing. (c) Automatic tuning process As described below, the image display control unit 11 performs processes such as acquiring optical characteristic data of a medical optical imaging device such as an endoscope, and automatically setting image display parameters (field of view, depth, magnification, etc.) based on the acquired data.

[0029] [Example of operation] I. Automatic Tuning Using Test Charts In one embodiment of the present invention, the medical image processing device 10 may be systemized in a configuration as exemplified in Fig. 3A and may have a function of synthesizing and displaying an endoscopic image and a 3D anatomical model. Fig. 3A shows an example in which an external display 13A is connected to the medical image processing device 10 and a medical image is displayed on the external display 13A, but other configurations are also possible in which the image is displayed on a display device integrated into the medical image processing device 10, or on two or more external displays 13A and 13B as shown in Fig. 3B. In the configuration of Fig. 3B, an L-side image for stereoscopic viewing may be displayed on one display 13A, and an R-side image may be displayed on the other display 13B.

[0030] The endoscope device 130 is not particularly limited, but in this example, it includes an endoscope 131 and a controller 132 that generates an endoscopic image based on a detection signal from the endoscope. Specifically, it may be a stereoscopic endoscope device that can generate an image that allows an inspection area to be observed three-dimensionally. The endoscope 131 has an illumination optical system that emits illumination light toward the inspection area and an imaging optical system that images the inspection area. Specifically, an imaging element such as a CCD image sensor or a CMOS image sensor is provided at the tip of the endoscope 131 as the imaging optical system. In addition, an objective lens unit is disposed in front of the imaging element. In the endoscope 131, the depth of field and the angle of view depend on the characteristics of the objective lens unit and the like provided at the tip of the endoscope. In the stereoscopic endoscope device, the imaging optical system is composed of independent optical systems for the L side and the R side, and generates an L side image and an R side image having parallax.

[0031] The medical image processing device 10 of this embodiment is configured to be connected to an endoscope device 130, read endoscopic images from the device, and display them on the display 13A and / or the display 13B. For ease of explanation, the following will be described using an example of an image displayed on the display 13A. An example of image synthesis and superimposition using a mixer or the like, rather than synthesis and superimposition of images in the medical image processing device 10, will be described later with reference to other drawings.

[0032] Fig. 4 is an example of an image displayed on the display 13A. As shown in Fig. 4, in this embodiment, a real-time endoscopic image (image of the affected area) 201 captured by the endoscope 131 and a 3D anatomical model 211 (hereinafter also referred to as a blood vessel model 211) are composited and displayed on one screen. The image in Fig. 4 also shows a forceps 205 inserted into the abdomen of the subject.

[0033] The blood vessel model 211 is created based on tomographic image data obtained by imaging the subject in advance using an imaging device such as a CT device, an MRI device, or an angiography device. Regarding imaging, imaging may be performed by injecting a contrast agent into the subject, and contrast agent injection using a liquid injector capable of controlling a drive mechanism to perform injection according to predetermined injection conditions is preferable in that it is possible to obtain a good image by contrasting the imaging target region of the subject.

[0034] With a configuration capable of synthesizing and displaying an endoscopic image 201 and a 3D anatomical model 211 as shown in FIG. 4, the surgeon can compare, for example, an actual anatomical structure (a blood vessel as an example) in the endoscopic image 201 with a virtual anatomical structure (a blood vessel as an example) to confirm the shape and course of the structure.

[0035] Regarding the composite display of the endoscopic image and the 3D anatomical model, several display functions may be provided depending on the purpose, and it is not necessarily limited to the mode in which the endoscopic image and the 3D anatomical model are aligned and displayed so as to be superimposed. As will be described in detail later, the 3D anatomical model may be imaged so as to appear in front of the image of the affected area in the endoscopic image.

[0036] Incidentally, in order to properly superimpose or synthesize the 3D anatomical model by matching its shape, size, etc. to the endoscopic image (image of the affected area), it is preferable that the 3D anatomical model is configured so that it can be deformed and fine-tuned in accordance with the characteristics of the endoscopic image of the endoscopic device 130 being used.

[0037] Therefore, in this embodiment, the following automatic tuning function is provided so as to be able to respond to the characteristics (optical characteristics) of the endoscope device 130 used. The following will be described with reference to FIGS.

[0038] (Automatic tuning function) First, in step S1, the test chart 136 is placed so as to face the tip of the endoscope 131, and in step S2, the test chart image is generated by capturing an image of the test chart 136 with the endoscope 131. The test chart 136 is not limited to a specific one, but may have a predetermined geometric pattern, such as a lattice pattern, a dot pattern, a circular pattern, or an appropriate combination thereof.

[0039] Then, in step S3, the endoscopic image acquired above is analyzed (for example, an analysis based on the degree of distortion, etc.) to acquire characteristic data of the endoscope 131. The characteristic data may be data related to the viewing angle, depth, magnification rate, aberration (distortion aberration, etc.) of the endoscope 131, etc.

[0040] Next, in step S4, the 3D anatomical model 211 is adjusted based on the characteristic data. For example, the adjustment may be performed so that the shape and / or size of the 3D anatomical model corresponds to an anatomical structure in the affected area image displayed as an endoscopic image. The above adjustment may be performed for each of the L-side image and the R-side image. The image of the adjusted 3D anatomical model may be automatically aligned to a certain anatomical structure in the endoscopic image, but this function is not essential.

[0041] According to the above-described configuration, the shape, size, etc. of the 3D anatomical model 211 can be adjusted according to the characteristics of the endoscope device 130 being used, thereby enabling excellent superimposition and synthesis with the endoscopic image.

[0042] The information of the medical optical imaging device to be used may be preset, and when one device to be used is selected, the characteristic data associated with the device may be automatically selected. The device may be selected by an operator, or the device to be used may be recognized by a predetermined sensor or equipment or by predetermined data communication, and the characteristic data may be automatically selected.

[0043] The above describes an example in which a medical image processing device is used together with an endoscopic device, but in addition to endoscopic devices, the device may also be used in connection with an imaging system in a procedure such as a brain surgery craniotomy in which the surgical field is photographed with a specified imaging system, and the image of the affected area is displayed on a monitor and the surgery is performed while viewing the image.

[0044] II. Movement of 3D anatomical models In the case of superimposing an endoscopic image and a 3D anatomical model, basically, it is sufficient that the position of the endoscopic image (image of the affected area) and the position of the image of the 3D anatomical model are consistent as shown in Fig. 7(a). On the other hand, in the medical image processing device 10 of this embodiment, the position of the image of the 3D anatomical model can be changed in the front-back direction (front-rear direction), and Fig. 7(b) shows a schematic diagram of a state in which the imaging position of the 3D anatomical model is moved forward. In the state of Fig. 7(b), the surgeon can visually recognize the image with a sensation that the 3D anatomical model is floating forward from the image of the affected area of ​​the subject.

[0045] Such a configuration has the advantage that, for example, in cases where one wishes to check certain details of a 3D anatomical model, the 3D anatomical model can be rotated and / or scaled at a position in front of the image of the affected area to check the details.

[0046] The image of the 3D anatomical model may be configured to be moved to the rear side of the endoscopic image. The imaging position of the 3D anatomical model is not necessarily limited to the front-rear direction (front-rear direction), and may be movable in any direction in three-dimensional space.

[0047] The medical image processing device 10 may be configured to be able to switch between a plurality of modes, such as an overlay mode in which a 3D anatomical model and an affected part image are superimposed, a 3D observation mode in which a 3D anatomical model is displayed in front of an affected part image, a 3D reference mode in which a 3D anatomical model is displayed as a small thumbnail on the screen and an affected part image is displayed mainly, for example. This switching may be performed by operating a predetermined image button on the screen, by voice input, or by detecting a gesture by the operator.

[0048] In this embodiment, the imaging position of the 3D anatomical model can be freely adjusted as described above, but the following problem may arise in relation to the position of the rotation axis of the 3D anatomical model. This will be explained with reference to the schematic diagram of FIG.

[0049] The 3D anatomical model 211 generated by the medical image processing device 10 can move the imaging position in a three-dimensional virtual space as shown in Fig. 8. Here, for example, consider moving the 3D anatomical model 211 from the position indicated by reference numeral 211 to the position indicated by reference numeral 211'. The 3D anatomical model 211 usually has a predetermined rotation axis set at a certain fixed position (see reference numeral Ax1), and when the 3D anatomical model 211 is rotated, the volume is rotated around this rotation axis.

[0050] On the other hand, even if the imaging position of the 3D anatomical model 211 is moved to the position of reference numeral 211' in Fig. 8, the axis of rotation usually remains at the original position. Therefore, when attempting to rotate the 3D anatomical model 211' at the moved position, the volume moves significantly around the initial setting axis of rotation, making such a display mode difficult to operate and also causing the 3D anatomical model 211' to go out of the visible range of the screen.

[0051] Therefore, in this embodiment, when the imaging position of the 3D anatomical model 211 is changed, the position of the rotation axis is changed accordingly. Specifically, in the example of Fig. 8, when the image of the 3D anatomical model is moved from the position of reference numeral 211 to the position of reference numeral 211', the rotation axis is changed from Ax1 to Ax2 accordingly. Since the rotation axis Ax2 is located near the 3D anatomical model 211', even when the 3D anatomical model 211' is rotated, the 3D anatomical model rotates around the rotation axis Ax2, which results in easy operation and easy confirmation of the 3D anatomical model after rotation.

[0052] In addition, the display magnification may be automatically adjusted when the imaging position is changed so that the display size of the 3D anatomical model does not change with the change of the imaging position. In addition, regarding the movement of the imaging position of the 3D anatomical model in Fig. 8 (reference numeral 211 → reference numeral 211'), the direction of the movement is not limited to the front-back direction (front-rear direction) and may be any direction in three-dimensional space, for example, the movement may be from the position of reference numeral 211 to reference numeral 211''. In this case, the rotation axis Ax3 is also configured to follow the 3D anatomical model 211'' (in other words, the relative positional relationship between the 3D anatomical model and the rotation axis is configured to not change and to be kept constant).

[0053] As described above, the user interface for manipulating the image is not particularly limited, but a specific example may be a GUI having a graphical slider as shown in FIG. 9. In this example, a slider display section 611 for moving the imaging position of the 3D anatomical model in the front-rear direction, a slider display section 612 for adjusting the parallax of the 3D anatomical model, and a slider display section 613 for adjusting the display viewing angle of the 3D anatomical model are displayed. All of these are not essential to the present invention, and only one or two may be displayed. Note that all or a part of the slider display sections 611 to 613 may be protected as designs of articles or images, and that this application also discloses the subject of protection under the Design Law.

[0054] The slider display section 611 includes a pointer 611a that slides along a straight line, and the imaging position of the 3D anatomical model is changed by the operator moving the position of this pointer 611a. Similarly, the slide display section 612 includes a pointer 612a that slides along a straight line, and the parallax of the 3D anatomical model is changed by the operator moving the position of this pointer 612a.

[0055] Similarly, the slider display section 613 includes a pointer 613a that slides along a straight line. Furthermore, in this example, an equilateral part 613b is displayed in such a manner that both ends of the pointer 613a are connected to the ends of the pointer movable range, and the display as a whole is in the shape of an approximately isosceles triangle. By moving the pointer 613a, the display view angle of the 3D anatomical model is also changed, and at that time, the shape of the approximately isosceles triangle changes in response to the position of the pointer 613a, so the operator can intuitively and visually grasp the change in the display view angle.

[0056] Although the embodiment of the present invention has been described above with reference to the drawings, the present invention is not limited to the above specific embodiments and can be modified as appropriate without departing from the spirit of the present invention. Furthermore, the disclosed technical features can be used in appropriate combinations without departing from the spirit of the present invention. Furthermore, this specification also discloses content disclosed as an apparatus invention, for example, expressed in the categories of a method invention, a computer program invention, and a program medium.

[0057] (Other embodiments) A medical image processing device according to an embodiment of the present invention may be used in a system as shown in FIG. 10. This medical image processing device 10A outputs data (data of a 3D anatomical model, etc.) in an image format corresponding to the image format output from an endoscope device 130A. The output data from the endoscope device 130A and the output data from the medical image processing device 10A are input to a mixer 121, where they are mixed and displayed on a display device 122. The mixer 121 may be any device capable of mixing a plurality of image data, and may be, for example, a conventionally known device. The display device 122 may be one or a plurality of displays, etc. The configuration and functions of the medical image processing device not mentioned here may be the same as those of the above-mentioned embodiment.

[0058] The image format may be any format, such as one that outputs data of left and right channels (LR), one that outputs data without distinction between left and right, etc. The medical image processing device 10A may be one that automatically detects the image format of the endoscope device 130A, or one that provides a user interface for the operator to select the format.

[0059] Even in such a system configuration, if the medical image processing device 10A has a function of changing the rotation axis position of the 3D anatomical model (Ax1 → Ax2), the same effect as the above-mentioned embodiment can be obtained. Also, if the medical image processing device 10A has a function of adjusting the display mode based on the characteristic data obtained by analyzing the test chart image, the same effect as the above-mentioned embodiment can be obtained.

[0060] (Appendix 1) This specification discloses the following inventions. The following inventions correspond to the configuration of FIG. 10, for example: A1. A medical image processing device (10A) having an image display control unit, The image display control unit is A process of reading a 3D anatomical model (211) that represents the anatomical structure of the subject in three dimensions; When the imaging position of the 3D anatomical model (for example, the position in the depth direction in the three-dimensional space) is changed, a process is performed to change the position of the rotation axis of the 3D anatomical model (Ax1 → Ax2) in accordance with the changed position (211'). Medical imaging equipment.

[0061] A2. The medical image processing device described above, wherein the image display control unit has a graphical user interface for accepting operations related to the display of the 3D anatomical model, and the graphical user interface includes a graphical slider (613) for moving the imaging position of the 3D anatomical model.

[0062] A3. The 3D anatomical model is displayed by combining it with an image (201) of the affected area of ​​the subject captured by a medical optical imaging device (130); The medical image processing apparatus according to any one of the above, wherein the image of the affected area is a stereoscopic image made up of a plurality of images having parallax.

[0063] A4. The medical image processing apparatus described above, wherein the medical optical imaging device is an endoscope apparatus.

[0064] A5. The medical image processing apparatus described above, wherein the medical optical imaging device is an imaging device that images an examination area of ​​a subject during a surgical procedure.

[0065] A process of reading a 3D anatomical model (211) that represents the anatomical structure of the subject in three dimensions; When the imaging position of the 3D anatomical model (for example, the position in the depth direction in the three-dimensional space) is changed, a process of changing the position of the rotation axis of the 3D anatomical model (Ax1 → Ax2) in accordance with the changed position (211'); A medical image processing method comprising: a step of: detecting a first image of a subject to be imaged;

[0066] A6. A medical image processing device (10A) having an image display control unit, The image display control unit (11) The apparatus is configured to read and output a 3D anatomical model (211) that represents the anatomical structure of the subject in a three-dimensional manner, and further configured to: e1: A process of capturing an image of a test chart using a medical optical imaging device to obtain an image of the test chart; e2: A process of analyzing the test chart image to obtain optical characteristic data of the medical optical imaging device; e3: adjusting the aspect of the 3D anatomical model (211) based on the optical characteristic data; A medical image processing device that performs the above steps.

[0067] A medical image processing method for reading and outputting a 3D anatomical model (211) that three-dimensionally represents an anatomical structure of a subject, the method further comprising: e1: A process of capturing an image of a test chart using a medical optical imaging device to obtain an image of the test chart; e2: A process of analyzing the test chart image to obtain optical characteristic data of the medical optical imaging device; e3: adjusting the aspect of the 3D anatomical model (211) based on the optical characteristic data; A medical image processing method comprising: a step of: detecting a first image of a subject to be imaged;

[0068] (Appendix 2) This specification also discloses the following invention: B1. A medical image processing device (10) having an image display control unit (11), The image display control unit (11) a) A process of reading an image (201) of an affected part of a subject captured by a medical optical imaging device (130); b: A process of reading a 3D anatomical model (211) that three-dimensionally represents the anatomical structures of the subject; c) A process of synthesizing and displaying the affected area image (201) and the 3D anatomical model; and further configured to: d: When the imaging position of the 3D anatomical model is changed, a process of changing the position of the rotation axis (Ax) of the 3D anatomical model according to the changed position is performed. Medical imaging equipment. Here, "also changing the position of the rotation axis (Ax)" specifically means that the position of the rotation axis is changed so that it follows the position of the 3D anatomical model, in other words, that the relative positional relationship between the rotation axis and the 3D anatomical model does not change.

[0069] One or more processing units (computers), a: A process of reading an image of an affected area of ​​a subject captured by a medical optical imaging device; b: A process of reading a 3D anatomical model that shows the subject's anatomical structures in three dimensions; c) A process of synthesizing and displaying the image of the affected area and the 3D anatomical model; d: when the imaging position of the 3D anatomical model is changed, a process of changing the position of the rotation axis of the 3D anatomical model according to the changed position; A medical image processing program that performs the following:

[0070] a: A process of reading an image of an affected area of ​​a subject captured by a medical optical imaging device; b: A process of reading a 3D anatomical model that shows the subject's anatomical structures in three dimensions; c) A process of synthesizing and displaying the image of the affected area and the 3D anatomical model; d: when the imaging position of the 3D anatomical model is changed, a process of changing the position of the rotation axis of the 3D anatomical model according to the changed position; A medical image processing method for performing the above.

[0071] B2. A medical image processing device (10) having an image display control unit (11), The image display control unit (11) a) A process of reading an image (201) of an affected part of a subject captured by a medical optical imaging device (130); b: A process of reading a 3D anatomical model (211) that three-dimensionally represents the anatomical structures of the subject; c) A process of synthesizing and displaying the affected area image (201) and the 3D anatomical model; and further configured to: e1: A process of capturing an image of a test chart using the medical optical imaging device to obtain an image of the test chart; e2: A process of analyzing the test chart image to obtain optical characteristic data of the medical optical imaging device; e3: A process of adjusting the display mode of the 3D anatomical model (211) based on the optical characteristic data; A medical image processing device that performs the above steps.

[0072] One or more processing units (computers), a) A process of reading an image (201) of an affected part of a subject captured by a medical optical imaging device (130); b: A process of reading a 3D anatomical model (211) that three-dimensionally represents the anatomical structures of the subject; c) A process of synthesizing and displaying the affected area image (201) and the 3D anatomical model; and further configured to: e1: A process of capturing an image of a test chart using the medical optical imaging device to obtain an image of the test chart; e2: A process of analyzing the test chart image to obtain optical characteristic data of the medical optical imaging device; e3: A process of adjusting the display mode of the 3D anatomical model (211) based on the optical characteristic data; A medical image processing program that performs the following:

[0073] a) A process of reading an image (201) of an affected part of a subject captured by a medical optical imaging device (130); b: A process of reading a 3D anatomical model (211) that three-dimensionally represents the anatomical structures of the subject; c) A process of synthesizing and displaying the affected area image (201) and the 3D anatomical model; and further configured to: e1: A process of capturing an image of a test chart using the medical optical imaging device to obtain an image of the test chart; e2: A process of analyzing the test chart image to obtain optical characteristic data of the medical optical imaging device; e3: A process of adjusting the display mode of the 3D anatomical model (211) based on the optical characteristic data; A medical image processing method for performing the above. [Explanation of symbols]

[0074] 1 Medical image processing system 10, 10A Medical image processing device 11 Image display control unit 12 Data storage section 13 Display Devices 13A, 13B Display 14 Input Devices 15 Data Input / Output Interface 18a~18c Objects 21 Medical information management device 22 Chemical injection device 23 Imaging device 121 Mixer 122 Display device 130 Endoscopic device 131 Endoscopy 132 Controller 136 Test Chart 201 Endoscopic images (images of affected areas) 205 Forceps 211, 211′ 3D anatomical model (vascular model) 611~613 Slider display 611a~613a Pointer 613b Equal sides

Claims

1. A medical image processing apparatus including an image display control unit, The image display control unit A process of reading a 3D anatomical model that three-dimensionally represents the anatomical structure of the subject; A process of imaging the 3D anatomical model in a stereoscopic virtual space so that the model can be freely moved; a process of combining the 3D anatomical model with an image of the affected area of ​​the subject, which is a stereoscopic image consisting of a plurality of images with parallax captured by a medical optical imaging device, and displaying the combined image; configured to: Medical imaging equipment.

2. A medical image processing device as described in Claim 1, wherein the image display control unit displays the imaging position of the 3D anatomical model so that it coincides with the position of the image of the affected area.

3. A medical image processing device as described in claim 2, configured to be able to change the imaging position of the 3D anatomical model in the forward and / or backward directions.

4. A medical image processing device as described in claim 3, having multiple display modes according to the imaging position of the 3D anatomical model, and the multiple display modes are configured to be switchable by an operator.

5. A medical image processing device described in any one of claims 1 to 4, wherein the image display control unit further performs processing to display a graphical user interface for accepting operations related to the display of the 3D anatomical model.

6. The medical image processing device described in Claim 5, wherein the graphical user interface includes a graphical slider for moving the imaging position for moving the imaging position of the 3D anatomical model.

7. A medical image processing device as described in Claim 6, wherein the graphical slider for moving the imaging position includes a pointer that can slide along a line.

8. A medical image processing device as described in any one of claims 5 to 7, wherein the graphical user interface includes a parallax adjustment graphical slider for adjusting the parallax of the 3D anatomical model.

9. A medical image processing device as described in Claim 8, wherein the parallax adjustment graphical slider includes a pointer that can be slid along a line.

10. A medical image processing method, comprising: A step of reading into a computer a 3D anatomical model that represents the anatomical structure of a subject in three dimensions; A step in which a computer images the 3D anatomical model so that the 3D anatomical model can be arbitrarily moved within a stereoscopic virtual space; a step in which a computer combines the 3D anatomical model with an image of the affected area of ​​the subject, the image being a stereoscopic image made up of a plurality of images with parallax, captured by a medical optical imaging device, and displays the combined image; A medical image processing method comprising:

11. A computer program for medical image processing, comprising: On the computer, reading a 3D anatomical model that represents the anatomical structure of the subject in three dimensions; A step of imaging the 3D anatomical model so that the 3D anatomical model can be arbitrarily moved within a stereoscopic virtual space; a step of combining the 3D anatomical model with an image of the affected area of ​​the subject, which is a stereoscopic image consisting of a plurality of images with parallax captured by a medical optical imaging device, and displaying the combined image; A computer program that executes

Images