Medical image processing system and medical image processing method

Abstract

The invention discloses a medical image processing system and method. An MR image captured in advance is aligned and superimposed with good accuracy on an X-ray image captured during an operation, and tubular tissue included in the MR image is displayed on the X-ray image. An MR image and a CT image, which are captured in advance, are aligned to obtain a correspondence relationship between coordinates. Next, the CT image and the X-ray image are aligned to obtain a correspondence between coordinates. The X-ray image and the MR image are aligned using the correspondence between the coordinates of the MR image and the CT image and the correspondence between the coordinates of the CT image and the X-ray image. Feature points indicating the position of a predetermined site of a subject in each of an MR image and a CT image are extracted. Feature points representing the positions of the corresponding parts in the X-ray image are extracted. The X-ray image and the MR image are further aligned using the coordinates of the feature points of the MR image and / or the CT image and the coordinates of the feature points of the X-ray image, and then superimposed, and displayed as a guide image.

Description

Technical Field

[0001] The present invention relates to a system for processing images taken by an X-ray imaging device. Background Art

[0002] Patent Document 1 discloses a technique of aligning and superimposing a projection image obtained by projecting a three-dimensional X-ray CT image on an X-ray image captured by an X-ray imaging device, and displaying the superimposed image.

[0003] On the other hand, Patent Document 2 discloses a technique for displaying a three-dimensional X-ray CT image (vascular image data) and a two-dimensional X-ray fluoroscopic image in an easily observable manner. In this technique, a focus point of the X-ray fluoroscopic image is extracted, a corresponding point on the three-dimensional vascular image data corresponding to the focus point is estimated, and a projection image including the corresponding point of the three-dimensional vascular image data is generated and displayed. At this time, a fluoroscopic image for the left eye and a fluoroscopic image for the right eye of the observer are generated so that the observer can observe in three dimensions.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2016-041166

[0005] Patent Document 2: Japanese Patent Application Publication No. 2013-059609

[0006] Although the three-dimensional X-ray CT images taken in advance can depict the bone structure of the subject's spine and other bones with high clarity, they cannot depict tubular tissues such as blood vessels and pancreatic ducts. If contrast agents are used, X-ray images can depict blood vessels and pancreatic ducts, but if contrast agents are not used, they cannot be depicted.

[0007] In recent years, it has been desired to grasp tubular tissues such as blood vessels and pancreatic ducts during surgery using pre-contrast X-ray images.

[0008] On the other hand, MRI images captured by an MRI (Magnetic Resonance Imaging) apparatus can depict tubular tissues depending on the imaging method. For example, MR cholangiopancreatography (MRCP) can depict bile ducts and pancreatic ducts using MRI without using contrast agents.

[0009] However, although MR images can depict protons in tissues, X-ray images can depict the high and low X-ray absorption rates. Therefore, the objects depicted in the two images are different, and it is not easy to align MR images and X-ray images with good accuracy. Summary of the invention

[0010] An object of the present invention is to accurately align and overlay an MR image captured in advance on an X-ray image captured during surgery, and to display a tubular tissue included in the MR image on the X-ray image.

[0011] In order to achieve the above-mentioned purpose, the medical image processing system of the present invention comprises a preliminary image acquisition unit, an intraoperative image acquisition unit, an MR-CT image alignment unit, a CT-X-ray image alignment unit, a preliminary image feature point extraction unit, an intraoperative image feature point extraction unit, an MR-X-ray image alignment unit and an image overlapping unit. The preliminary image acquisition unit reads an MR image obtained by previously photographing a subject using a magnetic resonance imaging device and a CT image obtained by previously photographing a subject using an X-ray CT device. The intraoperative image acquisition unit reads an X-ray image obtained by photographing a subject during surgery. The MR-CT image alignment unit aligns the MR image and the CT image to obtain the corresponding relationship between the coordinates of the MR image and the CT image. The CT-X-ray image alignment unit aligns the CT image and the X-ray image to obtain the corresponding relationship between the coordinates of the CT image and the X-ray image. The preliminary image feature point extraction unit extracts feature points representing the position of a predetermined part of the subject in the MR image and the CT image, respectively. The intraoperative image feature point extraction unit extracts feature points representing the position of a part of the subject in the X-ray image. The MR-X-ray image alignment unit aligns the X-ray image and the MR image. The image overlapping unit overlaps the X-ray image aligned by the MR-X-ray image alignment unit and the MR image to generate an overlapping image, and displays the overlapping image as a guide image. After the MR-X-ray image alignment unit aligns the X-ray image and the MR image using the corresponding relationship between the coordinates of the MR image and the CT image and the corresponding relationship between the coordinates of the CT image and the X-ray image, the X-ray image and the MR image are further aligned at least in the parallel translation direction using the coordinates of at least one of the feature points of the MR image and the feature points of the CT image and the coordinates of the feature points of the X-ray image.

[0012] Effects of the Invention

[0013] According to the present invention, an MR image captured in advance can be accurately aligned and superimposed on an X-ray image captured during surgery, so that a tubular tissue included in the MR image can be displayed on the X-ray image. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a functional block diagram showing the configuration of the medical image processing system 100 according to the first embodiment of the present invention.

[0015] Figure 2 This is a flowchart showing the operation of the medical image processing system 100 according to the first embodiment.

[0016] Figure 3 (a) to (f) are diagrams showing an overview of image processing by the medical image processing system 100 according to the first embodiment.

[0017] Figure 4 This is a functional block diagram showing the configuration of a medical image processing system 200 according to Embodiment 2 of the present invention.

[0018] Figure 5 This is a flowchart showing the operation of the medical image processing system 200 according to the second embodiment.

[0019] Figure 6 (a) to (f) are diagrams showing an overview of feature point existence regions and alignment in the medical image processing system 200 according to the second embodiment.

[0020] Figure 7 This is a diagram showing the relationship between the imaging (respiration) conditions and the shape of the existence region of the medical image processing system 200 according to the second embodiment.

[0021] Figure 8 (a) to (c) are diagrams showing a part of image processing by the medical image processing system 200 according to the second embodiment.

[0022] Explanation of symbols

[0023] 1-subject, 2-X-ray imaging device, 3-medical image server, 10-intraoperative two-dimensional image acquisition unit, 20-preliminary three-dimensional image acquisition unit, 30-input unit, 40-display unit, 50-MR-CT image alignment unit, 51-preliminary image feature point extraction unit, 52-projection calculation unit, 53-intraoperative image feature point extraction unit, 54-CT-X-ray image alignment unit, 55-MR-X-ray image alignment unit, 56-image overlapping unit, 57-feature point existence area calculation unit, 1 00-medical image processing system, 150-image processing unit, 200-medical image processing system, 201-CT image, 202-MR image, 203-X-ray image, 203a-morphology, 211-CT projection image, 211a-morphology, 212-MR projection image, 212a-tubular tissue, 220-overlapping image, 230-overlapping image, 250-existence area, 251-feature point, 252-feature point, 253-feature point, 301-treatment instrument. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present invention will be described using the drawings.

[0025] Implementation Method 1

[0026] The configuration of the medical image processing system 100 according to the first embodiment will be described.

[0027] The medical image processing system 100 generates a guide image by superimposing a previously captured MR image on an X-ray image of a subject undergoing surgery without the injection of a contrast agent. Thus, in the guide image, tubular structures such as blood vessels and pancreatic ducts depicted on the MR image are superimposed on the X-ray image, and the surgeon can grasp the tubular structure where the treatment instrument is to be inserted on the X-ray image.

[0028] Figure 1 is a diagram showing the configuration of the medical image processing system 100 . Figure 2 : is a flowchart showing the operation of the medical image processing system 100 . Figure 3 (a)~ Figure 3 (f) is a diagram showing an overview of image processing.

[0029] like Figure 1 Thus, the medical image processing system 100 is configured to include the intraoperative two-dimensional image acquisition unit 10 , the preliminary three-dimensional image acquisition unit 20 , the image processing unit 150 , the input unit 30 , and the display unit 40 .

[0030] The X-ray imaging device 2 is connected to the intraoperative two-dimensional image acquisition unit 10 . The medical image server 3 is connected to the preoperative three-dimensional image acquisition unit 20 .

[0031] The medical image server 3 stores an MR image 202 obtained by previously imaging the subject 1 using an MRI (magnetic resonance imaging) device and a CT image 201 obtained by previously imaging the subject 1 using an X-ray CT device. Figure 3 (a)). Here, both the MR image 202 and the CT image 201 are three-dimensional images.

[0032] The image processing unit 150 includes an MR-CT image alignment unit 50 , a preliminary image feature point extraction unit 51 , a projection calculation unit 52 , an intraoperative image feature point extraction unit 53 , a CT-X-ray image alignment unit 54 , the preliminary image feature point extraction unit 51 , an MR-X-ray image alignment unit 55 and an image overlapping unit 56 .

[0033] In advance, the three-dimensional image acquisition unit 20 reads the MR image 202 and the CT image 201 of the subject 1 from the medical image server 3 .

[0034] As is known, the X-ray CT image 201 can clearly depict the morphology (shape) 211a of the skeleton or contour of the subject 1, but it is difficult to depict tubular structures such as blood vessels and pancreatic ducts in the subject 1 without the injection of contrast agent. On the other hand, the MR image 202 has the following characteristics: even without the contrast agent, tubular tissues 212a such as the bile duct or pancreatic duct of the subject can be depicted, but it is difficult to depict the morphology (shape) of the skeleton or contour.

[0035] On the other hand, the intraoperative two-dimensional image acquisition unit 10 reads an X-ray image 203 ( Figure 3 (d)) The read X-ray image 203 may be one frame of a fluoroscopic image (dynamic image) taken during surgery, or may be an X-ray image (still image) taken during surgery.

[0036] It is known that the X-ray image 203 can generally clearly depict the morphology (shape) 203a of the skeleton or contour of the subject 1, similarly to the CT image 201. Furthermore, since it is during the operation, when a treatment instrument 301 such as an endoscope or a vascular catheter is inserted into the subject 1, the image of the treatment instrument can also be depicted on the X-ray image 203. However, in the X-ray image 203, it is difficult to depict tubular structures such as the bile duct and the pancreatic duct in the subject 1 without the injection of a contrast agent.

[0037] The MR-CT image alignment unit 50 aligns the MR image 202 and the CT image 201 acquired by the three-dimensional image acquisition unit 20 in advance, and obtains the corresponding relationship between the coordinates of the MR image 202 and the CT image 201. Here, the MR image 202 and the CT image 201 are three-dimensional images, so the MR-CT image alignment unit 50 aligns the MR image 202 and the CT image 201 in three-dimensional space. As the alignment method, a well-known method is used. For example, a conventional alignment method using pixel values, an alignment method using feature points as landmarks, or an alignment method using mutual information is used.

[0038] The image feature point extraction unit 51 extracts feature points 251 and 252 (hereinafter referred to as feature points) indicating the positions of predetermined parts of the subject 1 in the CT image 201 and the MR image 202, respectively. Figure 3 (a), (b)). For example, the "duodenal papilla" is used as a feature point. For example, the pre-image feature point extraction unit 51 receives the designation of positions in the three-dimensional CT image 201 and the three-dimensional MR image 202 from the user using the pointer of the input unit 30, and uses the received positions as feature points 251, 252. In addition, the pre-image feature point extraction unit 51 can also automatically extract the feature points 251, 252 by performing image processing on the CT image 201 and the MR image 202.

[0039] The projection calculation unit 52 projects the three-dimensional MR image 202 and the CT image 201 in a predetermined direction to generate a two-dimensional MR projection image 212 and a CT projection image 211. The projection direction is set to the direction of the optical axis of the X-ray emitted by the X-ray imaging device 2 to the subject 1. As a projection method, a ray summation (RaySum) method or a maximum projection (MIP) method, etc., which uses the sum of pixel values ​​arranged along the ray as the pixel value of the projection image, is used.

[0040] The MR-CT image alignment unit 50 can also generate an overlapped image 220 of the projected two-dimensional MR projection image 212 and the CT projection image 211, and display it together with a guide image (overlapped image 230) and the like described later. Figure 3 (b)).

[0041] On this overlapping image, the CT projection image 211 is used to depict the shape 211a of the bones and the like of the subject 1, and the MR projection image 212 is used to depict the tubular tissue 212a of the subject 1, thereby simulating an X-ray image taken by the X-ray imaging device 2 while contrast agent is used to contrast the tubular tissue during surgery.

[0042] On the other hand, the intraoperative image feature point extraction unit 53 extracts feature points 253 ( Figure 3 (d)). For example, on the X-ray image 203, the user places the tip of the treatment instrument 301 at a predetermined position (e.g., the nipple), and the intraoperative image feature point extraction unit 53 extracts the insertion position (e.g., the tip) of the treatment instrument 301 through image processing, and uses the extracted position as the feature point 253. In addition, the position of the feature point is the same position as the feature point extracted by the prior image feature point extraction unit 51 (e.g., "duodenal nipple"). In addition, the intraoperative image feature point extraction unit 53 can receive the designation of a position on the X-ray image 203 from the user using the indicator of the input unit 30, etc., in the same way as the prior image feature point extraction unit 51, and use the position as the feature point 253.

[0043] The CT-X-ray image alignment unit 54 aligns the projected two-dimensional CT projection image 211 and the X-ray image 203 to obtain the corresponding relationship (parameters) between the coordinates of the CT projection image 211 and the X-ray image 203 ( Figure 3(c), (d)). In this alignment, the portion of the CT projection image 211 whose contrast is higher than a predetermined value (the shape 211a of the skeleton or body contour, etc.) and the portion of the X-ray image 203 whose contrast is higher than a predetermined value (the shape 203a of the skeleton or body contour, etc.) are aligned in the parallel translation direction and the rotation direction, and the corresponding relationship (parameter) of the coordinates in the parallel translation direction and the rotation direction is calculated.

[0044] The MR-X-ray image alignment unit 55 aligns the MR projection image 212 and the X-ray image 203 using the parameters of the parallel translation direction and the rotation direction obtained by the CT-X-ray image alignment unit 54 .

[0045] Furthermore, the MR-X-ray image alignment unit 55 aligns the X-ray image 203 and the MR projection image 212 at least in the parallel translation direction so that the feature point 253 of the X-ray image 203 and the feature point 252 of the MR projection image 212 are consistent with each other or have a predetermined positional relationship. Figure 3 (e), (d)).

[0046] The MR projection image 212 is aligned with the CT projection image 211 in the three-dimensional space before projection by the MR-CT image alignment unit 50, and the CT projection image 211 is aligned with the X-ray image 203 in the parallel translation direction and the rotation direction by the CT-X-ray image alignment unit 54. Therefore, the MR projection image 212 and the X-ray image 203 are also aligned in the parallel translation direction and the rotation direction. The MR-X-ray image alignment unit 55 uses the feature points 252 and 253 to align the MR projection image 212 and the X-ray image 203 in the parallel translation direction, thereby further accurately aligning the two.

[0047] In addition, when the MR projection image 212 and the X-ray image 203 are aligned using the feature points 252 and 253 of both, the MR-X-ray image alignment unit 55 can display the feature point 251 of the CT projection image 211 on the MR projection image 212, and use it together with the feature point 252 or instead of the feature point 252 for alignment. The MR projection image 212 and the CT projection image 211 have been aligned, so the positional relationship between the feature points 252 and the feature points 252 is clear, and the feature point 251 of the CT projection image 211 can be displayed on the MR projection image 212.

[0048] The image superimposing unit 56 superimposes the X-ray image 203 aligned by the MR-X-ray image alignment unit 55 and the MR projection image 212 to generate a superimposed image 230, and displays the superimposed image 230 on the display unit 40 as a guide image ( Figure 3 (f)).

[0049] Thus, the operating doctor can confirm that the tubular tissue 212a such as the bile duct or pancreatic duct, which is not depicted on the pre-contrast X-ray image 203 during surgery, is aligned with the X-ray image 203 by observing the superimposed image 230 on the display unit 40. Therefore, the shape of the tubular tissue 212a and the positional relationship between the treatment instrument (e.g., endoscope) 301 depicted on the X-ray image 203 and the tubular tissue 212a can be grasped, and even if the tubular tissue 212a is not depicted on the X-ray image 203, the operating doctor can grasp the tubular tissue 212a on the X-ray image 203 and perform surgery using the treatment instrument 301.

[0050] Next, use Figure 2 The operation of each unit of the medical image processing system 100 is described in the following process.

[0051] The image processing unit 150 includes a CPU and a memory, and the CPU executes a program stored in advance in the memory to perform the operations of each unit (50 to 56) of the image processing unit 150 by software. In addition, a part or all of the image processing unit 150 can also be implemented using a hard disk. For example, a custom IC such as an ASIC (Application Specific Integrated Circuit) or a programmable IC such as an FPGA (Field-Programmable Gate Array) can be used to perform circuit design to implement the functions of each unit.

[0052] <Step S101>

[0053] First, the preliminary three-dimensional image acquisition unit 20 acquires a three-dimensional CT image 201 obtained by previously imaging the subject 1 from the medical image server 3 .

[0054] <Step S102>

[0055] Next, the preliminary three-dimensional image acquisition unit 20 acquires a three-dimensional MR image 202 obtained by previously imaging the subject 1 from the medical image server 3 .

[0056] <Step S103>

[0057] The MR-CT image alignment unit 50 aligns the MR image 202 and the CT image 201 in a three-dimensional space. As an alignment method, for example, a known image alignment method using mutual information is used.

[0058] <Step S104>

[0059] The pre-image feature point extraction unit 51 extracts feature points 251 representing the positions of predetermined parts of the subject 1 in the CT image 201. For example, the pre-image feature point extraction unit 51 displays the three-dimensional CT image 201 on the display unit 40, prompts the user to operate the pointer of the input unit 30 or the like to point to a predetermined part (for example, the papilla of the duodenum) on the image with a cursor, and reads the position pointed by the user as the feature point 251.

[0060] <Step S105>

[0061] Next, the preliminary image feature point extraction unit 51 extracts feature points 252 indicating the positions of predetermined parts of the subject 1 in the MR image 202 , similarly to the CT image 201 .

[0062] <Step S106>

[0063] The projection calculation unit 52 projects the three-dimensional CT image 201 from which the feature points 251 are extracted in step 104 in a predetermined direction to generate a two-dimensional CT projection image 211. The projection direction is set to the direction of the optical axis of the X-ray emitted by the X-ray imaging device 2 to the subject 1. Hereinafter, the projected CT projection image 211 is also referred to as a "virtual projection image (CT)".

[0064] <Step S107>

[0065] The projection calculation unit 52 projects the three-dimensional MR image 202 whose characteristic points 252 are extracted in step 105 in a predetermined direction to generate a two-dimensional MR projection image 212. Hereinafter, the projected MR projection image 212 is also referred to as a "virtual projection image (MR)".

[0066] <Step S108>

[0067] The MR-CT image alignment unit 50 creates an overlapped image 220 of a virtual projection image (CT) (CT projection image 211) and a virtual projection image (MR) (MR projection image 212). The overlapped image 220 depicts the morphology 211a of the bones and the like of the subject 1 and tubular tissue 212a, and is an image (virtual projection image) that simulates an X-ray image taken while contrast media is used to contrast the tubular tissue during surgery.

[0068] <Step S121>

[0069] On the other hand, the intraoperative two-dimensional image acquisition unit 10 reads an X-ray image 203 obtained by imaging the subject 1 by the X-ray imaging device 2 during the operation. Hereinafter, the X-ray image 203 is also referred to as an "intraoperative image."

[0070] <Step S122>

[0071] The intraoperative image feature point extraction unit 53 extracts feature points 253 ( Figure 3 (d)). For example, when the tip of the treatment instrument 301 is placed at a predetermined position (e.g., the nipple) by the user, the intraoperative image feature point extraction unit 53 extracts the tip of the treatment instrument 301 in the X-ray image 203 through image processing, and uses the extracted position as the feature point 253.

[0072] <Step S109>

[0073] The CT-X-ray image alignment unit 54 aligns the virtual projection image (CT) (CT projection image 211) and the intraoperative image (X-ray image 203) in the parallel translation direction and the rotation direction so that the high-contrast portion of the CT projection image 211 (the shape 211a of the skeleton or body contour, etc.) and the high-contrast portion of the X-ray image 203 (the shape 203a of the skeleton or body contour, etc.) are aligned, and calculates the parameters ( Figure 3 (c), (d)).

[0074] <Step S110>

[0075] The MR-X-ray image alignment unit 55 aligns the virtual projection image (MR) (MR projection image 212) and the intraoperative image (X-ray image 203) in the parallel translation direction and the rotation translation direction using the parameters obtained by the CT-X-ray image alignment unit 54 in step 109. In step 103, the MR image 202 and the CT image 201 have been aligned, so in step 109, the MR projection image 212 and the X-ray image 203 can be aligned based on the parameters obtained for the CT projection image 211.

[0076] <Step S111>

[0077] Next, the MR-X-ray image alignment unit 55 displays the feature points 251 of the CT image 201 extracted in step 104 and the feature points 251 of the MR image 202 extracted in step 105 on the virtual projection image (MR) (MR projection image 212 ).

[0078] <Step S112>

[0079] The MR-X-ray image alignment unit 55 uses one of the feature points 251 of the CT image 201 and the feature points 252 of the MR image 202 displayed on the virtual projection image (MR) (MR projection image 212) in step 111 and the feature points 253 of the intraoperative image (X-ray image 203) to further align the virtual projection image (MR) (MR projection image 212) and the intraoperative image (X-ray image 203) in the parallel movement direction in a manner that makes them consistent.

[0080] For example, the MR-X-ray image alignment unit 55 aligns the virtual projection image (MR) (MR projection image 212) with the intraoperative image (X-ray image 203) in such a manner that the feature point 251 of the CT image 201 coincides with the feature point 253 of the X-ray image 203. Furthermore, the MR-X-ray image alignment unit 55 may also align the virtual projection image (MR) (MR projection image 212) with the intraoperative image (X-ray image 203) in such a manner that the feature point 252 of the MR image 202 coincides with the feature point 253 of the X-ray image 203.

[0081] <Step S113>

[0082] The image superimposing unit 56 superimposes the intraoperative image (X-ray image 203 ) aligned by the MR-X-ray image alignment unit 55 in step 112 and the virtual projection image (MR) (MR projection image 212 ) to generate a superimposed image 230 .

[0083] <Step S114>

[0084] The image superimposing unit 56 displays the superimposed image 230 as a guide image on the display unit 40 ( Figure 3 (f)). At the same time, in step 108, a superimposed image 220 (virtual projection image) obtained by superimposing the virtual projection image (CT) (CT projection image 211) and the virtual projection image (MR) (MR projection image 212) generated by the MR-CT image alignment unit 50 is displayed on the display unit 40.

[0085] As described above, the medical image processing system 100 of the first embodiment can display the superimposed image 230 (guide image) obtained by superimposing a virtual image of the tubular tissue 212a that can be observed if contrast is performed on the intraoperative image (X-ray image 203) before contrast imaging taken by the operating surgeon using the X-ray imaging device 2 during surgery using the three-dimensional CT image 201 and MR image 202 acquired in advance. Thus, the operating surgeon can visually confirm the tubular tissue 212a that is the surgical target, etc., on the intraoperative image (X-ray image 203) before contrast is used.

[0086] On the display unit, in addition to the overlapping image 230 (guide image) of the intraoperative image (X-ray image 203), the overlapping image 220 (virtual projection image) of the virtual projection image (CT) (CT projection image 211) and the virtual projection image (MR) (MR projection image 212) generated in step 114 is also displayed, so that the surgeon can also compare the virtual projection image with the intraoperative X-ray image 203 with high precision.

[0087] On the superimposed image 220 (virtual projection image), the surgeon can understand the body contour and the morphology 211 a of the subject 1 , such as bones, from the CT projection image 211 acquired in advance, and can understand the tubular tissue 212 a from the MR projection image 212 .

[0088] Implementation Method 2

[0089] The configuration of the medical image processing system 200 according to the second embodiment will be described.

[0090] In embodiment 1, in steps 111 and 112, feature points 251 of the CT image 201 and feature points 252 of the MR image 202 are displayed on a virtual projection image (MR) (MR projection image 212), and are aligned in a manner such that the feature points 251 or 252 are consistent with feature points 253 of the intraoperative image (X-ray image 203).

[0091] Ideally, the position coordinates of feature points pointing to the same part on multiple 3D medical images should be consistent, but 3D medical images obtained by photographing the subject (human body) with different medical imaging equipment are not necessarily consistent due to body movements such as breathing. Specifically, this is because X-ray images during surgery are taken in a natural breathing state, while X-ray CT images are taken in a state of maximum inspiration and MR images are taken in a natural breathing or breath holding state.

[0092] In the second embodiment, the position coordinates of the feature points 251 of the CT image 201 and the feature points 252 of the MR image 202 and the imaging conditions such as the respiratory state when the feature points 251 and 252 are imaged are used to calculate the position or region where the feature points 253 of the X-ray image 203 have a high probability of existing, and the region with a high probability of existing is aligned with the feature points 253 of the X-ray image 203. This will be described in detail below.

[0093] Figure 4 This is a block diagram showing the configuration of a medical image processing system 200 according to the second embodiment. Figure 5 : is a flowchart showing the operation of the medical image processing system 200 . Figure 6 (a) and (c) are diagrams showing examples of characteristic points 251 and 252 and an existing area 250 .

[0094] from Figure 4 It is clear that the medical image processing system 200 has a configuration in which a feature point existing region calculating unit 57 is added to the medical image processing system 100 of the first embodiment.

[0095] The feature point existing region calculation unit 57 uses the coordinates of the feature point 252 of the MR projection image 212 and the feature point 251 of the CT projection image 211 to calculate an existing region 250 (see FIG. Figure 6 (a), (c)).

[0096] The feature point existence region calculation unit 57 further calculates the distribution of the existence probability of the feature points 253 in the existence region 250 based on the imaging conditions.

[0097] Specifically, the shape of the existence area 250 and the distribution of the existence probability of the feature point 253 can be predetermined depending on whether the combination of the respiratory state when the X-ray image 203, CT image 201 and MR image 202 are taken into account, normal body activity, or whether the heartbeat is taken into account.

[0098] Generally, the X-ray image 203 is captured under natural breathing, the CT image 201 is captured under maximum inhalation, and the MR image 202 is captured under natural breathing or breath-holding in many cases.

[0099] For example, Figure 7 As shown in the table, when normal body activities other than breathing are taken into account, the feature point 253 moves in all directions according to the body activities, so the range of movement (existence area 250) becomes a shape close to a circle, and the existence probability of the feature point 253 becomes roughly uniform within the shape.

[0100] Furthermore, when only natural breathing is considered, the feature point 253 moves back and forth in a specific direction, and the range of movement (existence area 250) becomes a shape close to an ellipse with the movement direction as the major axis, and the existence probability of the feature point 253 tends to be higher at both ends of the major axis of the ellipse.

[0101] Furthermore, the coordinates of the feature points of the image captured under natural respiration (e.g., the feature points 252 of the MR image 202), the feature points of the image captured at the maximum inhalation position (e.g., the feature points 251 of the CT image 201), and the existence area 250 of the feature points 253 of the X-ray image 203 relative to them reciprocate in a specific direction, and thus become an ellipse with the moving direction as the major axis, and the existence probability becomes low near the feature point 251 of the image at the maximum inhalation position. This is because the maximum inhalation position is not reached under natural respiration. On the other hand, the existence probability becomes high within the movement range of natural respiration.

[0102] Furthermore, the coordinates of the feature points of the image captured at the breath-holding position (for example, the feature points 252 of the breath-holding MR image 202), the coordinates of the feature points of the image captured at the maximum inhalation position (for example, the feature points 251 of the CT image 201), and the existence area 250 of the feature points 253 of the X-ray image 203 relative to them reciprocate in a specific direction, and thus become a shape close to an ellipse with the movement direction as the major axis, and the existence probability becomes higher at the feature points 252 of the image at the breath-holding position.

[0103] On the other hand, when only the jitter is considered, it becomes a reciprocating movement in a specific direction, and the range of movement (existence area 250) becomes an ellipse. In terms of the existence probability of the feature point 253, there are areas with high probability of existence at both ends of the reciprocating motion.

[0104] Taking the above characteristics into consideration, the shape of the existence area 250 and the distribution of the existence probability of the feature points 253 can be predetermined for each imaging condition.

[0105] The feature point existence area calculation unit 57 calculates the shape and existence probability of the feature point 253 using the coordinates of the feature points 251 and 252 based on the shape of the existence area 250 and the distribution of the existence probability of the feature point 253 under each predetermined imaging condition, and based on the imaging conditions of the MR projection image 212, the CT projection image 211 and the X-ray image.

[0106] The MR-X-ray image aligning unit 55 aligns the feature point 253 of the X-ray image 203 with a position having a high existence probability based on the existence region 250 calculated by the feature point existence region calculating unit 57 and the distribution of the existence probability in the existence region 250 .

[0107] The other structures ( 10 , 20 , 30 , 40 , 50 - 56 ) of the medical image processing system 200 are the same as those in the first embodiment, and thus the description thereof is omitted.

[0108] Next, use Figure 5 The operation of each part of the medical image processing system 200 is described in the following process. Figure 2 The same actions in the process are omitted and the explanation is focused on the different actions.

[0109] <Steps S101 to S110>

[0110] In the same manner as in the first embodiment, the virtual projection image (MR) (MR projection image 212 ) and the intraoperative image (X-ray image 203 ) are aligned in the parallel translation direction and the rotation translation direction.

[0111] <Step S131>

[0112] The feature point existing region calculation unit 57 calculates the shape and existence probability of the feature point 253 in the existing region 250 according to the shape of the existing region 250 and the distribution of the existence probability of the feature point 253 under each predetermined imaging condition, and according to the imaging conditions of the MR projection image 212, the CT projection image 211, and the X-ray image using the coordinates of the feature points 251 and 252 (see Figure 6 (a), (d)).

[0113] <Step S132>

[0114] The MR-X-ray image alignment unit 55 displays the feature points 251 of the CT image 201, the feature points 251 of the MR image 202, the existence region 250 calculated in step 131, and the probability distribution on the virtual projection image (MR) (MR projection image 212) (refer to Figure 8 (a)).

[0115] <Step S133>

[0116] The MR-X-ray image alignment unit 55 uses the feature points 251 of the CT image 201 and the feature points 252 of the MR image 202 displayed on the virtual projection image (MR) (MR projection image 212) and the probability distribution to align the virtual projection image (MR) (MR projection image 212) in the parallel translation direction so that the feature points 253 of the intraoperative image (X-ray image 203) coincide with the positions with high probability distribution. Figure 6 (b), (c), (e), (f), Figure 8 (b), (c)).

[0117] <Steps S113 to S114>

[0118] and Figure 2 Similarly, in steps S113 and S114 of the process, the image superimposition unit 56 superimposes the intraoperative image (X-ray image 203) aligned by the MR-X-ray image alignment unit 55 in step 133 with the virtual projection image (MR) (MR projection image 212) to generate a superimposed image 230 ( Figure 8 (c)).

[0119] The image superimposing unit 56 displays the superimposed image 230 as a guide image on the display unit 40 . At the same time, the superimposed image 220 (virtual projection image) is displayed on the display unit 40 .

[0120] According to the medical image processing system 200 of the second embodiment, in addition to the effects achieved by the medical image processing system 100 of the first embodiment, the following effects can be achieved. That is, based on the imaging conditions such as the respiratory state when the CT image 201 or MR image 202 captured in advance or the respiratory state when the X-ray image 203 during surgery is captured, the position where the feature point 253 of the X-ray image 203 has a high probability of existence can be calculated from the feature points 251 and 252 in consideration of the body movement of the subject 1. By aligning the MR projection image 212 and the X-ray image 203 so that the feature point 253 overlaps with the calculated position with a high probability of existence, the tubular tissue of the MR projection image 212 can be displayed on the X-ray image 203 with good accuracy.

Claims

1. A medical image processing system, characterized in that: have: a pre-image acquisition unit that reads an MR image of a subject obtained by pre-imaged using a magnetic resonance imaging device and a CT image of the subject obtained by pre-imaged using an X-ray CT device; an intraoperative image acquisition unit for reading an X-ray image obtained by photographing the subject during the operation; an MR-CT image alignment unit that aligns the MR image and the CT image to obtain a corresponding relationship between coordinates of the MR image and the CT image; a CT-X-ray image alignment unit that aligns the CT image and the X-ray image to obtain a corresponding relationship between coordinates of the CT image and the X-ray image; a pre-image feature point extraction unit for extracting feature points representing positions of predetermined parts of the subject in the MR image and the CT image, respectively; an intraoperative image feature point extraction unit that extracts feature points representing the position of the part of the subject in the X-ray image; An MR-X-ray image alignment unit, aligning the X-ray image with the MR image; and an image superimposing unit that superimposes the X-ray image aligned by the MR-X-ray image alignment unit and the MR image to generate a superimposed image, and displays the superimposed image as a guide image, The MR-X-ray image alignment unit uses the corresponding relationship between the coordinates of the MR image and the CT image and the corresponding relationship between the coordinates of the CT image and the X-ray image to align the X-ray image and the MR image, and then uses the coordinates of at least one of the feature points of the MR image and the feature points of the CT image and the coordinates of the feature points of the X-ray image to further align the X-ray image and the MR image at least in the parallel movement direction.

2. The medical image processing system according to claim 1, characterized in that: The MR image includes an image of a tubular tissue of the subject, and the X-ray image includes an image of a treatment instrument inserted into the subject.

3. The medical image processing system according to claim 1, characterized in that: When the MR-X-ray image alignment unit uses the coordinates of the feature points for alignment, a point representing the coordinates of at least one of the feature points of the MR image and the feature points of the CT image is displayed on the MR image, and a point representing the coordinates of the feature points of the X-ray image is displayed on the X-ray image, and alignment is performed using a plurality of the points.

4. The medical image processing system according to claim 1, characterized in that: It also has a feature point existing area calculation unit, The feature point existing region calculation unit calculates an existing region, which is a region where the feature point may exist, using the coordinates of the feature point of the MR image and the feature point of the CT image. When the MR-X-ray image alignment unit uses the coordinates of the feature points for alignment, the existence area and points representing the coordinates of at least one of the feature points of the MR image and the feature points of the CT image are displayed on the MR image, and points representing the coordinates of the feature points of the X-ray image are displayed on the X-ray image, and alignment is performed using a plurality of the points and the existence area.

5. The medical image processing system according to claim 4, characterized in that: The feature point existing region calculation unit further calculates the distribution of the existence probability of the feature points in the existing region according to the imaging conditions. The MR-X-ray image alignment unit aligns the X-ray image and the MR image based on the existence probability.

6. The medical image processing system according to claim 1, characterized in that: The CT-X-ray image alignment unit performs alignment at least in the rotation direction so that the shape of a portion of the CT image having a contrast higher than a predetermined value matches the shape of a portion of the X-ray image having a contrast higher than a predetermined value.

7. The medical image processing system according to claim 1, characterized in that: The MR image and the CT image are both three-dimensional images. The preliminary image feature point extraction unit extracts the feature points in each of the MR image and the CT image which are three-dimensional images.

8. The medical image processing system according to claim 1, characterized in that: The MR image and the CT image are both three-dimensional images. The MR-CT image registration unit projects the MR image and the CT image of three-dimensional images into two-dimensional images in a predetermined projection direction, and obtains a correspondence relationship between coordinates of the projected two-dimensional images of the MR image and the CT image.

9. The medical image processing system according to claim 1, characterized in that: The MR-CT image alignment unit generates a superimposed image of the MR image and the CT image, and displays the superimposed image together with the guide image.

10. The medical image processing system according to claim 1, characterized in that: having an input unit for receiving a designation of a position in the MR image and the CT image from a user, The advance image feature point extraction unit extracts positions designated by the input unit from the user on the MR image and the CT image as the feature points.

11. The medical image processing system according to claim 1, characterized in that: The intraoperative image feature point extraction unit detects an insertion position of a treatment instrument included in the X-ray image, and extracts the detected insertion position of the treatment instrument as the feature point.

12. A medical image processing method comprising the following steps: reading an MR image of the subject obtained by previously photographing the subject using a magnetic resonance imaging device and a CT image of the subject obtained by previously photographing the subject using an X-ray CT device; reading an X-ray image of the subject obtained by photographing the subject during surgery; aligning the MR image and the CT image to obtain a corresponding relationship between coordinates of the MR image and the CT image; aligning the CT image and the X-ray image to obtain a corresponding relationship between coordinates of the CT image and the X-ray image; extracting feature points representing positions of predetermined parts of the subject in the MR image and the CT image respectively; extracting feature points in the X-ray image that represent the position of the part of the subject; aligning the X-ray image with the MR image; and The X-ray image aligned by the MR-X-ray image alignment unit is overlapped with the MR image to generate an overlapped image, and the overlapped image is displayed as a guide image. The medical image processing method is characterized in that: In the step of aligning the X-ray image with the MR image, after aligning the X-ray image with the MR image using the correspondence between the coordinates of the MR image and the CT image and the correspondence between the coordinates of the CT image and the X-ray image, the X-ray image and the MR image are further aligned at least in the parallel movement direction using the coordinates of at least one of the feature points of the MR image and the feature points of the CT image and the coordinates of the feature points of the X-ray image.

Images