Medical image processing apparatus and medical image processing method

Abstract

The present application provides a medical image processing device and a medical image processing method, which reduces metal artifacts and maintains image quality in a region with less influence of metal artifacts. A medical image processing device includes an operation unit that reconstructs a tomographic image from projection data of a subject including metal, characterized in that the operation unit acquires a machine learning output image output when a machine learning engine that has learned to reduce metal artifacts is input with the tomographic image, and generates a composite image by compositing the machine learning output image and the tomographic image.

Description

Technical Field

[0001] This invention relates to a medical image processing apparatus and a medical image processing method for processing medical images obtained by medical imaging devices such as X-ray CT (Computed Tomography) devices, and to a technique for reducing metal artifacts that occur when metal is present in the body being examined. Background Technology

[0002] An X-ray CT scanner, as an example of a medical imaging device, works by irradiating the subject with X-rays from multiple angles to obtain projection data, and then reconstructing a tomographic image of the subject for diagnostic purposes. If the subject contains metal, such as plates used for bone fixation, artifacts caused by the metal, known as metal artifacts, are generated in the medical images, hindering diagnostic imaging. Techniques to reduce metal artifacts are called MAR (Metal Artifact Reduction). While various methods have been developed, including beam hardening correction, linear interpolation, and deep learning, each has its advantages and disadvantages.

[0003] Non-patent document 1 discloses a method that combines the advantages of various methods by using the original image and an image with reduced metal artifacts obtained through beam hardening correction and linear interpolation as input images and applying them to a deep learning method.

[0004] Prior art literature

[0005] Non-patent literature

[0006] Non-patent document 1: Y.Zhang and H.Yu, "Convolutional Neural Network BasedMetal Artifact Reduction in X-Ray Computed Tomography," in IEEE Transactionson Medical Imaging, vol.37, no.6, pp.1370-1381, June 201 Summary of the Invention

[0007] -The problem the invention aims to solve-

[0008] However, in Non-Patent Document 1, although metal artifacts are reduced, image quality is sometimes degraded in areas where the influence of metal artifacts is small, such as areas far from the metal.

[0009] Therefore, the object of the present invention is to provide a medical image processing apparatus and a medical image processing method that can reduce metal artifacts and maintain image quality even in areas where the influence of metal artifacts is small.

[0010] -Methods for solving problems-

[0011] To achieve the above objectives, the present invention provides a medical image processing apparatus comprising a computing unit that reconstructs a tomographic image from projection data of a subject containing metal. The computing unit acquires a machine learning output image when the tomographic image is input to a machine learning engine that has been machine-learned to reduce metal artifacts, and synthesizes the machine learning output image and the tomographic image to generate a synthesized image.

[0012] Furthermore, the present invention is a medical image processing method for reconstructing tomographic images from projection data of a subject containing metal, characterized by comprising: an acquisition step of acquiring a machine learning output image output when the tomographic image is input to a machine learning engine that has been machined to reduce metal artifacts; and a generation step of synthesizing the machine learning output image and the tomographic image to generate a synthesized image.

[0013] -Invention Effects-

[0014] According to the present invention, a medical image processing apparatus and a medical image processing method are provided that can reduce metal artifacts without losing the detailed structure of the metal region. Attached Figure Description

[0015] Figure 1 This is a diagram of the overall structure of a medical image processing device.

[0016] Figure 2 This is a diagram of the overall structure of an X-ray CT device, which is an example of a medical imaging device.

[0017] Figure 3 This is a diagram illustrating an example of the processing flow of Example 1.

[0018] Figure 4 This is an example of a metallic artifact.

[0019] Figure 5 This is a diagram illustrating an example of the processing flow of S303 in Example 1.

[0020] Figure 6 This is a diagram showing an example of the operation window in Embodiment 1.

[0021] Figure 7 This is a diagram illustrating an example of the processing flow of Example 2.

[0022] -Explanation of Figure Markers-

[0023] 1: Medical image processing device; 2: Computation unit; 3: Memory; 4: Storage device; 5: Network adapter; 6: System bus; 7: Display device; 8: Input device; 10: Medical image capturing device; 11: Medical image database; 12: Machine learning engine; 100: X-ray CT device; 200: Scanner; 210: Subject; 211: X-ray tube; 212: Detector; 213: Collimator; 214: Drive unit; 215: Central control unit; 216: X-ray control unit, 217: High voltage generation unit, 218: Scanner control unit, 219: Table control unit, 221: Collimator control unit, 222: Preamplifier, 223: A / D converter, 240: Table, 250: Operation unit, 251: Reconstruction processing unit, 252: Image processing unit, 254: Storage unit, 256: Display unit, 258: Input unit, 601: Input image display unit, 602: Composite image display unit, 603: Adjustment coefficient setting unit. Detailed Implementation

[0024] Hereinafter, embodiments of the medical image processing apparatus and medical image processing method according to the present invention will be described with reference to the accompanying drawings. Furthermore, in the following description and drawings, structural elements having the same functional structure are labeled with the same reference numerals, thereby omitting repeated descriptions.

[0025] [Example 1]

[0026] Figure 1 This diagram illustrates the hardware structure of the medical image processing device 1. The medical image processing device 1 is configured to connect to the arithmetic unit 2, memory 3, storage device 4, and network adapter 5 via a system bus 6, enabling signal transmission and reception. Furthermore, the medical image processing device 1 is connected to the medical image capturing device 10, the medical image database 11, and the machine learning engine 12 via a network 9, enabling signal transmission and reception. A display device 7 and an input device 8 are connected to the medical image processing device 1. Here, "capable of signal transmission and reception" means that it is capable of transmitting and receiving signals electrically or optically, either wired or wirelessly, to each other or from one side to the other.

[0027] The arithmetic unit 2 is a device that controls the operation of various structural elements; specifically, it includes a CPU (Central Processing Unit) or an MPU (Micro Processor Unit). The arithmetic unit 2 loads the program stored in the storage device 4 and the data required for program execution into the memory 3 and executes it, performing various image processing operations on the medical images. The memory 3 stores the program executed by the arithmetic unit 2 and the intermediate processing steps. The storage device 4 stores the program executed by the arithmetic unit 2 and the data required for program execution; specifically, it includes an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The network adapter 5 is used to connect the medical image processing device 1 to a network 9 such as a LAN, telephone line, or the Internet. Various data processed by the arithmetic unit 2 can also be transmitted and received externally to the medical image processing device 1 via a LAN (Local Area Network) or other network 9.

[0028] Display device 7 is a device that displays the processing results of medical image processing device 1, and more specifically, it is a liquid crystal display (LCD). Input device 8 is an operating device for the operator to give instructions on operating medical image processing device 1, and more specifically, it is a keyboard, mouse, touch panel, etc. The mouse can also be a touchpad, trackball, or other indicating device.

[0029] The medical imaging device 10 is, for example, an X-ray CT (Computed Tomography) device that acquires projection data of a subject and reconstructs tomographic images based on the projection data. Figure 2 As will be described later. The medical image database 11 is a database system that stores projection data, tomographic images, and corrected images of the tomographic images that have undergone image processing, etc., acquired by the medical image capturing device 10.

[0030] The machine learning engine 12 is generated by performing machine learning to reduce metal artifacts in tomographic images, for example, using a CNN (Convolutional Neural Network). In the generation of the machine learning engine 12, a tomographic image without metal is used as a teaching image, for example. Furthermore, a tomographic image containing metal artifacts is used as the input image, wherein a forward projection of an image with a metal region added to the teaching image is performed to generate projection data containing metal, and a backward projection of this projection data is performed to obtain the tomographic image containing metal artifacts.

[0031] use Figure 2 The overall structure of an X-ray CT apparatus 100, exemplified as a medical imaging device 10, is explained. Furthermore, in Figure 2 In this design, the horizontal axis is defined as the X-axis, the vertical axis as the Y-axis, and the direction perpendicular to the paper plane as the Z-axis. The X-ray CT apparatus 100 includes a scanner 200 and an operating unit 250. The scanner 200 includes an X-ray tube 211, a detector 212, a collimator 213, a drive unit 214, a central control unit 215, an X-ray control unit 216, a high-voltage generator 217, a scanner control unit 218, a bed control unit 219, a collimator control unit 221, a preamplifier 222, an A / D converter 223, and a bed 240, etc.

[0032] X-ray tube 211 is a device for irradiating X-rays onto a subject 210 placed on a bed 240. A high voltage generated by high voltage generating unit 217 according to a control signal sent from X-ray control unit 216 is applied to X-ray tube 211, thereby irradiating the subject with X-rays from X-ray tube 211.

[0033] Collimator 213 is a device that limits the irradiation range of X-rays emitted from X-ray tube 211. The irradiation range of X-rays is set according to a control signal sent from collimator control unit 221.

[0034] Detector 212 is a device that measures the spatial distribution of X-rays transmitted through the subject 210 by detecting X-rays transmitted through the subject 210. Detector 212 is positioned opposite X-ray tube 211, and a large number of detection elements are arranged two-dimensionally in the plane opposite to X-ray tube 211. The signal measured by detector 212 is amplified by preamplifier 222 and then converted into a digital signal by A / D converter 223. Afterwards, various correction processes are performed on the digital signal to obtain projection data.

[0035] The drive unit 214 rotates the X-ray tube 211 and detector 212 around the subject 210 according to control signals sent from the scanner control unit 218. By rotating the X-ray tube 211 and detector 212, X-ray irradiation and detection are performed, thereby acquiring projection data from multiple projection angles. Each data collection unit for each projection angle is called a view. Regarding the arrangement of the detection elements of the two-dimensionally arranged detector 212, the rotation direction of the detector 212 is called a channel, and the direction orthogonal to the channel is called a column. The projection data is identified by views, channels, and columns.

[0036] The table control unit 219 controls the movement of the table 240, keeping the table 240 stationary during X-ray irradiation and detection, or moving the table 240 at a constant speed in the Z-axis direction, which is the body axis of the subject 210. Scanning with the table 240 stationary is called axial scanning, and scanning while moving the table 240 is called helical scanning.

[0037] The central control unit 215 controls the operation of the scanner 200 described above based on instructions from the operation unit 250. Next, the operation unit 250 will be described. The operation unit 250 includes a reconstruction processing unit 251, an image processing unit 252, a storage unit 254, a display unit 256, an input unit 258, etc.

[0038] The reconstruction processing unit 251 reconstructs the tomographic image by back-projecting the projection data acquired by the scanner 200. The image processing unit 252 performs various image processing steps to make the tomographic image suitable for diagnosis. The storage unit 254 stores the projection data, the tomographic image, and the image after processing. The display unit 256 displays the tomographic image or the image after processing. The input unit 258 is used when the operator sets the conditions for acquiring the projection data (tube voltage, tube current, scanning speed, etc.) and the reconstruction conditions for the tomographic image (reconstruction filter, FOV size, etc.).

[0039] Alternatively, the operation unit 250 can also be Figure 1 The medical image processing apparatus 1 shown. In this case, the reconstruction processing unit 251 and the image processing unit 252 are equivalent to the arithmetic unit 2, the storage unit 254 is equivalent to the storage device 4, the display unit 256 is equivalent to the display device 7, and the input unit 258 is equivalent to the input device 8.

[0040] use Figure 3 An example of the process performed in Example 1 will be described step by step.

[0041] (S301)

[0042] The computation unit 2 acquires a tomographic image I_ORG of the subject containing metal. Since the subject contains metal, metal artifacts are present in the tomographic image I_ORG. Figure 4 An example of a metal artifact. Figure 4 The images were taken as tomographic images of the abdominal phantom, which showed a dark band between two metallic regions in the liver and streaks originating from each metallic region.

[0043] (S302)

[0044] The computing unit 2 obtains the machine learning output image I_MAR when the tomographic image I_ORG is input to the machine learning engine 12 which has machined to reduce metal artifacts. In the machine learning output image I_MAR, although metal artifacts are reduced, the image quality is sometimes reduced in areas where the influence of metal artifacts is small, such as areas far away from metal.

[0045] (S303)

[0046] The processing unit 2 synthesizes the machine learning output image I_MAR obtained in S302 and the tomographic image I_ORG obtained in S301. In the machine learning output image I_MAR, image quality is sometimes reduced in areas with minimal metal artifacts; conversely, in the tomographic image I_ORG, image quality is not reduced in areas with minimal metal artifacts. Therefore, by synthesizing the machine learning output image I_MAR and the tomographic image I_ORG, a composite image is generated that reduces metal artifacts and maintains image quality in areas with minimal metal artifact influence. The generated composite image is displayed on the display device 7 or stored in the storage device 4.

[0047] use Figure 5 An example of the process for handling S303 is explained step by step.

[0048] (S501)

[0049] The arithmetic unit 2 obtains a weight mapping that maps real numbers greater than or equal to 0 and less than or equal to 1, i.e., the weight coefficients w. The weight mapping I_w is generated, for example, by the following formula.

[0050] I_w = |I_ORG - I_BHC|...(Equation 1)

[0051] Here, I_BHC is a beam-hardened corrected image obtained by applying beam hardening correction to the tomographic image I_ORG.

[0052] The beam-hardened corrected image I_BHC is obtained, for example, through the following steps. First, metallic pixels are extracted from the tomographic image I_ORG. Next, in the projection data P_ORG used to generate the tomographic image I_ORG, projection data P_BHC is obtained by correcting the projection values ​​corresponding to the metallic pixels. In correcting the projection values ​​corresponding to the metallic pixels, the length of the metallic pixel in the projection line involved in the projection value and the projection value are used. That is, the longer the length of the metallic pixel in the projection line and the higher the projection value, the greater the correction strength. Then, the projection data P_BHC is back-projected and added to or subtracted from the tomographic image I_ORG, thereby obtaining the beam-hardened corrected image I_BHC.

[0053] In addition, the weight mapping I_w can also be generated by the following formula.

[0054] I_w=|I_ORG-I_LI|…(Equation 2)

[0055] Here, I_LI is a linearly interpolated image obtained by applying linear interpolation to the tomographic image I_ORG.

[0056] The linear interpolated image I_LI is obtained, for example, through the following steps. First, metallic pixels are extracted from the tomographic image I_ORG. Next, in the projection data P_ORG used to generate the tomographic image I_ORG, the projection values ​​obtained by linearly interpolating the projection values ​​corresponding to the metallic pixels with adjacent projection values ​​are replaced to obtain projection data P_LI. Then, the projection data P_LI is back-projected, and the extracted metallic pixels are synthesized to obtain the linear interpolated image I_LI.

[0057] Since the beam hardening correction image I_BHC and the linear interpolation image I_LI are images with reduced metal artifacts, the weight mapping I_w generated by (Equation 1) and (Equation 2) is also an artifact mapping that represents the distribution of the probability of the existence of metal artifacts.

[0058] (S502)

[0059] The computation unit 2 uses the weight coefficients w of the weight mapping I_w obtained in S501 to synthesize the machine learning output image I_MAR and the tomographic image I_ORG, generating a synthesized image I_CMP. The synthesis image I_CMP is generated, for example, using the following formula.

[0060] I_CMP=w·I_MAR+(1-w)·I_ORG...(Formula 3)

[0061] According to Equation 3, the values ​​obtained by multiplying each pixel value of the machine learning output image I_MAR by the pixel values ​​of the weight mapping I_w (i.e., the weight coefficient w) and multiplying each pixel value of the tomographic image I_ORG by (1-w) are added together. That is, in regions with many metal artifacts, the ratio of the machine learning output image I_MAR increases, and in regions with few metal artifacts, the ratio of the tomographic image I_ORG increases. As a result, metal artifacts are reduced in the synthesized image I_CMP, and image quality is maintained in regions where the influence of metal artifacts is minimal.

[0062] Since the beam-hardened corrected image I_BHC is obtained based on the correction of the projection values ​​corresponding to the metal pixels, the use of the weight mapping I_w in (Equation 1) can further reduce artifacts in areas where the influence of metal pixels is large. Furthermore, the linear interpolation image I_LI is obtained by linearly interpolating the projection values ​​corresponding to the metal pixels with adjacent projection values; therefore, the use of the weight mapping I_w in (Equation 2) can further reduce artifacts directly generated from the metal.

[0063] Furthermore, metal artifacts decrease with distance from the metal pixels extracted from the tomographic image I_ORG, therefore the weighting coefficient w also decreases with distance from the metal pixels. Additionally, larger pixel values ​​of metal pixels result in larger metal artifacts; therefore, larger pixel values ​​of metal pixels lead to larger weighting coefficients w.

[0064] Furthermore, the weighting coefficient w can also be any of the following: tomographic image I_ORG, machine learning output image I_MAR, beam hardening correction image I_BHC, or linear interpolation image I_LI, adjusting the weight mapping I_w based on the tissue, air, and other tissues within the subject. For example, known thresholding segmentation can also be used to segment the tomographic image I_ORG into regions of metal, non-metal subjects, and air, treating the metal region as in the machine learning output image I_MAR (w=1), the non-metal subjects as in the weight mapping I_w, and the air region as in the tomographic image I_ORG (w=0), using prior information from the image to adjust the weighting coefficient w.

[0065] Furthermore, the weight coefficients w can also be adjusted appropriately by the operator. For example, an adjustment coefficient set by the operator can also be used to adjust the weight coefficients w. The adjustment coefficient is a real number above 0 and below 1, and all weight coefficients w are adjusted simultaneously by multiplying the weight map I_w by the adjustment coefficient. That is, all weight coefficients w contained in the weight map I_w are multiplied by the same adjustment coefficient.

[0066] use Figure 6 Here is an example of an operation window used to set adjustment coefficients. Figure 6 The illustrated operation window includes an input image display unit 601, a composite image display unit 602, and an adjustment coefficient setting unit 603. The input image display unit 601 displays a tomographic image I_ORG containing metal artifacts and a machine learning output image I_MAR output from the machine learning engine 12. However, the input image display unit 601 is not mandatory. The composite image I_CMP generated in S502 is displayed in the composite image display unit 602. The adjustment coefficient setting unit 603 is used to set the adjustment coefficient multiplied by the weighting coefficient w, and is configured, for example, by a slider or a text box. Alternatively, the adjustment coefficient setting unit 603 can be configured to set the adjustment coefficient for the position of the subject 210 along its body axis, i.e., for each slice position.

[0067] Operator by using Figure 6 The illustrated operation screen allows you to view the composite image I_CMP, which is updated each time an adjustment factor is set. Furthermore, when the input image display unit 601 is displayed, the adjustment factor can be set while comparing the tomographic image I_ORG, the machine learning output image I_MAR, and the composite image I_CMP.

[0068] In addition, while limited to artifacts produced by metals, artifacts produced by high absorbers such as bone and contrast agents with high X-ray absorption coefficients other than metals, as well as artifacts produced by low absorbers such as lung fields and intestines with extremely low X-ray absorption coefficients relative to the tissues being examined, can also be reduced using the same method.

[0069] By following the processing steps described above, a composite image with reduced metal artifacts and maintained image quality in areas where the influence of metal artifacts is minimal can be obtained.

[0070] [Example 2]

[0071] In Example 1, the case of generating a synthetic image I_CMP using a synthetic tomographic image I_ORG and a machine learning output image I_MAR output from the machine learning engine 12 was described. In Example 2, the case of obtaining a corrected image with reduced metal artifacts by inputting an artifact mapping representing the distribution of the probability of the presence of metal artifacts and the tomographic image I_ORG into the machine learning engine 12 was described. Furthermore, the hardware structure of the medical image processing apparatus 1 in Example 2 is the same as that in Example 1, therefore, the description is omitted.

[0072] use Figure 7 An example of the process performed in Example 2 will be described step by step.

[0073] (S701)

[0074] Similar to S301, the arithmetic unit 2 acquires a tomographic image I_ORG of the subject containing metal.

[0075] (S702)

[0076] The computation unit 2 obtains an artifact map representing the distribution of the probability of the presence of metal artifacts. The artifact map can also be generated, for example, using (Equation 1) and (Equation 2).

[0077] (S703)

[0078] The computation unit 2 inputs the artifact mapping obtained in S702 and the tomographic image I_ORG obtained in S701 into the machine learning engine 12. The machine learning engine 12, which inputs the artifact mapping along with the tomographic image I_ORG, outputs a corrected image that reduces metal artifacts and maintains image quality in areas where the influence of metal artifacts is small.

[0079] (S704)

[0080] In S703, the arithmetic unit 2 acquires the corrected image output from the machine learning engine 12. The acquired corrected image is then displayed on the display device 7 or stored in the storage device 4.

[0081] Through the processing flow described above, a corrected image with reduced metal artifacts and maintained image quality in areas where the influence of metal artifacts is minimal can be obtained. Furthermore, in S703, the beam hardening correction image I_BHC and the linear interpolation image I_LI can be further input into the machine learning engine 12. By further inputting the beam hardening correction image I_BHC and the linear interpolation image I_LI, the metal artifacts in the corrected image output from the machine learning engine 12 can be further reduced.

[0082] The present invention has been described above with reference to several embodiments. However, the present invention is not limited to the above embodiments and can be further customized by modifying the structural elements without departing from the spirit of the invention. Furthermore, the various structural elements disclosed in the above embodiments can be appropriately combined. Moreover, several structural elements can be deleted from all the structural elements shown in the above embodiments.

Claims

1. A medical image processing apparatus comprising a computing unit for reconstructing tomographic images from projection data of a subject containing metal, characterized in that, The computing unit obtains the machine learning output image when the tomographic image is input into a machine learning engine that has machined to reduce metal artifacts, and synthesizes the machine learning output image and the tomographic image to generate a synthetic image. The computation unit obtains a weight map that maps the weight coefficients, and uses the weight map to synthesize the machine learning output image and the tomographic image. The weight mapping is the distribution of the absolute value of the difference between the beam-hardened image obtained by applying beam hardening correction to the tomographic image and the tomographic image, or the distribution of the absolute value of the difference between the linearly interpolated image obtained by applying linear interpolation to the tomographic image and the tomographic image.

2. The medical image processing device according to claim 1, wherein, The weighting coefficients decrease as they move away from the metallic pixels extracted from the tomographic image.

3. The medical image processing device according to claim 2, wherein, The larger the pixel value of the metal pixel, the larger the weighting coefficient.

4. The medical image processing device according to claim 1, wherein, The computation unit uses the value obtained by multiplying the adjustment coefficient set in the adjustment coefficient setting unit by the weight coefficient to synthesize the machine learning output image and the tomographic image.

5. The medical image processing device according to claim 4, wherein, The synthesized image is displayed in the same window as the adjustment coefficient setting unit and is updated whenever the adjustment coefficient is set in the adjustment coefficient setting unit.

6. A medical image processing method for reconstructing tomographic images from projection data of a subject containing metal, characterized in that, have: The acquisition step involves acquiring a machine learning output image when the tomographic image is input into a machine learning engine that has been machine-learned to reduce metal artifacts; and In the generation step, the machine learning output image and the tomographic image are synthesized to generate a synthetic image. A weight map containing the weight coefficients is obtained, and the weight map is used to synthesize the machine learning output image and the tomographic image. The weight mapping is the distribution of the absolute value of the difference between the beam-hardened image obtained by applying beam hardening correction to the tomographic image and the tomographic image, or the distribution of the absolute value of the difference between the linearly interpolated image obtained by applying linear interpolation to the tomographic image and the tomographic image.

7. A medical image processing apparatus, comprising a computing unit for reconstructing tomographic images from projection data of a subject containing metal, characterized in that, The computation unit obtains a corrected image with reduced metal artifacts by inputting an artifact map representing the distribution of the probability of the presence of metal artifacts and the tomographic image into a machine learning engine that has been machined to reduce metal artifacts. The artifact mapping represents the distribution of the absolute value of the difference between the beam-hardened image obtained by applying beam hardening correction to the tomographic image and the tomographic image, or the distribution of the absolute value of the difference between the linearly interpolated image obtained by applying linear interpolation to the tomographic image and the tomographic image.

8. The medical image processing apparatus according to claim 7, wherein, The computing unit further inputs a beam-hardened image obtained by applying beam hardening correction to the tomographic image or a linear interpolation image obtained by applying linear interpolation to the tomographic image into the machine learning engine.

Images