Medical Image Processing Apparatus and Medical Image Processing Method

Abstract

To provide a medical image processing device and a medical image processing method capable of improving extraction accuracy of a tumor region included in a diagnosis target image.SOLUTION: A medical image processing device for extracting a predetermined region from a diagnosis target image includes: an organ extraction unit for extracting an organ region from the diagnosis target image; and a tumor extraction unit generated by executing machine learning with a known tumor region included in each of medical image groups as teacher data and with an organ region extracted from each of the medical image groups and the medical image groups as input data. The tumor extraction unit extracts a tumor region from the diagnosis target image while using the organ region extracted from the diagnosis target image by the organ extraction unit.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a medical image processing apparatus and a medical image processing method for extracting tumors included in medical images.

Background Art

[0002] Medical image imaging apparatuses typified by X-ray CT (Computed Tomography) apparatuses are apparatuses that image the forms of lesions and the like, and the acquired medical images are used for image diagnosis and treatment planning. In order to perform appropriate image diagnosis and treatment planning, it is important to classify tissues and lesions with high accuracy.

[0003] Patent Document 1 discloses an image processing apparatus that can classify regions of tissues and lesions included in medical images with high accuracy. Specifically, a discriminator that has learned, as teacher data, tomographic images in which the types of tissues and lesions are known is used to identify the types of tissues and lesions to which each pixel of each tomographic image in different cross-sectional directions belongs, and the types of lesions and the like are re-identified by evaluating pixels common to a plurality of tomographic images.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in Patent Document 1, only discriminators that have individually learned the types of tissues and lesions are used, which is insufficient for extracting lesions such as tumors with higher accuracy.

[0006] Therefore, an object of the present invention is to provide a medical image processing apparatus and a medical image processing method capable of improving the extraction accuracy of tumor regions included in diagnostic target images.

Means for Solving the Problems

[0007] In order to achieve the above object, the present invention provides a medical image processing apparatus for extracting a predetermined region from a diagnostic target image, comprising: an organ extraction unit for extracting an organ region from the diagnostic target image; and a tumor extraction unit generated by machine learning using known tumor regions included in each of a group of medical images as teacher data, and organ regions extracted from each of the group of medical images and the group of medical images as input data, wherein the tumor extraction unit extracts a tumor region from the diagnostic target image while using the organ region extracted from the diagnostic target image by the organ extraction unit.

[0008] The present invention also provides a medical image processing method for extracting a predetermined region from a diagnostic target image, comprising: an organ extraction step for extracting an organ region from the diagnostic target image; and a tumor extraction step for extracting a tumor region from the diagnostic target image while using the organ region extracted from the diagnostic target image in the organ extraction step.

Advantages of the Invention

[0009] According to the present invention, it is possible to provide a medical image processing apparatus and a medical image processing method capable of improving the extraction accuracy of a tumor region included in a diagnostic target image.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 5E

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of a medical image processing apparatus and a medical image processing method according to the present invention will be described with reference to the accompanying drawings. In the following description and the accompanying drawings, components having the same functional configuration will be denoted by the same reference numerals, and redundant description will be omitted.

Examples

[0012] FIG. 1 is a diagram showing the hardware configuration of a medical image processing apparatus 1. The medical image processing apparatus 1 is configured such that an arithmetic unit 2, a memory 3, a storage device 4, and a network adapter 5 are connected to be able to transmit and receive signals via a system bus 6. The medical image processing apparatus 1 is also connected to be able to transmit and receive signals to and from a medical image imaging apparatus 10 and a medical image database 11 via a network 9. Further, a display device 7 and an input device 8 are connected to the medical image processing apparatus 1. Here, "able to transmit and receive signals" indicates a state in which signals can be transmitted and received to and from each other or from one to the other, regardless of whether it is electrical, optical, wired, or wireless.

[0013] The arithmetic unit 2 is a device that controls the operations of each component, specifically, a CPU (Central Processing Unit), an MPU (Micro Processor Unit), etc. The arithmetic unit 2 loads the programs stored in the storage device 4 and the data necessary for program execution into the memory 3 and executes them, performing various image processes on medical images. The memory 3 stores the programs executed by the arithmetic unit 2 and the progress of the arithmetic processes. The storage device 4 is a device that stores the programs executed by the arithmetic unit 2 and the data necessary for program execution, specifically, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc. The network adapter 5 is for connecting the medical image processing device 1 to a network 9 such as a LAN, a telephone line, or the Internet. Various data handled by the arithmetic unit 2 may be transmitted and received to and from the outside of the medical image processing device 1 via a network 9 such as a LAN (Local Area Network).

[0014] The display device 7 is a device that displays the processing results, etc. of the medical image processing device 1, specifically, a liquid crystal display, a touch panel, etc. The input device 8 is an operation device for an operator to give operation instructions to the medical image processing device 1, specifically, a keyboard, a mouse, a touch panel, etc. The mouse may be other pointing devices such as a track pad or a track ball.

[0015] The medical image imaging device 10 is a device that images tomographic images, etc. of a subject, for example, an X-ray CT device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device. The medical image database 11 is a database system that stores medical images such as tomographic images imaged by the medical image imaging device 10 and corrected images obtained by performing image processing on the tomographic images.

[0016] The functional block diagram of Example 1 will be described with reference to FIG. 2. Each function shown in FIG. 2 may be configured by dedicated hardware using an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like, or may be configured by software operating in the arithmetic unit 2. In the following description, the case where each function of Example 1 is configured by software will be described.

[0017] Example 1 includes an organ extraction unit 201 and a tumor extraction unit 202. The storage device 4 stores diagnostic target images and the like that are medical images captured by the medical imaging device 10 and are the targets of diagnosis. The diagnostic target image may be a tomographic image or a volume image. Each component will be described below.

[0018] The organ extraction unit 201 extracts an organ region from the diagnostic target image based on the pixel values of the pixels included in the diagnostic target image. The extracted organ region is classified into, for example, the heart, lungs, body surface, etc. Note that the organ extraction unit 201 may be configured by a CNN (Convolutional Neural Network) generated by machine learning using a large number of medical images including known organ regions as input data and known organ regions as teacher data.

[0019] The tumor extraction unit 202 is generated by machine learning using known tumor regions included in a large number of medical images as teacher data in order to extract a tumor region from the diagnostic target image, and is configured by, for example, a CNN.

[0020] An example of the process flow for generating the tumor extraction unit 202 will be described step by step with reference to FIG. 3.

[0021] (S301) A group of medical images including known tumor regions is acquired. That is, an identifier indicating whether each pixel included in each medical image is a tumor region is assigned.

[0022] (S302) An organ region is extracted from each of the medical images obtained in S301. That is, an identifier indicating whether each pixel included in each medical image is an organ region is assigned, and an identifier indicating which organ it is is further assigned to the pixels that are organ regions.

[0023] For the extraction of the organ region from each medical image, the organ extraction unit 201 may be used. Also, when a known organ region is included in each medical image, the identifier that each pixel has is used.

[0024] (S303) The tumor extraction unit 202 is generated by machine learning using the medical image group obtained in S301 and the organ regions extracted in S302 as input data, and the known tumor regions included in each medical image as teacher data. Specifically, the weight values of each path in the intermediate layer connecting the input layer and the output layer are adjusted so that the output data output from the output layer matches the teacher data when the input data is input to the input layer.

[0025] The tumor extraction unit 202 is generated by the processing flow described above. The generated tumor extraction unit 202 machine-learns not only the medical image group including the known tumor regions, but also the organ regions extracted from each of the medical image groups as input data, and thus includes the relationship between the tumor region and the organ region, for example, their relative positional relationship, as knowledge.

[0026] Using FIG. 4, an example of the processing flow for extracting the tumor region from the diagnostic target image in Example 1 will be described step by step.

[0027] (S401) A diagnostic target image, which is a medical image to be diagnosed, is obtained. The diagnostic target image is, for example, a tomographic image obtained by imaging the chest of a subject shown in FIG. 5A with the medical image imaging device 10, and is transmitted via the network adapter or read from the storage device 4. A tomographic image of the chest is illustrated in FIG. 5B. The obtained diagnostic target image is not limited to a single tomographic image, and may be a plurality of tomographic images or a volume image.

[0028] (S402) The organ extraction unit 201 extracts an organ region from the diagnostic target image obtained in S401. The extracted organ region is classified into a heart, lungs, body surface, etc. as illustrated in FIG. 5C.

[0029] (S403) The tumor extraction unit 202 extracts a tumor region from the diagnostic target image while using the organ region extracted in S402. A tumor region extracted from a tomographic image of the chest is illustrated in FIG. 5D. Note that, together with the extraction of the tumor region, the organ region extracted in S402 may be subdivided, for example, the spine may be extracted. Also, as illustrated in FIG. 5E, the extracted tumor region may be fused with the organ region.

[0030] As described above, by the processing flow, a tumor region can be extracted from the diagnostic target image. In particular, since the tumor extraction unit 202 includes the relationship between the tumor region and the organ region as knowledge, it can extract the tumor region with higher accuracy than a discriminator that has only been machine-learned using known tumor regions as teacher data.

[0031] An example of the model configuration of the organ extraction unit 201 and the tumor extraction unit 202 will be described with reference to FIG. 6. The organ extraction unit 201 and the tumor extraction unit 202 generated by machine learning are basically configured with a CNN. When 3D CT data, which is the diagnostic target image, is input to the input layer of the organ extraction unit 201, an organ region is extracted. The extracted organ region is incorporated into the tumor extraction unit 202 and is used when the tumor extraction unit 202 extracts a tumor region from the diagnostic target image. Note that the extracted organ region may be incorporated into the intermediate layer of the tumor extraction unit 202 or may be input to the input layer. Finally, the organ region extracted by the organ extraction unit 201 and the tumor region extracted by the tumor extraction unit 202 are fused.

[0032] Using FIG. 7, another example of the model configuration of the organ extraction unit 201 and the tumor extraction unit 202 will be described. In the example of FIG. 6, the model of the organ extraction unit 201 and the model of the tumor extraction unit 202 are separated, whereas in the example of FIG. 7, the organ extraction unit 201 and the tumor extraction unit 202 are configured by a single model, and the extraction result of the organ region is incorporated via the layers inside the network and used for the extraction of the tumor region. According to the configuration illustrated in FIG. 7, the size of the model can be reduced, facilitating the implementation of the medical image processing apparatus 1.

Example

[0033] In Example 1, the extraction of the tumor region from the diagnostic target image by the tumor extraction unit 202 generated by machine learning using, as input data, not only the medical image group including the known tumor region but also the organ region extracted from each of the medical image groups was described. In Example 2, the calculation of the feature amount related to the extracted tumor region will be described. The feature amount related to the tumor region can be used to support image diagnosis and treatment planning. Since the hardware configuration of the medical image processing apparatus 1 in Example 2 is the same as that in Example 1, the description thereof will be omitted.

[0034] The functional block diagram of Example 2 will be described with reference to FIG. 8. Each function shown in FIG. 8 may be configured by dedicated hardware using an ASIC, an FPGA, etc., in the same manner as in Example 1, or may be configured by software operating in the arithmetic unit 2. In the following description, the case where each function of Example 2 is configured by software will be described.

[0035] Example 2 includes an organ extraction unit 201 and a tumor extraction unit 202 as in Example 1, and further includes a feature amount calculation unit 403 and a state determination unit 404. Hereinafter, the feature amount calculation unit 403 and the state determination unit 404 added to the configuration of Example 1 will be described.

[0036] The feature quantity calculation unit 403 calculates feature quantities related to the tumor region extracted from the diagnostic target image. The feature quantities related to the tumor region include tumor property feature quantities that are feature quantities related to the tumor region itself, tumor-organ feature quantities that are feature quantities representing the relationship between the tumor region and the organ region, and intermediate layer feature quantities that are feature quantities used in the intermediate layer of the tumor extraction unit 202.

[0037] The state determination unit 404 determines the state of the tumor region based on the feature quantities calculated by the feature quantity calculation unit 403. The state of the tumor region includes the size of the tumor, the invasion stage, classification, progression stage, etc.

[0038] Using FIG. 9, an example of the processing flow for extracting the tumor region from the diagnostic target image and calculating the feature quantities related to the tumor region in Example 2 will be described step by step. Since S301 to S303 are the same as those in Example 1, the description will be omitted.

[0039] (S904) The feature quantity calculation unit 403 calculates feature quantities related to the tumor region extracted in S303, that is, tumor property feature quantities, tumor-organ feature quantities, and intermediate layer feature quantities.

[0040] Using FIG. 10, an example of the feature quantities calculated by the feature quantity calculation unit 403 will be described. The tumor property feature quantities are, for example, values indicating the size of each tumor region, values indicating the shape, values calculated from the luminance histogram, Radomics, etc. The values indicating the size of the tumor region include volume and maximum diameter. The values indicating the shape of the tumor region include circularity and index values indicating the presence or absence of cavities. The values calculated from the luminance histogram include the uniformity of the luminance values and the half-value width of the maximum peak. The tumor property feature quantities are used to assist in image diagnosis.

[0041] The tumor-organ feature quantities are, for example, the distance between the tumor region and the organ region, index values representing the presence or absence of adhesion, and index values representing the presence or absence of invasion. Also, the position, distribution, number, etc. of the tumor in the imaging site may be included in the tumor-organ feature quantities. The tumor-organ feature quantities are used to assist in image diagnosis and treatment planning.

[0042] The intermediate layer feature amount is a value indicating information shared by the tumor region and the organ region, etc. The intermediate layer feature amount is used for transfer learning and prediction of treatment effects.

[0043] Return to the description of FIG. 9.

[0044] (S905) Based on the feature amount calculated in S904, the state determination unit 404 determines the state of the tumor region. Note that S905 is not essential.

[0045] According to the processing flow described above, the tumor region is extracted from the diagnostic target image, and the feature amount related to the tumor region is calculated, or the state of the tumor is determined. Also in Example 2, as in Example 1, the tumor region is extracted with high accuracy. In addition, the feature amount related to the tumor region calculated by the feature amount calculation unit 403 is used for supporting image diagnosis and treatment planning.

[0046] An example of the input / output screen of Example 2 will be described with reference to FIG. 11. The input / output screen 500 illustrated in FIG. 11 includes an axial image display unit 511, a sagittal image display unit 512, a coronal image display unit 513, a three-dimensional image display unit 514, a feature amount display unit 515, and a determination result display unit 516.

[0047] The axial image display unit 511 displays the axial image of the diagnostic target image. The sagittal image display unit 512 displays the sagittal image of the diagnostic target image. The coronal image display unit 513 displays the coronal image of the diagnostic target image. Lines indicating the contours of the extracted organ region and tumor region may be superimposed and displayed on the axial image, sagittal image, and coronal image. The three-dimensional image display unit 514 displays the three-dimensional image of the diagnostic target image.

[0048] The feature quantity display unit 515 displays the feature quantities calculated by the feature quantity calculation unit 403. In the feature quantity display unit 515 illustrated in FIG. 11, it is displayed that the tumor size is 28 mm, the edge is lobulated, there is no aeration, the luminance distribution is solid, there is adhesion to the lung, and there is no adhesion to the heart, etc. The determination result display unit 516 displays the state of the tumor region determined by the state determination unit 404. In the determination result display unit 516 illustrated in FIG. 11, it is displayed that the stage is IIIb and the classification is large cell.

[0049] Also, the input / output screen 500 has an image selection button 521, a region extraction button 522, a region selection button 523, a region editing button 524, a feature quantity setting button 525, a result output button 526, and a state determination button 527.

[0050] The image selection button 521 is used when selecting a diagnostic target image. The selected diagnostic target image is displayed on the axial image display unit 511, the sagittal image display unit 512, the coronal image display unit 513, and the three-dimensional image display unit 514.

[0051] The region extraction button 522 is used when extracting an organ region or a tumor region from a diagnostic target image. The extracted organ region or tumor region is superimposed and displayed on the axial image display unit 511, the sagittal image display unit 512, and the coronal image display unit 513.

[0052] The region selection button 523 is used when selecting an extracted organ region or tumor region. The region editing button 524 is used when editing an extracted organ region or tumor region. The selection and editing of the organ region or tumor region are performed on the axial image display unit 511, the sagittal image display unit 512, and the coronal image display unit 513.

[0053] The feature quantity setting button 525 is used when setting the feature quantities related to the tumor region. The set feature quantities are calculated by the feature quantity calculation unit 403 and displayed on the feature quantity display unit 515.

[0054] The result output button 526 is used when outputting the extraction results of the organ region and tumor region, as well as the calculation results of the feature amounts, from the medical image processing apparatus 1.

[0055] The state determination button 527 is used when determining the state of the tumor region. The state of the tumor region is determined by the state determination unit 404 and displayed on the determination result display unit 516.

[0056] By using the input / output screen 500 illustrated in FIG. 11, the operator can select a diagnostic target image and confirm the tumor region extracted from the diagnostic target image together with the feature amounts related to the tumor region.

[0057] As described above, a plurality of embodiments of the present invention have been described. Note that the present invention is not limited to the above embodiments, and components can be modified and embodied without departing from the gist of the invention. Also, a plurality of components disclosed in the above embodiments may be appropriately combined. Furthermore, some components may be deleted from all the components shown in the above embodiments.

Explanation of Signs

[0058] 1: Medical image processing apparatus, 2: Arithmetic unit, 3: Memory, 4: Storage device, 5: Network adapter, 6: System bus, 7: Display device, 8: Input device, 10: Medical image imaging apparatus, 11: Medical image database, 201: Organ extraction unit, 202: Tumor extraction unit, 403: Feature amount calculation unit, 404: State determination unit, 500: Input / output screen, 511: Axial image display unit, 512: Sagittal image display unit, 513: Coronal image display unit, 514: Three-dimensional image display unit, 515: Feature amount display unit, 516: Determination result display unit, 521: Image selection button, 522: Region extraction button, 523: Region selection button, 524: Region editing button, 525: Feature amount setting button, 526: Result output button, 527: State determination button

Claims

1. A medical image processing apparatus for extracting a predetermined region from a diagnostic target image, comprising: an organ extraction unit that extracts an organ region from the diagnostic target image; a tumor extraction unit generated by machine learning using known tumor regions included in each of a group of medical images as teacher data, and organ regions extracted from each of the group of medical images and the group of medical images as input data; the tumor extraction unit extracts a tumor region from the diagnostic target image while using the organ region extracted from the diagnostic target image by the organ extraction unit; further comprising a feature amount calculation unit that calculates a feature amount related to the tumor region extracted by the tumor extraction unit; The medical image processing apparatus is characterized in that the feature amount calculated by the feature amount calculation unit includes a tumor-organ feature amount that is a feature amount representing the relationship between the organ region extracted by the organ extraction unit and the tumor region extracted by the tumor extraction unit.

2. The medical image processing apparatus according to claim 1, wherein the organ extraction unit is generated by machine learning using known organ regions included in each of a group of medical images as teacher data and the group of medical images as input data; The medical image processing apparatus is characterized in that the input data used for generating the tumor extraction unit includes the organ regions extracted from each of the group of medical images by the organ extraction unit.

3. The medical image processing apparatus according to claim 1, wherein the tumor-organ feature amount includes the distance between the organ region and the tumor region, the presence or absence of adhesion between the organ region and the tumor region, and the presence or absence of infiltration of the tumor region into the organ region.

4. The medical image processing apparatus according to claim 1, wherein the feature amount calculated by the feature amount calculation unit includes a tumor property feature amount that is a feature amount related to the tumor region itself extracted by the tumor extraction unit.

5. The medical image processing apparatus according to claim 4, wherein the tumor property feature amount includes the size, shape, and pixel value histogram of the tumor region.

6. The medical image processing apparatus according to claim 1, further comprising a state determination unit that determines the state of the tumor region based on the feature amount calculated by the feature amount calculation unit.

7. A medical image processing method for extracting a predetermined region from a diagnostic target image, comprising: An organ extraction step of extracting an organ region from the diagnostic target image; A tumor extraction step of extracting a tumor region from the diagnostic target image while using the organ region extracted from the diagnostic target image in the organ extraction step; A feature amount calculation step of calculating a feature amount related to the tumor region extracted in the tumor extraction step, and The feature amount calculated in the feature amount calculation step includes a tumor-organ feature amount that is a feature amount representing the relationship between the organ region extracted in the organ extraction step and the tumor region extracted in the tumor extraction step. A medical image processing method characterized by this.

Images