Medical image processing device, method of operating the medical image processing device, and program

Abstract

To provide a medical image processing apparatus, a method of operating the medical image processing apparatus and a program that achieve stable performance in characteristics analysis of a medical image using contrast state.SOLUTION: A medical image (IIN) generated by performing contrast radiography is acquired (P1), a contrast state of the medical image is estimated (P2) based on analysis of the medical image, and characteristics analysis of a region of interest included in the medical image is executed (P3) based on characteristics in the region of interest using contrast state information (INF) representing the contrast state of the medical image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a medical image processing apparatus, a method of operating the medical image processing apparatus, and a program. [Background technology]

[0002] A known imaging technique called dynamic contrast-enhanced CT combines angiography using contrast agents with X-ray CT. For example, in dynamic contrast-enhanced CT of the liver, multiple scans are performed with different contrast-enhancing phases while injecting a contrast agent to acquire CT images, and the changes in the degree of enhancement of lesions in the CT images are observed. Note that CT is an abbreviation for Computed Tomography.

[0003] Patent Document 1 describes an image discrimination device that automatically determines whether an image is contrast-enhanced or non-contrast-enhanced. The device described in the document detects a first region from acquired image data that is not affected by the contrast agent, identifies a second region that is in a predetermined relative position to the first region and is affected by the contrast agent, and determines whether the image data was acquired using contrast imaging based on whether the CT value of the second region is higher than a predetermined value.

[0004] Non-patent document 1 describes a deep learning model applied to the classification of liver tumor lesions. The deep learning model described in this document takes images in which the tumor region has been cropped from images with known contrast-enhanced phases (non-contrast, arterial phase, and equilibrium phase) as input images, and outputs classifications of five lesions: classical hepatocellular carcinoma, malignant tumors other than hepatocellular carcinoma, benign tumors, hemangiomas, and cysts. Note that the benign tumor category includes images where it is not possible to distinguish between benign and malignant tumors. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2011-136030 [Non-patent literature]

[0006] [Non-Patent Document 1] Yasaka K, Akai H, Abe O, Kiryu S. Deep Learning with Convolutional Neural Network for Differentiation of Liver Masses at Dynamic Contrast-enhanced CT: A Preliminary Study. Radiology. 2018 Mar;286(3):887-896. doi: 10.1148 / radiol.2017170706. Epub 2017 Oct 23. PMID: 29059036. [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] However, even if images are taken at the same time after the start of contrast agent injection, the contrast enhancement may differ depending on the subject's physique and physical condition. In such cases, when determining the contrast phase using information on the start time of contrast agent injection included in the metadata and performing characterization analysis, a decrease in the performance of the characterization analysis may occur due to variability in the contrast phase.

[0008] Furthermore, if the metadata does not include information on the start time of contrast agent injection, the characterization analysis may be performed without determining the contrast phase.

[0009] The system described in Patent Document 1 determines whether the acquired image data was obtained using contrast imaging, but it is difficult to determine the contrast-enhanced phase of the captured image.

[0010] The method described in Non-Patent Document 1 assumes that the contrast-enhancing phase is known. Furthermore, it cannot address the variability in the contrast-enhancing phase influenced by factors such as the subject's physique, or the lack of information regarding the identification of the contrast-enhancing phase.

[0011] This invention has been made in view of these circumstances, and aims to provide a medical image processing apparatus, an operating method for the medical image processing apparatus, and a program that achieve stable performance in the characterization of medical images using contrast-enhanced states. [Means for solving the problem]

[0012] The medical image processing apparatus according to the first embodiment comprises one or more processors and one or more memories that store programs to be executed by the one or more processors, wherein the one or more processors execute program instructions to acquire a medical image generated by performing contrast imaging, estimate the contrast enhancement state of the medical image based on the analysis of the medical image, and perform a characteristic analysis of the region of interest contained in the medical image using contrast enhancement state information representing the contrast enhancement state of the medical image.

[0013] According to the medical image processing apparatus of the first embodiment, the contrast enhancement state is estimated by image analysis of the medical image generated by performing contrast-enhanced imaging, and the characteristics of the region of interest contained in the medical image are analyzed using the estimated contrast enhancement state. This suppresses the performance degradation of the characteristics analysis caused by variability in the contrast enhancement state, and enables stable performance in the characteristics analysis of medical images using the contrast enhancement state. Furthermore, even if the medical image does not contain information that identifies the contrast enhancement state, the characteristics analysis of the medical image using the contrast enhancement state can be performed.

[0014] An example of estimating the contrast-enhanced state is the estimation of the contrast-enhanced phase. The contrast-enhanced state may include non-contrast-enhanced areas.

[0015] In the second embodiment, in the medical image processing apparatus of the first embodiment, one or more processors select medical images using contrast enhancement information.

[0016] According to this embodiment, medical images corresponding to contrast enhancement states suitable for characterization can be selected.

[0017] In the third embodiment, in the medical image processing apparatus of the second embodiment, one or more processors select medical images having contrast enhancement state information suitable for characterization analysis.

[0018] According to this embodiment, the application of medical images unsuitable for characterization analysis to characterization analysis is suppressed. This allows for stable performance of characterization analysis.

[0019] A fourth aspect is a medical image processing apparatus according to the second or third aspect, in which one or more processors select medical images according to limitations on the input images in the characterization analysis.

[0020] According to this embodiment, it is possible to select input images for property analysis that correspond to the limitations of input images in property analysis.

[0021] The fifth embodiment is a medical image processing apparatus according to any one embodiment of the second to fourth embodiments, in which one or more processors select medical images for each contrast enhancement state information corresponding to two or more predetermined contrast enhancement states.

[0022] According to this embodiment, even when the contrast enhancement state suitable for characterization analysis is limited, it is possible to select medical images corresponding to the contrast enhancement state suitable for characterization analysis.

[0023] The sixth embodiment is a medical image processing apparatus according to any one embodiment of the second to fifth embodiments, in which one or more processors select medical images having contrast-enhanced state information corresponding to contrast-enhanced states excluding non-contrast-enhanced states.

[0024] According to this embodiment, even when a characterization analysis that is not suitable for non-contrast imaging is applied, it is possible to select medical images that correspond to a contrast-enhanced state suitable for characterization analysis.

[0025] The seventh embodiment is a medical image processing apparatus according to any one embodiment of the first to sixth embodiments, wherein one or more processors extract regions of interest from acquired medical images and perform characterization analysis of the regions of interest.

[0026] According to this embodiment, even when a medical image in which the region of interest has not been extracted is obtained, characterization analysis of the region of interest can be performed.

[0027] The eighth aspect is a medical image processing apparatus according to any one of the first to seventh aspects, in which one or more processors estimate the contrast enhancement state of the acquired medical image using a trained model.

[0028] According to this configuration, an improvement in the accuracy of estimating the contrast enhancement state is expected.

[0029] Examples of pre-trained learning models include deep learning models such as neural networks.

[0030] The ninth embodiment is a medical image processing apparatus according to any one embodiment of the first to eighth embodiments, wherein one or more processors perform characterization of regions of interest contained in acquired medical images using a trained model.

[0031] According to this embodiment, the accuracy of property analysis can be improved.

[0032] The tenth embodiment is a medical image processing apparatus according to the ninth embodiment, in which one or more processors use a trained model to extract feature quantities from each of the regions of interest contained in the medical image for each contrast-enhancing state, and perform a characterization analysis of the regions of interest contained in the medical image based on feature data in which the feature quantities of the regions of interest for each contrast-enhancing state are concatenated.

[0033] According to this embodiment, characterization analysis can be performed that takes into account the characteristics of the region of interest included in medical images under multiple contrast enhancement conditions.

[0034] The 11th embodiment is a medical image processing apparatus of the 10th embodiment in which one or more processors use a feature extraction model as a trained learning model to extract features from each of the regions of interest contained in the medical image for each contrast-enhanced state, concatenate the features of the regions of interest contained in the medical image for each contrast-enhanced state, and perform character analysis of the regions of interest contained in the medical image using a classification model as a trained learning model to classify the feature data into which the features of the regions of interest contained in the medical image for each contrast-enhanced state have been concatenated.

[0035] According to this embodiment, feature extraction and characteristic analysis can be performed using individual pre-trained models for each process.

[0036] The twelfth embodiment is a medical image processing apparatus according to the eleventh embodiment, wherein one or more processors average a portion of the feature quantities of the region of interest included in the medical image for each contrast enhancement state.

[0037] According to this embodiment, the processing load is reduced when concatenating feature quantities of regions of interest contained in medical images for each contrast enhancement state.

[0038] The 13th embodiment is a medical image processing apparatus according to the 12th embodiment, in which one or more processors calculate weights for each of the features of the region of interest contained in the medical image for each contrast-enhanced state, using a weight calculation model as a trained learning model, which calculates weights used when calculating a weighted average of the features of the region of interest contained in the medical image for each contrast-enhanced state.

[0039] According to this embodiment, it is possible to estimate the contribution of the region of interest contained in the medical image for each contrast enhancement state in the characterization analysis.

[0040] The 14th embodiment is a medical image processing apparatus in the 10th embodiment or the 11th embodiment, in which, when a portion of the region of interest contained in the medical image for each contrast-enhanced state is missing, one or more processors use a region of interest contained in the medical image for a contrast-enhanced state that has similar features to the region of interest contained in the medical image for the contrast-enhanced state that is missing.

[0041] According to this embodiment, even if the region of interest is missing in some contrast-enhanced states, it is possible to perform characterization analysis of the region of interest included in the medical image based on the contrast-enhanced state.

[0042] The operation method of a medical image processing device according to the 15th embodiment is an operation method of a medical image processing device to which a computer is applied, wherein the medical image processing device performs the steps of: acquiring a medical image generated by performing contrast imaging; estimating the contrast state of the medical image based on the analysis of the medical image; and performing a characteristic analysis of a region of interest contained in the medical image using contrast state information representing the contrast state of the medical image.

[0043] According to the operating method of the medical image processing apparatus according to the 15th embodiment, it is possible to obtain the same effects as those of the medical image processing apparatus according to the present disclosure. The constituent elements of the medical image processing apparatus according to the other embodiments may be applied to the constituent elements of the operating method of the medical image processing apparatus according to the other embodiments.

[0044] The program according to the 16th embodiment is a program that enables a computer to perform the following functions: acquiring a medical image generated by performing contrast imaging; estimating the contrast enhancement state of the medical image based on the analysis of the medical image; and performing a characteristic analysis of a region of interest contained in the medical image using contrast enhancement state information representing the contrast enhancement state of the medical image.

[0045] According to the program of the 16th embodiment, it is possible to obtain the same effects as the medical image processing apparatus of this disclosure. The constituent elements of the medical image processing apparatus of the other embodiments may be applied to the constituent elements of the program of the other embodiments. [Effects of the Invention]

[0046] According to the present invention, the contrast enhancement state is estimated by image analysis of the medical image generated by contrast-enhanced imaging, and the characteristics of the region of interest contained in the medical image are analyzed using the estimated contrast enhancement state. This suppresses the degradation of the characteristics analysis performance caused by variability in the contrast enhancement state, and enables stable performance in the characteristics analysis of medical images using the contrast enhancement state. Furthermore, even if the medical image does not contain information that identifies the contrast enhancement state, the characteristics analysis of the medical image using the contrast enhancement state can be performed. [Brief explanation of the drawing]

[0047] [Figure 1] Figure 1 is an explanatory diagram of dynamic contrast-enhanced CT. [Figure 2] Figure 2 is a graph showing the relationship between the contrast-enhanced phase and CT values. [Figure 3] Figure 3 is a conceptual diagram showing an overview of the process applied to the property analysis apparatus according to the first embodiment. [Figure 4] Figure 4 is a functional block diagram showing an overview of the processing functions of the property analysis apparatus according to the first embodiment. [Figure 5] Figure 5 is a schematic block diagram showing an example of the hardware configuration of the property analysis apparatus according to the first embodiment. [Figure 6] Figure 6 is a flowchart showing the procedure for the property analysis method according to the first embodiment. [Figure 7] Figure 7 is a conceptual diagram showing an overview of the process applied to the property analysis apparatus according to the second embodiment. [Figure 8] Figure 8 is a functional block diagram showing an overview of the processing functions of the property analysis apparatus according to the second embodiment. [Figure 9] Figure 9 is a schematic block diagram showing an example of the hardware configuration of a property analysis apparatus according to the second embodiment. [Figure 10] Figure 10 is a flowchart showing the procedure for the property analysis method according to the second embodiment. [Figure 11] Figure 11 is a conceptual diagram showing a specific example of the sorting process. [Figure 12] Figure 12 is a conceptual diagram showing an example of contrast-enhancement phase estimation. [Figure 13] Figure 13 is a conceptual diagram showing a specific example of the property analysis process. [Figure 14] Figure 14 is a conceptual diagram showing a modified example of the property analysis process shown in Figure 13. [Figure 15] Figure 15 is a conceptual diagram illustrating a specific example of characterization processing when some CT images are missing during the contrast-enhanced phase. [Figure 16] Figure 16 is a conceptual diagram showing another specific example of the property analysis process. [Figure 17] Figure 17 is a block diagram showing an example configuration of a medical information system that uses a property analysis device. [Modes for carrying out the invention]

[0048] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this specification, identical components are denoted by the same reference numerals, and redundant descriptions are omitted where appropriate.

[0049] [About Dynamic Contrast-Enhanced CT] Figure 1 is an explanatory diagram of dynamic contrast-enhanced CT. Figure 1 schematically illustrates CT images taken at different contrast-enhanced phases using dynamic contrast-enhanced CT. Dynamic contrast-enhanced CT is a technique in which a contrast agent, such as iodine contrast agent, is injected into a vein in the arm, and then the same area is repeatedly scanned multiple times to observe the changes in the CT images over time.

[0050] In other words, dynamic contrast-enhanced CT is a scanning method that involves scanning multiple organs at timings that take into account the hemodynamics of those organs. The timing of the scans is determined according to the organ. For example, when scanning the liver, the scans are performed at timings that take into account the hemodynamics of the liver. In addition, in dynamic contrast-enhanced CT, the scans may be performed before the injection of the contrast agent begins, and non-contrast CT images similar to those obtained in a simple CT scan may be acquired.

[0051] Figure 1 illustrates non-contrast CT image I1, arterial phase CT image I2, portal venous phase CT image I3, and equilibrium phase CT image I4. Figure 1 shows any single slice image from a group of slice images containing multiple slice images for each contrast-enhanced phase as a CT image.

[0052] The horizontal axis in Figure 1 represents the time axis, with the start time of contrast agent injection set to 0 seconds, and the unit of time is seconds. Figure 1 shows CT images of the arterial phase (I2) taken approximately 35 seconds after contrast agent injection, the portal venous phase (I3) taken approximately 80 seconds after contrast agent injection, and the equilibrium phase (I4) taken approximately 150 seconds after contrast agent injection.

[0053] Tumors appear differently in CT images due to differences in the contrast-enhancing phase. Therefore, accurate information on the contrast-enhancing phase is necessary for characterization analysis. In the example shown in Figure 1, enhancement is observed in CT image I2 during the arterial phase, which can lead to the characterization analysis result of early enhancement.

[0054] Here, the term "image" in this specification may include not only the meaning of the image itself, but also the meaning of image data, which is a signal representing the image. Furthermore, the term "injection start time" may be interpreted as "injection start timing."

[0055] Figure 2 is a graph showing the relationship between contrast-enhanced phase and CT values. The horizontal axis of the graph in Figure 2 is the time axis, and the unit of time is seconds. The vertical axis of the same figure is the CT value axis. Curve 1 shows the change in CT values ​​over time in arteries. Curve 2 shows the change in CT values ​​over time in portal veins. Curve 3 shows the change in CT values ​​over time in livers.

[0056] The contrast-enhanced phase refers to the state after a specific amount of time has elapsed since the injection of the contrast agent. In dynamic contrast-enhanced CT of the liver, the arterial phase, portal venous phase, and equilibrium phase are defined. For example, the arterial phase indicates a state in which the contrast agent is flowing mostly into the arteries.

[0057] When contrast agent is injected intravenously, it reaches the abdominal arteries 30 to 40 seconds after the start of injection. Period t1 shown in Figure 2 corresponds to the arterial phase. The contrast agent injected intravenously also reaches the portal vein 60 to 80 seconds after the start of injection. Period t2 shown in Figure 2 corresponds to the portal venous phase. Furthermore, the contrast agent injected intravenously reaches equilibrium in terms of contrast concentration between the intravascular and extracellular fluid 150 to 200 seconds after the start of injection. This state is the equilibrium phase. Period t3 shown in Figure 2 corresponds to the equilibrium phase.

[0058] Figure 2 illustrates the contrast-enhanced phases of the liver, but examples of contrast-enhanced phases for the kidney include the corticomedullary phase, parenchymal phase, and excretion phase. Similar to the liver, the relationship between the contrast-enhanced phase and the elapsed time from the start of contrast agent injection is also defined for the kidney.

[0059] [Challenges in analyzing characteristics] Analyzing the characteristics of CT images acquired using dynamic contrast-enhanced CT requires accurate information on the contrast-enhanced phase, but identifying the contrast-enhanced phase presents challenges such as missing metadata and differences in the physique of the subjects.

[0060] A missing metadata refers to the absence of information regarding the start time of contrast agent injection in the metadata. If the information regarding the start time of contrast agent injection is missing or corrupted in the metadata, it becomes difficult to determine the elapsed time from the start of contrast agent injection, and as a result, it becomes difficult to accurately determine the contrast phase.

[0061] Furthermore, differences in the physique of the subjects mean that the spread of the contrast agent differs due to differences in body size and heart rate among subjects, and even if the start time of contrast agent injection is known, it is not possible to accurately determine the contrast phase based on the elapsed time from the start time of contrast agent injection.

[0062] For example, the greater the volume of blood circulating per unit time, such as cardiac output which represents the volume of blood pumped out per unit time, the faster the contrast agent reaches the target object, the lower the maximum CT value, and the faster the time to reach the maximum CT value.

[0063] However, it is difficult to predict the blood circulation rate per unit time in advance, and during the examination, the blood circulation rate per unit time may increase compared to when the subject is at rest due to factors such as anxiety. In this way, variations in the blood circulation rate per unit time can also lead to variations in the movement time of the contrast agent.

[0064] The following describes a property analysis device that acquires accurate information on the contrast phase, which is less affected by variations in contrast agent migration time due to the subject's physique, etc., and which performs property analysis using the contrast phase information, without relying on information on the injection start time of the contrast agent in the metadata.

[0065] [Property analysis apparatus according to the first embodiment] Figure 3 is a conceptual diagram showing an overview of the process applied to the property analysis apparatus according to the first embodiment. In the process applied to the property analysis apparatus shown in the figure, the CT image acquisition process P1 acquires 3D CT data generated by applying dynamic contrast-enhanced CT to image acquisition.

[0066] Figure 3 shows how the 3D CT data acquired in the CT image acquisition process P1 is converted to a 2D CT image I. IN This is schematically illustrated. Below, CT image I IN The term can be interpreted as 3D CT data.

[0067] In CT image acquisition process P1, CT image I, where the contrast-enhanced phase is not specified, IN The CT image acquisition process P1 retrieves the CT image I from the memory device where medical images are stored. IN This is obtained.

[0068] Figure 3 shows four types of CT images taken before the start of contrast agent injection, t1 second after the start of contrast agent injection, t2 seconds after the start of contrast agent injection, and t3 seconds after the start of contrast agent injection, as shown in Figure 1. IN An example of how this can be obtained is shown. CT image I IN This is from the start of contrast agent injection to t iIt is any one CT image in a group of CT images including a plurality of CT images acquired after IN seconds. Here, i represents the number of shooting times in chronological order, and an integer of 1 or more is applicable. Further, the CT image I shown in FIG. 3

[0069] In the contrast phase estimation process P2, for each of the CT images I acquired in the CT image acquisition process P1 IN image analysis is performed, the contrast phase is estimated, and contrast phase information I IN is acquired for each CT image I NF Here, the image analysis may include the meaning of a process using pixel values of pixels constituting the image. Further, the acquisition of the contrast phase information I NF may include the meaning of the generation of the contrast phase information I NF

[0070] For the CT image I for which the contrast phase has been estimated IN the property analysis process P3 of the region of interest R NF is performed using the corresponding contrast phase information I OI and the information output process P4 is performed, and the analysis result A R is output. In FIG. 3, as the analysis result A of the property analysis R early enhancement, washout, and capsule are exemplified.

[0071] In the CT image acquisition process P1 shown in FIG. 3, any one slice image included in a plurality of slice images sampled at equal intervals from the three-dimensional CT data of the patient taken using a CT device may be acquired as the CT image I IN . Note that the slice image may be referred to as a tomographic image. That is, the slice image may be understood as a cross-sectional image that is substantially a two-dimensional image.

[0072] FIG. 4 is a functional block diagram showing an outline of the processing functions of the property analysis apparatus according to the first embodiment. The property analysis apparatus 10 can be realized using computer hardware and software.

[0073] The property analysis device 10 comprises a CT image acquisition unit 12, a contrast-enhancing phase estimation unit 14, a region of interest extraction unit 16, a property analysis unit 18, and an information output unit 20.

[0074] The CT image acquisition unit 12 acquires the CT image I shown in Figure 3. IN The CT image acquisition unit 12 acquires the CT image I IN For example, regarding a group of slice images that includes multiple slice images generated for each specified number of shots, one slice image may be acquired, or multiple slice images may be acquired.

[0075] The contrast-enhanced phase estimation unit 14 uses the CT image acquisition unit 12 to estimate the CT image I IN Image analysis was performed on the CT image I IN The contrast-enhancing phase is estimated. The contrast-enhancing phase estimation unit 14 estimates the CT image I IN R of the region of interest extracted from OI Image analysis was performed on the CT image I IN The contrast-enhancing phase may be estimated.

[0076] In other words, the contrast-enhanced phase estimation unit 14 estimates the CT image I IN For each of these, see the contrast-enhanced phase information I shown in Figure 3. NF Obtain contrast-enhanced phase information I NF CT image I IN The data is then sent to the property analysis unit 18 in correspondence with the CT image I. IN and contrast-enhanced phase information I NF You can also memorize them by associating them with each other.

[0077] Note: CT image I IN The contrast-enhanced phase refers to the CT image I IN This is synonymous with the contrast-enhanced phase of a group of slice images that include the slice image. Furthermore, the contrast-enhanced phase described in the embodiment is an example of the contrast-enhanced state. Contrast-enhanced phase information I described in the embodiment NF This is an example of contrast enhancement status information.

[0078] The region of interest extraction unit 16 extracts CT images I obtained using the CT image acquisition unit 12. IN From the R of the area of ​​interestOI Extracts the region of interest R. The region of interest extraction unit 16 extracts the region of interest R. OI This allows for the extraction of lesion regions, including tumors and other lesions. The region of interest extraction unit 16 processes the CT image I IN and the area of ​​interest R OI Using a pre-trained model that has learned the relationship with CT image I IN From the R of the area of ​​interest OI The region of interest extraction unit 16 extracts the CT image I specified by the user. IN Using the positional information in CT image I IN From the R of the area of ​​interest OI You may extract it.

[0079] Examples of learning models include deep learning models such as convolutional neural networks. Convolutional neural networks are abbreviated as CNNs.

[0080] The CT image acquisition unit 12 pre-selects the region of interest R OI Extracted CT image I IN If the region of interest R is obtained, the processing of the region of interest extraction unit 16 is omitted. OI Extracted CT image I IN When acquiring this, an embodiment without the region of interest extraction unit 16 is also possible.

[0081] The region of interest extraction unit 16 extracts CT image I IN Area of ​​interest R OI Obtain information representing the specified conditions, and the region of interest R OI Based on the specified conditions, CT image I IN From the R of the area of ​​interest OI You may extract it.

[0082] The property analysis unit 18 analyzes CT image I IN Contrast-enhancing phase information I NF Using CT image I IN R of the region of interest extracted from OI The property analysis will be performed. The property analysis unit 18 will analyze the area of ​​interest R OI and the area of ​​interest R OIA pre-trained model that has learned the relationship with the properties can be applied. A Convolutional Neural Network (CNN) is an example of a pre-trained model. The property analysis unit 18 to which the pre-trained model is applied is expected to show improved accuracy in property analysis.

[0083] The information output unit 20 outputs the analysis results of the property analysis performed using the property analysis unit 18. The information output unit 20 outputs the acquired CT image I IN The results of the property analysis may be associated with the data and stored in the memory unit.

[0084] The information output unit 20 outputs the CT image I to be processed. IN Area of ​​interest R OI It functions as an output interface that outputs information representing the characteristics of the CT image I IN Area of ​​interest R OI It may also function as an output interface that provides the properties of the object to other processing units.

[0085] The information output unit 20 may include at least one processing unit, such as a processing unit for generating data for display and a data conversion processing unit for transmitting data to an external source. The analysis results of the property analysis device 10 may be displayed using a display device or the like.

[0086] The material properties analyzer 10 may be incorporated into a medical image processing device for processing medical images acquired at a medical institution such as a hospital. Furthermore, the processing functions of the material properties analyzer 10 may be provided as a cloud service.

[0087] [Explanation of medical images used for input] In the DICOM standard, which defines the format and communication protocol for medical images, the series ID is defined within the study ID, which is an identification code used to identify the type of examination. ID is an abbreviation for identification. Furthermore, "medical image" is synonymous with "healthcare image."

[0088] For example, when performing contrast-enhanced imaging of a patient's liver using dynamic contrast-enhanced CT, multiple CT scans of the area including the liver are performed at different timings. For example, the first scan might be performed before contrast agent injection, the second scan 35 seconds after injection, the third scan 80 seconds after injection, and the fourth scan 150 seconds after injection.

[0089] These four scans are performed, yielding four types of CT data. CT data, in this context, is three-dimensional data composed of multiple consecutive slice images. A collection of these slice images, which constitutes the three-dimensional data, is called an image series.

[0090] The four types of CT data obtained from the series of scans, including the four scans described above, are each assigned the same study ID and a different series ID.

[0091] For example, a study ID of a liver contrast imaging examination in a specific patient might be assigned Study 1. The series ID for CT data obtained from imaging before contrast agent injection would be Series 1, Series 2 for CT data obtained 35 seconds after contrast agent injection, Series 3 for CT data obtained 80 seconds after contrast agent injection, Series 4 for CT data obtained 150 seconds after contrast agent injection, and so on, with each series being assigned a unique ID.

[0092] Therefore, CT data can be identified by combining the study ID and series ID. On the other hand, in actual CT data, the correspondence between the series ID and the timing of the scans is not always clearly understood. The timing of the scans referred to here can be interpreted as the elapsed time after contrast agent injection.

[0093] Furthermore, because 3D CT data is large in size, processing such as estimating the contrast-enhanced phase using CT data directly as input data can be difficult in terms of processing time and processing load. Therefore, the characterization analyzer 10 can use one or more slice images from the same image series as input and perform contrast-enhanced phase estimation based on image analysis.

[0094] [Example Hardware Configuration of a Property Analysis Device] Figure 5 is a schematic block diagram showing an example of the hardware configuration of a property analysis device according to the first embodiment. The property analysis device 10 can be implemented by a computer system consisting of one or more computers. Here, an example is shown in which one computer executes a program to realize various functions of the property analysis device 10.

[0095] The form of the computer that functions as the property analysis device 10 is not particularly limited; it may be a server computer, a workstation, a personal computer, or a tablet terminal. Furthermore, the computer may be a virtual machine.

[0096] The property analysis device 10 comprises a processor 30, a non-temporary tangible computer-readable medium 32, a communication interface 34, an input / output interface 36, and a bus 38. Note that IF in Figure 5 represents an interface.

[0097] The processor 30 includes a CPU (Central Processing Unit). The processor 30 may also include a GPU (Graphics Processing Unit). The processor 30 is connected to a computer-readable medium 32, a communication interface 34, and an input / output interface 36 via a bus 38. The processor 30 reads various programs and data stored in the computer-readable medium 32 and performs various processes.

[0098] The computer-readable medium 32 includes a main memory 40 and an auxiliary storage 42. The storage 42 can be configured using a hard disk drive, a solid-state drive, an optical disk, a magneto-optical disk, and a semiconductor memory. The storage 42 can be configured using an appropriate combination of hard disk drives, etc. Various programs and data are stored in the storage 42.

[0099] Furthermore, hard disk drives can be referred to as HDDs, an abbreviation of the English term Hard Disk Drive. Similarly, solid state drives can be referred to as SSDs, an abbreviation of the English term Solid State Drive.

[0100] Memory 40 is used as a workspace for the processor 30 and as a storage unit that temporarily stores programs and various data read from storage 42. Programs stored in storage 42 are loaded into memory 40, and program instructions are executed using the processor 30. The processor 30 functions as a processing unit that performs various operations defined by the program. Memory 40 stores the contrast-enhancing phase estimation program 50, the region of interest extraction program 52, the property analysis program 54, and various data, which are executed using the processor 30.

[0101] The contrast-enhancement phase estimation program 50 causes the processor 30 to execute the contrast-enhancement phase estimation process, which is performed using the contrast-enhancement phase estimation unit 14 shown in Figure 4. The contrast-enhancement phase estimation program 50 may include a pre-trained model.

[0102] The region of interest extraction program 52 causes the processor 30 to execute the region of interest extraction process performed by the region of interest extraction unit 16. The region of interest extraction program 52 may include a pre-trained model.

[0103] The property analysis program 54 causes the processor 30 to execute the property analysis process performed using the property analysis unit 18. The property analysis program 54 may include a trained model. Each program shown in Figure 5 contains one or more instructions. The processor 30 executes the instructions contained in each program to realize the function corresponding to each program.

[0104] The communication interface 34 performs communication processing with external devices using wired or wireless connections and exchanges information with external devices. The property analysis device 10 is connected to a communication line via the communication interface 34. The communication line may be a local area network or a wide area network. The communication interface 34 can also act as a data acquisition unit that accepts input data such as images. Note that the diagram of the communication line is omitted.

[0105] The property analysis device 10 may include an input device 60 and a display device 62. The input device 60 and the display device 62 are connected to the bus 38 via an input / output interface 36. Examples of the input device 60 include a keyboard, mouse, multi-touch panel, other pointing devices, and voice input devices. The input device 60 may be an appropriate combination of keyboards and the like.

[0106] The display device 62 is an output interface on which various types of information are displayed. Examples of the display device 62 include liquid crystal displays, organic EL displays, and projectors. The display device 62 may also be an appropriate combination of liquid crystal displays, etc. Organic EL is abbreviated as OEL, using the English term organic electro-luminescence.

[0107] [Procedure for analyzing the properties of the material] Figure 6 is a flowchart showing the procedure of the property analysis method according to the first embodiment. In the CT image acquisition step S10, the CT image acquisition unit 12 shown in Figure 4 acquires the CT image I shown in Figure 3. INThe CT image acquisition process S10 shown in Figure 6 corresponds to the CT image acquisition process P1 shown in Figure 3. After the CT image acquisition process S10, the process proceeds to the contrast phase estimation process S12.

[0108] In contrast-enhanced phase estimation step S12, the contrast-enhanced phase estimation unit 14 analyzes the CT image I IN Based on the image analysis, the acquired CT image I IN The contrast-enhancement phase is estimated. The contrast-enhancement phase estimation step S12 shown in Figure 6 corresponds to the contrast-enhancement phase estimation process P2 shown in Figure 3. After the contrast-enhancement phase estimation step S12, the process proceeds to the region of interest extraction step S14.

[0109] In the region of interest extraction process S14, the region of interest extraction unit 16 extracts CT image I IN From the R of the area of ​​interest OI The region of interest is extracted. After the region of interest extraction step S14, the process proceeds to the property analysis step S16. The region of interest extraction step S14 may be performed in parallel with the contrast-enhancement phase estimation step S12, or the order of the contrast-enhancement phase estimation step S12 may be reversed.

[0110] Furthermore, in the CT image acquisition process S10, the region of interest R is set in advance. OI CT image I from which the extracted information is extracted. IN If obtained, the region of interest extraction step S14 is omitted. ,sex Proceed to the condition analysis process S16.

[0111] In the property analysis process S16, the property analysis unit 18 analyzes the CT image I IN Area of ​​interest R OI A property analysis will be performed. In the property analysis step S16, CT image I IN The results of the property analysis are associated with the data, and the results of the property analysis may be stored. The property analysis process S16 shown in Figure 6 corresponds to the property analysis process P3 shown in Figure 3. After the property analysis process S16, the process proceeds to the information output process S18.

[0112] In the information output step S18, the information output unit 20 outputs the analysis results of the property analysis performed in the property analysis step S16. The output of the analysis results in the information output step S18 can be in a form that visualizes the analysis results, such as displaying them on a display device 62. The information output step S18 shown in Figure 6 corresponds to the information output process P4 shown in Figure 3. The procedure for the property analysis method is completed after the information output step S18.

[0113] After the information output process S18, the next CT image I IN The system waits for input and then proceeds to the next CT image I IN If this is input, each step from the CT image acquisition step S10 to the information output step S18 may be executed. After the information output step S18, the next CT image I IN The system waits for input and then, within the specified period, the next CT image I IN If no input is provided, the procedure for the property analysis method may be terminated.

[0114] The property analysis method described in the embodiment is an example of how a medical image processing device to which a computer is applied operates.

[0115] [Effects of the First Embodiment] The property analysis apparatus and property analysis method according to the first embodiment can achieve the following effects.

[0116] CT image I IN Image analysis was performed on CT image I IN The contrast-enhanced phase is estimated, and CT image I IN Contrast-enhancing phase information I NF Using CT image I IN Area of ​​interest R OI A characterization analysis is performed. This results in CT image I IN Even if information on the start time of contrast agent injection is missing in the metadata, the contrast phase can be estimated, and CT image I IN Area of ​​interest R OI The performance of characterization analysis using contrast-enhanced phases becomes more stable.

[0117] In addition, variations in the contrast phase due to differences in the physique of the subject are suppressed, and the performance of the property analysis using the contrast phase information I IN for the region of interest R OI in the CT image I NF is stabilized.

[0118] [Property Analysis Device According to the Second Embodiment] FIG. 7 is a conceptual diagram showing an overview of the processing applied to the property analysis device according to the second embodiment. Hereinafter, mainly the differences from the first embodiment will be described, and the description of matters common to the first embodiment will be omitted as appropriate.

[0119] The processing function of the property analysis device according to the second embodiment uses the contrast phase information I IN of the CT image I NF to perform a selection process P5 for selecting the CT image I IN that is the target of the property analysis process P3. In FIG. 7, the property analysis process P3 does not correspond to the non-contrast CT image I IN and the selection process P5 in which the non-contrast CT image I IN is excluded is shown.

[0120] In other words, as the target of the property analysis process P3 shown in FIG. 7, from the non-contrast CT image I IN , the arterial-phase CT image I IN , the portal-phase CT image I IN and the equilibrium-phase CT image I IN , the arterial-phase CT image I IN , the portal-phase CT image I IN and the equilibrium-phase CT image I IN are selected.

[0121] FIG. 8 is a functional block diagram showing an overview of the processing function of the property analysis device according to the second embodiment. The property analysis device 10A shown in FIG. 8 has a selection unit 22 added to the property analysis device 10 shown in FIG. 1.

[0122] The selection unit 22 uses the contrast phase information I IN for each CT image I NF estimated using the contrast phase estimation unit 14 to input the CT image IIN The sorting unit 22 sorts the CT image I based on the pre-set sorting conditions for the contrast-enhanced phase. IN The sorting unit 22 can select the CT image I. IN You may select them.

[0123] [Example Hardware Configuration of a Property Analysis Device] Figure 9 is a schematic block diagram showing an example of the hardware configuration of a property analysis apparatus according to the second embodiment. In the property analysis apparatus 10A shown in the figure, the memory 40A included in the computer-readable medium 32A stores the sorting program 56.

[0124] The selection program 56 is performed using the selection unit 22 shown in Figure 8 to collect contrast-enhanced phase information I NF The processor 30 is instructed to perform a sorting process based on the selected CT images I. IN Area of ​​interest R OI The processor 30 is instructed to perform a property analysis process on the data.

[0125] [Procedure for analyzing the properties of the material] Figure 10 is a flowchart showing the procedure for the property analysis method according to the second embodiment. The flowchart shown in this figure has a sorting step S13 added to the flowchart shown in Figure 6.

[0126] In other words, after the contrast-enhanced phase estimation process S12, the process proceeds to the sorting process S13. In the sorting process S13, the sorting unit 22 shown in Figure 8 is used to sort CT images I IN Contrast-enhancing time-phase information for each case I NF Using this, CT image I is applied to the property analysis step S16. IN The selected samples are then sorted. After the sorting step S13, the process proceeds to the region of interest extraction step S14. The procedure for the region of interest extraction step S14 is the same as that of the property analysis method according to the first embodiment, so its explanation is omitted here.

[0127] The sorting step S13 may be performed in a different order from the region of interest extraction step S14, or it may be performed in parallel with the region of interest extraction step S14. In other words, the sorting step S13 should be performed after the contrast-enhancement phase estimation step S12 and before the property analysis step S16.

[0128] In the sorting process S13, CT image I IN If sorting is performed, in the information output step S18, the information output unit 20 may output the sorting results from the sorting step S13 when outputting the analysis results of the property analysis. The sorting results output in the information output step S18 may be displayed on the display device 62.

[0129] [Specific examples of sorting processes] Figure 11 is a conceptual diagram showing a specific example of the sorting process. Sorting process P shown in Figure 11 51 CT image I IN Contrast-enhancing time-phase information for each case I NF Using this, a CT image I consisting of a predetermined set of contrast-enhanced phases is created. IN Select the sets. In other words, selection process P 51 This refers to contrast-enhancing phase information I, which corresponds to each of two or more predetermined contrast-enhancing states. NF Each CT image I IN Select the following. Note that the contrast-enhancing phase information I described in the embodiment NF This is an example of information for each contrast enhancement state.

[0130] Figure 11 shows the CT image I to which the property analysis process P3 shown in Figure 7 is applied. IN It is limited to two types, and CT image I IN The contrast-enhanced phase of the property analysis process P is limited. 31 This is an example. Figure 11 shows the property analysis process P. 31 Examples of combinations of input 1 and input 2 include combinations of the arterial phase and the equilibrium phase, and combinations of the arterial phase and the portal venous phase.

[0131] Figure 11 shows different contrast-enhancing time-phase information I. NF Two types of CT images I IN Selection process P in which the selected items are selected. 51As an example, the same contrast-enhancing phase information I is sent to both Input 1 and Input 2. NF CT image type I IN This may be entered.

[0132] The sorting process P5 shown in Figure 7 includes contrast-enhanced phase information I, which includes non-contrast-enhanced information. NF CT image I obtained IN One or more types of CT images I IN You may select the non-contrast-enhanced contrast-enhanced phase information I NF CT image I obtained IN One or more types of CT images I IN You may select them.

[0133] [Specific examples of contrast-enhanced phase estimation] Figure 12 is a conceptual diagram showing an example of contrast-enhancement phase estimation. The figure shows the contrast-enhancement phase estimation process P. 21 In the trained model L M The trained model L is applied. M As an example, we will use 3DCNN. 3DCNN combines 3D spatial information and performs 3D convolution. Pre-trained model L M The contrast-enhanced phase estimation to which this is applied is expected to show improved accuracy. Note that 3D in 3DCNN refers to three dimensions.

[0134] Contrast-enhanced phase estimation process P 21 So, CT image I IN is the learning model L M When input is entered, information on the class representing the contrast-enhanced phase and the probability p for each class is derived, and the contrast-enhanced phase with the highest probability p is estimated result E. R It will be output as follows.

[0135] Figure 12 shows the derives of probability p for each class: non-contrast-enhanced (p) is 0.03, arterial phase (p) is 0.87, portal venous phase (p) is 0.06, and equilibrium phase (p) is 0.04, based on CT image I IN Estimated result of contrast-enhancing phase E R As such, the learning model L outputs the arterial phase. M To give an example:

[0136] Note that the contrast-enhanced phase estimation process P shown in Figure 12 is also shown. 21 The 3DCNN applied is an example, and the contrast-enhanced phase estimation process P2 shown in Figure 3, etc., uses the trained learning model L. M Therefore, any classification model can be applied.

[0137] Furthermore, the time phase estimation of contrast enhancement is performed using CT image I IN Image analysis is performed for each CT image I IN The ground truth value, which is the elapsed time from the start of contrast agent injection for each image, is estimated, and the relationship between the elapsed time from the start of contrast agent injection and the contrast phase is defined in a table, and CT image I IN The contrast-enhancing phase may be estimated for each individual contrast-enhancing phase. The specific example of contrast-enhancing phase estimation described with reference to Figure 12 is also applicable to the first embodiment.

[0138] [Effects of the second embodiment] The property analysis apparatus and property analysis method according to the second embodiment can achieve the following effects.

[0139] [1] The property analysis device 10A uses CT image I IN Contrast-enhancing time-phase information for each case I NF Using this, the CT image I that matches the input of the property analysis unit 18 IN It includes a sorting unit 22 for sorting out CT images I that do not conform to the input of the property analysis unit 18. IN The input to the property analysis unit 18 is suppressed, and the performance of the property analysis in the property analysis unit 18 can be stabilized.

[0140] [2] The sorting unit 22 receives non-contrast-enhanced contrast-enhanced phase information I NF CT image I IN CT image I to exclude IN This sorts the CT images I that do not match the input of the property analysis unit 18 when the property analysis process P3, which is not compatible with non-contrast imaging, is performed. IN The input to the property analysis unit 18 is suppressed, and the performance of the property analysis in the property analysis unit 18 can be stabilized.

[0141] [3] The sorting unit 22 uses predefined contrast-enhancing time-phase information I NF CT image I IN This selects the CT image I that does not match the input of the characterization analysis unit 18 when the characterization analysis process P3 is performed, which has a limited number of contrast-enhanced phases that match the input. IN The input to the property analysis unit 18 is suppressed, and the performance of the property analysis in the property analysis unit 18 can be stabilized.

[0142] [4] The sorting unit 22 uses two types of contrast-enhancing time-phase information I that are predetermined. NF Two types of CT images I each have IN Select a set of these. This will result in two types of CT images corresponding to the two specified contrast-enhanced phases. IN The following is input to the properties analysis unit 18, and the CT image I that does not conform to the input of the properties analysis unit 18 IN The input to the property analysis unit 18 is suppressed, and the performance of the property analysis in the property analysis unit 18 can be stabilized.

[0143] [First modified example of the input image from the property analysis device] In the first and second embodiments, a 3D image, which is 3D CT data, was used as input. However, slice images obtained by cutting slices at equal intervals from the 3D CT data may also be used as input.

[0144] Alternatively, instead of slice images, MIP images configured at equal intervals and average images generated from multiple slice images may be used. MIP is an abbreviation for Maximum Intensity Projection.

[0145] [Second variation of the input image from the property analysis device] The input to the CT image acquisition unit 12, as shown in Figure 6, may be a combination of multiple data elements. For example, at least one of the following can be used as input: a 3D image, a slice image, a MIP image, and an average image, which are partial images of CT data from the same image series. A combination of these multiple image types can be input to the CT image acquisition unit 12, and output can be obtained from the information output unit 20.

[0146] For example, a combination of the average image and the MIP image may be input to the CT image acquisition unit 12, and output may be obtained from the information output unit 20. Here, a 3D image refers to a set of multiple slice images.

[0147] In this embodiment, dynamic contrast-enhanced CT is used as an example of dynamic contrast enhancement, but the characterization analysis shown in this embodiment can also be applied to modalities other than CT to which dynamic contrast enhancement is applicable, such as dynamic contrast-enhanced MRI.

[0148] [Specific examples of property analysis processing] Figure 13 is a conceptual diagram showing a specific example of the property analysis process. In the property analysis process shown in Figure 13, the acquired CT images for each contrast-enhanced phase are processed. IN R of the region of interest extracted from OI Extracting feature quantities from CT images for each contrast-enhanced phase. IN R of the region of interest extracted from OI The feature quantities are concatenated, and a property classification process is performed as a property analysis. This allows for a property analysis process that takes into account the feature quantities of multiple contrast-enhanced phases.

[0149] The property analysis apparatus used to perform the property analysis process shown in Figure 13 is the property analysis apparatus 10 shown in Figure 4, and the property analysis section 18 performs the property analysis process shown in Figure 13.

[0150] First, CT image acquisition process P1 is performed, and CT image I IN Next, contrast-enhanced phase estimation processing P2 is performed, and the CT image I is obtained. IN The contrast-enhanced phase is estimated, and CT image I IN Contrast-enhancing time-phase information for each case I NFThis is obtained. Also, CT image I IN From the R of the area of ​​interest OI The following is extracted. The processing up to this point is the same as that performed for the CT image acquisition process P1 and the contrast-enhanced phase estimation process P2 shown in Figure 3, etc.

[0151] Area of ​​interest R OI CT image I from which the extract was not obtained IN If obtained, CT image I IN From the R of the area of ​​interest OI This is extracted. Region of interest R OI Extracted CT image I IN If obtained, CT image I IN From the R of the area of ​​interest OI The process of extracting this information is not performed.

[0152] Next, feature extraction process P 101 The contrast-enhanced phase was estimated from CT image I. IN The respective areas of interest R OI Therefore, using the feature extraction network 100, CT image I IN Each feature is extracted.

[0153] The feature extraction network 100 outputs a feature vector 102 representing non-contrast features, a feature vector 104 representing arterial phase features, a feature vector 106 representing portal venous phase features, and a feature vector 108 representing equilibrium phase features.

[0154] Feature vectors 102, 104, 106, and 108 are to which one-dimensional feature vectors are applied. That is, feature extraction process P 101 The feature extraction network 100 uses the input region of interest R OI This process outputs the same number of feature vectors (102) as the number of elements.

[0155] Next, the concatenation process P 102 This process is performed, and feature vectors 102, 104, 106, and 108 are concatenated to generate feature data 110.

[0156] Furthermore, property analysis process P 103 The classification network 120 performs characteristic classification processing based on the feature data 110. Feature data 110 Property analysis processing P based on 103 This corresponds to the property analysis process P3 shown in Figure 3.

[0157] Furthermore, information output processing P 104 The classification network 120 performed the analysis and determined the classification result of the characteristics. R The output is as follows. The feature extraction network 100 and the classification network 120 can be modified to apply neural networks.

[0158] The characterization process shown in Figure 13 involves the region of interest R in multiple contrast-enhanced phases. OI A characteristic analysis is performed that takes these features into consideration. Furthermore, contrast-enhancing phase estimation, feature extraction, and characteristic classification processes are carried out, and each of the following areas of interest R is considered. OI After feature vectors are extracted, each feature vector is concatenated, and the concatenated feature data 110 is used to analyze the region of interest R OI The configuration in which this classification is implemented can reduce the effects of positional displacement between contrast-enhanced phases, as well as the effects of patient respiration and patient movement.

[0159] Figure 13 shows the non-contrast region of interest R OI , Arterial phase region of interest R OI , portal venous phase area of ​​interest R OI and the region of interest R of the equilibrium phase OI Although an example was given using four feature extraction networks 100 for each, a single feature extraction network 100 may be used in common. When comparing multiple feature vectors in a common feature space, it is desirable to use a single feature extraction network.

[0160] The feature extraction network 100 and classification network 120 shown in Figure 13 may be components that make up a single trained model. For example, the network that performs feature extraction and the network that performs property analysis can be integrated to form a model with a region of interest R OI This can form a single network for performing characteristic analysis.

[0161] Figure 14 is a conceptual diagram showing a modified version of the property analysis process shown in Figure 13. In the property analysis process shown in Figure 14, some of the contrast-enhanced phase features are averaged compared to the property analysis process shown in Figure 13.

[0162] Specifically, in the property analysis process shown in Figure 14, the feature extraction process P is applied to the property analysis process shown in Figure 13. 101 Following this, an averaging process P generates a feature vector 109 by averaging the portal phase feature vector 106 and the equilibrium phase feature vector 108. 105 This will be added.

[0163] Area of ​​interest R in the portal venous phase OI Features in and the region of interest of the equilibrium phase R OI Since the features are similar, the two feature vectors are averaged and combined into one. This can reduce the number of feature vectors when generating feature data 111. Figure 14 shows an example of a CT image obtained by performing dynamic contrast-enhanced CT of the liver, but similar processing can be applied to other organs as well, as long as the contrast-enhanced phase has similar features.

[0164] Figure 15 is a conceptual diagram showing a specific example of characterization processing when some CT images are missing during the contrast-enhanced phase. The figure shows CT images I in the equilibrium phase. IN Let's look at an example of when it's missing.

[0165] The linking process P shown in the figure. 102 So, equilibrium phase This represents the characteristics of Since feature vector 108 is missing, feature vector 106, which represents the features of the portal phase, is treated as a feature vector obtained by averaging feature vector 106, which represents the features of the portal phase, and feature vector 108 in the equilibrium phase.

[0166] In other words, the linking process P shown in the figure. 102 Then, the feature vector 102 representing the non-contrast features, the feature vector 104 representing the arterial phase features, and the feature vector 106 representing the portal venous phase features are concatenated to generate feature data 111A.

[0167] The characterization process shown in Figure 15 is performed on CT images I in some contrast-enhanced phases. IN Even if the data is missing, a characterization analysis can be performed based on the estimated results of the contrast-enhanced phase. The figure shows the equilibrium phase CT image I IN Although an example of a case where it is missing was given, the portal venous phase CT image I IN CT images of the equilibrium phase I IN Other CT images I IN It is acceptable if it is missing.

[0168] Figure 16 is a conceptual diagram showing another specific example of the property analysis process. In the property analysis process shown in this figure, the region of interest R for each contrast-enhancing phase is defined. OI Extract the feature quantities and define the region of interest R for each contrast-enhancing phase. OI The features are weighted and averaged, and then the characteristic classification process is performed.

[0169] Specifically, the region of interest R for each contrast-enhancing phase. OI Weight W is applied when weighted averaging of features. ei It includes a weight calculation network 130 that calculates the weights. A neural network is applied to the weight calculation network 130.

[0170] The weight calculation network 130 calculates the region of interest R for each contrast-enhancing phase. OI Weight W represents how much each feature contributed to the classification of the characteristics. ei Weight calculation process P that calculates the weights 106 The following will be performed: Weight calculation process P 106 So, we'll assign weights W to each feature separately. ei You may also calculate this.

[0171] The weight calculation network 130 calculates the region of interest R for each contrast-enhancing phase. OIWhen the features are input, the region of interest R OI Dynamic weights W for features ei The output is shown. Note that the region of interest R for each contrast-enhancing phase is shown in Figure 16. OI Weights W for each feature ei This is an example, and any number is shown in the diagram.

[0172] Weighted averaging P 112 So, what is the region of interest R for each contrast-enhancing phase? OI Weight W for each feature ei Weighted averaging is performed using the following: Characteristic analysis process P 103 So, the classification network 120 has a region of interest R for each contrast-enhancing phase. OI The characteristic classification process is performed based on the weighted average of the feature quantities. Information output process P 104 Now, analysis result A R The output is performed.

[0173] The feature extraction network 100, the weight calculation network 130, and the classification network 120 are used to estimate the contrast-enhancing time phase in the region of interest R OI The pair of the characteristic classification result can be used as training data, and training can be performed all at once. Backpropagation can be applied to the training algorithms of the feature extraction network 100, the weight calculation network 130, and the classification network 120.

[0174] In other words, the feature extraction network 100, the weight calculation network 130, and the classification network 120 perform learning that minimizes the output loss of the classification network 120 for each input.

[0175] For the training of the feature extraction network 100 and classification network 120 shown in Figures 13 to 15, backpropagation can be applied as the training algorithm to perform training that minimizes the output loss of the classification network 120 for each input.

[0176] Note that the feature extraction network 100 described in the embodiment is an example of a feature extraction model. The classification network 120 described in the embodiment is an example of a property analysis model. The weight calculation network 130 described in the embodiment is an example of a weight calculation model.

[0177] [Example configuration of a medical imaging system] Figure 17 is a block diagram showing an example configuration of a medical information system in which a property analysis device is used. The property analysis device 10 and other components described in the first and second embodiments can be incorporated into the medical image processing device 220 shown in Figure 17.

[0178] The medical information system 200 is a computer network built in a medical institution such as a hospital. The medical information system 200 includes a modality 230 for capturing medical images, a DICOM server 240, a medical image processing device 220, an electronic medical record system 244, and a viewer terminal 246. The elements of the medical information system 200 are connected via a communication line 248. The communication line 248 may be an in-house communication line within the medical institution. In addition, a portion of the communication line 248 may be a wide-area communication line.

[0179] Specific examples of modality 230 include CT scanners 231, MRI scanners 232, ultrasound diagnostic equipment 233, PET scanners 234, X-ray diagnostic equipment 235, X-ray fluoroscopy diagnostic equipment 236, and endoscopes 237. The types of modalities 230 connected to the communication line 248 can vary from one medical institution to another. Note that MRI is an abbreviation for Magnetic Resonance Imaging, and PET is an abbreviation for Positron Emission Tomography.

[0180] The DICOM server 240 is a server that operates in accordance with the DICOM specification. The DICOM server 240 is a computer that stores and manages various types of data, including images captured using modality 230. The DICOM server 240 is equipped with a large-capacity external storage device and a database management program.

[0181] The DICOM server 240 communicates with other devices via the communication line 248 to send and receive various types of data, including image data. The DICOM server 240 receives image data and other data generated using modality 230 via the communication line 248, and stores and manages them on a recording medium such as a large-capacity external storage device. The storage format of the image data and communication between devices via the communication line 248 are based on the DICOM protocol.

[0182] The medical image processing device 220 can acquire data from a DICOM server 240 or the like via a communication line 248. The medical image processing device 220 performs image analysis and other various processing on medical images taken using modality 230. In addition to the processing functions of the characterization analyzer 10, the medical image processing device 220 may be configured to perform various computer-aided diagnostic analysis processes, such as processing to recognize lesion areas from images, processing to identify classifications such as disease names, and segmentation processing to recognize areas of organs, etc. Computer-aided diagnosis may be referred to as CAD, using the abbreviations Computer Aided Diagnosis or Computer Aided Detection.

[0183] Furthermore, the medical image processing device 220 can send processing results to the DICOM server 240 and the viewer terminal 246. The processing functions of the medical image processing device 220 may be implemented in either the DICOM server 240 or the viewer terminal 246.

[0184] Various information, including data stored in the database of the DICOM server 240 and processing results generated by the medical image processing device 220, can be displayed on the viewer terminal 246.

[0185] The viewer terminal 246 is a terminal for image viewing called a PACS viewer or a DICOM viewer. A plurality of viewer terminals 246 can be connected to the communication line 248. Note that PACS is an abbreviation for Picture Archiving and Communication System. The form of the viewer terminal 246 is not particularly limited and may be a personal computer, a workstation, or a tablet terminal, etc.

[0186] [Regarding the program for operating the computer] The processing functions in the property analysis device 10 etc. of It is possible to record the program to be realized on a computer-readable medium which is a tangible non-temporary information storage medium such as an optical disk, a magnetic disk, and a semiconductor memory, and provide the program through this information storage medium.

[0187] Instead of the mode of storing and providing the program in such a tangible non-temporary computer-readable medium, it is also possible to provide the program signal as a download service using a telecommunication line such as the Internet.

[0188] Furthermore, part or all of the processing functions in the property analysis device 10 etc. may be realized by cloud computing, and it is also possible to provide them as a SasS service. Note that SasS is an abbreviation for Software as a Service.

[0189] [Regarding the hardware configuration of each processing unit] The hardware structure of the processing unit that executes various processes such as the CT image acquisition unit 12, the contrast phase estimation unit 14, the region of interest extraction unit 16, the property analysis unit 18, and the information output unit 20 in the property analysis device 10 etc. is, for example, various processors as shown below.

[0190] Various processors include a CPU, which is a general-purpose processor that executes programs and functions as various processing units, a GPU, which is a processor specialized in image processing, and a processor such as an FPGA (Field Programmable Gate Array) whose circuit configuration can be changed after manufacture. It also includes dedicated electric circuits and the like, which are processors having a circuit configuration specifically designed to execute specific processes such as programmable logic devices and ASICs.

[0191] Note that a programmable logic device can be referred to as a PLD, which is an abbreviation of Programmable Logic Device in English notation. ASIC is an abbreviation of Application Specific Integrated Circuit.

[0192] One processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types. For example, one processing unit may be composed of a plurality of FPGAs, or may be composed of a combination of a CPU and an FPGA, or a combination of a CPU and a GPU.

[0193] Also, a plurality of processing units may be composed of one processor. Examples of composing a plurality of processing units with one processor include, firstly, a form in which one processor is composed of a combination of one or more CPUs and software, as represented by computers such as clients and servers, and this processor functions as a plurality of processing units. Secondly, there is a form in which a processor that realizes the functions of an entire system including a plurality of processing units with one IC chip is used, as represented by a system on a chip. Note that a system on a chip can be referred to as an SoC, which is an abbreviation of System On a Chip. IC is an abbreviation of Integrated Circuit.

[0194] Thus, each processing unit is configured using one or more of the above-mentioned processors as its hardware structure. Furthermore, the hardware structure of these various processors is more specifically an electrical circuit (circuitry) that combines circuit elements such as semiconductor elements. That is the case.

[0195] The technical scope of the present invention is not limited to the scope described in the embodiments above. The configurations and other elements in each embodiment can be appropriately combined with those in each embodiment without departing from the spirit of the present invention. [Explanation of symbols]

[0196] 1. Curve showing the temporal changes in CT values ​​in arteries. 2. Curve showing the time-dependent changes in CT values ​​in the portal vein. 3. Curve showing the time course of CT values ​​in the liver. 10 Property analyzer 10A property analyzer 12 CT image acquisition unit 14 Contrast temporal phase estimation section 16. Area of ​​Interest Extraction Unit 18 Property analysis department 20 Information Output Unit 22 Sorting Department 30 processors 32 Computer-readable media 32A Computer-readable media 34 Communication Interfaces 36 Input / Output Interfaces 38 bus 40 memory 40A Memory 42 storage 50 Contrast-enhanced phase estimation program 52. Program for Extracting Areas of Interest 54. Property Analysis Program 56 Selection Program 60 Input devices 62 Display device 100 Feature Extraction Networks 102 Non-contrast feature vector 104 Arterial phase feature vector 106 Portal venous phase feature vector 108 Equilibrium phase feature vector 109 Feature vector 110 Feature data 111 Feature data 111A Feature data 120 Classification network 130 Weight calculation network 200 Medical information system 220 Medical image processing device 230 Modality 231 CT device 232 MRI device 233 Ultrasonic diagnostic device 234 PET device 235 X-ray diagnostic device 236 Fluoroscopic X-ray diagnostic device 237 Endoscope device 240 DICOM server 244 Electronic medical record system 246 Viewer terminal 248 Communication line I1 Non-contrast CT image I2 Arterial phase CT image I3 Portal venous phase CT image I4 Equilibrium phase CT image I IN CT image t1 Period corresponding to the arterial phase t2 Period corresponding to the portal venous phase t3 Period corresponding to the equilibrium phase P1 CT image acquisition process P2 Contrast phase estimation process P3 Characteristic analysis process P 31 Characteristic analysis process P4 Information output process P5 Selection process P 51 Selection process P 101 Feature extraction process P 102 Linking process P 103 Property analysis processing P 105 Averaging P 106 Weight calculation process P 112 Weighted averaging p probability W ei weight S10-S18 Steps in the property analysis method

Claims

1. One or more processors, One or more memory locations in which programs to be executed by the one or more processors are stored, Equipped with, The one or more processors execute the instructions of the program, By performing contrast-enhanced imaging and obtaining the resulting medical images, Based on the analysis of the aforementioned medical images, the contrast-enhanced state representing the non-contrast or contrast-enhanced phase of the medical images is estimated. A medical image processing device that performs a characteristic analysis to classify the state of a region of interest extracted from a medical image using contrast enhancement state information representing the contrast enhancement state of the medical image, and using image analysis of the medical image for each contrast enhancement state.

2. The medical image processing apparatus according to claim 1, wherein one or more processors select the medical image using the contrast state information.

3. The medical image processing apparatus according to claim 2, wherein one or more processors select the medical image having contrast-enhanced state information suitable for the characteristic analysis.

4. The medical image processing apparatus according to claim 2 or 3, wherein the one or more processors select the medical images according to the limitations on the input images in the characteristic analysis.

5. The medical image processing apparatus according to claim 2, wherein one or more processors select the medical image for each contrast enhancement state information corresponding to each of two or more predetermined contrast enhancement states.

6. The medical image processing apparatus according to claim 2, wherein one or more processors select the medical images having contrast-enhanced state information corresponding to contrast-enhanced states other than non-contrast-enhanced states.

7. The medical image processing apparatus according to claim 1, wherein the one or more processors extract regions of interest from the acquired medical image and perform characterization analysis of the regions of interest.

8. The medical image processing apparatus according to claim 1, wherein the one or more processors estimate the contrast enhancement state of the acquired medical image using a trained model.

9. The medical image processing apparatus according to claim 1, wherein the one or more processors perform the characteristic analysis of the region of interest contained in the acquired medical image using a trained learning model.

10. The one or more processors described above are: Using the aforementioned trained model, feature quantities are extracted from each of the regions of interest contained in the medical images for each contrast enhancement state. The medical image processing apparatus according to claim 9, which performs a characteristic analysis of the region of interest contained in the medical image based on feature data obtained by combining and concatenating the feature quantities of the region of interest for each contrast enhancement state.

11. The one or more processors described above are: As the aforementioned trained model, a feature extraction model is used to extract features from each of the regions of interest contained in the medical images for each contrast-enhanced state, The feature quantities of the region of interest for each contrast enhancement state are concatenated. The medical image processing apparatus according to claim 10, which performs a characteristic analysis of a region of interest contained in a medical image using a classification model that classifies feature data in which feature quantities of the region of interest contained in the medical image for each contrast enhancement state are concatenated, as the pre-trained model.

12. The one or more processors described above are: The medical image processing apparatus according to claim 10 or 11, which averages a portion of the feature quantities of the region of interest included in the medical image for each contrast enhancement state.

13. The one or more processors described above are: The medical image processing apparatus according to claim 12, wherein the pre-trained model is a weight calculation model that calculates weights used when calculating a weighted average of feature quantities of regions of interest included in the medical image for each contrast-enhanced state, and the weight is calculated for each of the feature quantities of regions of interest included in the medical image for each contrast-enhanced state.

14. The one or more processors described above are: The medical image processing apparatus according to claim 12, wherein, when some of the predetermined number of features to be averaged are missing, the number of features that are not missing is used as the number of features to be averaged.

15. A method for operating a medical image processing device to which a computer is applied, The aforementioned medical image processing device The process of acquiring medical images generated by performing contrast-enhanced imaging. A step of estimating the contrast-enhanced state, which represents the non-contrast or contrast-enhanced phase of the medical image, based on the analysis of the medical image. A method for operating a medical image processing device, which performs a step of performing a characteristic analysis to classify the state of a region of interest extracted from a medical image using contrast enhancement state information representing the contrast enhancement state of the medical image, and using image analysis of the medical image for each contrast enhancement state.

16. On the computer, A function to acquire medical images generated by performing contrast-enhanced imaging. A function to estimate the contrast-enhanced state, which represents the non-contrast or contrast-enhanced phase of the medical image, based on the analysis of the medical image, and A program that implements a function to perform characteristic analysis that classifies the state of a region of interest extracted from a medical image using contrast enhancement state information representing the contrast enhancement state of the medical image, and using image analysis of the medical image for each contrast enhancement state.

Images