Image determination device, method, and program

Abstract

To enable a doctor to make easy-to-understand explanation to a patient using a feature amount by extracting a feature amount (image) necessary for explaining a treatment plan to the patient.SOLUTION: An image determination device includes: image acquisition means for acquiring a dynamic image of a part including a diagnosis object region of a patient; feature amount extraction means for extracting a feature amount by first processing on the basis of the dynamic image; determination means for performing determination on diagnosis by second processing based on a result of machine learning on the basis of the dynamic image; explanation data generation means for generating explanation data on the basis of the feature amount and the determination on the diagnosis; and an output part or a communication part for outputting the explanation data to the outside.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an image determination apparatus, method, and program.

Background Art

[0002] Estimation of the disease level of a specific disease by machine learning from medical images has been performed (for example, Patent Document 1).

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For informed consent, doctors, nurses, etc. need to explain the disease situation (symptoms and conditions) and treatment policies to patients. The device of Patent Document 1 performs class determination based on feature amounts, but the feature amounts that are the basis of the class determination are not output. Therefore, even if the device of Patent Document 1 is used for disease determination, explanatory materials for informed consent are not output. Medical images are images taken using radiation or ultrasonic waves and are images used by doctors, etc., but are unfamiliar to patients. Therefore, when doctors, etc. explain the grasped disease situation and determined treatment policy to patients, even if they explain using the medical images that are the basis for determining the disease situation and treatment policy, it is often not an easy-to-understand explanation for patients.

[0005] Recently, a radiation generating device repeats radiation pulses at a cycle of multiple times per unit time (e.g., 15 times per second) for a predetermined time (duration) while a radiation instruction is being given, and a radiation detecting device reads out the amount of charge generated according to the dose of radiation received through a subject as a signal value (intensity), so that a doctor makes a judgment on a disease based on a dynamic image composed of a plurality of still images. That is, a dynamic image is a series of still images that capture the temporal changes of a subject. A dynamic image may be a 2D image or a 3D image as long as it captures temporal changes. When a medical image is a dynamic image, it is more difficult for a doctor or the like to give an easy-to-understand explanation because the patient is less familiar with it.

[0006] Therefore, when estimating the disease level of a specific disease by machine learning, while improving the accuracy of estimation by making a judgment based on a medical image, by extracting the feature amounts (images) necessary for explaining the treatment policy to a patient, it is required that a doctor or the like can use the feature amounts to give an easy-to-understand explanation to the patient.

Means for Solving the Problem

[0007] An image determination device according to an embodiment of the present disclosure includes an image acquisition unit that acquires a dynamic image of a site including a diagnosis target region of a patient, a feature amount extraction unit that extracts a feature amount by a first process based on the dynamic image, a determination unit that makes a determination regarding diagnosis by a second process based on the result of machine learning based on the dynamic image, an explanation data generation unit that generates explanation data based on the feature amount and the determination regarding diagnosis, and an output unit or a communication unit that outputs the explanation data to the outside.

[0008] An image determination method according to an embodiment of the present disclosure includes an image acquisition step of acquiring a dynamic image of a part including a diagnostic target area of a patient, a feature amount extraction step of extracting a feature amount by a first process based on the dynamic image, a determination step of making a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image, an explanatory data generation step of generating explanatory data based on the feature amount and the determination related to diagnosis, and a step of outputting the explanatory data to the outside.

[0009] <* An image determination program according to an embodiment of the present disclosure causes a computer to execute an image acquisition step of acquiring a dynamic image of a part including a diagnostic target area of a patient, a feature amount extraction step of extracting a feature amount by a first process based on the dynamic image, a determination step of making a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image, an explanatory data generation step of generating explanatory data based on the feature amount and the determination related to diagnosis, and a step of outputting the explanatory data to the outside.

[0010] These general or specific aspects may be implemented by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be implemented by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

Effects of the Invention

[0011] According to the present disclosure, it is possible to provide an image determination apparatus that extracts feature amounts useful when a doctor or the like explains to a patient.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings.

[0014] <1. Image Determination Device>[ First, the schematic configuration of an image determination device 100 according to an embodiment of the present disclosure will be described.

[0015] [Configuration] FIG. 1 is a diagram showing the configuration of the image determination device 100.

[0016] The image determination device 100 includes a processing circuit 110, an input / output unit 120, a communication unit 130, and a memory 140. The input / output unit 120 includes an input unit 121 and an output unit 122. The input unit 121 and the output unit 122 may be integrated.

[0017] The processing circuit 110 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc., and includes a neural network. The processing circuit 110 extracts feature amounts based on the input medical image and estimates the disease level of a specific disease. Details of the processing circuit 110 will be described later.

[0018] The input unit 121 includes at least one of a touch panel, a keyboard, a mouse, a microphone, etc., and is input based on the operations of a user (such as a doctor, a radiologic technologist, etc.).

[0019] The output unit 122 includes at least one of a display, a speaker, a printer, etc., and outputs the explanatory data generated by the processing circuit 110 to the outside.

[0020] The communication unit 130 communicates with an external device by wireless or wired means, such as a bus, a LAN (Local Area Network), the Internet, a VPN (Virtual Private Network), a public line, etc. The communication unit 130 communicates with a hospital information system (HIS), a radiology information system (RIS), a picture archiving and communication system (PACS), a dynamic analysis device, etc.

[0021] The memory 140 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically EPROM), an HDD (Hard Disk Drive), etc., and stores medical images, learning data (teacher data), various programs, etc.

[0022] Figure 2 is a functional block diagram of the processing circuit 110.

[0023] The processing circuit 110 includes an image acquisition means 111, a feature quantity extraction means 112, a determination means 113, and an explanatory data generation means 114. The processing circuit 110 estimates the disease level of a disease by means of two stages of the feature quantity extraction means 112 and the determination means 113.

[0024] The image acquisition means 111 acquires a medical image. The medical image is, for example, a dynamic image.

[0025] The feature quantity extraction means 112 extracts a feature quantity (image) effective for a doctor or the like to explain to a patient based on the medical image acquired by the image acquisition means 111. The feature quantity extraction means 112 outputs the extracted feature quantity. The feature quantity may be numerical data or image data. The feature quantity may be one still image constituting a dynamic image. In the present disclosure, the feature quantity extracted by the feature quantity extraction means 112 is information used when a doctor or the like explains to a patient, and is information that is easy for humans to understand. Since the disease level determined by the determination means 113 is determined using a dynamic image, it is not necessarily required to use the feature quantity extracted by the feature quantity extraction means 112. When the feature quantity extracted by the feature quantity extraction means 112 is input to the determination means 113, the feature quantity extraction means 112 may separately extract the feature quantity necessary for the determination means 113 to perform a highly accurate estimation and output it to the determination means 113. The feature quantity separately extracted by the feature quantity extraction means 112 does not necessarily have to be information that is easy for humans to understand. The content of the feature quantity extraction means 112 will be described later. A doctor or the like uses the output feature quantity to explain the disease situation and treatment policy to the patient.

[0026] The determination means 113 estimates the disease level of a specific disease based on the medical image acquired by the image acquisition means 111. The determination means 113 may estimate the disease level of a specific disease based on the medical image acquired by the image acquisition means 111 and the feature quantity extracted by the feature quantity extraction means 112. The determination means 113 outputs the estimated disease level. The content of the determination means 113 will be described later.

[0027] Explanation data generation means 114 generates explanation data which is information that enables a doctor to easily grasp the condition of a disease or information useful when a doctor or the like explains to a patient, based on the feature amounts extracted by feature amount extraction means 112 and the disease level estimated by determination means 113. In the present disclosure, the explanation data is information that enables a doctor or the like to easily grasp the condition of a disease or information useful when a doctor or the like explains to a patient. A doctor or the like can use the generated explanation data to grasp the condition of the disease and explain the condition of the disease and the treatment policy to the patient. The explanation data generated by explanation data generation means 114 is output from output unit 122.

[0028] <Diagnosis of COPD: First Embodiment> FIG. 3 is a diagram showing a flowchart of processing circuit 110 when diagnosing COPD (Chronic Obstructive Pulmonary Disease).

[0029] COPD affects the bronchi and alveoli. When determining the disease level of COPD, a doctor refers to, for example, the lung field area, the change rate of the lung field area, the bronchial diameter, the change rate of the bronchial diameter, the displacement amount of the diaphragm, the change amount of the alveoli, the image density, the variance of each change amount, etc. The disease level of COPD may be the disease stage (stage I, stage II, stage III, stage IV). The feature amount is, for example, at least one of the lung field area, the change rate of the lung field area, the bronchial diameter, the change rate of the bronchial diameter, the displacement amount of the diaphragm, the change amount of the alveoli, the image density, the variance of each change amount, or one or more still images (a part of the dynamic image) constituting the dynamic image for understanding these values or a processed version of one or more of these still images. For example, a feature amount may be one with a line indicating the area of the lung field added to the still image or a numerical value such as the lung field area added.

[0030] [Image acquisition means] Image acquisition means 111 acquires a medical image (step S301). The medical image is, for example, a dynamic image. Image acquisition means 111 may read out the dynamic image stored in memory 140 or may receive the dynamic image from an RIS or the like via communication unit 130.

[0031] [Feature quantity extraction means] The feature quantity extraction means 112 extracts feature quantities from the moving image acquired by the image acquisition means 111 in step S301 based on machine learning or rules, and outputs the extracted feature quantities to the outside from the output unit 122, or outputs (transmits) them to an external device via the communication unit 130.

[0032] The feature quantity extraction means 112 performs lung contour recognition processing based on machine learning on the moving image, and obtains the lung field area and the change rate of the lung field area (step S302). Edge processing on the moving image may be performed as preprocessing. The feature quantity extraction means 112 may obtain the lung field area of each still image constituting the moving image based on machine learning, and obtain the change rate of the lung field area by arithmetic processing. The average, maximum, minimum, median, or mode value of the lung field area of each still image may be used as the lung field area.

[0033] The feature quantity extraction means 112 outputs the lung field area and the change rate of the lung field area extracted in step S302 to the outside (step S303). The feature quantity extraction means 112 may output the maximum lung field area and / or the minimum lung field area.

[0034] The feature quantity extraction means 112 performs trachea recognition processing based on machine learning on the moving image, and obtains the trachea diameter and the change rate of the trachea diameter (step S304). Edge processing on the moving image may be performed as preprocessing. The feature quantity extraction means 112 may obtain the trachea diameter of each still image constituting the moving image based on machine learning, and obtain the change rate of the trachea diameter by arithmetic processing. The average, maximum, minimum, median, or mode value of the trachea diameter of each still image may be used as the trachea diameter.

[0035] The feature quantity extraction means 112 outputs the trachea diameter and the change rate of the trachea diameter extracted in step S304 to the outside (step S305). The feature quantity extraction means 112 may output the maximum trachea diameter and / or the minimum trachea diameter to the outside.

[0036] The feature extraction means 112 performs diaphragm recognition processing based on rules on the moving image (step S306). The feature extraction means 112 obtains the position of the diaphragm based on rules for each still image constituting the moving image, and obtains the displacement amount of the diaphragm by arithmetic processing. For example, line fitting is used for diaphragm recognition. The feature extraction means 112 may obtain the position of the diaphragm based on machine learning. In this case, the feature extraction means 112 may perform edge processing on the moving image as preprocessing.

[0037] The feature extraction means 112 outputs the displacement amount of the diaphragm extracted in step S306 to the outside (step S307).

[0038] The feature extraction means 112 executes at least one of steps S302, S304, and S306. The feature extraction means 112 may determine the processing to be executed according to the feature amount to be extracted. For example, when the feature extraction means 112 extracts the lung field area, the change rate of the lung field area, the tracheal diameter, the change rate of the tracheal diameter, and the displacement amount of the diaphragm as feature amounts, it performs three processes: lung contour recognition processing, trachea recognition processing, and diaphragm recognition processing.

[0039] The machine learning performed by the feature extraction means 112 will be described.

[0040] In the learning phase, the feature extraction means 112 performs learning using learning data with the feature amounts for the moving image as teacher data. The feature extraction means 112 performs learning using learning data corresponding to each process, for example, learning data for "lung contour recognition processing" and learning data for "trachea recognition processing". The learning may be learning corresponding to a doctor or the like. The learning may be learning corresponding to the attributes of a patient, such as whether the patient is an adult or a child.

[0041] In the inference phase, the feature extraction means 112 extracts feature amounts from the moving image based on the results learned in the learning phase.

[0042] [Judgment means] The determination means 113 estimates the disease level of COPD based on machine learning based on the moving image acquired by the image acquisition means 111 (step S308). The determination means 113 may estimate the disease level of COPD based on the moving image acquired by the image acquisition means 111 and the feature amounts extracted by the feature amount extraction means 112 in steps S302, S304, and S306. The feature amounts input to the determination means 113 may be feature amounts extracted separately from the feature amounts extracted by the feature amount extraction means 112 in steps S303, S305, and S307. The separately extracted feature amounts do not necessarily have to be information that is easy for humans to understand. When the feature amounts extracted by the feature amount extraction means 112 are impossible numerical values (for example, the tracheal diameter is 10 cm, etc.), the determination means 113 may exclude the feature amounts before making the estimation.

[0043] The determination means 113 outputs the disease level of COPD estimated in step S308 to the outside (step S309). The determination means 113 may output the estimated disease level of COPD from the output unit 122 to the outside, or may output (transmit) it to an external device via the communication unit 130.

[0044] The machine learning performed by the determination means 113 will be described.

[0045] In the learning phase, the determination means 113 learns using learning data with the disease level of COPD for the moving image as teacher data. Alternatively, the determination means 113 learns using learning data with the disease level of COPD for the moving image and the feature amounts as teacher data.

[0046] In the inference phase, the determination means 113 estimates the disease level of COPD from the moving image based on the result learned in the learning phase. Alternatively, the determination means 113 estimates the disease level of COPD from the moving image and the feature amounts based on the result learned in the learning phase.

[0047] [Explanation data generation means] The explanation data generation means 114 generates data that enables doctors and the like to easily understand the disease situation and explanation data used when doctors and the like explain the disease situation and treatment policy to patients, based on the feature amounts extracted by the feature amount extraction means 112 and the disease level estimated by the determination means (step S310).

[0048] The generated explanation data may be a still image or a moving image obtained by performing image processing on the moving image acquired by the image acquisition means 111, or may be data to which annotations, markings, numerical values of feature amounts, etc. are added. For example, data obtained by adding a line indicating the area of the lung field to a still image or adding numerical values such as the lung field area may be used as the explanation data.

[0049] <Diagnosis of COPD: Second Embodiment> FIG. 4 is a diagram showing another flowchart of the processing circuit 110 when diagnosing COPD. The disease level of COPD may be the disease stage (stage I, stage II, stage III, stage IV). The feature amount is, for example, at least one of the ratio of the magnitude of movement at each point in the lung field, the area of the entire lung field, the area of the lung field where the movement has decreased, and the ratio of the area of the lung field where the movement has decreased to the area of the entire lung field, or one or more still images (parts of the moving image) constituting the moving image or those obtained by processing one or more of those still images for understanding these values. For example, data obtained by adding a line indicating the area of the lung field where the movement has decreased to a still image or adding numerical values such as the ratio and area may be used as the feature amount.

[0050] [Image Acquisition Means] The image acquisition means 111 acquires a medical image (step S401). The medical image is, for example, a moving image. The image acquisition means 111 may read out the moving image stored in the memory 140, or may receive the moving image from an RIS or the like via the communication unit 130.

[0051] [Feature Amount Extraction Means] The feature extraction means 112 extracts features from the moving image acquired by the image acquisition means 111 in step S401 based on machine learning or rules, and outputs the extracted features to the outside from the output unit 122 or outputs (transmits) them to an external device via the communication unit 130.

[0052] The feature extraction means 112 calculates an optical flow (difference) from the moving image acquired by the image acquisition means 111 in step S401 (step S402). The difference may be a difference with respect to a reference frame (still image) or a difference with respect to the immediately preceding frame. The reference frame may be the first frame. The difference is calculated as a vector. Step S402 is executed as preprocessing.

[0053] The feature extraction means 112 identifies the lung field based on machine learning based on the optical flow calculated in step S402. The feature extraction means 112 calculates the horizontal / vertical size of the entire lung field. Also, since the optical flow is a vector, when the optical flow is a difference with respect to the reference frame, the magnitude of the movement can be grasped by the magnitude of the maximum vector over a predetermined period, and when the optical flow is a difference with respect to the immediately preceding frame, the magnitude of the movement can be grasped by the sum of the magnitudes of the vectors over a predetermined period. The feature extraction means 112 calculates the magnitude of the movement in the horizontal / vertical direction (i.e., the horizontal / vertical magnitude of the vector) at each point in the lung field based on rules. The feature extraction means 112 calculates the ratio of the magnitude of the movement in the horizontal / vertical direction at each point in the lung field to the magnitude of the movement in the horizontal / vertical direction of the entire lung field at each point based on the horizontal / vertical size of the entire lung field and the magnitude of the movement in the horizontal / vertical direction at each point (step S403).

[0054] The feature extraction means 112 outputs to the outside the ratio of the magnitude of the movement at each point in the lung field calculated in step S403 (step S404). The feature extraction means 112 may output the ratio of the magnitude of the movement at some points.

[0055] The feature extraction means 112 calculates the ratio of the lung field area with reduced movement based on machine learning based on the optical flow calculated in step S402 (step S405). The feature extraction means 112 may calculate the ratio of the lung field area with reduced movement based on the ratio of the magnitude of movement for each point in the lung field calculated in step S403. For example, when the ratio of the magnitude of movement is equal to or less than a predetermined threshold, it is determined that the movement has decreased, and the ratio of the lung field area with reduced movement is calculated based on the number of points with reduced movement and the number of points in the entire lung field.

[0056] The feature extraction means 112 outputs to the outside the area of the entire lung field, the area of the lung field with reduced movement, and the ratio of the area of the lung field with reduced movement to the area of the entire lung field calculated in step S405 (step S406).

[0057] The machine learning performed by the feature extraction means 112 will be described.

[0058] In the learning phase, the feature extraction means 112 performs learning using learning data with features of the optical flow as teacher data. The feature extraction means 112 performs learning using learning data corresponding to each process, for example, learning data for "lung contour recognition process" and learning data for "calculating the lung field area with reduced movement". The learning may be learning corresponding to a doctor or the like. The learning may be learning corresponding to the attributes of a patient, such as whether the patient is an adult or a child.

[0059] In the inference phase, the feature extraction means 112 extracts features from the optical flow based on the results learned in the learning phase.

[0060] [Determination means] The determination means 113 estimates the disease level of COPD based on machine learning based on the optical flow calculated in step S402 (step S407). The determination means 113 may estimate the disease level of COPD based on the optical flow calculated in step S402 and the feature amounts extracted by the feature amount extraction means 112 in steps S403 and S405. The feature amounts input to the determination means 113 may be feature amounts extracted separately from the feature amounts extracted by the feature amount extraction means 112 in steps S404 and S406. The separately extracted feature amounts do not necessarily have to be information that is easily understandable to humans. When the feature amounts extracted by the feature amount extraction means 112 are impossible numerical values (for example, the area of the entire lung field is 5 cm 2 etc.), the determination means 113 may exclude the feature amounts before making the estimation.

[0061] The determination means 113 outputs the disease level of COPD estimated in step S407 to the outside (step S408). The determination means 113 may output the estimated disease level of COPD to the outside from the output unit 122, or may output (transmit) it to an external device via the communication unit 130.

[0062] The machine learning performed by the determination means 113 will be described.

[0063] In the learning phase, the determination means 113 learns using learning data with the disease level of COPD for the optical flow as teacher data. Alternatively, the determination means 113 learns using learning data with the disease level of COPD for the optical flow and feature amounts as teacher data.

[0064] In the inference phase, the determination means 113 estimates the disease level of COPD from the optical flow based on the results learned in the learning phase. Alternatively, the determination means 113 estimates the disease level of COPD from the optical flow and feature amounts based on the results learned in the learning phase.

[0065] [Explanation data generation means] The explanatory data generation means 114 generates data that enables doctors and the like to easily understand the disease situation and explanatory data used when doctors and the like explain the disease situation and treatment policy to patients, based on the feature amounts extracted by the feature amount extraction means 112 and the disease level estimated by the determination means (step S409).

[0066] The explanatory data to be generated may be a still image or a moving image obtained by performing image processing on the moving image acquired by the image acquisition means 111, or may be data to which annotations, markings, numerical values of feature amounts, etc. are added. For example, data obtained by adding a line indicating the area of the lung field with reduced movement to a still image or adding numerical values such as ratios and areas may be used as explanatory data.

[0067] <Diagnosis of Fallot: Third Embodiment> FIG. 5 is a diagram showing a flowchart of the processing circuit 110 when diagnosing Fallot (tetralogy of Fallot).

[0068] Fallot (tetralogy of Fallot) affects the heart. When making a determination of Fallot, doctors and the like use feature amounts such as the pulmonary artery waveform and the heartbeat waveform. The disease level of Fallot may be the regurgitation rate or the NYHA (New York Heart Association) cardiac function classification (stage I, II, III, IV). The feature amount may be, for example, at least one of the pulmonary artery waveform and the heartbeat waveform, or one or more still images (parts of the moving image) constituting the moving image or those obtained by processing one or more of those still images for understanding these values. For example, a still image with a line indicating the pulmonary artery added may be used as the feature amount.

[0069] [Image Acquisition Means] The image acquisition means 111 acquires a medical image (step S501). The medical image is, for example, a moving image. The image acquisition means 111 may read out the moving image stored in the memory 140, or may receive the moving image from an RIS or the like via the communication unit 130.

[0070] [Feature Amount Extraction Means] The feature extraction means 112 extracts features based on machine learning from the moving image acquired by the image acquisition means 111 in step S501, and outputs the extracted features to the outside from the output unit 122, or outputs (transmits) them to an external device via the communication unit 130.

[0071] The feature extraction means 112 performs a recognition process of the pulmonary artery based on machine learning on the moving image, and extracts the waveform of the pulmonary artery (step S502). The feature extraction means 112 may extract the waveforms of the pulmonary artery at a plurality of points, such as a point close to the heart and a point close to the lung, among the recognized pulmonary arteries. The feature extraction means 112 recognizes the pulmonary artery in each still image constituting the moving image based on machine learning, selects a plurality of points of the recognized pulmonary artery, and detects the waveform of the pulmonary artery at each point as a change in density corresponding to time. The feature extraction means 112 may recognize a plurality of points of the pulmonary artery in each still image constituting the moving image based on machine learning.

[0072] The feature extraction means 112 outputs the waveform of the pulmonary artery extracted in step S502 to the outside (step S503). The feature extraction means 112 may output the waveforms of the pulmonary artery at some points to the outside.

[0073] The feature extraction means 112 performs a recognition process of the cardiac apex based on machine learning on the moving image, and extracts the waveform of the cardiac apex, that is, the heartbeat waveform (step S504). The feature extraction means 112 obtains the cardiac apex in each still image constituting the moving image based on machine learning, and detects the heartbeat waveform as a change in density corresponding to time.

[0074] The feature extraction means 112 outputs the heartbeat waveform extracted in step S504 to the outside (step S505).

[0075] The feature extraction means 112 may execute only one of steps S502 and S504. When extracting the waveforms at a plurality of points of the pulmonary artery, the recognition process of the pulmonary artery may be performed for each point.

[0076] Describe the machine learning performed by the feature extraction means 112.

[0077] In the learning phase, the feature extraction means 112 performs learning using learning data with the features of the dynamic image as the teacher data. The feature extraction means 112 performs learning using learning data corresponding to each process, for example, learning data for "recognition process of pulmonary artery" and learning data for "recognition process of cardiac apex". The feature may be numerical data or image data. The learning may be learning corresponding to a doctor or the like. The learning may be learning corresponding to the attributes of the patient, such as whether the patient is an adult or a child.

[0078] In the inference phase, the feature extraction means 112 extracts features from the dynamic image based on machine learning.

[0079] [Determination means] The determination means 113 estimates the disease level of Fallot based on the dynamic image acquired by the image acquisition means 111 in step S501, based on machine learning. The determination means 113 may estimate the disease level of Fallot based on the dynamic image acquired by the image acquisition means 111 in step S501 and the features extracted by the feature extraction means 112 in steps S502 and S504. The features input to the determination means 113 may be features extracted separately from the features extracted by the feature extraction means 112 in steps S503 and S505. The separately extracted features do not necessarily have to be information that is easy for humans to understand. When the features extracted by the feature extraction means 112 are impossible numerical values (for example, a heart rate of 500 beats per minute, etc.), the determination means 113 may exclude those features before making an estimate.

[0080] ]The determination means 113 outputs the estimated disease level of Fallot to the outside in step S506 (step S507). The determination means 113 may output the estimated disease level of Fallot to the outside from the output unit 122, or may output (transmit) it to an external device via the communication unit 130.

[0081] In the learning phase, the determination means 113 learns using learning data with the disease level of PH as teacher data for the moving image. Alternatively, the determination means 113 learns using learning data with the disease level of PH as teacher data for the moving image and the feature amount.

[0082] In the inference phase, the determination means 113 estimates the disease level of PH from the moving image based on the result learned in the learning phase.

[0083] [Explanation data generation means] The explanation data generation means 114 generates data that makes it easy for doctors and the like to grasp the disease situation and explanation data used when doctors and the like explain the disease situation and treatment policy to patients, based on the feature amount extracted by the feature amount extraction means 112 and the disease level estimated by the determination means (step S508).

[0084] The generated explanation data may be a still image or a moving image obtained by performing image processing on the moving image obtained by the image acquisition means 111, or may be data to which annotations, markings, numerical values of feature amounts, etc. are added. For example, data obtained by adding a line indicating the pulmonary artery to a still image may be used as the explanation data.

[0085] <Diagnosis of CTEPH: Fourth Embodiment> FIG. 6 is a diagram showing a flowchart of the processing circuit 110 when diagnosing CTEPH (Chronic ThromboEmbolic Pulmonary Hypertension).

[0086] CTEPH affects the lungs and the heart. When making a diagnosis of CTEPH, doctors and the like use blood flow images, the amount of phase change and the amount of amplitude change for each measurement location. The disease level of CTEPH may be the confidence level, or it may be the NYHA cardiac function classification (Class I, II, III, IV) and / or the WHO (World Health Organization) pulmonary hypertension functional classification (Degree I, II, III, IV). The feature quantity may be, for example, at least one of the amount of phase change and the amount of amplitude change of the blood flow image at each measurement location, or one or more still images (a part of the dynamic image) constituting the dynamic image for understanding these values, or a processed version of one or more of these still images. For example, a still image with a mark indicating the measurement location added thereto may be used as the feature quantity.

[0087] [Image acquisition means] The image acquisition means 111 acquires a medical image (step S601). The medical image is, for example, a dynamic image. The image acquisition means 111 may read out the dynamic image stored in the memory 140, or may receive the dynamic image from a RIS or the like via the communication unit 130.

[0088] [Feature quantity extraction means] The feature quantity extraction means 112 extracts a feature quantity from the dynamic image acquired by the image acquisition means 111 in step S601 based on machine learning, and outputs the extracted feature quantity to the outside from the output unit 122, or outputs (transmits) it to an external device via the communication unit 130.

[0089] The feature quantity extraction means 112 identifies the positions of the organs and the lung fields in the dynamic image based on machine learning, and identifies the reference positions and the measurement positions for extracting the amplitude information and the phase information based on the identified positions of the organs and the lung fields (step S602). For example, the reference position may be the hilum of the lung (the position of high blood flow connected to the heart), and the measurement positions may be the right upper lobe, the right middle lobe, the right lower lobe, the left upper lobe, and the left lower lobe. The reference position may be the location with the maximum signal change in the whole.

[0090] The feature extraction means 112 outputs the position extracted in step S602 to the outside (step S603). The feature extraction means 112 may output the positions of some points to the outside.

[0091] The feature extraction means 112 performs an extraction process of signal changes synchronized with the heartbeat cycle based on machine learning on the dynamic image, and generates a blood flow image (step S604). The feature extraction means 112 obtains the heartbeat cycle from the dynamic image based on machine learning, and extracts the changes in the signal synchronized with the heartbeat cycle.

[0092] The feature extraction means 112 outputs the blood flow image generated in step S604 to the outside (step S605).

[0093] The feature extraction means 112 performs a phase information extraction process of the blood flow image (changes in the signal synchronized with the heartbeat cycle) generated in step S604 for each position specified in step S602, and extracts the phase change amount (step S606). The feature extraction means 112 extracts the phase change amount based on rules. The feature extraction means 112 compares the phase data of the reference position with the phase data of each measurement position, and calculates the phase change amount (that is, the delay time) for each measurement position.

[0094] The feature extraction means 112 outputs the phase change amount generated in step S606 to the outside (step S607). The feature extraction means 112 may output the phase change amounts of some points to the outside.

[0095] The feature extraction means 112 performs an amplitude information extraction process of the blood flow image (changes in the signal synchronized with the heartbeat cycle) generated in step S604 for each position specified in step S602, and extracts the amplitude change amount (step S608). The feature extraction means 112 extracts the amplitude change amount based on rules. The feature extraction means 112 compares the amplitude data of the reference position with the amplitude data of each measurement position, and calculates the amplitude change amount (that is, the attenuation) for each measurement position.

[0096] The feature extraction means 112 outputs the amount of amplitude change generated in step S608 to the outside (step S609). The feature extraction means 112 may output the amount of amplitude change at some points to the outside.

[0097] Describe the machine learning performed by the feature extraction means 112.

[0098] In the learning phase, the feature extraction means 112 performs learning using learning data with the features of the dynamic image as teacher data. The features may be numerical data or image data. The learning may be learning corresponding to a doctor or the like. The learning may be learning corresponding to the attributes of a patient, such as whether the patient is an adult or a child.

[0099] In the inference phase, the feature extraction means 112 extracts features from the dynamic image based on the results learned in the learning phase.

[0100] [Determination means] The determination means 113 estimates the disease level of CTEPH based on machine learning based on the dynamic image acquired by the image acquisition means 111 in step S601 (step S610). The determination means 113 may estimate the disease level of CTEPH based on machine learning based on the dynamic image acquired by the image acquisition means 111 in step S601 and the features extracted by the feature extraction means 112 in steps S602 and S604. The features input to the determination means 113 may be features extracted separately from the features extracted by the feature extraction means 112 in steps S607 and S609. The separately extracted features do not necessarily have to be information that is easy for humans to understand. The determination means 113 may exclude the features if the features extracted by the feature extraction means 112 are impossible numerical values before performing the estimation.

[0101] The determination means 113 outputs the disease level of CTEPH estimated in step S610 to the outside (step S611). The determination means 113 may output the estimated disease level of CTEPH from the output unit 122 to the outside, or may output (transmit) it to an external device via the communication unit 130.

[0102] Describe the machine learning performed by the determination means 113.

[0103] In the learning phase, the determination means 113 learns using learning data with the disease level of CTEPH for the dynamic image as teacher data. Alternatively, the determination means 113 learns using learning data with the disease level of Fallot for the dynamic image and feature amounts as teacher data.

[0104] In the inference phase, the determination means 113 estimates the disease level of CTEPH from the medical image based on the result learned in the learning phase.

[0105] [Explanation data generation means] Based on the feature amounts extracted by the feature amount extraction means 112 and the disease level estimated by the determination means, the explanation data generation means 114 generates data that enables doctors and the like to easily grasp the disease situation, and explanation data used when doctors and the like explain the disease situation and treatment policy to patients (step S612).

[0106] The generated explanation data may be a still image or a dynamic image that constitutes the dynamic image acquired by the image acquisition means 111, or an image on which image processing has been performed, or data to which annotations, markings, numerical values of feature amounts, etc. have been added. For example, data with a mark indicating the measurement position added to a still image may be used as the explanation data.

[0107] [Output] Figures 7 to 9 are diagrams showing output examples of the output unit 122. It may be transmitted to an external device via the communication unit 130, and Figures 7, 8, or 9 may be output by the external device. The output unit 122 may output any output as long as it is explanation data for a doctor to explain to a patient.

[0108] The disease for which the image determination device determines the disease level may be a disease selected by a doctor or the like, may be a disease determined corresponding to a patient (medical record), or may be all diseases that can be determined.

[0109] When the feature amount cannot be extracted due to the resolution of medical information or the like, a message indicating that the determination cannot be made may be output from the output unit 122 or the communication unit 130 to the outside.

[0110] (Output Example 1) FIG. 7 is a diagram showing an output example of the output unit 122. A window 700 may be displayed on the display. FIG. 7 shows an example in which the disease levels of COPD and CTEPH are determined.

[0111] The disease level of the disease determined by the determination means 113 is output as the window 700. The disease levels of a plurality of diseases may be output.

[0112] (Output Example 2) FIG. 8 is a diagram showing another output example of the output unit 122. A window 800 may be displayed on the display.

[0113] The feature amount extracted by the feature amount extraction means 112 is output as the window 800. The explanation data generation means 114 may output an image obtained by performing image processing on the dynamic image and a plurality of feature amounts. Among the plurality of feature amounts, a selected feature amount may be displayed largely. The feature amount may be displayed on the display for the patient based on the operation of a doctor or the like.

[0114] (Output Example 3) FIG. 9 is a diagram showing still another output example of the output unit 122. A window 900 may be displayed on the display.

[0115] The disease level 901 of the disease determined by the determination means 113, the medical image 902, the image 903 obtained by performing image processing on the dynamic image, and the feature amount (numerical data) may be displayed in the window 900. The disease level 901 of the disease, the medical image 902, the image 903 obtained by performing image processing on the dynamic image, and the feature amount may be displayed in one window 900 or in separate windows. The image obtained by performing image processing on the dynamic image and the feature amount may be displayed so that the images and feature amounts obtained by performing a plurality of image processes on the dynamic image can be selectively displayed. FIG. 9 shows an example in which the image obtained by performing the selected image process on the dynamic image is displayed large and the images obtained by performing other image processes on the dynamic image are displayed small.

[0116] The output unit 122 may include a plurality of displays. For example, different information may be displayed on different displays, such that FIG. 9 is displayed on the doctor's display and FIG. 8 is displayed on the patient's display. The same information may be displayed on different displays.

[0117] When a doctor or the like explains a treatment policy or the like to a patient, an easy-to-understand explanation can be made by using the output feature amount. The feature amount may be at least one of the examples, but by extracting and outputting many feature amounts, the accuracy of the determination process can be further improved.

[0118] As described above, the embodiments have been described with reference to the drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive of various modification examples or correction examples within the scope described in the claims. Such modification examples or correction examples are also understood to belong to the technical scope of the present disclosure. Also, within the scope not departing from the gist of the present disclosure, the components in the embodiments may be arbitrarily combined.

[0119] (1) In one embodiment of the present disclosure, an image determination device includes an image acquisition unit that acquires a dynamic image obtained by photographing a site including a diagnostic target region of a patient, a feature amount extraction unit that extracts a feature amount by a first process based on the dynamic image, a determination unit that makes a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image, an explanation data generation unit that generates explanation data based on the feature amount and the determination related to diagnosis, and an output unit or a communication unit that outputs the explanation data to the outside.

[0120] (2) In one embodiment of the present disclosure, the image determination device further includes an output unit or a communication unit that outputs the feature amount to the outside, in the image determination device of (1).

[0121] (3) In one embodiment of the present disclosure, in the image determination device of (1), the determination unit makes a determination related to diagnosis based on the feature amount and the dynamic image.

[0122] (4) In one embodiment of the present disclosure, in the image determination device of (1), the first process is a process based on machine learning or a process based on a rule.

[0123] (5) In one embodiment of the present disclosure, in the image determination device of (1), the dynamic image is a radiation image obtained by a radiation imaging device.

[0124] (6) In one embodiment of the present disclosure, in the image determination device of (1), the feature amount includes one or more still images constituting the dynamic image.

[0125] (7) In one embodiment of the present disclosure, in the image determination device of (1), the determination related to diagnosis is a determination of the disease level of a specific disease.

[0126] (8) In an embodiment of the present disclosure, the image determination device in (7) is such that the specific disease is COPD, the disease level is the stage of COPD, and the feature amount is at least one of the lung field area, the change rate of the lung field area, the tracheal diameter, the change rate of the tracheal diameter, the displacement amount of the diaphragm, the change amount of the alveoli, the image density, the variance of each change amount, one or more still images constituting the dynamic image, and the processed one or more still images.

[0127] (9) In an embodiment of the present disclosure, the image determination device in (7) is such that the specific disease is COPD, the disease level is the stage of COPD, and the feature amount is at least one of the ratio of the magnitude of movement at each point in the lung field, the area of the entire lung field, the area of the lung field where the movement has decreased, and the ratio of the area of the lung field where the movement has decreased to the area of the entire lung field, one or more still images constituting the dynamic image, and the processed one or more still images.

[0128] (10) In an embodiment of the present disclosure, the image determination device in (7) is such that the specific disease is tetralogy of Fallot, the disease level is the regurgitation rate, and the feature amount is at least one of the waveform of the pulmonary artery, the heartbeat waveform, one or more still images constituting the dynamic image, and the processed one or more still images.

[0129] (11) In an embodiment of the present disclosure, the image determination device in (7) is such that the specific disease is CTEPH, the disease level is the confidence level, and the feature amount is at least one of the phase change amount and the amplitude change amount of the blood flow image at each measurement position, one or more still images constituting the dynamic image, and the processed one or more still images.

[0130] (12) An image determination method according to an embodiment of the present disclosure includes an image acquisition step of acquiring a dynamic image of a part including a diagnostic target area of a patient, a feature amount extraction step of extracting a feature amount by a first process based on the dynamic image, a determination step of making a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image, an explanation data generation step of generating explanation data based on the feature amount and the determination related to diagnosis, and a step of outputting the explanation data to the outside.

[0131] (13) An image determination program according to an embodiment of the present disclosure causes a computer to execute an image acquisition step of acquiring a dynamic image of a part including a diagnostic target area of a patient, a feature amount extraction step of extracting a feature amount by a first process based on the dynamic image, a determination step of making a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image, an explanation data generation step of generating explanation data based on the feature amount and the determination related to diagnosis, and a step of outputting the explanation data to the outside.

Industrial Applicability

[0132] The present disclosure is useful for an image determination device.

Explanation of Signs

[0133] 100 Image determination device 110 Processing circuit 111 Image acquisition means 112 Feature amount extraction means 113 Determination means 120 Input / output unit 121 Input unit 122 Output unit 130 Communication unit 140 Memory

Claims

1. An image acquisition means for acquiring a dynamic image obtained by photographing a site including a diagnostic target area of a patient; A feature quantity extraction means for extracting a feature quantity by a first process based on the dynamic image; A determination means for making a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image; An explanatory data generation means for generating explanatory data based on the feature quantity and the determination related to diagnosis; An output unit or a communication unit for outputting the explanatory data to the outside; An image determination device comprising the above.

2. An output unit or a communication unit for outputting the feature quantity to the outside; The image determination device according to claim 1, further comprising the above.

3. The determination means makes a determination related to diagnosis based on the feature quantity and the dynamic image. The image determination device according to claim 1.

4. The first process is a process based on machine learning or a process based on a rule. The image determination device according to claim 1.

5. The dynamic image is a radiation image obtained by a radiation imaging device. The image determination device according to claim 1.

6. The feature quantity includes one or more still images constituting the dynamic image. The image determination device according to claim 1.

7. The determination related to diagnosis is a determination of the disease level of a specific disease. The image determination device according to claim 1.

8. The specific disease is COPD; The disease level is the stage of COPD; The feature quantity is at least one of the lung field area, the change rate of the lung field area, the tracheal diameter, the change rate of the tracheal diameter, the displacement amount of the diaphragm, the change amount of the alveoli, the image density, the variance of each change amount, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination device according to claim 7.

9. The specific disease is COPD; The disease level is the stage of COPD; The feature quantity is at least one of the ratio of the magnitude of movement at each point in the lung field, the area of the entire lung field, the area of the lung field where the movement is decreased, and the ratio of the area of the lung field where the movement is decreased to the area of the entire lung field, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination device according to claim 7.

10. The specific disease is tetralogy of Fallot; The disease level is the regurgitation rate; The feature quantity is at least one of the waveform of the pulmonary artery, the heartbeat waveform, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination device according to claim 7.

11. The specific disease is CTEPH, The disease level is the confidence level, The feature amount is at least one of the amount of phase change and the amount of amplitude change of the blood flow image at each measurement position, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination device according to claim 7.

12. An image acquisition step of acquiring a dynamic image obtained by photographing a site including a diagnostic target region of a patient; A feature amount extraction step of extracting a feature amount by a first process based on the dynamic image; A determination step of making a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image; An explanatory data generation step of generating explanatory data based on the feature amount and the determination related to diagnosis; A step of outputting the explanatory data to the outside; An image determination method comprising:

13. Causing a computer to An image acquisition step of acquiring a dynamic image obtained by photographing a site including a diagnostic target region of a patient; A feature amount extraction step of extracting a feature amount by a first process based on the dynamic image; A determination step of making a determination related to diagnosis by a second process based on the result of machine learning based on the dynamic image; An explanatory data generation step of generating explanatory data based on the feature amount and the determination related to diagnosis; A step of outputting the explanatory data to the outside; An image determination program for causing the above to be executed.

Images