Image determination device, method, and program

Abstract

To enable a doctor to make easy-to-understand explanation to a patient using a feature amount by extracting a feature amount (image) necessary for explaining a treatment plan to the patient.SOLUTION: An image determination device includes: image acquisition means for acquiring a dynamic image of a part including a diagnosis object region of a patient; feature amount extraction means for extracting a feature amount by first processing on the basis of the dynamic image; determination means for performing determination on diagnosis by second processing based on a result of machine learning on the basis of the feature amount; explanation data generation means for generating explanation data on the basis of the feature amount and the determination on the diagnosis; and an output part or a communication part for outputting the explanation data to the outside.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an image determination device, method, and program.

Background Art

[0002] It has been performed to perform class determination as to whether medical images are taken with appropriate positioning by machine learning (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For informed consent, doctors, nurses, etc. need to explain the disease situation (symptoms and conditions), treatment policies, etc. to patients. The device of Patent Document 1 performs class determination based on feature amounts, but the feature amounts that are the basis of the class determination are not output. Therefore, even if the device of Patent Document 1 is used for disease judgment, explanatory materials for informed consent are not output. Medical images are images taken using radiation or ultrasonic waves and are images used by doctors and the like, but are unfamiliar to patients. Therefore, when doctors and the like explain the grasped disease situation and the determined treatment policy to patients, even if they explain using medical images that are the basis for determining the disease situation and treatment policy, it often does not result in an easy-to-understand explanation for patients.

[0005] Recently, a radiation generating device repeats radiation pulses at a cycle of multiple times per unit time (for example, 15 times per second) during the period when a radiation instruction is given, and emits radiation over a predetermined time (duration). A radiation detection device reads out the amount of charge generated according to the dose of radiation received through a subject as a signal value (intensity), and based on a dynamic image composed of a plurality of still images, a doctor makes a judgment on a disease. That is, a dynamic image is a series of still images that capture temporal changes of a subject. The dynamic image may be a 2D image or a 3D image as long as it captures temporal changes. When a medical image is a dynamic image, since the dynamic image itself is a new image, it may be difficult for a doctor to grasp image features caused by a pathological condition. Since it is even less familiar to a patient, it is more difficult for a doctor or the like to give an easy-to-understand explanation to the patient.

[0006] Therefore, when estimating the disease level of a specific disease by machine learning, it is required to assist a doctor's judgment and enable a doctor or the like to give an easy-to-understand explanation to a patient by providing data that is easy for the doctor to understand the pathological condition or data necessary for explaining a treatment policy to the patient.

Means for Solving the Problem

[0007] An image determination device according to an embodiment of the present disclosure includes an image acquisition unit that acquires a dynamic image obtained by photographing a site including a diagnosis target region of a patient, a feature amount extraction unit that extracts a feature amount by a first process based on the dynamic image, a determination unit that makes a determination related to diagnosis by a second process based on a result of machine learning based on the feature amount, an explanation data generation unit that generates explanation data based on the feature amount and the determination related to diagnosis, and an output unit or a communication unit that outputs the explanation data to the outside.

[0008] An image determination method according to an embodiment of the present disclosure includes an image acquisition step of acquiring a dynamic image of a site including a diagnostic target region of a patient, a feature amount extraction step of extracting a feature amount by a first process based on the dynamic image, a determination step of making a determination related to diagnosis by a second process based on a machine learning result based on the feature amount, an explanatory data generation step of generating explanatory data based on the feature amount and the determination related to diagnosis, and a step of outputting the explanatory data to the outside.

[0009] An image determination program according to an embodiment of the present disclosure causes a computer to execute an image acquisition step of acquiring a dynamic image of a site including a diagnostic target region of a patient, a feature amount extraction step of extracting a feature amount by a first process based on the dynamic image, a determination step of making a determination related to diagnosis by a second process based on a machine learning result based on the feature amount, an explanatory data generation step of generating explanatory data based on the feature amount and the determination related to diagnosis, and a step of outputting the explanatory data to the outside.

[0010] These general or specific aspects may be implemented in a system, apparatus, method, integrated circuit, computer program, or recording medium, or may be implemented in any combination of a system, apparatus, method, integrated circuit, computer program, and recording medium.

Advantages of the Invention

[0011] According to the present disclosure, it is possible to provide an image determination device that extracts feature amounts useful when a doctor or the like explains to a patient.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings.

[0014] <1. Image Determination Apparatus> First, the schematic configuration of an image determination apparatus 100 according to an embodiment of the present disclosure will be described.

[0015] [Configuration] FIG. 1 is a diagram showing the configuration of the image determination apparatus 100.

[0016] The image determination apparatus 100 includes a processing circuit 110, an input / output unit 120, a communication unit 130, and a memory 140. The input / output unit 120 includes an input unit 121 and an output unit 122. The input unit 121 and the output unit 122 may be integrated.

[0017] The processing circuit 110 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc., and includes a neural network. The processing circuit 110 extracts feature amounts based on the input medical images and estimates the disease level of a specific disease. Details of the processing circuit 110 will be described later.

[0018] The input unit 121 includes at least one of a touch panel, a keyboard, a mouse, a microphone, etc., and is input based on the operations of a user (such as a doctor, a radiological technologist, etc.).

[0019] The output unit 122 includes at least one of a display, a speaker, a printer, etc., and outputs the explanatory data generated by the processing circuit 110 to the outside.

[0020] The communication unit 130 communicates with an external device wirelessly or by wire, such as via a bus, a LAN (Local Area Network), the Internet, a VPN (Virtual Private Network), a public line, etc. The communication unit 130 communicates with a hospital information system (HIS), a radiology information system (RIS), a picture archiving and communication system (PACS), a dynamic analysis device, etc.

[0021] The memory 140 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically EPROM), an HDD (Hard Disk Drive), etc., and stores medical images, learning data (teacher data), various programs, etc.

[0022] Figure 2 is a functional block diagram of the processing circuit 110.

[0023] The processing circuit 110 includes an image acquisition means 111, a feature extraction means 112, a determination means 113, and an explanatory data generation means 114. The processing circuit 110 estimates the disease level of a disease by means of two stages of means, namely the feature extraction means 112 and the determination means 113.

[0024] The image acquisition means 111 acquires a medical image. The medical image is, for example, a dynamic image.

[0025] The feature amount extraction means 112 extracts a feature amount (image) effective for a doctor or the like to explain to a patient based on the medical image acquired by the image acquisition means 111. The feature amount extraction means 112 extracts a feature amount. The feature amount may be numerical data or image data. The feature amount may be a single still image constituting the dynamic image. Since the feature amount extracted by the feature amount extraction means 112 is information necessary for the determination means 113 to estimate the disease level, it is important that it is information necessary for the determination means 113 to perform a highly accurate estimation, and it is not necessarily information that is easy for humans to understand. The content of the feature amount extraction means 112 will be described later.

[0026] The determination means 113 estimates the disease level of a specific disease based on the feature amount extracted by the feature amount extraction means 112. The determination means 113 outputs the estimated disease level. The content of the determination means 113 will be described later.

[0027] The explanation data generation means 114 generates explanation data that is information that enables a doctor to easily grasp the disease situation or information useful when a doctor or the like explains to a patient, based on the feature amount extracted by the feature amount extraction means 112 and the disease level estimated by the determination means 113. In the present disclosure, the explanation data is information that enables a doctor or the like to easily grasp the disease situation or information useful when a doctor or the like explains to a patient. Even if the feature amount extracted by the feature amount extraction means 112 is not information that is easy for humans to understand, the explanation data generation means 114 can change it to information that is easy for humans to understand. A doctor or the like can use the generated explanation data to grasp the disease situation and explain the disease situation and treatment policy to the patient. The explanation data generated by the explanation data generation means 114 is output from the output unit 122.

[0028] <Diagnosis of COPD: First Embodiment> FIG. 3 is a diagram showing a flowchart of the processing circuit 110 when diagnosing COPD (Chronic Obstructive Pulmonary Disease).

[0029] COPD affects the bronchi and alveoli. When determining the disease level of COPD, doctors refer to lung field area, rate of change of lung field area, tracheal diameter, rate of change of tracheal diameter, diaphragm displacement, alveolar change, image density, and variance of each change. The disease level of COPD may be a disease stage (stage I, stage II, stage III, or stage IV). The feature may be, for example, a numerical value indicating at least one of the lung field area, rate of change of lung field area, tracheal diameter, rate of change of tracheal diameter, diaphragm displacement, alveolar change, image density, and variance of each change, or one or more still images (parts of the dynamic image) constituting a dynamic image or processed versions of the one or more still images to interpret these numerical values.

[0030] [Image acquisition method] The image acquisition means 111 acquires a medical image (step S301). The medical image is, for example, a dynamic image. The image acquisition means 111 may read out the dynamic image stored in the memory 140, or may receive the dynamic image from the RIS or the like via the communication unit 130.

[0031] [Feature extraction method] The feature extraction means 112 extracts features from the dynamic image acquired by the image acquisition means 111 in step S301 based on machine learning or rules, and outputs the extracted features to the outside from the output unit 122 or outputs (transmits) them to an external device via the communication unit 130.

[0032] The feature extraction means 112 performs lung contour recognition processing based on machine learning on the dynamic image, and obtains the lung field area and the change rate of the lung field area (step S302). Edge processing on the dynamic image may be performed as preprocessing. The feature extraction means 112 may obtain the lung field area of each still image constituting the dynamic image based on machine learning, and obtain the change rate of the lung field area by arithmetic processing. The average, maximum, minimum, median, or mode value of the lung field area of each still image may be used as the lung field area.

[0033] The feature extraction means 112 outputs the lung field area and the change rate of the lung field area extracted in step S302 to the outside (step S303). The feature extraction means 112 may output the maximum lung field area and / or the minimum lung field area.

[0034] The feature extraction means 112 performs trachea recognition processing based on machine learning on the dynamic image, and obtains the trachea diameter and the change rate of the trachea diameter (step S304). Edge processing on the dynamic image may be performed as preprocessing. The feature extraction means 112 may obtain the trachea diameter of each still image constituting the dynamic image based on machine learning, and obtain the change rate of the trachea diameter by arithmetic processing. The average, maximum, minimum, median, or mode value of the trachea diameter of each still image may be used as the trachea diameter.

[0035] The feature extraction means 112 outputs the trachea diameter and the change rate of the trachea diameter extracted in step S304 to the outside (step S305). The feature extraction means 112 may output the maximum trachea diameter and / or the minimum trachea diameter to the outside.

[0036] The feature extraction means 112 performs diaphragm recognition processing based on rules on the dynamic image (step S306). The feature extraction means 112 obtains the position of the diaphragm based on rules for each still image constituting the dynamic image, and obtains the displacement amount of the diaphragm by arithmetic processing. For example, line fitting is used for diaphragm recognition. The feature extraction means 112 may obtain the position of the diaphragm based on machine learning. In this case, the feature extraction means 112 may perform edge processing on the dynamic image as preprocessing.

[0037] The feature quantity extraction means 112 outputs the displacement amount of the diaphragm extracted in step S306 to the outside (step S307).

[0038] The feature quantity extraction means 112 executes at least one of steps S302, S304, and S306. The feature quantity extraction means 112 may determine the process to be executed according to the feature quantity to be extracted. For example, when the feature quantity extraction means 112 extracts the lung field area, the change rate of the lung field area, the tracheal diameter, the change rate of the tracheal diameter, and the displacement amount of the diaphragm as feature quantities, three processes of lung contour recognition processing, trachea recognition processing, and diaphragm recognition processing are performed.

[0039] The machine learning performed by the feature quantity extraction means 112 will be described.

[0040] In the learning phase, the feature quantity extraction means 112 performs learning using learning data with the feature quantity for the dynamic image as teacher data. The feature quantity extraction means 112 performs learning using learning data corresponding to each process, for example, learning data for "lung contour recognition processing" and learning data for "trachea recognition processing". The learning may be learning corresponding to a doctor or the like. The learning may be learning corresponding to the attributes of a patient, such as whether the patient is an adult or a child.

[0041] In the inference phase, the feature quantity extraction means 112 extracts feature quantities from the dynamic image based on the results learned in the learning phase.

[0042] [Determination means] The determination means 113 estimates the disease level of COPD based on machine learning based on the feature quantities extracted by the feature quantity extraction means 112 in steps S302, S304, and S306 (step S308). When the feature quantity extracted by the feature quantity extraction means 112 is an impossible numerical value (for example, the tracheal diameter is 10 cm, etc.), the determination means 113 may exclude the feature quantity before performing the estimation.

[0043] The determination means 113 outputs the COPD disease level estimated in step S308 to the outside (step S309). The determination means 113 may output the estimated COPD disease level from the output unit 122 to the outside, or may output (transmit) it to an external device via the communication unit 130.

[0044] Describe the machine learning performed by the determination means 113.

[0045] In the learning phase, the determination means 113 learns using learning data with the COPD disease level for the feature amount as teacher data.

[0046] In the inference phase, the determination means 113 estimates the COPD disease level from the feature amount based on the result learned in the learning phase.

[0047] [Explanation data generation means] The explanation data generation means 114 generates data that makes it easy for a doctor or the like to grasp the disease situation and explanation data used when a doctor or the like explains the disease situation and treatment policy to a patient, based on the feature amount extracted by the feature amount extraction means 112 and the disease level estimated by the determination means (step S310).

[0048] The generated explanation data may be a still image or a moving image obtained by performing image processing on the moving image acquired by the image acquisition means 111, or may be data to which annotations, markings, numerical values of feature amounts, etc. are added. For example, data with a line indicating the lung field area added to a still image or a numerical value such as the lung field area added may be used as the explanation data.

[0049] <Diagnosis of COPD: Second Embodiment> FIG. 4 is a diagram showing another flowchart of the processing circuit 110 when diagnosing COPD. The disease level of COPD may be a disease stage (stage I, stage II, stage III, stage IV). The feature amount indicates, for example, at least one numerical value of the ratio of the magnitude of movement for each point in the lung field, the area of the entire lung field, the area of the lung field where the movement has decreased, and the ratio of the area of the lung field where the movement has decreased to the area of the entire lung field, or one or more still images (parts of the dynamic image) constituting the dynamic image for understanding these numerical values, or those obtained by processing one or more of the still images.

[0050] [Image acquisition means] The image acquisition means 111 acquires a medical image (step S401). The medical image is, for example, a dynamic image. The image acquisition means 111 may read out the dynamic image stored in the memory 140, or may receive the dynamic image from a RIS or the like via the communication unit 130.

[0051] [Feature amount extraction means] The feature amount extraction means 112 extracts a feature amount based on machine learning from the dynamic image acquired by the image acquisition means 111 in step S401, and outputs the extracted feature amount to the outside from the output unit 122, or outputs (transmits) it to an external device via the communication unit 130.

[0052] The feature amount extraction means 112 calculates an optical flow (difference) from the dynamic image acquired by the image acquisition means 111 in step S401 (step S402). The difference may be a difference with respect to a reference frame (still image) or a difference with respect to the immediately preceding frame. The reference frame may be the first frame. The difference is calculated as a vector. Step S402 is executed as preprocessing.

[0053] Based on the optical flow calculated in step S402, the feature extraction means 112 identifies the lung field based on machine learning. The feature extraction means 112 calculates the horizontal / vertical size of the entire lung field. Also, since the optical flow is a vector, when the optical flow is the difference with respect to the reference frame, the magnitude of the movement can be grasped by the magnitude of the maximum vector during a predetermined period, and when the optical flow is the difference with respect to the immediately preceding frame, the magnitude of the movement can be grasped by the sum of the magnitudes of the vectors during a predetermined period. The feature extraction means 112 calculates the magnitude of the movement in the horizontal / vertical direction (i.e., the horizontal / vertical magnitude of the vector) based on rules for each point in the lung field. The feature extraction means 112 calculates the ratio of the magnitude of the movement in the horizontal / vertical direction for each point to the magnitude of the movement in the horizontal / vertical direction of the entire lung field for each point based on the horizontal / vertical size of the entire lung field and the magnitude of the movement in the horizontal / vertical direction for each point (step S403).

[0054] The feature extraction means 112 outputs externally the ratio of the magnitude of the movement for each point in the lung field calculated in step S403 (step S404). The feature extraction means 112 may output the ratio of the magnitude of the movement for some points.

[0055] Based on the optical flow calculated in step S402, the feature extraction means 112 calculates the ratio of the area of the lung field where the movement has decreased based on machine learning (step S405). The feature extraction means 112 may calculate the ratio of the area of the lung field where the movement has decreased based on the ratio of the magnitude of the movement for each point in the lung field calculated in step S403. For example, when the ratio of the magnitude of the movement is equal to or less than a predetermined threshold, it is determined that the movement has decreased, and the ratio of the area of the lung field where the movement has decreased may be calculated based on the number of points where the movement has decreased and the number of points in the entire lung field.

[0056] The feature extraction means 112 outputs externally the area of the entire lung field, the area of the lung field where the movement has decreased, and the ratio of the area of the lung field where the movement has decreased to the area of the entire lung field calculated in step S405 (step S406).

[0057] Describe the machine learning performed by the feature extraction means 112.

[0058] In the learning phase, the feature extraction means 112 performs learning using learning data with features for the optical flow as teacher data. The feature extraction means 112 performs learning using learning data corresponding to each process, for example, learning data for "lung contour recognition process" and learning data for "calculating the area of the lung field with reduced movement". The learning may be learning corresponding to a doctor or the like. The learning may be learning corresponding to the attributes of a patient, such as whether the patient is an adult or a child.

[0059] In the inference phase, the feature extraction means 112 extracts features from the optical flow based on the results learned in the learning phase.

[0060] [Determination means] Based on the features extracted by the feature extraction means 112 in steps S403 and S405, the determination means 113 estimates the disease level of COPD based on machine learning (step S407). If the features extracted by the feature extraction means 112 are impossible numerical values (for example, the area of the entire lung field is 5 cm 2 etc.), the determination means 113 may exclude those features before performing the estimation.

[0061] The determination means 113 outputs the disease level of COPD estimated in step S407 to the outside (step S408). The determination means 113 may output the estimated disease level of COPD to the outside from the output unit 122, or may output (transmit) it to an external device via the communication unit 130.

[0062] Describe the machine learning performed by the determination means 113.

[0063] In the learning phase, the determination means 113 learns using learning data with the disease level of COPD for the features as teacher data.

[0064] In the inference phase, the determination means 113 estimates the disease level of COPD from the feature quantities based on the results learned in the learning phase.

[0065] [Explanation data generation means] The explanation data generation means 114 generates data that is easy for doctors and the like to understand the disease situation and explanation data used when doctors and the like explain the disease situation and treatment policy to patients, based on the feature quantities extracted by the feature quantity extraction means 112 and the disease level estimated by the determination means (step S409).

[0066] The generated explanation data may be a still image or a moving image obtained by performing image processing on the moving image acquired by the image acquisition means 111, or data to which annotations, markings, numerical values of feature quantities, etc. are added. For example, data with a line indicating the area of the lung field with reduced movement added to a still image or numerical values such as ratios and areas added may also be used as the explanation data.

[0067] <Diagnosis of Fallot: The third embodiment> FIG. 5 is a diagram showing a flowchart of the processing circuit 110 when diagnosing Fallot (tetralogy of Fallot).

[0068] Fallot (tetralogy of Fallot) affects the heart. When doctors and the like make a determination of Fallot, they use feature quantities such as the pulmonary artery waveform and the heartbeat waveform. The disease level of Fallot may be the regurgitation rate or the NYHA (New York Heart Association) cardiac function classification (stage I, II, III, IV). The feature quantity may be, for example, a numerical value indicating at least one of the pulmonary artery waveform and the heartbeat waveform, or one or more still images (parts of the moving image) constituting the moving image or those obtained by processing one or more of these still images for understanding these numerical values.

[0069] [Image acquisition means] The image acquisition means 111 acquires a medical image (step S501). The medical image is, for example, a dynamic image. The image acquisition means 111 may read out a dynamic image stored in the memory 140, or may receive a dynamic image from an RIS or the like via the communication unit 130.

[0070] [Feature quantity extraction means] The feature quantity extraction means 112 extracts feature quantities based on machine learning from the dynamic image acquired by the image acquisition means 111 in step S501, and outputs the extracted feature quantities to the outside from the output unit 122, or outputs (transmits) them to an external device via the communication unit 130.

[0071] The feature quantity extraction means 112 performs a recognition process of the pulmonary artery based on machine learning on the dynamic image, and extracts the waveform of the pulmonary artery (step S502). The feature quantity extraction means 112 may extract the waveforms of the pulmonary artery at a plurality of points, such as a point close to the heart and a point close to the lung, among the recognized pulmonary arteries. The feature quantity extraction means 112 recognizes the pulmonary artery in each still image constituting the dynamic image based on machine learning, selects a plurality of points of the recognized pulmonary artery, and detects the waveform of the pulmonary artery at each point as a change in density corresponding to time. The feature quantity extraction means 112 may recognize a plurality of points of the pulmonary artery in each still image constituting the dynamic image based on machine learning.

[0072] The feature quantity extraction means 112 outputs the waveform of the pulmonary artery extracted in step S502 to the outside (step S503). The feature quantity extraction means 112 may output the waveforms of the pulmonary artery at some points to the outside.

[0073] The feature quantity extraction means 112 performs a recognition process of the cardiac apex based on machine learning on the dynamic image, and extracts the waveform of the cardiac apex, that is, the heartbeat waveform (step S504). The feature quantity extraction means 112 obtains the cardiac apex in each still image constituting the dynamic image based on machine learning, and detects the heartbeat waveform as a change in density corresponding to time.

[0074] The feature quantity extraction means 112 outputs the heartbeat waveform extracted in step S504 to the outside (step S505).

[0075] The feature quantity extraction means 112 may execute only one of steps S502 and S504. When extracting waveforms at a plurality of points of the pulmonary artery, the recognition process of the pulmonary artery may be performed for each point.

[0076] Describe the machine learning performed by the feature quantity extraction means 112.

[0077] In the learning phase, the feature quantity extraction means 112 performs learning using learning data with the feature quantity for the dynamic image as the teacher data. The feature quantity extraction means 112 performs learning using learning data corresponding to each process, for example, learning data for "recognition process of the pulmonary artery" and learning data for "recognition process of the cardiac apex". The feature quantity may be numerical data or image data. The learning may be learning corresponding to a doctor or the like. The learning may be learning corresponding to the attributes of a patient, such as whether the patient is an adult or a child.

[0078] In the inference phase, the feature quantity extraction means 112 extracts a feature quantity from the dynamic image based on machine learning.

[0079] [Determination means] The determination means 113 estimates the disease level of Fallot based on machine learning based on the feature quantities extracted by the feature quantity extraction means 112 in steps S502 and S504 (step S506). If the feature quantity extracted by the feature quantity extraction means 112 is an impossible numerical value (for example, a heart rate of 500 beats per minute, etc.), the determination means 113 may exclude the feature quantity before making the estimation.

[0080] The determination means 113 outputs the disease level of Fallot estimated in step S506 to the outside (step S507). The determination means 113 may output the estimated disease level of Fallot to the outside from the output unit 122, or may output (transmit) it to an external device via the communication unit 130.

[0081] In the learning phase, the determination means 113 learns using learning data with the disease level of Fallot for the feature amount as teacher data.

[0082] In the inference phase, the determination means 113 estimates the disease level of Fallot from the feature amount based on the result learned in the learning phase.

[0083] [Explanation data generation means] The explanation data generation means 114 generates data that is easy for a doctor or the like to understand the disease situation and explanation data used when a doctor or the like explains the disease situation and treatment policy to a patient, based on the feature amount extracted by the feature amount extraction means 112 and the disease level estimated by the determination means (step S508).

[0084] The generated explanation data may be a still image or a moving image obtained by performing image processing on the moving image acquired by the image acquisition means 111, or may be data to which annotations, markings, numerical values of feature amounts, etc. are added. For example, data obtained by adding a line indicating the pulmonary artery to a still image may be used as the explanation data.

[0085] <Diagnosis of CTEPH: The fourth embodiment> FIG. 6 is a diagram showing a flowchart of the processing circuit 110 when diagnosing CTEPH (Chronic ThromboEmbolic Pulmonary Hypertension).

[0086] CTEPH affects the lungs and the heart. When diagnosing CTEPH, doctors and the like use blood flow images, the amount of phase change and the amount of amplitude change for each measurement position. The disease level of CTEPH may be the confidence level, or may be the NYHA cardiac function classification (Class I, II, III, IV) and / or the WHO (World Health Organization) pulmonary hypertension functional classification (Degree I, II, III, IV). The feature quantity is, for example, a numerical value indicating at least one of the amount of phase change and the amount of amplitude change of the blood flow image at each measurement position, or one or more still images (a part of the dynamic image) constituting the dynamic image for understanding these numerical values, or a processed image of one or more of these still images.

[0087] [Image acquisition means] The image acquisition means 111 acquires a medical image (step S601). The medical image is, for example, a dynamic image. The image acquisition means 111 may read out the dynamic image stored in the memory 140, or may receive the dynamic image from an RIS or the like via the communication unit 130.

[0088] [Feature quantity extraction means] The feature quantity extraction means 112 extracts feature quantities from the dynamic image acquired by the image acquisition means 111 in step S601 based on machine learning, and outputs the extracted feature quantities to the outside from the output unit 122, or outputs (transmits) them to an external device via the communication unit 130.

[0089] The feature quantity extraction means 112 identifies the positions of the organs and the lung fields in the dynamic image based on machine learning, and identifies the reference positions and the measurement positions for extracting the amplitude information and the phase information based on the identified positions of the organs and the lung fields (step S602). For example, the reference position may be the hilum of the lung (the position of high blood flow connected to the heart), and the measurement positions may be the upper right lobe, the right middle lobe, the lower right lobe, the upper left lobe, and the lower left lobe. The reference position may be the location with the largest overall signal change.

[0090] The feature extraction means 112 outputs the positions extracted in step S602 to the outside (step S603). The feature extraction means 112 may output the positions of some of the points to the outside.

[0091] The feature extraction means 112 extracts signal changes synchronized with the cardiac cycle from the dynamic image based on machine learning, and generates a blood flow image (step S604). The feature extraction means 112 obtains the cardiac cycle from the dynamic image based on machine learning, and extracts signal changes synchronized with the cardiac cycle.

[0092] The feature extraction means 112 outputs the blood flow image generated in step S604 to the outside (step S605).

[0093] The feature extraction means 112 extracts phase information from the blood flow image (signal changes synchronized with the cardiac cycle) generated in step S604 for each position identified in step S602, and extracts the amount of phase change (step S606). The feature extraction means 112 extracts the amount of phase change based on rules. The feature extraction means 112 compares the phase data of the reference position with the phase data for each measurement position, and calculates the amount of phase change (i.e., delay time) for each measurement position.

[0094] The feature extraction means 112 outputs the phase change amount generated in step S606 to the outside (step S607). The feature extraction means 112 may output the phase change amount of some points to the outside.

[0095] The feature extraction means 112 extracts amplitude information from the blood flow image (signal changes synchronized with the cardiac cycle) generated in step S604 for each position identified in step S602, and extracts the amount of amplitude change (step S608). The feature extraction means 112 extracts the amount of amplitude change based on rules. The feature extraction means 112 compares the amplitude data at the reference position with the amplitude data at each measurement position, and calculates the amount of amplitude change (i.e., attenuation) for each measurement position.

[0096] The feature extraction means 112 outputs the amount of amplitude change generated in step S608 to the outside (step S609). The feature extraction means 112 may output the amount of amplitude change at some points to the outside.

[0097] Describe the machine learning performed by the feature extraction means 112.

[0098] In the learning phase, the feature extraction means 112 performs learning using learning data with the features for the dynamic image as teacher data. The features may be numerical data or image data. The learning may be learning corresponding to a doctor or the like. The learning may be, for example, learning corresponding to the attributes of a patient such as an adult or a child.

[0099] In the inference phase, the feature extraction means 112 extracts features from the dynamic image based on the results learned in the learning phase.

[0100] [Determination means] The determination means 113 estimates the disease level of CTEPH based on machine learning based on the features extracted by the feature extraction means 112 (step S610). If the features extracted by the feature extraction means 112 are impossible numerical values, the determination means 113 may exclude those features before performing the estimation.

[0101] The determination means 113 outputs the disease level of CTEPH estimated in step S610 to the outside (step S611). The determination means 113 may output the estimated disease level of CTEPH to the outside from the output unit 122, or may output (transmit) it to an external device via the communication unit 130.

[0102] Describe the machine learning performed by the determination means 113.

[0103] In the learning phase, the determination means 113 learns using learning data with the disease level of CTEPH for the features as teacher data.

[0104] In the inference phase, the determination means 113 estimates the disease level of CTEPH from the feature amount based on the results of learning in the learning phase.

[0105] [Method for generating explanatory data] The explanation data generation means 114 generates data that makes it easy for doctors and others to understand the condition of the disease, and explanation data that doctors and others can use when explaining the condition of the disease and treatment plan to patients, based on the features extracted by the feature extraction means 112 and the disease level estimated by the judgment means (step S612).

[0106] The explanation data to be generated may be a still image constituting the dynamic image acquired by the image acquisition means 111 or an image obtained by performing image processing on the dynamic image, or may be data to which annotations, markings, numerical values of feature quantities, etc. are added. For example, a still image to which marks indicating measurement positions are added may be used as explanation data.

[0107] [output] 7 to 9 are diagrams showing examples of output from the output unit 122. The data may be transmitted to an external device via the communication unit 130, and the external device may output Fig. 7, 8 or 9. The output unit 122 may output any type of explanation data that allows a doctor to explain to a patient.

[0108] The diseases for which the image assessment device determines the disease level may be selected by a doctor or other medical professional, may be determined based on the patient's medical record, or may be all diseases that can be assessed.

[0109] If extraction of a feature amount is not possible due to the resolution of the medical information or the like, a message indicating that a determination cannot be made may be output from the output unit 122 or the communication unit 130 to the outside.

[0110] (Example output 1) Fig. 7 is a diagram showing an example of output from the output unit 122. A window 700 may be displayed on the display. Fig. 7 shows an example in which the disease levels of COPD and CTEPH are determined.

[0111] The disease level of the disease determined by the determining means 113 is output as a window 700. The disease levels of a plurality of diseases may be output.

[0112] (Example output 2) 8 is a diagram showing another example of output from the output unit 122. A window 800 may be displayed on the display.

[0113] The feature quantity extracted by the feature quantity extraction means 112 is output as a window 800. The explanation data generation means 114 may output an image obtained by performing image processing on the dynamic image and a plurality of feature quantities. A selected feature quantity from the plurality of feature quantities may be displayed in a large size. The feature quantity may be displayed on a display for the patient based on an operation by a doctor or the like.

[0114] (Example output 3) 9 is a diagram showing yet another example of output from the output unit 122. A window 900 may be displayed on the display.

[0115] A disease level 901 of the disease determined by the determination means 113, a medical image 902, an image 903 obtained by image processing on a dynamic image, and feature amounts (numerical data) may be displayed in a window 900. The disease level 901 of the disease, the medical image 902, the image 903 obtained by image processing on a dynamic image, and feature amounts may be displayed in a single window 900 or in separate windows. Images and feature amounts obtained by image processing on a dynamic image may be displayed in a manner that allows selection of images and feature amounts obtained by multiple image processing on a dynamic image. Figure 9 shows an example in which an image obtained by selected image processing on a dynamic image is displayed large, and images obtained by other image processing on a dynamic image are displayed small.

[0116] The output unit 122 may include multiple displays. Different information may be displayed on different displays, such as Fig. 9 being displayed on the doctor's display and Fig. 8 being displayed on the patient's display. The same information may also be displayed on different displays.

[0117] When a doctor or the like explains a treatment policy or the like to a patient, an easy-to-understand explanation can be made by using the output feature amount. The feature amount may be at least one of the examples, but by extracting and outputting many feature amounts, the accuracy of the determination process can be further improved.

[0118] As described above, the embodiments have been described with reference to the drawings, but the present disclosure is not limited to such examples. It is obvious that a person skilled in the art can conceive various modification examples or correction examples within the scope described in the claims. Such modification examples or correction examples are also understood to belong to the technical scope of the present disclosure. Further, within the scope not departing from the gist of the present disclosure, the components in the embodiments may be arbitrarily combined.

[0119] (1) An image determination device according to an embodiment of the present disclosure includes an image acquisition unit that acquires a dynamic image obtained by photographing a site including a diagnostic target region of a patient, a feature amount extraction unit that extracts a feature amount by a first process based on the dynamic image, a determination unit that makes a determination regarding a diagnosis by a second process based on a result of machine learning based on the feature amount, an explanation data generation unit that generates explanation data based on the feature amount and the determination regarding the diagnosis, and an output unit or a communication unit that outputs the explanation data to the outside.

[0120] (2) An image determination device according to an embodiment of the present disclosure is the image determination device according to (1), wherein the first process is a process based on machine learning or a process based on a rule.

[0121] (3) An image determination device according to an embodiment of the present disclosure is the image determination device according to (1), wherein the dynamic image is a radiation image obtained by a radiation imaging device.

[0122] (4) An image determination device according to an embodiment of the present disclosure is the image determination device according to (1), wherein the feature amount includes one or more still images constituting the dynamic image.

[0123] (5) In the image determination device according to an embodiment of the present disclosure, in the image determination device of (1), the determination regarding the diagnosis is a determination of the disease level of a specific disease.

[0124] (6) In the image determination device according to an embodiment of the present disclosure, in the image determination device of (5), the specific disease is COPD, the disease level is a determination of the disease stage, and the feature amount is at least one of a lung field area, a change rate of the lung field area, a tracheal diameter, a change rate of the tracheal diameter, a displacement amount of the diaphragm, a change amount of alveoli, an image density, a variance of each change amount, one or more still images constituting the dynamic image, and the processed one or more still images.

[0125] (7) In the image determination device according to an embodiment of the present disclosure, in the image determination device of (5), the specific disease is COPD, the disease level is a determination of the disease stage, and the feature amount is at least one of a ratio of the magnitude of movement at each point in the lung field, an area of the entire lung field, an area of the lung field where the movement has decreased, a ratio of the area of the lung field where the movement has decreased to the area of the entire lung field, one or more still images constituting the dynamic image, and the processed one or more still images.

[0126] (8) In the image determination device according to an embodiment of the present disclosure, in the image determination device of (5), the specific disease is tetralogy of Fallot, the disease level is a regurgitation rate, and the feature amount is at least one of a waveform of the pulmonary artery, a heartbeat waveform, one or more still images constituting the dynamic image, and the processed one or more still images.

[0127] (9) In the image determination device according to an embodiment of the present disclosure, in the image determination device of (5), the specific disease is CTEPH, the disease level is a confidence level, and the feature amount is at least one of a phase change amount and an amplitude change amount of a blood flow image at each measurement position, one or more still images constituting the dynamic image, and the processed one or more still images.

[0128] (10) An image determination method according to an embodiment of the present disclosure includes an image acquisition step of acquiring a dynamic image obtained by photographing a site including a diagnostic target region of a patient, a feature quantity extraction step of extracting a feature quantity by a first process based on the dynamic image, a determination step of making a determination regarding diagnosis by a second process based on the result of machine learning based on the feature quantity, an explanatory data generation step of generating explanatory data based on the feature quantity and the determination regarding diagnosis, and a step of outputting the explanatory data to the outside.

[0129] (11) An image determination program according to an embodiment of the present disclosure causes a computer to execute an image acquisition step of acquiring a dynamic image obtained by photographing a site including a diagnostic target region of a patient, a feature quantity extraction step of extracting a feature quantity by a first process based on the dynamic image, a determination step of making a determination regarding diagnosis by a second process based on the result of machine learning based on the feature quantity, an explanatory data generation step of generating explanatory data based on the feature quantity and the determination regarding diagnosis, and a step of outputting the explanatory data to the outside.

Industrial Applicability

[0130] The present disclosure is useful for an image determination device.

Explanation of Signs

[0131] 100 Image determination device 110 Processing circuit 111 Image acquisition means 112 Feature quantity extraction means 113 Determination means 120 Input / output unit 121 Input unit 122 Output unit 130 Communication unit 140 Memory

Claims

1. An image acquisition means for acquiring a dynamic image obtained by photographing a site including a diagnostic target area of a patient; A feature quantity extraction means for extracting a feature quantity by a first process based on the dynamic image; A determination means for making a determination related to diagnosis by a second process based on the result of machine learning based on the feature quantity; An explanatory data generation means for generating explanatory data based on the feature quantity and the determination related to diagnosis; An output unit or a communication unit for outputting the explanatory data to the outside; An image determination apparatus comprising the above.

2. The first process is a process based on machine learning or a process based on a rule. The image determination apparatus according to Claim 1.

3. The dynamic image is a radiation image obtained by a radiation imaging apparatus. The image determination apparatus according to Claim 1.

4. The feature quantity includes one or more still images constituting the dynamic image. The image determination apparatus according to Claim 1.

5. The determination related to diagnosis is a determination of the disease level of a specific disease. The image determination apparatus according to Claim 1.

6. The specific disease is COPD; The disease level is a determination of the disease stage; The feature quantity is at least one of a lung field area, a change rate of the lung field area, a tracheal diameter, a change rate of the tracheal diameter, a displacement amount of the diaphragm, a change amount of alveoli, an image density, a variance of each change amount, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination apparatus according to Claim 5.

7. The specific disease is COPD; The disease level is a determination of the disease stage; The feature quantity is at least one of a ratio of the magnitude of movement at each point in the lung field, an area of the entire lung field, an area of the lung field where the movement has decreased, a ratio of the area of the lung field where the movement has decreased to the area of the entire lung field, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination apparatus according to Claim 5.

8. The specific disease is tetralogy of Fallot; The disease level is a regurgitation rate; The feature quantity is at least one of a waveform of the pulmonary artery, a heartbeat waveform, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination apparatus according to Claim 5.

9. The specific disease is CTEPH; The disease level is a confidence level; The feature quantity is at least one of the phase change amount and the amplitude change amount of the blood flow image at each measurement position, one or more still images constituting the dynamic image, and the processed one or more still images. The image determination device according to claim 5.

10. An image acquisition step of acquiring a dynamic image obtained by photographing a site including a diagnostic target region of a patient; A feature quantity extraction step of extracting a feature quantity by a first process based on the dynamic image; A determination step of making a determination regarding diagnosis by a second process based on the result of machine learning based on the feature quantity; An explanatory data generation step of generating explanatory data based on the feature quantity and the determination regarding diagnosis; A step of outputting the explanatory data to the outside; An image determination method comprising the steps.

11. A computer is caused to perform an image acquisition step of acquiring a dynamic image obtained by photographing a site including a diagnostic target region of a patient; perform a feature quantity extraction step of extracting a feature quantity by a first process based on the dynamic image; perform a determination step of making a determination regarding diagnosis by a second process based on the result of machine learning based on the feature quantity; perform an explanatory data generation step of generating explanatory data based on the feature quantity and the determination regarding diagnosis; perform a step of outputting the explanatory data to the outside; An image determination program.

Images