Medical image processing equipment

Abstract

To improve the image reading efficiency.SOLUTION: A medical image processing apparatus according to an embodiment comprises: an acquisition unit; a control unit; and a presentation unit. The acquisition unit acquires a medical image obtained by imaging a subject. The control unit extracts a difference object image being a difference object between the subject and the medical image among the past medical images captured at a plurality of different times prior to the medical image on the basis of first inspection information being information about an inspection in which the medical image is captured and second inspection information being information about an inspection in which the past medical images are captured. The presentation unit presents the difference object image and the medical image together with the second inspection information and the first inspection information of the difference object image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The embodiments disclosed in the present specification and drawings relate to a medical image processing apparatus. [Background technology]

[0002] For example, when a user such as a radiologist wants to determine whether a tumor in a subject has metastasized to a bone, the user needs medical images taken in a previous examination to compare with the medical images taken in the current examination. In this case, for example, the user performs a search using a patient ID or the like to obtain medical images of the subject taken in the previous examination from a server. At this time, the user visually determines whether the obtained medical images are to be compared with the medical images of the subject taken in the current examination. In this case, time is wasted searching for medical images to be compared with before performing radiology interpretation, and the efficiency of radiology interpretation is not improved. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-6130 A Summary of the Invention [Problem to be solved by the invention]

[0004] One of the problems that the embodiments disclosed in this specification and the drawings aim to solve is to improve the efficiency of image interpretation. However, the problems solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0005] The medical image processing apparatus according to the present embodiment includes an acquisition unit, a control unit, and a presentation unit. The acquisition unit acquires a medical image of a subject. The control unit extracts a subtraction target image to be subtracted from the medical image from past medical images taken of the subject at multiple different times prior to the medical image, based on first examination information, which is information on the examination in which the medical image was taken, and second examination information, which is information on the examination in which the past medical image was taken. The presentation unit presents the subtraction target image and the medical image together with the second examination information of the subtraction target image and the first examination information. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a medical image processing system including a medical image processing apparatus according to this embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of a medical image processing apparatus according to this embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of conditions related to photography in this embodiment. [Figure 4] FIG. 4 is a diagram showing an example of information relating to the purpose of the examination in this embodiment. [Diagram 5] FIG. 5 is a diagram for explaining an example of learning using AI in this embodiment. [Figure 6] FIG. 6 is a diagram for explaining an example of a process using a trained model in this embodiment. [Figure 7] FIG. 7 is a flowchart showing the procedure of processing by the medical image processing apparatus 100 according to this embodiment. [Figure 8] FIG. 8 is a diagram for explaining the processing by the medical image processing apparatus 100 according to this embodiment. [Figure 9] FIG. 9 is a diagram for explaining the processing by the medical image processing apparatus 100 according to this embodiment. [Figure 10] FIG. 10 is a diagram for explaining the processing by the medical image processing apparatus 100 according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an embodiment of a medical image processing device will be described in detail with reference to the accompanying drawings. Note that the following description will be given by taking a medical image processing system including the medical image processing device as an example. In the medical image processing system shown in FIG. 1, each device is shown one by one, but in reality, the system may further include multiple devices.

[0008] 1 includes a Hospital Information System (HIS), a Radiology Information System (RIS), and a Picture Archiving and Communication System (PACS). The medical image processing system includes an HIS server 10, a RIS server 20, a medical image diagnostic device 30, a PACS server 40, a terminal 50, an application server 60, and a medical image processing device 100.

[0009] The HIS server 10, the RIS server 20, the medical image diagnostic device 30, the PACS server 40, the terminal 50, the medical image processing device 100, and the application server 60 are connected to, for example, an in-hospital LAN (Local Area Network) installed in a hospital, and transmit information to a predetermined device and receive information transmitted from the predetermined device. The HIS server 10 may be connected to an external network in addition to the in-hospital LAN.

[0010] For example, the terminal 50 is used by a doctor who interprets images using medical images, etc. For example, the terminal 50 includes a PC (Personal Computer), a tablet PC, a PDA (Personal Digital Assistant), a mobile terminal, etc. A viewer (software) is installed in the terminal 50 to display images (medical images) and medical reports (image interpretation reports) based on medical image data on the terminal's own display.

[0011] The HIS is a system that manages information generated within a hospital. Here, we will omit the description of the terminals (HIS terminals) that medical staff use to perform various tasks within the hospital.

[0012] In the HIS, the HIS server 10 shown in Fig. 1 manages information generated within the hospital. Information generated within the hospital includes patient information, test order information, etc. Each record included in the patient information has items such as a patient ID, patient name (first and last name), age (date of birth), sex, height, weight, and blood type. Each record included in the test order information has items such as a test ID that can identify the test, a patient ID, information indicating hospitalization or outpatient care, a test code, a medical department, a type of test, a part of the test, and a scheduled test date and time.

[0013] The examination ID is issued when the examination order information is input, and is an identifier for uniquely identifying the examination order information within, for example, one hospital. The patient ID is assigned to each patient, and is an identifier for uniquely identifying the patient within, for example, one hospital. The examination code is an identifier for uniquely identifying an examination, which is defined within, for example, one hospital. The medical department indicates, for example, a specialized field of medical treatment in medicine. Specifically, the medical department is internal medicine, surgery, etc. The examination type indicates an examination using medical images. For example, the examination types include X-ray examination, CT (Computed Tomography) examination, MRI (Magnetic Resonance Imaging) examination, etc. The examination site includes the brain, kidneys, lungs, liver, bones, etc.

[0014] For example, when test order information is input by a requesting doctor, the HIS server 10 transmits the input test order information and patient information specified by the test order information to the RIS. In this case, the HIS server 10 also transmits the patient information to the PACS.

[0015] The RIS is a system that manages examination reservation information related to radiological examination work. Here, the description of the terminal (RIS terminal) used by, for example, radiological technologists to carry out various tasks in the radiology department is omitted.

[0016] In the RIS, the RIS server 20 shown in Fig. 1 manages information related to radiological examination services. For example, the RIS server 20 receives examination order information transmitted from the HIS server 10, adds various setting information to the received examination order information, accumulates it, and manages the accumulated information as examination reservation information. Specifically, when the RIS server 20 receives patient information and examination order information transmitted from the HIS server 10, it generates examination reservation information required to operate the medical image diagnostic device 30 based on the received patient information and examination order information. The examination reservation information includes information required to perform the examination, such as an examination ID, a patient ID, an examination type, and an examination site. The RIS server 20 transmits the generated examination reservation information to the medical image diagnostic device 30.

[0017] 1 is a device with which a clinical technician performs an examination by photographing a patient. The medical image diagnostic device 30 includes, for example, an X-ray diagnostic device, an X-ray computed tomography device (X-ray CT device), a magnetic resonance imaging device (MRI device), a nuclear medicine diagnostic device, and an ultrasonic diagnostic device. The medical image diagnostic device 30 is also called a modality device.

[0018] The medical image diagnostic apparatus 30 performs an examination based on examination reservation information transmitted from the RIS server 20, for example. Then, the medical image diagnostic apparatus 30 generates examination implementation information indicating the implementation of the examination, and transmits it to the RIS server 20. In this case, the RIS server 20 receives the examination implementation information from the medical image diagnostic apparatus 30, and outputs the received examination implementation information to the HIS server 10 and the terminal 50 as the latest examination implementation information. For example, the HIS server 10 receives the latest examination implementation information and manages the received examination implementation information. For example, the terminal 50 of the image interpretation doctor receives the latest examination implementation information, and creates an image interpretation report, which is a medical report, according to the received examination implementation information. The examination implementation information includes examination reservation information (examination ID, patient ID, examination type, and examination site), and the implementation date and time of the examination, etc.

[0019] Furthermore, the medical image diagnostic device 30 generates medical image data by having a clinical laboratory technician capture an image of a subject (patient) during an examination. The medical image data is, for example, X-ray CT image data, X-ray image data, MRI image data, nuclear medicine image data, and ultrasound image data. The medical image diagnostic device 30 converts the generated medical image data into a format that complies with, for example, the DICOM (Digital Imaging and Communication in Medicine) standard. That is, the medical image diagnostic device 30 generates medical image data to which a DICOM tag is added as supplementary information.

[0020] The incidental information includes, for example, a patient ID, an examination ID, an apparatus ID, an image series ID, conditions related to imaging, and the like, and is standardized according to the DICOM standard. The apparatus ID is information for identifying the medical image diagnostic apparatus 30. The image series ID is information for identifying one imaging by the medical image diagnostic apparatus 30, and includes, for example, the part of the subject (patient) imaged, the time of image generation, slice thickness, slice position, and the like. For example, by performing a CT examination or an MRI examination, tomographic images at each of a plurality of slice positions are obtained as medical image data. The conditions related to imaging will be described later.

[0021] The medical image diagnostic apparatus 30 transmits the generated medical image data to the PACS, which is a system for managing various types of medical image data.

[0022] In the PACS, the PACS server 40 shown in FIG. 1 receives, for example, patient information transmitted from the HIS server 10 and manages the received patient information. The PACS server 40 includes a storage circuit for managing patient information. The PACS server 40 receives, for example, medical image data transmitted from the medical image diagnostic device 30, associates the received medical image data with the patient information, and stores the medical image data in its own storage circuit. The PACS server 40 reads, for example, medical image data in response to an acquisition request from the medical image processing device 100 from its own storage circuit and transmits the medical image data to the medical image processing device 100. The medical image data stored in the PACS server 40 is added with additional information such as a patient ID, an examination ID, an apparatus ID, and an image series ID. Therefore, a user can obtain necessary patient information from the PACS server 40 by performing a search using the patient ID, etc. Here, the user includes a medical professional such as a radiology doctor. In addition, a user can obtain necessary medical image data from the PACS server 40 by performing a search using the patient ID, an examination ID, an apparatus ID, an image series ID, etc.

[0023] The medical image processing device 100 shown in FIG. 1 is a workstation for displaying an image (medical image) based on medical image data and a medical report (image interpretation report) on the display of a terminal 50. An image interpretation doctor performs image interpretation using the medical image and the like displayed on the display of the terminal 50 by the medical image processing device 100. Here, when performing image interpretation, the image interpretation doctor may compare the medical image data of the subject captured in the current examination with the medical image data of the subject captured in a previous examination that precedes the current examination. For example, the image interpretation doctor judges whether or not a tumor in the subject has metastasized to a bone by comparing the medical image data of the subject captured in the current examination with the medical image data of the subject captured in the previous examination.

[0024] A bone subtraction application (program) is installed in the application server 60 shown in Fig. 1. The bone subtraction application can be read by the medical image processing device 100. The bone subtraction application is an application that supports the detection of bone metastasis.

[0025] For example, when the medical image data is X-ray CT image data, the bone subtraction application performs binarization processing such as segmentation on the medical image data in the current examination and the medical image data in the past examination to generate a bone extraction image in which the bone parts are extracted. The bone subtraction application also performs preprocessing such as contrast conversion on the medical image data in the current examination and the medical image data in the past examination to generate a bone-enhanced image in which the bone parts are enhanced as a medical image based on the medical image data. The bone subtraction application then performs registration between the bone extraction images and reflects the result of the registration in the registration between the bone-enhanced images. The bone subtraction application then generates a difference image in which the area of ​​bone metastasis is enhanced by taking the difference between the bone-enhanced image in the current examination and the bone-enhanced image in the past examination, which have been registered.

[0026] For example, in a bone subtraction application, a difference is taken between a medical image of a subject taken in a past examination before a tumor was formed and a medical image of the subject taken in a current examination after a tumor was formed, in which the CT value of the bone part has changed, and a difference image is generated to represent the difference. By referring to the difference image, a radiologist can determine the type of bone metastasis, such as whether the area depicted in the difference image is osteolytic metastasis with an area where bone has been dissolved, osteoblastic metastasis with an area where bone has been created, or mixed metastasis, which is a mixture of osteolytic metastasis and osteoblastic metastasis.

[0027] The medical image processing device 100 according to this embodiment will be described in detail below. Fig. 2 is a diagram showing an example of the configuration of the medical image processing device 100 according to this embodiment. As shown in Fig. 2, the medical image processing device 100 has an input interface 110, displays 120 and 123, a communication interface 130, a memory circuitry 140, and a processing circuitry 150.

[0028] The input interface 110 has a pointing device such as a mouse, a keyboard, etc., and receives input of various operations for the medical image processing device 100 from a user, and transfers the instructions and setting information received from the user to the processing circuitry 150. Here, the user refers to a medical professional including an image interpretation doctor.

[0029] The display 120 is a monitor viewed by the user, and displays image data to the user under the control of the processing circuit 150, and displays a GUI (Graphical User Interface) for receiving various instructions and settings from the user via the input interface 110. The communication interface 130 is a NIC (Network Interface Card) or the like, and communicates with other devices.

[0030] The memory circuit 140 is, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Here, the memory circuit 140 is an example of a memory unit.

[0031] The processing circuitry 150 controls the components of the medical image processing device 100. For example, the processing circuitry 150 executes a control function 151 and a display control function 152 as shown in FIG. 2. Here, for example, each processing function executed by the control function 151 and the display control function 152, which are components of the processing circuitry 150, is recorded in the storage circuitry 140 in the form of a program executable by a computer. The processing circuitry 150 is a processor that realizes a function corresponding to each program by reading each program from the storage circuitry 140 and executing it. In other words, the processing circuitry 150 in a state in which each program is read has each function shown in the processing circuitry 150 in FIG. 2. The control function 151 is an example of an acquisition unit and a control unit, and the display control function 152 is an example of a presentation unit.

[0032] The term "processor" used in the above description means a circuit such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). When the processor is a CPU, for example, the processor realizes a function by reading and executing a program stored in the memory circuit 140. On the other hand, when the processor is an ASIC, for example, instead of storing a program in the memory circuit 140, the program is directly incorporated in the circuit of the processor. Note that each processor in this embodiment is not limited to being configured as a single circuit for each processor, and may be configured as one processor by combining multiple independent circuits to realize its function. Furthermore, the multiple components in FIG. 2 may be integrated into one processor to realize its function.

[0033] The above describes the overall configuration of the medical image processing system including the medical image processing apparatus 100 according to this embodiment. With this configuration, the medical image processing apparatus 100 improves the efficiency of image interpretation.

[0034] For example, when a user such as an image interpreter determines whether a tumor in a subject has metastasized to a bone, the user needs medical images taken in a previous examination to compare with the medical images taken in the current examination. In this case, for example, the user performs a search using a patient ID or the like to obtain the medical images of the subject taken in the previous examination from the PACS server 40. At this time, the user visually determines whether the obtained medical image is to be used as a difference target for the medical image of the subject taken in the current examination.

[0035] The reason for this is that the medical image of the subject taken in the current examination and the medical image of the subject taken in the past examination must satisfy the imaging conditions. The imaging conditions include, for example, whether the medical image taken in the current examination and the medical image taken in the past examination are taken in the same body position. For example, if the medical image taken in the current examination is a medical image taken with the subject's arms raised, the medical image taken in the past examination must also be a medical image taken with the subject's arms raised. The imaging conditions include, for example, whether the medical image taken in the current examination and the medical image taken in the past examination include an area including a landmark for facilitating alignment, such as the scapula and pelvis. For example, if the medical image taken in the current examination is a medical image of an area including the scapula of the subject, it is preferable for alignment that the medical image taken in the past examination is also a medical image of an area including the scapula of the subject. In either case, the user visually determines whether the medical image acquired from the PACS server 40 and the medical image captured in the current examination satisfy the imaging conditions.

[0036] Another reason is that the purpose of the examination in which the medical image was taken in the past examination is important in the medical image of the subject taken in the past examination. For example, when the user refers to the medical image of the subject taken in the current examination and finds that the tumor of the subject may have metastasized to the bone, the user wants to refer to the medical image before the bone metastasis occurred as the medical image of the subject taken in the past examination. The medical image before the bone metastasis occurs is, for example, the medical image of the subject whose primary lesion was taken in the past examination. In addition, for example, the user wants to check the prognosis of the treatment for the subject who has already been treated for bone metastasis, so the user refers to the medical image taken after the treatment of bone metastasis as the medical image of the subject taken in the current examination and wants to refer to the medical image taken before the treatment of bone metastasis. In either case, the user visually judges for what examination purpose the medical image of the subject acquired from the PACS server 40 was taken.

[0037] In this way, the user visually judges whether or not the medical image acquired from the PACS server 40 is a medical image to be subtracted from the medical image of the subject captured in the current examination. In this case, time is wasted searching for the medical image to be subtracted before interpretation, and the efficiency of interpretation is not improved.

[0038] Therefore, in the medical image processing apparatus 100 according to this embodiment, the following process is performed to improve the efficiency of image interpretation. First, in the medical image processing apparatus 100 according to this embodiment, the control function 151 acquires a medical image of a subject. The control function 151 extracts a subtraction target image to be subtracted from the acquired medical image from among past medical images of the subject taken at multiple different times before the medical image, based on first examination information, which is information on the examination in which the medical image was taken, and second examination information, which is information on the examination in which the past medical image was taken. The display control function 152 causes the terminal 50 to present the subtraction target image and the medical image together with the second examination information and the first examination information of the subtraction target image. The past medical image is an example of a medical image of the subject taken in a past examination. The acquired medical image is an example of a medical image of the subject taken in the current examination. The subtraction target image is an example of a medical image to be subtracted from the medical image of the subject taken in the current examination.

[0039] Here, the medical images to be subjected to subtraction are automatically extracted by learning or machine learning using AI (Artificial Intelligence) based on a neural network, for example.

[0040] For example, in this embodiment, the conditions for photographing medical images taken in an examination are learned using a model described later. FIG. 3 is a diagram showing an example of the conditions for photographing in this embodiment. The conditions for photographing include information required for positioning between images. The conditions for photographing include, for example, reconstruction conditions, photographing conditions, and photographing range. The reconstruction conditions include, for example, a condition as to whether the medical image of the subject photographed in the current examination and the medical image of the subject photographed in the past examination are created using the same reconstruction function. Examples of the reconstruction function include a reconstruction function for soft tissue and a reconstruction function for lung fields. The photographing conditions include, for example, a condition as to whether the medical image of the subject photographed in the current examination and the medical image of the subject photographed in the past examination are photographed with the same parameters. Examples of the parameters include tube voltage and tube current. The photographing conditions include, for example, a condition as to whether the medical image of the subject photographed in the current examination and the medical image of the subject photographed in the past examination are photographed in the same position. The body position may be a subject with arms raised or lowered, a subject lying on the ground, etc. The imaging range may include conditions such as whether or not an area including landmarks such as the shoulder blades and pelvis is imaged in the medical image of the subject taken in the current examination and the medical image of the subject taken in the previous examination. The above-mentioned reconstruction conditions, imaging conditions, imaging range, etc. are constraints for aligning the images.

[0041] Also, for example, in this embodiment, information on the purpose of the examination in which the medical image was taken is learned using a model described later. FIG. 4 is a diagram showing an example of information on the purpose of the examination in this embodiment. As shown in FIG. 4, for example, the examination is performed in chronological order, such as "initial examination", "preoperative plan", "radiotherapy", "follow-up 1", "radiotherapy", "follow-up 2", and "present". For example, if the subject is suspected of having a tumor, the medical image of the subject is taken by the medical image diagnostic device 30 in the "initial examination" as the purpose of the examination. Then, it is assumed that "preoperative plan" is implemented and "radiotherapy" is performed on the subject. In this case, since it is desired to confirm the prognosis of the treatment, the medical image of the subject is taken by the medical image diagnostic device 30 in the "follow-up" of the first radiation therapy as the purpose of the examination. Then, it is assumed that "radiotherapy" is performed on the subject again. In this case, since it is desired to confirm the prognosis of the treatment, the medical image of the subject is taken by the medical image diagnostic device 30 in the "follow-up" of the second radiation therapy as the purpose of the examination. Then, at the "present", a medical image of the subject is taken by the medical image diagnostic device 30 so that a doctor can determine whether or not the tumor in the subject has metastasized to the bone.

[0042] The flow of the examination is not limited to the example shown in Fig. 4. In Fig. 4, "radiotherapy" is performed on the subject after "preoperative planning" is performed, but "radiotherapy" may be performed after surgery on the subject after "preoperative planning". Also, in the "follow-up" of the second radiation therapy, it may be found that the subject's tumor has metastasized to the bone, so the next examination may be performed "currently".

[0043] In addition, for example, in this embodiment, the time when the medical image was taken is also learned using a model described later. For example, even if the medical image of the subject is a medical image before bone metastasis occurs, if the medical image is a medical image taken when the subject is in a traffic accident, the medical image may not be useful for the user in interpreting the image, so it is excluded from the extraction target. For example, if the medical image of the subject is a medical image taken more than 10 years ago, it may not be useful for the user in interpreting the image, so it is excluded from the extraction target.

[0044] Fig. 5 is a diagram for explaining an example of learning using AI in this embodiment. Fig. 5 shows a block diagram assuming deep learning using AI. Fig. 6 is a diagram for explaining an example of processing using a trained model in this embodiment.

[0045] In this embodiment, the trained model is generated, for example, by the control function 151 of the medical image processing device 100 and stored in the memory circuitry 140 in the medical image processing device 100. Here, the trained model may be realized by an in-hospital device or an external device.

[0046] In the process of generating a trained model, as shown in FIG. 5, first, the control function 151 inputs a training dataset 501 obtained when an examination was performed in the past. The training dataset 501 includes data that associates a plurality of medical images taken in a past examination with information on the examination in which the plurality of medical images were taken. The data is stored in the memory circuitry 140. For example, the plurality of medical images include medical images of a subject taken in each of examinations performed on the subject at a plurality of different times. Note that, since algorithms such as AI learn from experience, the dataset used for learning does not have to be of the same subject.

[0047] Here, the information on the examination includes an examination ID, a patient ID, an examination type, an examination site, the date and time of the examination, conditions for taking the medical image taken in the examination, information on the purpose of the examination for which the medical image was taken, and the like. As described above, the examination type indicates an examination using a medical image. For example, the examination type includes an X-ray examination, a CT examination, and an MRI examination. The examination site includes the brain, the kidney, the lung, and the liver, and the like. The information on the examination includes information on findings, and the findings include information on the type of tumor, the location of the primary lesion, age, and the like. The conditions on the imaging include information required for positioning between images, and include, for example, the above-mentioned reconstruction conditions, imaging conditions, imaging range, and the like. The information on the purpose of the examination for which the medical image was taken includes, for example, information on whether the medical image of the subject taken in the past examination is a medical image taken at the "initial consultation", a medical image taken as a "follow-up" of the first radiation therapy, or a medical image taken as a "follow-up" of the second radiation therapy. In addition, the information regarding the purpose of the examination for which the medical image was taken includes, for example, information as to whether the medical image of the subject was taken when the subject was in a traffic accident, or whether the image was taken more than 10 years ago.

[0048] The control function 151 has a learning program 502 as an AI algorithm. The control function 151 uses the learning program 502 to learn from the learning dataset 501 a plurality of medical images taken in past examinations and information on the examinations in which the plurality of medical images were taken, and generates a trained model 503 that is functionalized to receive input data described below and output medical images extracted from the plurality of medical images and information on the examinations of the medical images as output data. For example, the generated trained model 503 is stored in the memory circuitry 140. Here, the learning method may be "supervised learning" in which learning data is trained with a correct answer given, or "unsupervised learning" in which learning data is trained without a correct answer given.

[0049] 6, in a process using the trained model 503, for example, the control function 151 inputs a medical image of the subject captured in the current examination from the terminal 50 as input data 504. At this time, the control function 151 reads out the trained model 503 from the memory circuitry 140.

[0050] Then, the control function 151 inputs the medical image of the subject taken in the current examination and information related to the examination of the medical image as input data 504 from the terminal 50 to the trained model 503 read out from the storage circuitry 140, thereby extracting, from the multiple medical images taken in the past examinations, a difference target image to be a difference target with the medical image of the subject taken in the current examination. Then, the control function 151 outputs the extracted medical image and information related to the examination of the medical image as output data 505 to the terminal 50. For example, the display control function 447 displays the output data 505 on the display of the terminal 50 together with the medical image of the subject taken in the current examination and information related to the examination of the medical image.

[0051] In this way, the medical image processing apparatus 100 uses the learning program 502 to learn a plurality of medical images taken in past examinations together with information about the examinations, and generates a trained model 503. As a result, the medical image processing apparatus 100 inputs the medical images of the subject taken in the current examination and information about the examination of the medical images to the trained model 503, and can automatically extract, from among the plurality of medical images taken in past examinations, a difference target image that is to be a difference target with the medical image of the subject taken in the current examination.

[0052] Below, each function of the control function 151 and the display control function 152 will be described with reference to Fig. 7 to Fig. 10. Fig. 7 is a flowchart showing the procedure of processing by the medical image-processing device 100 according to this embodiment. Figs. 8 to 10 are diagrams for explaining the processing by the medical image-processing device 100 according to this embodiment.

[0053] First, a case where one subtraction target image to be subjected to subtraction between medical images of the subject captured in the current examination will be explained with reference to FIGS. 7 and 8. FIG.

[0054] In step S101 of FIG. 7, the control function 151 executes an extraction process.

[0055] First, the control function 151 acquires the medical image of the subject taken in the current examination and information related to the examination of the medical image. For example, the control function 151 acquires the medical image of the subject taken in the current examination and information related to the examination of the medical image from the PACS server 40. Then, the control function 151 inputs the medical image of the subject taken in the current examination and information related to the examination of the medical image as input data 504 from the terminal 50 to the trained model 503 read from the storage circuitry 140, thereby extracting one difference target image that is to be the difference target of the medical image of the subject taken in the current examination from multiple medical images taken in past examinations. At this time, the control function 151 outputs the extracted medical image and information related to the examination of the medical image as output data 505.

[0056] Next, in step S102 of FIG. 7, the display control function 152 executes a presentation process.

[0057] For example, the display control function 152 displays, on the display of the terminal 50, a subtraction target image which is a medical image extracted as the output data 505 and which is a subtraction target of the medical image of the subject captured in the current examination, and information related to the examination of the medical image, together with the medical image of the subject captured in the current examination and information related to the examination of the medical image. For example, the display control function 152 displays a screen 200 shown in FIG. 8 on the display of the terminal 50.

[0058] 8 displays a medical image 230 of the subject captured in the current examination, and a subtraction target image 220 that is to be subtracted from the medical image 230. The screen 200 shown in Fig. 8 also includes display fields 210 and 240, and the display field 240 displays information related to the examination of the subtraction target image 220 and information related to the examination of the medical image 230. The display field 210 displays the examination corresponding to the extracted subtraction target image 220 in an identifiable manner in a schematic diagram showing a plurality of examinations performed on the subject in chronological order.

[0059] For example, when a subject is suspected of having a tumor, the subject is assumed to have been photographed by the medical image diagnostic device 30 at the "initial consultation". Then, after the "preoperative plan" is implemented and the subject is subjected to "radiotherapy", the subject is assumed to have been photographed by the medical image diagnostic device 30 as "follow-up 1" of the first radiation therapy. Also, after the subject is subjected to "radiotherapy" again, the subject is assumed to have been photographed by the medical image diagnostic device 30 as "follow-up 2" of the second radiation therapy. Then, at the "present", the subject is assumed to have been photographed by the medical image diagnostic device 30 at the current examination so that the radiologist can determine whether the subject's tumor has metastasized to the bone. Here, the subtraction target image 220 that is the subtraction target of the medical image 230 is assumed to be the medical image of the subject photographed at the examination "initial consultation".

[0060] In this case, the display control function 152 displays the examination "initial examination" corresponding to the extracted subtraction target image 220 in the display field 210 in a identifiable manner in a schematic diagram showing a plurality of examinations "initial examination", "preoperative plan", "radiotherapy", "follow-up 1", "radiotherapy", "follow-up 2", and "current" performed on the subject in chronological order. Here, the display control function 152 controls the display form of the examination "initial examination" corresponding to the subtraction target image 220 and the examination "current" corresponding to the medical image 230 in the display field 210. For example, as a control of the display form, the examination "initial examination" corresponding to the subtraction target image 220 and the examination "current" corresponding to the medical image 230 are displayed by hatching. Alternatively, as a control of the display form, control such as changing the color or blinking may be performed. In addition, the display control function 152 displays information on the examination "initial examination" of the subtraction target image 220 and information on the examination "current" of the medical image 230 in the display field 240.

[0061] 8, a difference image 250 between the difference target image 220 and the medical image 230 is displayed. For example, when one difference target image 220 is extracted, the control function 151 reads out a bone subtraction application implemented in the application server 60, and generates a difference image 250 between the difference target image 220 and the medical image 230 using the read out bone subtraction application. The display control function 152 displays the difference image 250 on the screen 200. Here, by referring to the difference image 250, the user can determine the type of bone metastasis, such as whether the area depicted in the difference image 250 is osteolytic metastasis, osteoblastic metastasis, or mixed metastasis. Furthermore, by referring to information on the examination "initial examination" of the difference target image 220 displayed in the display field 240 and information on the examination "current" of the medical image 230, the user can determine whether the comparison between the difference target image 220 and the medical image 230 is appropriate.

[0062] In step S103 of FIG. 7, the control function 151 executes an update process.

[0063] For example, the screen 200 shown in FIG. 8 includes an acceptance button "Accept" 260 and an acceptance button "Dismiss" 270. The acceptance button "Accept" 260 is operated by a user when the difference image 250 displayed on the screen 200 is used for image interpretation. When the acceptance button "Accept" 260 is operated, the control function 151 accepts information indicating that the difference image 250 displayed on the screen 200 is used for image interpretation. The acceptance button "Dismiss" 270 is operated by a user when the difference image 250 displayed on the screen 200 is not used for image interpretation. When the acceptance button "Dismiss" 270 is operated, the control function 151 accepts information indicating that the difference image 250 displayed on the screen 200 is not used for image interpretation. When the control function 151 accepts information indicating whether or not the difference image 250 displayed on the screen 200 is used for image interpretation, the control function 151 updates the trained model 503 based on the information. That is, information indicating whether or not the difference image 250 is to be used for interpretation is fed back to the trained model 503.

[0064] In this manner, the medical image processing apparatus 100 according to this embodiment inputs the medical image 230 of the subject captured in the current examination and information related to the examination of the medical image 230 to the trained model 503, and automatically extracts the difference target image 220, which is to be the difference target with the medical image 230 of the subject captured in the current examination, from among multiple medical images captured in past examinations, thereby improving the efficiency of image interpretation.

[0065] Next, a case where a plurality of subtraction target images are extracted as subtraction targets for the medical images of the subject captured in the current examination will be described with reference to FIGS.

[0066] First, in step S101 of FIG. 7, the control function 151 executes an extraction process.

[0067] For example, the control function 151 acquires the medical image of the subject taken in the current examination and information on the examination of the medical image from the PACS server 40. Then, the control function 151 inputs the medical image of the subject taken in the current examination and information on the examination of the medical image as input data 504 from the terminal 50 to the trained model 503 read from the storage circuitry 140, thereby extracting multiple subtraction target images to be subtracted from the medical image of the subject taken in the current examination from multiple medical images taken in past examinations. Here, the trained model 503 associates accuracy with the subtraction target images to be subtracted from the medical image of the subject taken in the current examination based on the shooting conditions and information on the purpose of the examination, and scores them according to the accuracy. Then, the trained model 503 extracts the subtraction target images to be subtracted from the medical image of the subject taken in the current examination, for example, in descending order of accuracy. In FIG. 9, three medical images are extracted. At this time, the control function 151 outputs the extracted medical images and information related to the examination of the medical images as output data 505.

[0068] Next, in step S102 of FIG. 7, the display control function 152 executes a presentation process.

[0069] For example, the display control function 152 displays, on the display of the terminal 50, a plurality of subtraction target images to be subtracted from the medical image of the subject captured in the current examination, which are medical images extracted as the output data 505, together with the medical image of the subject captured in the current examination and information regarding the examination of the medical images. For example, the display control function 152 displays a screen 200 shown in FIG. 9 on the display of the terminal 50.

[0070] 9 displays a medical image 230 of the subject captured in the current examination, and a subtraction target image 221 that is to be subtracted from the medical image 230. The screen 200 shown in FIG. 9 also includes display fields 210 and 240, an acceptance button "Accept" 260, an acceptance button "Dismiss" 270, and an acceptance button 280 "Subtraction". The acceptance button 280 "Subtraction" is operated by the user when generating a subtraction image between the subtraction target image 221 and the medical image 230. The operation of the acceptance button 280 "Subtraction" will be described later.

[0071] Here, when a plurality of subtraction target images are extracted as subtraction targets for the medical images of the subject captured in the current examination, the subtraction images are not displayed on the screen 200 shown in FIG. 9. For example, when subtraction images are generated for all the extracted medical images, it takes time to execute the bone subtraction application. Therefore, in this embodiment, a subtraction image is generated for a selected subtraction target image among the plurality of extracted subtraction target images. Note that a subtraction image may be generated and displayed for all the extracted subtraction target images.

[0072] For example, in the display field 210, in a schematic diagram showing a plurality of examinations performed on a subject in chronological order, examinations corresponding to the plurality of extracted subtraction target images 220 are displayed in an identifiable manner. Here, the plurality of extracted subtraction target images are assumed to be medical images of the subject taken in the examinations "initial examination", "follow-up 1", and "follow-up 2". In this case, the display control function 152 causes the display field 210 to identifiablely display the examinations "initial examination", "follow-up 1", and "follow-up 2" corresponding to the plurality of extracted subtraction target images in a schematic diagram showing a plurality of examinations performed on the subject in chronological order, namely "initial examination", "preoperative plan", "radiotherapy", "follow-up 1", "radiotherapy", "follow-up 2", and "present".

[0073] For example, when the medical image of the subject taken in the examination "follow-up 1" is the medical image with the highest accuracy, the display control function 152 displays the medical image of the subject taken in the examination "follow-up 1" on the screen 200 as a subtraction target image 221. In this case, the display control function 152 controls the display form of the examination "follow-up 1" corresponding to the subtraction target image 221 and the examination "current" corresponding to the medical image 230 in the display field 210.

[0074] Furthermore, the display control function 152 causes information relating to the examination “follow-up 1” of the subtraction target image 221 and information relating to the examination “current” of the medical image 230 to be displayed in the display field 240. Note that the display control function 152 may cause information relating to the examinations of the multiple extracted subtraction target images to be displayed in the display field 240 in descending order of accuracy.

[0075] Here, the display control function 152 accepts switching of display for the extracted subtraction target images. For example, when the user refers to the subtraction target image 221, the medical image 230, the information on the examination in the display field 240, and the schematic diagram in the display field 210 displayed on the screen 200, and judges that the medical image of the subject captured in the examination "follow-up 2" is more suitable as the subtraction target image to be the subtraction target of the medical image of the subject captured in the current examination than the medical image of the subject captured in the examination "follow-up 1", the user switches the display.

[0076] Specifically, the display control function 152 accepts, in the display field 210, the selection of an examination corresponding to one subtraction target image to be a subtraction target of the medical image of the subject captured in the current examination, among the multiple extracted subtraction target images. For example, in the display field 210, the examination "follow-up 2" is selected by a user's operation. At this time, the display control function 152 accepts the selection of the examination "follow-up 2" and displays a screen 200 shown in FIG. 10 on the display of the terminal 50. In this case, the display control function 152 displays the medical image of the subject captured in the examination "follow-up 2" as a subtraction target image 221 on the screen 200, and controls the display form of the examination "follow-up 2" corresponding to the subtraction target image 221 and the examination "current" corresponding to the medical image 230 in the display field 210. In addition, the display control function 152 displays information on the examination "follow-up 2" of the subtraction target image 221 and information on the examination "current" of the medical image 230 in the display field 240.

[0077] In step S103 of FIG. 7, the control function 151 executes an update process.

[0078] For example, when the examination "follow-up 2" is selected by a user operation in the display field 210, the control function 151 accepts the selection of the examination "follow-up 2". Upon accepting information representing the selection of the examination "follow-up 2", the control function 151 updates the trained model 503 based on the selected information. That is, the control function 151 feeds back information representing the selection of a medical image by the user to the trained model 503.

[0079] Here, since the user's selection of medical images is reflected in the trained model 503, for example, when a medical image of the subject is captured in an examination subsequent to the current examination, the display control function 152 may display information regarding the examination of multiple extracted difference target images in the display area 240 in order of the user's tendency to select medical images.

[0080] In addition, in the screen 200 shown in FIG. 10, when the user operates the reception button 280 "Subtraction" after the medical image of the subject taken in the examination "follow-up 2" is selected, step S102 in FIG. 7 is executed. For example, when the reception button 280 "Subtraction" is operated, the control function 151 reads out a bone subtraction application implemented in the application server 60, and generates a difference image between the subtraction target image 221 and the medical image 230 using the read bone subtraction application. The display control function 152 displays the difference image on the screen 200. Here, the user can determine the type of bone metastasis, such as whether the area depicted in the difference image is osteolytic metastasis, osteoblastic metastasis, or mixed metastasis, by referring to the difference image. Furthermore, the user can determine whether the comparison between the difference target image 221 and the medical image 230 is appropriate by referring to information regarding the examination "Follow-up 2" of the difference target image 221 displayed in the display field 240 and information regarding the examination "Current" of the medical image 230.

[0081] Here, when the acceptance button "Accept" 260 or the acceptance button "Dismiss" 270 is operated, step S103 in FIG. 7 is executed. For example, when the acceptance button "Accept" 260 is operated, the control function 151 accepts information indicating that the difference image displayed on the screen 200 is used for interpretation. When the acceptance button "Dismiss" 270 is operated, the control function 151 accepts information indicating that the difference image displayed on the screen 200 is not used for interpretation. When the control function 151 accepts information indicating whether or not the difference image displayed on the screen 200 is used for interpretation, the control function 151 updates the trained model 503 based on the information. That is, the information indicating whether or not the difference image is used for interpretation is fed back to the trained model 503.

[0082] In this way, the medical image processing apparatus 100 according to this embodiment inputs the medical image 230 of the subject captured in the current examination and information related to the examination of the medical image 230 to the trained model 503, and automatically extracts a plurality of subtraction target images to be subtracted from the medical image 230 of the subject captured in the current examination from a plurality of medical images captured in past examinations, so that the user can select a medical image suitable for interpretation from the plurality of extracted subtraction target images. Therefore, the medical image processing apparatus 100 according to this embodiment can improve the efficiency of interpretation.

[0083] (Other embodiments) Although the embodiment has been described above, the present invention may be embodied in various different forms other than the above-described embodiment.

[0084] In the above-described embodiment, when the control function 151 extracts a plurality of subtraction target images to be subtraction targets of the medical image of the subject captured in the current examination using the trained model 503, the user selects a medical image suitable for interpretation from the plurality of extracted subtraction target images, but this is not limited thereto. For example, the user may select a medical image that has not been extracted as a subtraction target of the medical image 230 of the subject captured in the current examination as a subtraction target of the medical image 230 of the subject captured in the current examination.

[0085] For example, it is assumed that the extracted subtraction target images are medical images of the subject taken in the examinations "follow-up 1" and "follow-up 2", and that the medical image of the subject taken in the examination "initial examination" is not extracted. Here, for example, in the display field 210, the examination "initial examination" is selected by the user's operation. At this time, the display control function 152 accepts the selection of the examination "initial examination". In this case, the display control function 152 displays the medical image of the subject taken in the examination "initial examination" on the screen 200 as, for example, the subtraction target image 220 shown in FIG. 8, and controls the display form of the examination "initial examination" corresponding to the subtraction target image 220 and the examination "current" corresponding to the medical image 230 in the display field 210. In addition, the display control function 152 displays information on the examination "initial examination" of the subtraction target image 220 and information on the examination "current" of the medical image 230 in the display field 240.

[0086] Furthermore, when the examination "initial examination" is selected by the user's operation in the display field 210, the control function 151 accepts the selection of the examination "initial examination". When the control function 151 accepts information representing the selection of the examination "initial examination", it updates the trained model 503 based on the information. That is, it feeds back information representing the selection of the medical image by the user to the trained model 503. In this way, when a medical image that has not been extracted as a difference target of the medical image 230 of the subject captured in the current examination is selected as a difference target of the medical image 230 of the subject captured in the current examination, the control function 151 updates the trained model 503 based on the selected medical image.

[0087] In the above-described embodiment, the case of determining whether or not a tumor in a subject has metastasized to a bone is taken as an example. That is, the medical image is an image of a bone. However, the present invention is not limited to this. For example, the above-described embodiment is applicable to any tissue that is easy to align two images and to subtract the two images. As an example, the above-described embodiment is applicable to the brain, which is surrounded by bones. That is, the medical image may be an image of the brain.

[0088] For example, when the subject's brain is photographed in the current examination, the control function 151 inputs the medical image of the subject photographed in the current examination and information on the examination of the medical image to the trained model 503, and extracts a difference target image to be a difference target of the medical image of the subject photographed in the current examination from a plurality of medical images photographed in past examinations. Even in this case, the display control function 152 displays the extracted medical image and information on the examination of the medical image on the display of the terminal 50 together with the medical image of the subject photographed in the current examination and information on the examination of the medical image. Then, the user judges, for example, whether a tumor in an organ has metastasized to the brain by referring to the difference image between the extracted medical image and the medical image of the subject photographed in the current examination.

[0089] In the above-described embodiment, the control function 151 extracts a subtraction target image to be a subtraction target of the medical image of the subject captured in the current examination by using the trained model 503, but is not limited to this. For example, when the medical image of the subject captured in the current examination and information related to the examination of the medical image are input from the terminal 50 as input parameters, the control function 151 may refer to the PACS server 40 and extract a medical image corresponding to the input parameters from a plurality of medical images captured in past examinations as a subtraction target image to be a subtraction target of the medical image of the subject captured in the current examination.

[0090] In addition, each component of each device illustrated in the present embodiment is a functional concept, and does not necessarily have to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to that illustrated, and all or a part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0091] The method described in this embodiment can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. This program can also be recorded on a non-transitory computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, or a DVD, and executed by being read from the recording medium by a computer.

[0092] According to at least one of the embodiments described above, the efficiency of image interpretation can be improved.

[0093] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0094] 100 Medical image processing device 151 Control Functions 152 Display control function

Claims

1. an acquisition unit for acquiring medical images of a subject; a control unit that extracts a subtraction target image to be subtracted from the medical image from among past medical images taken of the subject at multiple different times prior to the medical image, based on first examination information, which is information regarding the examination in which the medical image was taken, and second examination information, which is information regarding the examination in which the past medical image was taken; a presentation unit that presents the subtraction target image, the medical image, second examination information of the subtraction target image, and the first examination information, and presents the examination corresponding to the subtraction target image in a identifiable manner in a schematic diagram showing a plurality of examinations performed on the subject in chronological order; A medical image processing device comprising:

2. An acquisition unit for acquiring medical images of a subject; a control unit that extracts a subtraction target image to be subtracted from the medical image from among past medical images taken of the subject at multiple different times prior to the medical image, based on first examination information, which is information regarding the examination in which the medical image was taken, and second examination information, which is information regarding the examination in which the past medical image was taken; a presentation unit that presents the subtraction target image and the medical image together with second examination information of the subtraction target image and the first examination information; Equipped with The control unit inputs the medical images and the first examination information as input data to a trained model using a plurality of medical images and information on the examinations in which the plurality of medical images were taken, and outputs the difference target image extracted from the plurality of past medical images taken at different times and second examination information of the difference target image as output data. Medical imaging equipment.

3. The information regarding the examination includes conditions for taking medical images taken during the examination and information regarding the purpose of the examination; The photographing conditions include information required for image registration. The medical image processing device according to claim 2 .

4. The presentation unit causes a difference image between the difference target image and the medical image to be presented, When the control unit receives information indicating whether the presented difference image is to be used for image interpretation, the control unit updates the trained model based on the information. The medical image processing device according to claim 2 or 3.

5. when a plurality of difference targets are extracted, the presentation unit presents tests corresponding to the plurality of extracted difference targets in a identifiable manner in a schematic diagram illustrating a plurality of tests performed on the subject in a chronological order; When the control unit receives, from the schematic diagram, a selection of an examination corresponding to one difference target to be a difference target of the medical image from among the plurality of extracted difference targets, the control unit updates the trained model based on the selected information. The medical image processing device according to any one of claims 2 to 4.

6. the presentation unit, after the one difference target is selected, receives a request to perform a difference between the one difference target image, which is the medical image selected as the one difference target, and the medical image, and presents a difference image between the one difference target image and the medical image; When the control unit receives information indicating whether the presented difference image is to be used for image interpretation, the control unit updates the trained model based on the information. The medical image processing device according to claim 5 .

7. When the past medical image that has not been extracted as the subtraction target image is selected as the subtraction target of the medical image, the control unit updates the trained model based on the selected image. The medical image processing device according to any one of claims 2 to 6.

8. The medical image is an image of a bone. The medical image processing device according to any one of claims 1 to 7.

9. The medical image is an image of the brain. The medical image processing device according to any one of claims 1 to 7.

Images