Document preparation support device, document preparation support method and program

Abstract

To generate a progress report capturing a change in a lesion, etc., hard to capture just by a local area of a medical image.SOLUTION: A medical information system includes an imaging apparatus, an image server, an image database, a report server, a report database, a medical terminal and an interpretation terminal connected to one another interposing a network. In the medical information system, a document preparation support device 20 includes a medical information acquisition part 32, a progress information acquisition part 33 and a report generation part 35. The medical information acquisition part is configured to acquire a medical image in a time-sequence, an interpretation report of each medical image, and medical information. The progress information acquisition part is configured to acquire progress information of the entire medical image in the time-sequence according to the medical image in the time-sequence, an interpretation report of each medical image, and the medical information. The report generation part is configured to generate a sentence and words showing a progress observation status of a patient according to the medical image in the time-sequence, an interpretation report of each medical image, the medical information and the progress information to generate a progress report showing a change in time-sequence in the entire medical image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a document creation support device, a document creation support method, and a program. [Background technology]

[0002] Conventionally, image diagnosis and the creation of interpretation reports are performed using medical images obtained by imaging devices such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices. The medical images are transmitted to a terminal of a doctor who interprets the medical images. The doctor uses his / her own terminal to interpret the medical images and creates an interpretation report.

[0003] In recent years, various companies have been offering functions that analyze medical images using classifiers trained by deep learning or the like, detect regions of interest such as structures and lesions, and automatically generate radiology reports (see, for example, Patent Documents 1 and 2). This is called computer-aided detection / diagnostics (CAD). In recent years, devices that automatically generate progress reports using generation AI have been proposed. Note that, in the past, input was often only a single X-ray image, or the results of computer-aided diagnosis, with the report describing local changes over time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2023-143436 [Patent Document 2] Japanese Patent Application Publication No. 2023-130986 Summary of the Invention [Problem to be solved by the invention]

[0005] Generative AI (Artificial Intelligence) can analyze features and generate reports simply by inputting image and linguistic information. Since the accuracy of generative AI tends to improve when multimodal information is included as input, it is desirable for multimodal information to be input automatically.

[0006] Furthermore, there are technologies that create progress reports without capturing changes over time based on the characteristics of the entire medical image, i.e., without analyzing changes over time from the past to the present in the entire medical image, such as the conventional technologies described in Patent Documents 1 and 2. However, such conventional technologies may miss changes such as lesions that cannot be captured (interpreted) within a localized area of ​​the medical image. In such cases, the user must specify the region of interest again and perform a detailed analysis.

[0007] Therefore, it is desirable to generate a progress report by analyzing changes over time from the past to the present in the entire medical image, regardless of local changes such as regions of interest in the medical image or computer-aided diagnostic analysis results.

[0008] The present invention has been made in consideration of the problems with the above-mentioned conventional technology, and an object of the present invention is to provide a document creation support device, document creation support method, programming method, and program for generating progress reports that capture changes in lesions, etc. that cannot be captured within the localized range of medical images. [Means for solving the problem]

[0009] The above-mentioned problems of the present invention can be solved by the following means.

[0010] (1) A document creation support device comprising: a medical information acquisition unit that acquires time-series medical images, interpretation reports for each of the medical images, and medical information; a progress information acquisition unit that acquires progress information for the entire time-series medical images based on the time-series medical images, interpretation reports for each of the medical images, and medical information; and a report generation unit that generates sentences or phrases that represent a patient's follow-up status based on the time-series medical images, interpretation reports for each of the medical images, the medical information, and the progress information, and generates a progress report that represents changes over time in the entire medical images.

[0011] (2) The document creation support device described in (1) above, wherein the medical information acquisition unit acquires from outside as the medical images one or more types of images selected from the group consisting of plain X-ray images, dynamic images, computed tomography images, magnetic resonance images, ultrasound images, and moving images.

[0012] (3) The document creation support device described in (1) above, wherein the medical information acquisition unit acquires, from an external source, patient information about the patient, including the patient's gender, age, medical history, medication history, and radiology report, and equipment information about the medical equipment, as the medical information.

[0013] (4) The document creation support device described in (1) above, wherein the progress information acquisition unit acquires the progress information by performing medical analysis processing including computer-assisted diagnosis analysis processing, difference analysis processing, and bone removal analysis processing using the medical images.

[0014] (5) The document creation support device according to (4) above, wherein the progress information acquisition unit selects all, some, or none of the results of the medical analysis processing and acquires the selected results as the progress information.

[0015] (6) The document creation support device according to (1) above, wherein the progress information includes information on disease scars, structural changes in organs and bones, and changes in measurement values ​​in the region of interest.

[0016] (7) The document creation support device described in (1) above, wherein the report generation unit generates sentences and phrases including follow-up observations of the entire or localized medical images based on the time-series medical images, the interpretation reports of each medical image, the medical information, and the progress information, thereby generating the progress report.

[0017] (8) The document creation support device described in (1) above, wherein the report generation unit analyzes the characteristics of the medical images, numerical information, and linguistic information using a deep learning model that has learned the relationship between the medical image information, numerical information, and linguistic information and the analysis results, and generates the progress report.

[0018] (9) The document creation support device according to (1) above, wherein the report generation unit emphasizes a progress observation portion of the generated progress report.

[0019] (10) The document creation support device according to (1), wherein the report generation unit generates the progress report including one or more images.

[0020] (11) The document creation support device according to (1) above, wherein the report generation unit, when displaying the generated progress report on a display unit, highlights the basis for generating the progress report.

[0021] (12) A document creation support method comprising: a medical information acquisition step of acquiring time-series medical images, interpretation reports for each of the medical images, and medical information; a progress information acquisition step of acquiring progress information for the entire time-series medical images based on the time-series medical images, interpretation reports for each of the medical images, and medical information; and a report generation step of generating sentences or phrases representing the patient's follow-up status based on the time-series medical images, interpretation reports for each of the medical images, the medical information, and the progress information, to generate a progress report representing the time-series changes for the entire medical images.

[0022] (13) A program that causes a computer to function as a medical information acquisition unit that acquires time-series medical images, interpretation reports for each of the medical images, and medical information; a progress information acquisition unit that acquires progress information for the entire time-series medical images based on the time-series medical images, interpretation reports for each of the medical images, and medical information; and a report generation unit that generates sentences or phrases that represent the patient's follow-up status based on the time-series medical images, interpretation reports for each of the medical images, the medical information, and the progress information, and generates a progress report that represents the time-series changes for the entire medical images. [Effects of the Invention]

[0023] According to the present invention, it is possible to generate a progress report that captures changes in lesions and the like that cannot be captured only within a local range of a medical image. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a configuration diagram of a medical information system including a document creation support device according to an embodiment. [Figure 2] FIG. 2 is a block diagram of a hardware configuration of the document creation support device. [Figure 3] FIG. 2 is a block diagram of a functional configuration of the document creation support device. [Figure 4] FIG. 1 is an explanatory diagram of a medical image. [Figure 5] 3 is a flowchart showing the operation of the document creation support device. [Figure 6] FIG. 2 is a diagram illustrating the operation of the document creation support device. [Figure 7] FIG. 10 is an explanatory diagram of a difference image. [Figure 8] FIG. 10 is an explanatory diagram of progress status items. [Figure 9] FIG. 10 is an explanatory diagram illustrating an example of a progress report. [Figure 10] FIG. 10 is an explanatory diagram showing an example of a medical image incorporated into a progress report. [Figure 11] FIG. 10 is an explanatory diagram showing an example of an image interpretation report. DETAILED DESCRIPTION OF THE INVENTION

[0025] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that each drawing is merely a schematic illustration to allow a sufficient understanding of the present invention. Therefore, the present invention is not limited to the illustrated examples. Furthermore, in each drawing, common or similar components are designated by the same reference numerals, and redundant explanations thereof will be omitted.

[0026] <Configuration of a medical information system including a document creation support device> 1 to 3, a medical information system including a document creation support device according to an embodiment of the present invention will be described below. Fig. 2 is a block diagram of the hardware configuration of the document creation support device. Fig. 3 is a block diagram of the functional configuration of the document creation support device.

[0027] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. First, the configuration of a medical information system 100 to which a document creation support device 20 of the present disclosure is applied will be described. FIG. 1 is a diagram showing a schematic configuration of the medical information system 100. The medical information system 100 shown in FIG. 1 photographs the examination target area of ​​a subject and stores the medical images obtained by photographing, based on an examination order from a doctor of a medical department using a known ordering system. In addition, the medical images are interpreted by a radiologist and an interpretation report is created, and the interpretation report is viewed by the doctor of the requesting medical department.

[0028] 1, the medical information system 100 includes an imaging device 11, an image server 12, an image database 13, a report server 14, a report database 15, a medical terminal 16, and an image interpretation terminal 17. The imaging device 11, the image server 12, the image database 13, the report server 14, the report database 15, the medical terminal 16, and the image interpretation terminal 17 are connected to each other via a wired or wireless network 18 in a state where they can communicate with each other.

[0029] Each device is a computer installed with an application program that causes the device to function as a component of the medical information system 100. The application program is recorded on a recording medium such as a DVD-ROM (Digital Versatile Disc Read Only Memory) or a CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed on the computer from the recording medium. Alternatively, the application program may be stored in a storage device or network storage of a server computer connected to the network 18, and downloaded to the computer upon request and installed.

[0030] The imaging device 11 is a device (modality) that captures an image of a region of a subject to be diagnosed, thereby generating a medical image T representing the region to be diagnosed. Examples of the imaging device 11 include a plain X-ray imaging device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, an ultrasound diagnostic device, an endoscope, and a fundus camera. The medical image generated by the imaging device 11 is transmitted to an image server 12 and stored in an image database 13.

[0031] The image server 12 is a general-purpose computer installed with a software program that provides the functions of a database management system (DBMS). The image server 12 is connected to an image database 13. The connection between the image server 12 and the image database 13 is not particularly limited, and they may be connected by a data bus or via a network such as a NAS (Network Attached Storage) or a SAN (Storage Area Network).

[0032] The image database 13 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. In the image database 13, medical images acquired by the imaging device 11 and accompanying information attached to the medical images are registered in association with each other.

[0033] The incidental information may include, for example, identification information such as an image ID (identification) for identifying the medical image, a cross-sectional ID assigned to each cross-sectional image included in the medical image, a subject ID for identifying the subject, and an examination ID for identifying the examination. The incidental information may also include, for example, information related to imaging, such as the imaging method, imaging conditions, and imaging date and time for capturing the medical image. Examples of the "imaging method" and "imaging conditions" include the type of imaging device 11, the imaging region, imaging protocol, imaging sequence, imaging technique, whether or not a contrast agent was used, and the slice thickness in tomography. The incidental information may also include information related to the subject, such as the subject's name, date of birth, age, and gender. The incidental information may also include information related to the purpose of capturing the medical image.

[0034] Furthermore, when the image server 12 receives a request to register a medical image from the imaging device 11, it converts the medical image into a database format and registers it in the image database 13. Furthermore, when the image server 12 receives a request to view from the image interpretation terminal 17 or the medical terminal 16, it searches for medical images registered in the image database 13 and transmits the searched medical image to the image interpretation terminal 17 or the medical terminal 16 that sent the request to view.

[0035] The report server 14 is a general-purpose computer installed with a software program that provides the functions of a database management system. The report server 14 is connected to a report database 15. The connection between the report server 14 and the report database 15 is not particularly limited, and they may be connected by a data bus or via a network such as a NAS or SAN.

[0036] The report database 15 is realized by a storage medium such as an HDD, an SSD, a flash memory, etc. In the report database 15, image interpretation reports created in the image interpretation terminal 17 are registered.

[0037] Furthermore, when the report server 14 receives a request to register an interpretation report from the interpretation terminal 17, it converts the interpretation report into a database format and registers it in the report database 15. Furthermore, when the report server 14 receives a request to view an interpretation report from the interpretation terminal 17 or the medical terminal 16, it searches for the interpretation report registered in the report database 15 and transmits the retrieved interpretation report to the interpretation terminal 17 or the medical terminal 16 that sent the view request.

[0038] Furthermore, the report server 14 may store results of analysis by computer-aided detection / diagnosis (CAD) or the like when creating the radiology report, even though these results were not written in the radiology report.

[0039] The image interpretation terminal 17 is a computer used by a doctor, such as a radiologist, to interpret medical images and prepare image interpretation reports. The image interpretation terminal 17 is communicably connected to the document creation support device 20 according to this embodiment. Alternatively, the image interpretation terminal 17 may have the document creation support device 20 built in. The document creation support device 20 is a computer that provides a doctor with a report on the progress of a patient (hereinafter referred to as a "progress report") as reference information when the doctor prepares an image interpretation report. The computer functions as the document creation support device 20 by, for example, installing a program 91 recorded in a storage medium 90.

[0040] In addition, the computer may be configured to download and install the program 91 recorded in, for example, a storage device of a server computer connected to the network 18 or a network storage. As a result, the computer may function as the document creation support device 20.

[0041] The image interpretation terminal 17 sends requests to the image server 12 to view medical images, performs various image processing on medical images received from the image server 12, displays the medical images, and accepts input of text related to the medical images. The image interpretation terminal 17 also performs analysis processing on medical images, supports the creation of image interpretation reports based on the analysis results, requests the report server 14 to register and view image interpretation reports, and displays image interpretation reports received from the report server 14. These processes are performed by the image interpretation terminal 17 by executing software programs for each process.

[0042] The medical terminal 16 is a computer used by, for example, a doctor in a medical department to observe medical images in detail, view radiology reports, and create electronic medical records, and is composed of a processing device, a display device such as a monitor, and input devices such as a keyboard and a mouse. The medical terminal 16 requests the image server 12 to view medical images, displays medical images received from the image server 12, requests the report server 14 to view radiology reports, and displays radiology reports received from the report server 14. These processes are performed by the medical terminal 16 by executing software programs for each process.

[0043] The network 18 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network). The imaging device 11, image interpretation terminal 17, medical terminal 16, image server 12, image database 13, report server 14, and report database 15 included in the medical information system 100 may be located in the same medical institution, or may be located in different medical institutions. Furthermore, the number of each device, the imaging device 11, image interpretation terminal 17, medical terminal 16, image server 12, image database 13, report server 14, and report database 15, is not limited to the number shown in Fig. 1. Each device may be composed of multiple devices having similar functions.

[0044] Medical images are analyzed using computer-aided diagnosis (CAD) techniques that use classifiers trained by deep learning or other methods to detect and / or diagnose regions of interest contained in the medical images. The creator of the image interpretation report, such as a radiologist, creates the image interpretation report by referring to the results of the computer-aided diagnosis analysis of the medical image. However, even if the results of the medical image analysis are obtained using computer-aided diagnosis, the results may not actually be included in the image interpretation report, depending on the judgment of the creator of the image interpretation report. On the other hand, a lesion that was initially determined not to require inclusion in the image interpretation report may worsen over time or the creator may change, and it may be determined that inclusion is necessary during a subsequent examination. In such cases, comparative image interpretation requires the effort of re-obtaining analysis results from a point in time that were not included in the image interpretation report.

[0045] Therefore, the document creation support device 20 according to this embodiment has a function of supporting comparative interpretation of medical images at a certain time point, even if the interpretation report at that time point does not include the analysis results of the medical images. The document creation support device 20 will be described below. As described above, the document creation support device 20 is communicably connected to the image interpretation terminal 17. Alternatively, the document creation support device 20 may be built into the image interpretation terminal 17.

[0046] <Hardware configuration of document creation support device> An example of the hardware configuration of the document creation support device 20 according to this embodiment will be described below with reference to Fig. 2. As shown in Fig. 2, the document creation support device 20 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 serving as a temporary storage area. The document creation support device 20 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I / F (Interface) 26. The network I / F 26 is connected to the network 18 and performs wired or wireless communication. The CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I / F 26 are connected via a bus 27 such as a system bus and a control bus so as to be able to exchange various information with one another.

[0047] The storage unit 22 is realized by a storage medium such as an HDD, an SSD, or a flash memory. A program 91 for the document creation support device 20 is stored in the storage unit 22. The CPU 21 reads the program 91 from the storage unit 22, expands it in the memory 23, and executes the expanded program 91. The CPU 21 is an example of a processor of the present disclosure. For example, a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, or the like can be appropriately applied as the document creation support device 20.

[0048] <Functional Configuration of Document Creation Support Device> Next, an example of the functional configuration of the document creation support device 20 according to this embodiment will be described with reference to Fig. 3. As shown in Fig. 3, the document creation support device 20 includes a control unit 31, a medical information acquisition unit 32, a progress information acquisition unit 33, a determination unit 34, a report generation unit 35, and a storage unit 36. When the CPU 21 (Fig. 3) executes a program 91, the CPU 21 functions as the control unit 31, the medical information acquisition unit 32, the progress information acquisition unit 33, the determination unit 34, and the report generation unit 35.

[0049] The control unit 31 controls the overall operation of the document creation support device 20 . The medical information acquisition unit 32 acquires medical information 37b, medical information 37b, and image interpretation reports 37c that can be used to generate a progress report 39 regarding the progress status of the patient. The medical images 37a are images of the inside of the patient (subject) in chronological order from the past to the present. The medical information 37b is various information related to medical care. The image interpretation reports 37c are image interpretation reports in which a doctor has previously interpreted each medical image 37a.

[0050] The progress information acquisition unit 33 acquires progress information 38 representing the progress of the patient (subject) throughout the entire time-series of medical images based on the time-series medical images, the interpretation reports for each medical image, and medical information. In this embodiment, the progress information 38 is described as including at least a subtraction image 38a and a CAD result 38b. The subtraction image 38a is an image representing the difference between any of the time-series medical images 37a of the patient (subject) from the past to the present. The CAD result 38b represents the results of analyzing the medical images 37a using computer-aided detection / diagnosis (CAD) to detect and / or diagnose regions of interest, including structures and lesions, contained in the medical images 37a. In this embodiment, for example, the progress information acquisition unit 33 analyzes the medical images 37a from the past to the present based on the information acquired by the medical information acquisition unit 32, and acquires progress information 38 for the entire medical image 37a, taking into account not only changes in local areas but also interrelationships between local areas.

[0051] Based on the medical information 37b acquired by the medical information acquisition unit 32 and the medical image 37a acquired by the progress information acquisition unit 33, the determination unit 34 determines sentences and phrases that represent progress observation for the entire medical image 37a.

[0052] The report generation unit 35 generates sentences and phrases representing the follow-up status of the patient and generates a progress report 39 representing the time-series changes in the entire medical images 37a. At this time, the report generation unit 35 generates sentences and phrases representing the follow-up status of the patient based on the time-series medical images 37a, the interpretation report 37c for each medical image 37a, the medical information 37b, and the progress information 38.

[0053] The storage unit 36 ​​stores medical images 37 a, medical information 37 b, an interpretation report 37 c, progress information 38 such as subtraction images 38 a and CAD results 38 b, and a progress report 39.

[0054] Fig. 4 is an explanatory diagram of medical image 37a. As shown in Fig. 4, medical image 37a includes tomographic images 41a, 41b, ..., 41m, and 41n of the inside of a patient's body. Here, the description will be given assuming that tomographic images 41a, 41b, ... are past images, tomographic image 41m is the previous image, and tomographic image 41n is the current (present) image.

[0055] <Operation of the document creation support device> FIG. 5 is a flowchart showing the operation of the document creation support device 20. FIG. 6 is an explanatory diagram of the operation of the document creation support device 20. As shown in FIG. 5, the document creation support device 20 executes a medical information acquisition step (step S110), a progress information acquisition step (step S120), and a report generation step (step S130). In the medical information acquisition step (step S110), the document creation support device 20 acquires medical information 37b, medical information 37b, and radiology report 37c used to create the report. In the progress information acquisition step (step S120), the document creation support device 20 acquires progress information 38 of the patient. In the report generation step (step S130), the document creation support device 20 generates a progress report 39.

[0056] Then, in the progress information acquisition step (step S120), the document creation support device 20 creates a subtraction image 38a of the patient between past and present medical images 37a of the patient and a CAD result 38b (temporal subtraction result) of the patient. At this time, the document creation support device 20 creates the subtraction image 38a and the CAD result 38b (temporal subtraction result) based on the medical images 37a, medical information 37b, and radiology report 37c.

[0057] Furthermore, in the report generation step (step S130), the document creation support device 20 generates sentences and phrases that represent the follow-up status of the patient, and generates a progress report 39 that represents the time-series changes in the entire medical image 37a. At this time, the document creation support device 20 generates sentences and phrases that represent the follow-up status of the patient based on the time-series medical images 37a, the radiology report 37c for each medical image 37a, the medical information 37b, and the progress information 38.

[0058] 6 shows an example of the operation of the document creation support device 20 in steps S120 and S130. As shown in FIG. 6, in step S120, the document creation support device 20 creates a subtraction image 38a of the patient and a CAD result 38b (temporal subtraction result) of the patient. At this time, the progress information acquisition unit 33 of the document creation support device 20 performs a difference analysis process using past and present medical images 37a of the patient (step S120a).

[0059] Specifically, the progress information acquisition unit 33 analyzes the difference between any images in the past and present medical images 37a of the patient (for example, between any of the past tomographic images 41a, 41b, ... and the previous tomographic image 41m and the current tomographic image 41n). Then, the progress information acquisition unit 33 creates a difference image 38a (FIG. 7) that represents the difference. FIG. 7 is an explanatory diagram of the difference image 38a. FIG. 7 shows an example in which the difference between the current tomographic image 41n and the past tomographic image 41d is analyzed to acquire the difference image 38a.

[0060] Furthermore, the progress information acquiring unit 33 performs computer-aided diagnosis analysis processing (CAD analysis processing) (step S120b). The computer-aided diagnosis analysis processing is processing for analyzing changes in the patient's lesion, etc. over time from the past to the present by analyzing the medical image 37a using CAD. The progress information acquiring unit 33 acquires CAD results 38b (temporal subtraction results) through the computer-aided diagnosis analysis processing.

[0061] In step S130, the document creation support device 20 generates a progress report 39. At this time, the report generation unit 35 of the document creation support device 20 generates sentences and phrases representing the follow-up observation status of the patient based on the time-series medical images 37a, the radiology report 37c for each medical image 37a, the medical information 37b, and the progress information 38. Then, the report generation unit 35 generates the progress report 39 representing the time-series changes in the entire medical images 37a.

[0062] Note that the document creation support device 20 learns the V&L model 96 by deep learning or the like before the processing of step S130. The V&L model 96 is an image original language model including sentences and phrases representing the follow-up observation status. The report generation unit 35 (or the determination unit 34) of the document creation support device 20 learns the V&L model 96 (image original language model) by deep learning or the like based on the medical image 37a, medical information 37b, and radiology report 37c. Therefore, the document creation support device 20 can acquire the V&L model 96 including the latest information by deep learning. The V&L model 96 includes sentences and phrases representing the progress status 60 of an item, for example, as shown in FIG. 8. FIG. 8 is an explanatory diagram of the items of the progress status 60. In the example shown in FIG. 8, the progress status 60 includes the following items: Appearance / disappearance of disease Changes in the disease over time (size, shape, etc.) Progression of the disease Prognosis prediction Presence or absence of complications Presence or absence of illness, type Functional changes Treatment response Signs of relapse

[0063] 9 and 10 show an example of the progress report 39 created in step S130. FIG. 9 is an explanatory diagram showing an example of the progress report 39. FIG. 10 is an explanatory diagram showing an example of a medical image 37a incorporated into the progress report 39. In the example shown in FIG. 9, the progress report 39 includes information on any item in the progress status 60 shown in FIG. 8 and a follow-up observation section 86. The progress report 39 emphasizes the follow-up observation section 86 by adding color or the like to make it easier to see. The progress report 39 also includes a medical image 37a as shown in FIG. 10. In the example shown in FIG. 10, the medical image 37a includes a region of interest 81 and a grounds portion 87. The region of interest 81 is an area extracted by a doctor or the medical information system 100 as a possible site of a lesion. The region of interest 81 includes an abnormal shadow region 82, which may be a lesion, and a structure region 83, such as skin. The basis section 87 is the part that served as the basis for creating the follow-up observation section 86.

[0064] The doctor operates the image interpretation terminal 17 (FIG. 1) to view the progress report 39 created by the document creation support device 20. The doctor then creates an image interpretation report 37c (FIG. 11) by referring to the contents of the progress report 39. FIG. 11 is an explanatory diagram showing an example of the image interpretation report 37c. In the example shown in FIG. 11, the image interpretation report 37c includes a subject information input field 71, a findings input field 72, a medical image display field 73, and an image analysis result display field 74. Patient information such as the patient's identification number, name, sex, date of birth, and purpose of the examination is entered in the subject information input field 71. Findings determined by the doctor are entered in the findings input field 72. The medical image display field 73 displays the medical image 37a used to determine the findings. The medical image 37a displayed in the medical image display field 73 includes an abnormal shadow region 82. Furthermore, the medical image 37a is displayed with emphasis by adding color or the like to the basis portion 87 that was the basis for creating the follow-up observation portion 86 so that the basis portion 87 can be easily seen. The image analysis result display field 74 displays the analysis results of the medical image 37a.

[0065] <Main features of the document creation support device, document creation support method, and program> (1) As shown in FIGS. 1 and 6, the document creation support device 20 according to this embodiment includes a medical information acquisition unit 32, a progress information acquisition unit 33, and a report generation unit 35. The medical information acquisition unit 32 acquires time-series medical images 37a, interpretation reports 37c for each medical image 37a, and medical information 37b. The progress information acquisition unit 33 acquires progress information 38 for the entire time-series medical images 37a based on the time-series medical images 37a, interpretation reports 37c for each medical image 37a, and medical information 37b. The report generation unit 35 generates sentences and phrases representing the follow-up status of the patient based on the time-series medical images 37a, interpretation reports 37c for each medical image 37a, medical information 37b, and progress information 38. The report generation unit 35 then generates a progress report 39 representing the time-series changes for the entire medical images 37a.

[0066] Prior art devices do not generate sentences and phrases describing a patient's follow-up status based on the time-series medical images 37a, the interpretation reports 37c for each medical image 37a, the medical information 37b, and the progress information 38. Furthermore, prior art devices do not use such sentences and phrases to generate a progress report 39 describing the time-series changes in the entire medical image 37a. Such prior art devices create progress reports without capturing the temporal changes based on the characteristics of the entire medical image, that is, without analyzing the temporal changes from the past to the present in the entire medical image. Therefore, prior art devices may miss changes in lesions, etc., that cannot be captured (interpreted) within a local area of ​​the medical image. In contrast, the document creation support device 20 according to this embodiment uses a progress information acquisition unit 33 to acquire progress information 38 for the entire time-series medical images 37a based on the time-series medical images 37a, the interpretation reports 37c for each medical image 37a, and the medical information 37b. Thereafter, the document creation support device 20 uses the report generation unit 35 to generate sentences and phrases representing the patient's follow-up status based on the time-series medical images 37a, the interpretation reports 37c for each medical image 37a, the medical information 37b, and the progress information 38. The document creation support device 20 then uses the report generation unit 35 to generate a progress report 39 representing the time-series changes in the entire medical image 37a. As a result, the document creation support device 20 can generate a progress report 39 that captures changes in the lesion, etc. that cannot be captured (interpreted) within a local area of ​​the medical image 37a, without overlooking such changes. The document creation support device 20 according to this embodiment can generate a highly accurate progress report 39 based on the overall characteristics of the medical image 37a.

[0067] (2) In the document creation support device 20 configured as described in (1) above, the medical information acquisition unit 32 externally acquires one or more of the following types of images as the medical image 37a: The images acquired externally include, for example, plain X-ray images, dynamic images, computed tomography images, magnetic resonance images, ultrasound images, and video images.

[0068] The document creation support device 20 configured as described above can generate a progress report 39 using plain X-ray images, dynamic images, computed tomography images, magnetic resonance images, ultrasound images, and video images as medical images 37a.

[0069] (3) In the document creation support device 20 configured as described in (1) above, the medical information acquisition unit 32 acquires the following information from outside as medical information 37b. The information acquired from outside is, for example, patient information about the patient and device information about the medical device. The patient information is, for example, the patient's gender, age, medical history, medication history, and radiology report 37c. The device information is, for example, an acquisition device that acquired medical images 37a such as plain X-ray images, dynamic images, computed tomography images, magnetic resonance images, ultrasound images, and moving images, a database in which the medical images 37a are registered, a database in which medical information is registered, and the like.

[0070] The document creation support device 20 configured in this manner can generate a progress report 39 based on patient information and device information, which captures changes in lesions and the like that cannot be captured within the local range of the medical image 37a.

[0071] (4) As shown in FIG. 6, in the document creation support device 20 configured as described in (1) above, the progress information acquisition unit 33 acquires progress information 38 by performing medical analysis processing using a medical image 37a. The medical analysis processing includes a computer-assisted diagnosis analysis processing (step S120b), a difference analysis processing (step S120a), and a bone removal analysis processing (not shown). The computer-assisted diagnosis analysis processing is a processing for analyzing an arbitrary region (particularly a region that may be a lesion) of the medical image 37a using computer-assisted diagnosis. The difference analysis processing is a processing for extracting differences between medical images 37a in a time series from the past to the present, and analyzing an arbitrary region (particularly a region that may be a lesion) based on the differences. The bone removal analysis processing is a processing for acquiring an image (bone-removed image) from which bone has been removed from the medical image 37a, and analyzing an arbitrary region (particularly a region that may be a lesion) based on the bone-removed image.

[0072] The document creation support device 20 configured as described above can perform medical analysis processes such as computer-aided diagnosis analysis processing (step S120b), difference analysis processing (step S120a), bone removal analysis processing (not shown), etc. By performing medical analysis processes such as computer-aided diagnosis analysis processing, difference analysis processing, and bone removal analysis processing, the document creation support device 20 configured as described above can provide a user (doctor) with an image suitable for analyzing any region (especially a region that may contain a lesion).

[0073] (5) In the document creation support device 20 configured as described in (4) above, the progress information acquisition unit 33 selects all, some, or no information from the results of the medical analysis process and acquires it as progress information 38.

[0074] The document creation support device 20 configured as described above selects all, some, or no information from the results of the medical analysis processing and acquires it as progress information 38. Then, the document creation support device 20 generates a progress report 39 based on the progress information 38. This allows the document creation support device 20 configured as described above to generate a progress report 39 that reflects the medical analysis processing.

[0075] (6) In the document creation support device 20 configured as described in (1) above, the progress information 38 includes information on disease scars, structural changes in organs and bones, and changes in measurement values ​​in the region of interest.

[0076] The document creation support device 20 configured as described above acquires progress information 38 including information on disease scars, structural changes in organs and bones, and changes in measurement values ​​in the region of interest, and generates a progress report 39 based on the progress information 38. This allows the document creation support device 20 configured as described above to generate a progress report 39 that reflects this information.

[0077] (7) In the document creation support device 20 configured as described in (1) above, the report generation unit 35 generates sentences and phrases based on time-series medical images 37a, interpretation reports 37c for each medical image 37a, medical information 37b, and progress information 38. The sentences and phrases contain progress observations of the entire or localized portion of the medical image 37a. The report generation unit 35 then generates a progress report 39 using the generated sentences and phrases.

[0078] The document creation support device 20 configured as described above can generate a progress report 39 using sentences and phrases including progress observations of the entire or localized portion of the medical image 37a.

[0079] (8) As shown in Figure 6, in the document creation support device 20 configured as described in (1) above, the report generation unit 35 analyzes the characteristics of the medical image, numerical information, and linguistic information using a deep learning model that has learned the relationship between the medical image information, numerical information, and linguistic information and the analysis results. Then, the report generation unit 35 generates a progress report 39 based on the analyzed characteristics.

[0080] The document creation support device 20 configured in this manner can generate a progress report 39 using sentences and phrases that represent progress observations learned through deep learning.

[0081] (9) As shown in FIG. 9, in the document creation support device 20 configured as in (1) above, the report generation unit 35 emphasizes the progress observation portion 86 of the generated progress report 39.

[0082] The document creation support device 20 configured as described above displays the progress report 39 on the display unit, emphasizing the follow-up observation portion 86. By looking at the progress report 39, the user (doctor) can focus on interpreting the area related to the follow-up observation portion 86 in the medical image 37a. This allows the user to interpret the image without missing any changes in the lesion or other lesions that are difficult to interpret. This allows the user to refer to the progress report 39 and create an interpretation report 37c that reflects highly accurate information on changes in the lesion or other lesions.

[0083] (10) As shown in FIG. 10, in the document creation support device 20 configured as in (1) above, the report generation unit 35 generates a progress report 39 including one or more images.

[0084] The document creation support device 20 of this configuration includes one or more images in the progress report 39, and therefore can provide the user (doctor) with a progress report 39 that makes it easy to visually identify any area (especially an area that may be a lesion).

[0085] (11) As shown in FIG. 10, in the document creation support device 20 having the configuration described in (1) above, when the report generation unit 35 displays the generated progress report 39 on the display unit, it highlights the basis portion 87 for the generation of the progress report 39.

[0086] The document creation support device 20 configured as described above displays the progress report 39 on the display unit with the evidence portion 87 emphasized. By looking at the evidence portion 87, the user (doctor) can focus on interpreting the area of ​​the medical image 37a that is related to the evidence portion 87. This allows the user to interpret the image without missing any changes in the lesion or other lesions that are difficult to interpret. This allows the user to refer to the progress report 39 and create an interpretation report 37c that reflects highly accurate information about changes in the lesion or other lesions.

[0087] (12) As shown in FIG. 5, the document creation support method according to this embodiment includes a medical information acquisition step (step S110), a progress information acquisition step (step S120), and a report generation step (step S130). In the medical information acquisition step (step S110), time-series medical images 37a, and interpretation reports 37c and medical information 37b for each medical image 37a are acquired. In the progress information acquisition step (step S120), progress information 38 of the patient for the entire time-series medical images 37a is acquired based on the time-series medical images 37a, and interpretation reports 37c and medical information 37b for each medical image 37a. In the report generation step (step S130), sentences or phrases representing the patient's follow-up status are generated based on the time-series medical images 37a, and interpretation reports 37c, medical information 37b, and progress information 38 for each medical image 37a. Then, in the report generation step (step S130), a progress report 39 is generated that shows the time series changes in the entire medical image 37a. The document creation support method according to this embodiment can generate a progress report 39 that captures changes in lesions and the like that cannot be captured within a localized range of the medical image 37a.

[0088] (13) The program 91 according to this embodiment is a program for causing a computer to function as a medical information acquisition unit 32, a progress information acquisition unit 33, and a report generation unit 35. The medical information acquisition unit 32 acquires time-series medical images 37a, and the interpretation reports 37c and medical information 37b for each medical image 37a. The progress information acquisition unit 33 acquires progress information 38 for the entire time-series medical images 37a based on the time-series medical images 37a, the interpretation reports 37c for each medical image 37a, and the medical information 37b. The report generation unit 35 generates sentences and phrases representing the follow-up status of the patient based on the time-series medical images 37a, the interpretation reports 37c for each medical image 37a, the medical information 37b, and the progress information 38. The report generation unit 35 then generates a progress report 39 representing the time-series changes for the entire medical images 37a. The program 91 according to this embodiment can realize the document creation support device 20 configured as described in (1) above.

[0089] The present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the spirit of the present invention.

[0090] For example, the above-described embodiment has been described in detail to clearly explain the gist of the present invention. Therefore, the present invention is not necessarily limited to an embodiment including all of the components described. Furthermore, the present invention may include some components by adding other components to them, or by replacing some components with other components. Furthermore, the present invention may also include some components by deleting them. [Explanation of symbols]

[0091] 11 Imaging equipment 12 Image Server 13 Image Database 14 Report Server 15 Report Database 16 Medical Devices 17 Image reading terminal 18 Network 20 Document creation support device 21 CPU 22 Memory section 23 Memory 24 displays 25 Input section 26 Network I / F 27 Bus 31 Control Unit 32 Medical Information Acquisition Department 33 Progress information acquisition unit 34 Judgment section 35 Report Generation Unit 36 Memory section 37a Medical Imaging 37b Medical Information 37c Reading Report 38 Progress Information 38a Difference image 38b CAD results (temporal subtraction results) 39 Progress Report 41a, 41b Tomographic images 41m fault image 41n tomographic image 60 Progress 71 Subject information input field 72 Observation entry field 73 Medical Image Display Column 74 Image analysis result display field 81 Areas of Interest 82 Abnormal shadow area 83 Structure area 86 Follow-up observation section 87 Basis section 90 Storage medium 91 Programs 96 V&L model 100 Medical Information Systems

Claims

1. a medical information acquisition unit that acquires time-series medical images, interpretation reports for each of the medical images, and medical information; a progress information acquisition unit that acquires progress information for the entire time-series medical images based on the time-series medical images, interpretation reports for each of the medical images, and the medical information; a report generation unit that generates sentences or phrases that represent a follow-up observation status of a patient based on the time-series medical images, the interpretation reports of each of the medical images, the medical information, and the progress information, and generates a progress report that represents a time-series change in the entirety of the medical images; Document creation support device.

2. the medical information acquisition unit externally acquires, as the medical images, one or more types of images selected from the group consisting of plain X-ray images, dynamic images, computed tomography images, magnetic resonance images, ultrasound images, and moving images; 2. The document creation support device according to claim 1.

3. The medical information acquisition unit acquires, from an external source, patient information relating to the patient, including the patient's gender, age, medical history, medication history, and radiology report, and device information relating to the medical device, as the medical information.

2. The document creation support device according to claim 1.

4. the progress information acquisition unit acquires the progress information by performing medical analysis processing including computer-aided diagnosis analysis processing, difference analysis processing, and bone removal analysis processing using the medical images; 2. The document creation support device according to claim 1.

5. the progress information acquisition unit selects all, some, or no information from the results of the medical analysis processing and acquires it as the progress information; 5. The document creation support device according to claim 4.

6. The progress information includes information on disease scars, structural changes of organs and bones, and changes in measurement values ​​in the region of interest.

2. The document creation support device according to claim 1.

7. the report generation unit generates sentences and phrases including a progress observation of the entire or a local portion of the medical image based on the time-series medical images, the interpretation report of each of the medical images, the medical information, and the progress information, and generates the progress report.

2. The document creation support device according to claim 1.

8. the report generation unit analyzes features of the medical image, numerical information, and linguistic information using a deep learning model that has learned the relationship between the medical image information, numerical information, and linguistic information and the analysis results, and generates the progress report; 2. The document creation support device according to claim 1.

9. The report generation unit emphasizes a progress observation portion of the generated progress report.

2. The document creation support device according to claim 1.

10. the report generator generates the progress report including one or more images.

2. The document creation support device according to claim 1.

11. the report generation unit, when displaying the generated progress report on a display unit, highlights the basis for generating the progress report.

2. The document creation support device according to claim 1.

12. a medical information acquisition step of acquiring time-series medical images, interpretation reports for each of the medical images, and medical information; a progress information acquisition step of acquiring progress information for the entire time-series medical images based on the time-series medical images, the interpretation reports for each of the medical images, and the medical information; and a report generation step of generating a progress report that represents a time-series change in the entire medical images by generating sentences or phrases that represent a follow-up observation status of the patient based on the time-series medical images, the interpretation reports of each medical image, the medical information, and the progress information. Document creation support methods.

13. Computer, a medical information acquisition unit that acquires time-series medical images, interpretation reports for each of the medical images, and medical information; a progress information acquisition unit that acquires progress information for the entire time-series medical images based on the time-series medical images, interpretation reports for each of the medical images, and the medical information; A program for functioning as a report generation unit that generates sentences or phrases representing the patient's follow-up observation status based on the time-series medical images, the interpretation reports for each of the medical images, the medical information, and the progress information, and generates a progress report that represents the time-series changes in the entire medical images.

Images