Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device, an information processing method, and an information processing program capable of supporting comparative diagnostic reading.SOLUTION: An information processing device 10 includes at least one processor. The processor acquires a first measurement value measured on a first region of interest included in a first image obtained by imaging a subject at a first time point, acquires a second document in which diagnostic contents based on a second image obtained by imaging the subject at a second time point are described, determines whether or not the second document includes a second measurement value which is of the same kind as the first measurement value, measured on a second region of interest included in the second image and the same as the first region of interest, and if the determination is negative, acquires the second measurement value.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, image diagnosis has been performed using medical images obtained by imaging devices such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices. Furthermore, medical images have been analyzed by CAD (Computer Aided Detection / Diagnosis) using classifiers trained by deep learning or the like to detect and / or diagnose regions of interest, including structures and lesions, contained in the medical images. The medical images and the results of the CAD analysis are transmitted to a terminal of a medical professional, such as a radiologist, who interprets the medical images. The medical professional, such as a radiologist, uses his or her own terminal to refer to the medical images and the analysis results, interprets the medical images, and creates a radiology report.

[0003] Furthermore, various methods for supporting the creation of radiology reports have been proposed to reduce the burden of radiology work. For example, Patent Document 1 discloses a technology for creating a radiology report based on keywords entered by a radiologist and the analysis results of medical images. In the technology described in Patent Document 1, a recurrent neural network trained to generate sentences from input characters is used to create sentences to be included in the radiology report.

[0004] Furthermore, for example, in periodic health checkups and follow-up observations after treatment, the same subject may be examined multiple times, and changes in the condition over time may be confirmed by comparative interpretation of medical images taken at each time point. Therefore, various methods for comparative interpretation have been proposed. For example, Patent Document 2 discloses a method for comparing first diagnosis identification information acquired at a first diagnosis with second diagnosis identification information acquired at a second diagnosis different from the first diagnosis. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-153250 [Patent Document 2] International Publication No. 2018 / 159363 Summary of the Invention [Problem to be solved by the invention]

[0006] Even if CAD-based analysis results are obtained for medical images, the results may not actually be included in the radiology report, depending on the judgment of the radiology report creator. On the other hand, a lesion that was initially deemed unnecessary to include in the radiology report may worsen over time or the creator may change, making it necessary to include it in a subsequent examination. In such cases, comparative interpretation requires the effort of re-obtaining analysis results from a point in time that were not included in the radiology report.

[0007] The present disclosure provides an information processing device, an information processing method, and an information processing program that can support comparative image interpretation. [Means for solving the problem]

[0008] A first aspect of the present disclosure is an information processing device comprising at least one processor, wherein the processor acquires a first measurement value measured for a first region of interest included in a first image obtained by photographing a subject at a first time point, acquires a second document describing a diagnosis based on a second image obtained by photographing the subject at a second time point, determines whether the second document includes a second measurement value of the same type as the first measurement value measured for a second region of interest included in the second image and identical to the first region of interest, and if the determination is negative, acquires the second measurement value.

[0009] In the first aspect, the processor may acquire a second measurement value that has been measured based on the second image if the determination is negative.

[0010] In the first aspect, the processor may measure a second measurement value based on the second image if the determination is negative.

[0011] In the first aspect, if the judgment is a negative judgment, the processor may acquire the second measurement value that has been measured based on the second image if there is a second measurement value that has been measured, and may measure the second measurement value based on the second image if there is no second measurement value that has been measured.

[0012] In the first aspect, the processor may cause the display to display the second measurement value in different display formats when the second measurement value has been obtained and when the second measurement value has been measured based on the second image.

[0013] In the first aspect above, if the determination is negative, the processor may acquire a second image, extract a second region of interest from the acquired second image, and measure a second measurement value based on the extracted second region of interest.

[0014] In the first aspect, the processor may cause the display to display the second measurement value obtained when the judgment is a negative judgment in a display format different from that when the second measurement value is included in the second document.

[0015] In the first aspect, the processor may create a plot of the acquired first and second measurement values ​​and display the plot on the display.

[0016] In the first aspect, the first and second measurement values ​​may be at least one of the size and signal value of the first and second regions of interest, respectively.

[0017] A second aspect of the present disclosure is an information processing method, which includes a process of acquiring a first measurement value measured for a first region of interest included in a first image obtained by photographing a subject at a first time point, acquiring a second document describing a diagnosis based on a second image obtained by photographing the subject at a second time point, determining whether the second document includes a second measurement value of the same type as the first measurement value measured for a second region of interest included in the second image and identical to the first region of interest, and acquiring the second measurement value if the determination is negative.

[0018] A third aspect of the present disclosure is an information processing program that causes a computer to execute a process of acquiring a first measurement value measured for a first region of interest included in a first image obtained by photographing a subject at a first time point, acquiring a second document describing a diagnosis based on a second image obtained by photographing the subject at a second time point, determining whether the second document includes a second measurement value of the same type as the first measurement value that is included in the second image and measured for a second region of interest that is identical to the first region of interest, and if the determination is negative, acquiring the second measurement value. [Effects of the Invention]

[0019] According to the above aspects, the information processing device, information processing method, and information processing program of the present disclosure can support comparative image interpretation. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing system. [Figure 2] FIG. 1 is a diagram illustrating an example of a medical image. [Figure 3] FIG. 1 is a diagram illustrating an example of a medical image. [Figure 4] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device. [Figure 5] FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing device. [Figure 6] FIG. 10 is a diagram illustrating an example of a screen displayed on a display. [Figure 7] FIG. 10 is a diagram illustrating an example of a screen displayed on a display. [Figure 8] FIG. 10 is a diagram illustrating an example of information registered in a report DB. [Figure 9] FIG. 10 is a diagram illustrating an example of a screen displayed on a display. [Figure 10] FIG. 10 is a diagram illustrating an example of a screen displayed on a display. [Figure 11] FIG. 10 is a diagram illustrating an example of a screen displayed on a display. [Figure 12] FIG. 10 is a diagram showing an example of a plot diagram. [Figure 13] 10 is a flowchart illustrating an example of information processing. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. First, the configuration of an information processing system 1 to which an information processing device of the present disclosure is applied will be described. FIG. 1 is a diagram showing a schematic configuration of the information processing system 1. The information processing system 1 shown in FIG. 1 photographs the examination target region of a subject and stores the medical images acquired by photographing, based on an examination order from a doctor of a medical department using a known ordering system. In addition, the information processing system 1 performs image interpretation work on the medical images by a radiologist and creates an interpretation report, and the requesting doctor of the medical department views the interpretation report.

[0022] 1, the information processing system 1 includes an imaging device 2, an interpretation WS (Workstation) 3 which is an interpretation terminal, a medical treatment WS 4, an image server 5, an image DB (DataBase) 6, a report server 7, and a report DB 8. The imaging device 2, the interpretation WS 3, the medical treatment WS 4, the image server 5, the image DB 6, the report server 7, and the report DB 8 are connected via a wired or wireless network 9 in a state where they can communicate with each other.

[0023] Each device is a computer installed with an application program that causes the device to function as a component of the information processing system 1. The application program may be recorded on a recording medium such as a DVD-ROM (Digital Versatile Disc Read Only Memory) or a CD-ROM (Compact Disc Read Only Memory) and distributed, and then installed on the computer from the recording medium. Alternatively, the application program may be stored in a storage device or network storage of a server computer connected to the network 9 in an externally accessible state, and downloaded to the computer upon request and installed.

[0024] The imaging device 2 is a device (modality) that captures an image of a region of a subject to be diagnosed, thereby generating a medical image T representing the region to be diagnosed. Examples of the imaging device 2 include a plain X-ray imaging device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, an ultrasound diagnostic device, an endoscope, and a fundus camera. The medical image generated by the imaging device 2 is transmitted to an image server 5 and stored in an image DB 6.

[0025] The image interpretation WS3 is a computer used by a medical professional, such as a radiologist, to interpret medical images and create image interpretation reports, and includes the information processing device 10 according to this embodiment. The image interpretation WS3 issues requests for viewing medical images to the image server 5, performs various image processing on medical images received from the image server 5, displays the medical images, and accepts input of text related to the medical images. The image interpretation WS3 also performs analysis processing on medical images, supports the creation of image interpretation reports based on the analysis results, requests the report server 7 to register and view image interpretation reports, and displays image interpretation reports received from the report server 7. These processes are performed by the image interpretation WS3 executing software programs for each process.

[0026] The medical WS4 is a computer used by medical personnel such as doctors in a medical department to perform detailed observations of medical images, view interpretation reports, and create electronic medical records, and is composed of a processing device, a display device such as a monitor, and input devices such as a keyboard and a mouse. The medical WS4 sends requests to the image server 5 to view medical images, displays medical images received from the image server 5, requests to the report server 7 to view interpretation reports, and displays interpretation reports received from the report server 7. These processes are performed by the medical WS4 executing software programs for each process.

[0027] The image server 5 is a general-purpose computer installed with a software program that provides the functions of a database management system (DBMS). The image server 5 is connected to an image DB 6. The connection between the image server 5 and the image DB 6 is not particularly limited, and may be via a data bus or a network such as a NAS (Network Attached Storage) or a SAN (Storage Area Network).

[0028] The image DB 6 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. In the image DB 6, medical images acquired by the imaging device 2 and accompanying information attached to the medical images are registered in association with each other.

[0029] The incidental information may include, for example, identification information such as an image ID (identification) for identifying the medical image, a tomographic ID assigned to each tomographic image included in the medical image, a subject ID for identifying the subject, and an examination ID for identifying the examination. The incidental information may also include, for example, information related to imaging, such as the imaging method, imaging conditions, and imaging date and time for capturing the medical image. Examples of the "imaging method" and "imaging conditions" include the type of imaging device 2, the imaging region, the imaging protocol, the imaging sequence, the imaging technique, whether or not a contrast agent was used, and the slice thickness in tomography. The incidental information may also include information related to the subject, such as the subject's name, date of birth, age, and gender. The incidental information may also include information related to the purpose of capturing the medical image.

[0030] Furthermore, when the image server 5 receives a request to register a medical image from the imaging device 2, it converts the medical image into a database format and registers it in the image DB 6. Furthermore, when the image server 5 receives a request to view from the image interpretation WS 3 and the medical treatment WS 4, it searches for medical images registered in the image DB 6 and transmits the searched medical images to the image interpretation WS 3 and the medical treatment WS 4 that have sent the request to view.

[0031] The report server 7 is a general-purpose computer on which a software program that provides the functions of a database management system is installed. The report server 7 is connected to a report DB 8. The connection between the report server 7 and the report DB 8 is not particularly limited, and they may be connected by a data bus or via a network such as a NAS or SAN.

[0032] The report DB 8 is realized by a storage medium such as an HDD, an SSD, a flash memory, etc. The report DB 8 registers the interpretation reports created in the interpretation WS 3.

[0033] Furthermore, when the report server 7 receives a request to register an interpretation report from the image interpretation WS 3, it formats the interpretation report for a database and registers it in the report DB 8. Furthermore, when the report server 7 receives a request to view an interpretation report from the image interpretation WS 3 and the medical care WS 4, it searches for the interpretation report registered in the report DB 8 and transmits the searched interpretation report to the image interpretation WS 3 and the medical care WS 4 that have made the view request.

[0034] In addition, the report server 7 may also store the results of analysis performed by CAD (Computer Aided Detection / Diagnosis) or the like when the radiology report was created, even if they were not included in the radiology report (details will be described later).

[0035] The network 9 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network). The imaging device 2, the interpretation WS 3, the medical treatment WS 4, the image server 5, the image DB 6, the report server 7, and the report DB 8 included in the information processing system 1 may be located in the same medical institution, or in different medical institutions. The number of each device of the imaging device 2, the interpretation WS 3, the medical treatment WS 4, the image server 5, the image DB 6, the report server 7, and the report DB 8 is not limited to the number shown in Fig. 1, and each device may be composed of multiple devices having similar functions.

[0036] Fig. 2 is a diagram schematically illustrating an example of a medical image acquired by the imaging device 2. The medical image T shown in Fig. 2 is, for example, a CT image made up of a plurality of tomographic images T1 to Tm (m is 2 or more) each representing a cross-sectional plane from the head to the waist of a single subject (human body).

[0037] FIG. 3 is a diagram schematically illustrating an example of one tomographic image Tx among the multiple tomographic images T1 to Tm. The tomographic image Tx illustrated in FIG. 3 represents a cross-sectional plane including the lungs. Each of the tomographic images T1 to Tm may include a structure region SA representing various organs and tissues of the human body (e.g., lungs and liver) and various tissues constituting the organs and tissues (e.g., blood vessels, nerves, muscles, etc.). Each tomographic image may also include an abnormal shadow region AA representing a lesion such as a nodule, tumor, injury, defect, or inflammation. In the tomographic image Tx illustrated in FIG. 3, the lung region is the structure region SA, and the nodule region is the abnormal shadow region AA. Note that a single tomographic image may include multiple structure regions SA and / or abnormal shadow regions AA. Hereinafter, at least one of the structure region SA and the abnormal shadow region AA will be referred to as a "region of interest."

[0038] Medical images are analyzed using CAD with a classifier trained by deep learning or other methods to detect and / or diagnose regions of interest contained in the medical images. The interpreter of the image report, such as a radiologist, creates the image report by referring to the CAD analysis results of the medical image. However, even if the CAD analysis results are obtained, the interpreter may not actually include the analysis results in the image report, depending on the interpreter's judgment. On the other hand, a lesion that was initially determined not to require inclusion in the image report may worsen over time or the interpreter may change, making it necessary to include the results in a subsequent examination. In such cases, comparative interpretation requires the time and effort of re-obtaining analysis results from a point in time that were not included in the image report.

[0039] Therefore, the information processing device 10 according to this embodiment has a function of supporting comparative interpretation of medical images at a certain point in time even if the interpretation report at that point in time does not include the analysis results of the medical images. The information processing device 10 will be described below. As described above, the information processing device 10 is included in the interpretation WS3.

[0040] First, an example of the hardware configuration of an information processing device 10 according to this embodiment will be described with reference to Fig. 4. As shown in Fig. 4, the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I / F (Interface) 26. The network I / F 26 is connected to a network 9 and performs wired or wireless communication. The CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I / F 26 are connected via a bus 28 such as a system bus and a control bus so as to be able to exchange various information with one another.

[0041] The storage unit 22 is realized by a storage medium such as an HDD, an SSD, or a flash memory. The storage unit 22 stores an information processing program 27 for the information processing device 10. The CPU 21 reads the information processing program 27 from the storage unit 22, loads it into the memory 23, and executes the loaded information processing program 27. The CPU 21 is an example of a processor of the present disclosure. For example, a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, etc. can be used as the information processing device 10, as appropriate.

[0042] Next, an example of the functional configuration of the information processing device 10 according to this embodiment will be described with reference to Fig. 5. As shown in Fig. 5, the information processing device 10 includes an acquisition unit 30, a determination unit 32, a reacquisition unit 34, and a control unit 36. The CPU 21 executes the information processing program 27, causing the CPU 21 to function as the acquisition unit 30, the determination unit 32, the reacquisition unit 34, and the control unit 36.

[0043] In the following description, a medical image obtained by imaging a subject at the current time, i.e., a medical image for which a current radiology report is being created, is referred to as a "current image." Also, a medical image obtained by imaging the same subject at a time point prior to the current time point is referred to as a "past image." The current time point is an example of a first time point in the present disclosure, and the current image is an example of a first image in the present disclosure. A past time point is an example of a second time point in the present disclosure, and the past image is an example of a second image in the present disclosure.

[0044] The acquiring unit 30 acquires finding information including at least measurement values ​​measured for a region of interest included in a current image. Specifically, the acquiring unit 30 may acquire the current image from the image server 5, extract a region of interest from the current image, and perform image analysis on the region of interest to acquire the finding information.

[0045] The finding information includes at least the measurement values ​​measured for the region of interest, and may also include information indicating various other findings, such as the name (type), characteristics, location, and suspected disease name of the region of interest. The measurement values ​​are values ​​that can be quantitatively measured from medical images, such as at least one of the size and signal value of the region of interest. The size is expressed, for example, by the major axis, minor axis, area, and volume of the region of interest. The signal value is expressed, for example, by the pixel value of the region of interest and the CT value in units of HU.

[0046] Examples of names (types) include the names of structures such as "lung" and "liver," and the names of abnormal shadows such as "nodule." Characteristics primarily refer to the characteristics of abnormal shadows. For example, in the case of pulmonary nodules, these include absorption values ​​such as "solid" and "ground-glass," margin shapes such as "clear / indistinct," "smooth / irregular," "spicule," "lobulated," and "serrated," and findings indicating the overall shape such as "near-circular" and "irregular." Other examples include the relationship with surrounding tissues such as "pleural contact" and "pleural indentation," and findings regarding the presence or absence of contrast enhancement and washout.

[0047] The location refers to the anatomical location, the location in the medical image, and the relative positional relationship with other regions of interest such as "inside," "periphery," and "surroundings." The anatomical location may be indicated by the organ name such as "lung" or "liver," or may be expressed by subdividing the lung into the "right lung," "upper lobe," and apical segment ("S1"). The estimated disease name is an evaluation result estimated based on the abnormal shadow, and examples include disease names such as "cirrhosis," "cancer," and "inflammation," as well as evaluation results regarding the disease name and characteristics such as "negative / positive," "benign / malignant," and "mild / severe."

[0048] Methods for extracting a region of interest from a current image may include, as appropriate, methods using known CAD technology and AI (Artificial Intelligence) technology. For example, the acquisition unit 30 may input a medical image and extract a region of interest from the current image using a learning model such as a convolutional neural network (CNN) that has been trained to extract and output a region of interest contained in the medical image. Methods for acquiring finding information through image analysis may include, as appropriate, methods using known CAD technology and AI technology. For example, the acquisition unit 30 may input a region of interest extracted from a medical image and acquire measurement values ​​using a learning model such as a CNN that has been trained in advance to output finding information about the region of interest.

[0049] Furthermore, when multiple regions of interest are extracted for the same subject, the acquiring unit 30 may acquire finding information for each of the multiple regions of interest. For example, the acquiring unit 30 may acquire finding information for each of multiple regions of interest included in a single medical image. Also, for example, the acquiring unit 30 may acquire finding information for each of multiple regions of interest included in multiple tomographic images each representing a different tomographic plane.

[0050] The acquisition unit 30 also inquires of the report server 7 whether a past radiology report describing diagnostic content based on past images is registered in the report DB 8. If a past radiology report is registered in the report DB 8, the acquisition unit 30 acquires the past radiology report. A past radiology report describing diagnostic content based on past images is an example of a second document of the present disclosure.

[0051] The determination unit 32 determines whether or not a past radiology report acquired by the acquisition unit 30 includes a measurement value measured for a region of interest included in a past image that is the same as a region of interest included in a current image. As a method for determining whether or not a measurement value is included, for example, the method may use words related to the measurement value (such as "major axis", "minor axis", "volume", "cm", "mm", "cm"). 3 ", etc.) may be stored in advance in the storage unit 22, and it may be determined whether or not these words are included in the radiology report. Alternatively, for example, a known named entity extraction method using a natural language processing model such as BERT (Bidirectional Encoder Representations from Transformers) may be applied as appropriate.

[0052] Here, the region of interest included in the previous image is the same as the region of interest included in the current image. Correspondence between the regions of interest may be achieved, for example, by aligning the previous image with the current image. Hereinafter, the region of interest included in the current image will be referred to as the "first region of interest," and the region of interest included in the previous image that is the same as the first region of interest will be referred to as the "second region of interest."

[0053] Furthermore, the measurement value measured for the second region of interest included in the past image is of the same type as the measurement value measured for the first region of interest included in the current image, such as at least one of the size and signal value of the second region of interest. Hereinafter, the measurement value for the first region of interest will be referred to as the "first measurement value," and the measurement value for the second region of interest that is the same type as the first measurement value will be referred to as the "second measurement value." In other words, the second measurement value is a past value of the first measurement value.

[0054] For example, assume that the acquisition unit 30 acquires the major axis (first measurement value) of a first region of interest included in the current image. In this case, the determination unit 32 determines whether or not the previous image includes a second region of interest that is identical to the first region of interest. If the previous image includes the same second region of interest, the determination unit 32 further determines whether or not the major axis (second measurement value) of the second region of interest has been measured and the major axis is included in the radiology report.

[0055] The reacquisition unit 34 acquires the second measurement value when the determination unit 32 makes a negative determination. For example, when the determination unit 32 makes a negative determination, the reacquisition unit 34 may acquire the second measurement value that has already been measured based on a past image. For example, the reacquisition unit 34 may acquire the second measurement value stored in various storage media such as the storage unit 22, the report server 7, the report DB 8, the image server 5, the image DB 6, and other external devices.

[0056] Furthermore, for example, if the determination by the determination unit 32 is a negative determination, the reacquisition unit 34 may measure the second measurement value based on the past image. Specifically, if the determination by the determination unit 32 is a negative determination, the reacquisition unit 34 may acquire the past image from the image server 5, extract a second region of interest from the acquired past image, and measure the second measurement value based on the extracted second region of interest. As a method for extracting the second region of interest from the past image and a method for acquiring the second measurement value, methods using known CAD technology and AI technology, etc., can be appropriately applied, similar to the method described above for the current image.

[0057] Furthermore, for example, the reacquisition unit 34 may acquire the second measurement value by combining these. Specifically, when the determination by the determination unit 32 is negative and a second measurement value that has been measured based on a past image exists in various storage media, the reacquisition unit 34 may acquire the measured second measurement value from various storage media, or may measure the second measurement value based on the past image if a second measurement value that has been measured does not exist.

[0058] The control unit 36 ​​controls the display of the second measurement value on the display 24. Specifically, if the determination by the determination unit 32 is a positive determination, i.e., if the second measurement value is included in the past radiology report, the control unit 36 ​​extracts the second measurement value from the past radiology report acquired by the acquisition unit 30 and displays it on the display 24. On the other hand, if the determination by the determination unit 32 is a negative determination, i.e., if the second measurement value is not included in the past radiology report, the control unit 36 ​​displays the second measurement value acquired by the reacquisition unit 34 on the display 24.

[0059] Furthermore, the control unit 36 ​​may cause the display 24 to display the second measurement value acquired by the reacquisition unit 34 when the determination by the determination unit 32 is a negative determination in a display format different from that used when the second measurement value is included in a past radiology report (i.e., when the determination by the determination unit 32 is a positive determination). For example, the control unit 36 ​​may change the underline, bold, italics, size, color, and type of the character string representing the second measurement value, as well as the background color of the character string, or may cause the character string to blink.

[0060] Furthermore, the control unit 36 ​​may cause the display 24 to display the second measurement values ​​in different display formats depending on whether the reacquisition unit 34 has acquired the second measurement values ​​that have already been measured from various storage media or whether the reacquisition unit 34 has measured the second measurement values ​​based on past images. This is because the source of the second measurement values ​​(past radiology reports, storage media, or past images) may be a factor in determining whether or not the user should include them in the findings.

[0061] Hereinafter, examples of the information processing device 10 according to this embodiment will be described with reference to FIGS.

[0062] (For the first test) First, the function of the information processing device 10 when creating an image interpretation report for an initial examination of a certain subject will be described with reference to Figures 6 to 8. Figure 6 shows an example of a screen D1A for creating an image interpretation report, which is displayed on the display 24 by the control unit 36. As shown in Figure 6, the screen D1A includes subject information 60, a finding statement L1, a medical image T001, and an image analysis result 62.

[0063] The medical image T001 is a medical image acquired from the image server 5 by the acquisition unit 30, and includes two regions of interest R1 and R2. Furthermore, in the medical image T001, the regions of interest R1 and R2 are each highlighted by being surrounded by a bounding box 90. The subject information 60 is information included in the supplementary information of the medical image T001, such as the subject ID, the subject's name, sex, date of birth, and age, as well as the purpose of the examination. The image analysis result 62 indicates the finding information for the regions of interest R1 and / or R2 included in the medical image T001. Note that in the case of an initial examination, no previous radiology reports have been registered in the report DB8, so nothing is displayed in the "Previous Findings" column.

[0064] The user selects a region of interest on the medical image T001 for which the user wishes to confirm the finding information by operating the mouse pointer 92 via the input unit 25. The acquisition unit 30 acquires finding information including at least measurement values ​​for the region of interest selected by the user. The control unit 36 ​​displays the finding information acquired by the acquisition unit 30 on the display 24 as the image analysis result 62. Screen D1A shows a state in which the region of interest R1 is selected.

[0065] Next, the user checks the image analysis results 62 (finding information), and if they decide to include a finding statement for the selected region of interest in the radiology report, they select the find statement creation button 94. When the find statement creation button 94 is selected, the control unit 36 ​​generates a finding statement based on the finding information for the selected region of interest and displays it on the display 24. Screen D1A shows a state in which a finding statement L1 for the region of interest R1 is displayed.

[0066] Specifically, the control unit 36 ​​may generate a finding statement including the finding information acquired by the acquisition unit 30 by machine learning. As a method for generating a finding statement using machine learning, for example, a method using a recurrent neural network as described in Patent Document 1 above can be applied as appropriate. Furthermore, for example, the control unit 36 ​​may generate a finding statement by a known method for generating a finding statement using a predetermined template.

[0067] The control unit 36 ​​may also accept user corrections to the finding statements displayed on the display 24. The control unit 36 ​​may also acquire, not only finding statements generated using machine learning, but also finding statements stored in advance in the report DB 8, the storage unit 22, other external devices, etc., and display them on the display 24. The control unit 36 ​​may also accept manual input of finding statements by the user.

[0068] Furthermore, if the user wishes to check the finding information for another region of interest, the user may select the other region of interest on the medical image T001. When the user selects another region of interest, the control unit 36 ​​controls the display 24 to display the finding information for the other region of interest. Figure 7 is an example of a screen D1B transitioning from the screen D1A (see Figure 6) in which the region of interest R1 is selected to a screen D1B in which the region of interest R2 is selected.

[0069] Furthermore, if the user checks the image analysis results 62 (finding information) and determines that the finding statement for the selected region of interest does not need to be included in the radiology report, the user does not need to select the finding statement creation button 94. In this case, the control unit 36 ​​does not generate a finding statement. On screen D1B, the finding statement creation button 94 is not selected for the region of interest R2, and the finding statement L1 does not include finding information for the region of interest R2.

[0070] When the generation of the radiology report is completed as described above, the control unit 36 ​​requests the image server 5 to register the medical image in the image DB 6, and requests the report server 7 to register the radiology report including the finding information and the finding text in the report DB 8. FIG. 8 shows an example of the contents registered in the report DB 8. As shown in FIG. 8, the report DB 8 stores the finding information and the finding text L1 for the region of interest R1 described in the radiology report, and the finding information for the region of interest R2 not described in the radiology report. In addition, the report DB 8 may store the examination ID, the subject ID, the image ID, and other supplementary information (not shown), as appropriate.

[0071] (If stated in previous radiology reports) Next, the function of the information processing device 10 when creating an image interpretation report for the second or subsequent examination of a subject will be described with reference to Fig. 9. In the case of the second or subsequent examination, unlike the case of the initial examination, past image interpretation reports are registered in the report DB8, so comparative image interpretation can be performed while referring to the past image interpretation reports.

[0072] Below, an example will be described in which an interpretation report describing a diagnosis based on medical image T001 created in Figures 6 to 8 is treated as a past interpretation report, and an interpretation report describing a diagnosis based on medical image T002 is created on screen D2A in Figure 9. Note that some of the explanations overlapping with the explanation for the initial examination will be omitted.

[0073] Fig. 9 is an example of a screen D2A for creating an interpretation report, which is displayed on the display 24 by the control unit 36. As shown in Fig. 9, the screen D2A includes subject information 60, a finding statement L2A, a medical image T002, an image analysis result 62, and a previous finding statement L1.

[0074] The medical image T002 is a medical image acquired from the image server 5 by the acquisition unit 30, and includes two regions of interest R1 and R2. The previous finding L1 is a finding included in a past radiology report acquired from the report server 7 by the acquisition unit 30.

[0075] The user selects a region of interest on the medical image T002 for which the user wishes to check the finding information by operating the mouse pointer 92 via the input unit 25. The acquisition unit 30 acquires finding information including at least the measurement value (first measurement value) for the region of interest selected by the user. The determination unit 32 determines whether or not the previous finding statement L1 (past radiology report) includes the previous measurement value (second measurement value) for the region of interest selected by the user.

[0076] Screen D2A shows a state in which region of interest R1 is selected. In this case, determination unit 32 determines that previous finding statement L1 includes a past measurement value for region of interest R1. When determination unit 32 determines that previous finding statement L1 includes a past measurement value, control unit 36 ​​extracts the past measurement value from previous finding statement L1 and displays it on display 24. Screen D2A displays the past measurement value "(previously 23 mm)" in image analysis result 62.

[0077] Next, the user checks the image analysis results 62 (finding information) and the previous finding statement L1, and if the user decides to include a finding statement for the selected region of interest in the radiology report, selects the find statement creation button 94. When the find statement creation button 94 is selected, the control unit 36 ​​generates a finding statement based on the finding information for the selected region of interest and past measurement values, and displays it on the display 24. Specifically, the control unit 36 ​​may generate a finding statement that represents the results of a comparison with past measurement values. Screen D2A shows a state in which a finding statement L2A is displayed for the region of interest R1, including a statement that represents the results of a comparison with past measurement values, such as "slightly increased from last time."

[0078] (If not mentioned in previous radiology reports) 10 is an example of a screen D2B that transitions from a screen D2A (see FIG. 9) in which the region of interest R1 is selected to a screen D2B in which the region of interest R2 is selected. In this case, the determination unit 32 determines that the previous finding statement L1 does not include a past measurement value for the region of interest R2.

[0079] When the determination unit 32 determines that the previous finding statement L1 does not include a past measurement value, the re-acquisition unit 34 acquires the past measurement value that has been measured based on the medical image T001 from the finding information (see FIG. 8) registered in the report DB8. The control unit 36 ​​causes the display 24 to display the past measurement value acquired by the re-acquisition unit 34. On the screen D2B, the past measurement value "(previously 3 mm)" is displayed in the image analysis result 62.

[0080] Furthermore, "(previous 3 mm)" in the image analysis result 62 on screen D2B is underlined, while "(previous 23 mm)" in the image analysis result 62 on screen D2A is not underlined. In this way, the control unit 36 ​​may determine whether or not to underline the past measurement values ​​("(previous 23 mm)" and "(previous 3 mm)") in the image analysis result 62, depending on whether or not the second measurement value was included in the past radiology report.

[0081] When the user selects the find statement button 94 while the region of interest R2 is selected, the control unit 36 ​​adds a find statement based on the finding information and past measurement values ​​for the selected region of interest R2 and displays it on the display 24. Screen D2B shows a state in which a find statement L2B including a statement about the region of interest R1 and a statement about the region of interest R2 is displayed.

[0082] (When displaying a plot of measured values) The control unit 36 ​​may create a plot of the first measurement values ​​acquired by the acquisition unit 30 and the second measurement values ​​acquired by the reacquisition unit 34, and may perform control to display the plot on the display 24. FIG. 11 shows an example of a screen D3 on which the plot is displayed among the screens for creating an interpretation report, which is displayed on the display 24 by the control unit 36. The screen D3 is a screen for treating the interpretation report created in FIGS. 9 and 10, which describes a diagnosis based on the medical image T002, as a past interpretation report, and creating an interpretation report that describes a diagnosis based on the medical image T003. As shown in FIG. 11, the screen D3 includes subject information 60, a finding statement L3, the medical image T003, a previous finding statement L2B, and a plot P.

[0083] Specifically, the control unit 36 ​​may create a plot diagram P that shows a line graph with the measured values ​​on the vertical axis and the measurement time points (time axis) of the measured values ​​on the horizontal axis. By presenting the transition of the measured values ​​as a plot diagram P in this way, it becomes easier to understand the changes in the measured values ​​over time.

[0084] 12, the control unit 36 ​​may display the plot in a different form depending on whether the second measurement value acquired by the reacquisition unit 34 is a negative determination by the determination unit 32 or a case where the second measurement value is included in a past radiology report. FIG. 12 is a modified example of the plot diagram P in FIG. 11, in which the plot as of March 2021 acquired from the report DB8 by the reacquisition unit 34 is displayed in a different form from other plots extracted from the findings by the control unit 36.

[0085] In addition, the control unit 36 ​​may change the display format of the plot depending on whether the reacquisition unit 34 acquires a second measurement value that has already been measured from various storage media or whether the reacquisition unit 34 measures based on a past image.

[0086] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 13. In the information processing device 10, the CPU 21 executes the information processing program 27, thereby executing the information processing shown in Fig. 13. The information processing is executed, for example, when a user issues an instruction to start execution via the input unit 25.

[0087] In step S10, the acquisition unit 30 acquires a first measurement value measured for a first region of interest included in a first image (current image) obtained by imaging the subject at a first time point (current time point). In step S12, the acquisition unit 30 acquires a second document (past image interpretation report) describing a diagnosis based on a second image (past image) obtained by imaging the subject at a second time point (past time point).

[0088] In step S14, the determination unit 32 determines whether the second document acquired in step S12 contains second measurement values ​​of the same type as the first measurement values ​​acquired in step S10, which are measured for a second region of interest that is included in the second image and is identical to the first region of interest. If the determination in step S14 is positive, the process proceeds to step S16. In step S16, the control unit 36 ​​extracts the second measurement values ​​from the second document acquired in step S12.

[0089] On the other hand, if the determination in step S14 is negative, the process proceeds to step S18. In step S18, the reacquisition unit 34 determines whether or not the second measurement values ​​that have been measured based on the second image are present in various storage media (storage unit 22, report server 7, report DB 8, image server 5, image DB 6, other external devices, etc.). If the determination in step S18 is positive, the process proceeds to step S20. In step S20, the reacquisition unit 34 acquires the second measurement values ​​that have been measured and stored in various storage media.

[0090] On the other hand, if the determination in step S18 is negative, the process proceeds to step S22. In step S22, the re-acquisition unit 34 acquires a second image from the image server 5 and measures a second measurement value based on the second image. In step S24, the control unit 36 ​​controls the display 24 to display the second measurement value acquired in any of step S16, step S18, and step S22, and then ends this information processing.

[0091] As described above, an information processing device 10 according to one embodiment of the present disclosure includes at least one processor, which acquires a first measurement value measured for a first region of interest included in a first image obtained by photographing a subject at a first time point, acquires a second document describing a diagnosis based on a second image obtained by photographing the subject at a second time point, determines whether the second document includes a second measurement value of the same type as the first measurement value measured for a second region of interest that is identical to the first region of interest included in the second image, and if the determination is negative, acquires the second measurement value.

[0092] That is, with the information processing device 10 according to this embodiment, even if the measurement values ​​for the region of interest of the object currently being interpreted are not included in the previous interpretation report, it is possible to obtain the measurement values ​​at a previous time point. Therefore, for example, even if a lesion that was once determined not to need to be included in the interpretation report worsens over time or the preparer changes, and it is now determined that it is necessary to include it in the next examination, it is possible to obtain the measurement values ​​at a previous time point. That is, with the information processing device 10 according to this embodiment, it is possible to support comparative interpretation.

[0093] In the above embodiment, the acquiring unit 30 acquires finding information including at least measurement values ​​by performing image analysis on medical images, but the present invention is not limited to this. For example, the acquiring unit 30 may acquire finding information stored in advance in the storage unit 22, the image server 5, the image DB 6, the report server 7, the report DB 8, or other external devices. Furthermore, for example, the acquiring unit 30 may acquire finding information manually input by a user via the input unit 25.

[0094] In the above embodiment, the description is given assuming a situation where an interpretation report is created in the interpretation WS 3, but the present invention is not limited to this. For example, the information processing device 10 may be applied to a situation where two interpretation reports are compared and viewed in the interpretation WS 3 and / or the medical care WS 4.

[0095] Furthermore, in the above embodiment, a form assuming an interpretation report of a medical image has been described, but the present disclosure is not limited thereto. The information processing device 10 of the present disclosure is applicable to various documents describing diagnostic content based on images obtained by photographing a subject. For example, the information processing device 10 may be applied to documents describing diagnostic content based on images obtained from equipment, buildings, pipes, welds, etc., as subjects in non-destructive testing such as radiographic testing and ultrasonic flaw detection testing.

[0096] Furthermore, in the above embodiment, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the acquisition unit 30, the determination unit 32, the reacquisition unit 34, and the control unit 36. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0097] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0098] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0099] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0100] In the above embodiment, the information processing program 27 is pre-stored (installed) in the storage unit 22, but this is not limiting. The information processing program 27 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 27 may also be downloaded from an external device via a network. Furthermore, the technology disclosed herein extends to not only information processing programs but also storage media that non-temporarily store information processing programs.

[0101] The technology of the present disclosure can also be appropriately combined with the above-described exemplary embodiments and examples. The above-described description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure and are merely examples of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or new elements may be replaced with other parts from the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. [Explanation of symbols]

[0102] 1. Information Processing Systems 2. Imaging equipment 3 Image Reading Workshop 4. Medical Workshop 5 Image Server 6. Image Database 7 Report Server 8 Report DB 9 Network 10. Information processing equipment 21 CPU 22 Memory section 23 Memory 24 displays 25 Input section 26 Network I / F 27 Information Processing Program 28 Bus 30 Acquisition Department 32 Judgment section 34 Reacquisition Department 36 Control Unit 60 Subject Information 62 Image analysis results 90 Bounding Box 92 Mouse Pointer 94 Observation statement creation button AA Area of ​​abnormal shadow D1A, D1B, D2A, D2B, D3 screens L1, L2A, L2B, L3 Observations P plot R, R1, R2 Regions of Interest SA Structure Area T Medical Imaging T1~Tm, T001~T003, Tx tomographic images

Claims

1. at least one processor; The processor: acquiring a first measurement value measured for a first region of interest included in a first image obtained by imaging the subject at a first time point; acquiring a second document describing a diagnosis based on a second image obtained by imaging the subject at a second time point; determining whether the second document includes a second region of interest included in the second image, the second region of interest being the same as the first region of interest, and a second measurement value of the same type as the first measurement value; If the determination is negative, if there are second measurement values ​​that have been measured based on the second image, the second measurement values ​​that have been measured are acquired; If the second measurement value has not been measured, the second measurement value is measured based on the second image. Information processing device.

2. The processor: The second measurement value is displayed on the display in different display formats when the second measurement value is acquired and when the second measurement value is measured based on the second image. The information processing device according to claim 1 .

3. The processor: If the determination is negative, the second image is acquired; extracting the second region of interest from the acquired second image; measuring the second measurement value based on the extracted second region of interest; 3. The information processing device according to claim 1.

4. The processor: If the determination is negative, the acquired second measurement value is displayed on the display in a display format different from that when the second measurement value is included in the second document. The information processing device according to any one of claims 1 to 3.

5. The processor: creating a plot of the first measurement value and the second measurement value obtained; Display the plot on the display The information processing device according to any one of claims 1 to 4.

6. The first and second measurements are at least one of a size and a signal value of the first and second regions of interest, respectively. The information processing device according to any one of claims 1 to 5.

7. acquiring a first measurement value measured for a first region of interest included in a first image obtained by imaging the subject at a first time point; acquiring a second document describing a diagnosis based on a second image obtained by photographing the subject at a second time point; determining whether the second document includes a second region of interest included in the second image, the second region of interest being the same as the first region of interest, and a second measurement value of the same type as the first measurement value; If the determination is negative, if there are second measurement values ​​that have been measured based on the second image, the second measurement values ​​that have been measured are acquired; If the second measurement value has not been measured, the second measurement value is measured based on the second image. An information processing method in which processing is performed by a computer.

8. acquiring a first measurement value measured for a first region of interest included in a first image obtained by imaging the subject at a first time point; acquiring a second document describing a diagnosis based on a second image obtained by photographing the subject at a second time point; determining whether the second document includes a second region of interest included in the second image, the second region of interest being the same as the first region of interest, and a second measurement value of the same type as the first measurement value; If the determination is negative, if there are second measurement values ​​that have been measured based on the second image, the second measurement values ​​that have been measured are acquired; If the second measurement value has not been measured, the second measurement value is measured based on the second image. An information processing program that causes a computer to execute a process.

Images