Information processing apparatus, information processing method, and information processing program

Abstract

To provide an information processing apparatus, an information process method, and an information process program capable of supporting creation of a diagnostic reading report.SOLUTION: An information processing apparatus 10 includes at least one processor, and the processor acquires an opinion sentence including a plurality of pieces of elemental information about at least one region of interest, performs first determination as to whether the opinion sentence is written about a plurality of regions of interest or written about one region of interest based on the elemental information, and determines a presenting method of the opinion sentence based on a result of the first determination.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, image diagnosis is performed using medical images obtained by imaging devices such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices. A person who creates an image interpretation report, such as a doctor who reads the medical images, interprets the medical images and creates an image interpretation report that includes a statement of findings.

[0003] In order to support the creation of radiology reports, a method has been proposed for searching and presenting appropriate findings from a group of previously created findings. For example, Patent Document 1 discloses a method for executing a search process of a structured document database based on word-based information input by a user, and executing a process for generating a sentence explaining the morphological description of an image based on the input information.

[0004] For example, Patent Document 2 discloses a method for searching and outputting a reference reading report from among multiple reading reports created during previous readings, which are expressed in structured tag format, using input tag information as a search key. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2008-146220 A [Patent Document 2] JP 2006-155002 A Summary of the Invention [Problem to be solved by the invention]

[0006] In some cases, new findings are created by rewriting parts of past findings obtained as search results. In order to improve the efficiency of creating findings in this case, it is desirable to present, as search results, findings that require fewer rewritings.

[0007] The present disclosure provides an information processing device, an information processing method, and an information processing program capable of supporting the creation of an image interpretation report. [Means for solving the problem]

[0008] A first aspect of the present disclosure is an information processing device comprising at least one processor, which acquires an observation statement including multiple element information for at least one region of interest, makes a first determination based on the element information as to whether the observation statement is described for multiple regions of interest or for a single region of interest, and determines a method for presenting the observation statement based on a result of the first determination.

[0009] In the first aspect described above, when presenting at least one of the multiple finding sentences, the processor may lower the presentation priority of a finding sentence determined to describe multiple regions of interest.

[0010] In the above first aspect, the processor may perform a second determination on a finding statement determined in the first determination to describe multiple regions of interest, as to whether the finding statement describes multiple types of regions of interest or multiple regions of interest of one type, and may determine a method of presenting the finding statement based on the result of the second determination.

[0011] In the first aspect described above, when presenting at least one of the multiple finding sentences, the processor may lower the presentation priority of a finding sentence that is determined to describe multiple types of multiple regions of interest.

[0012] In the above first aspect, when presenting at least one of the multiple finding statements, the processor may increase the presentation priority of a finding statement that describes a number and type of areas of interest that match the number and type of new areas of interest for which a new finding statement is to be created.

[0013] In the first aspect, the processor may make the first determination based on the number of element information pieces indicating the type of the region of interest included in the finding statement.

[0014] In the first aspect above, the processor may make the first determination based on the number of groups when the element information is divided into groups for each region of interest.

[0015] In the above first aspect, the observation sentence is composed of a plurality of partial sentences, and the processor may perform a first judgment for each partial sentence, and determine a presentation method for dividing the observation sentence into partial sentences and presenting them based on the result of the first judgment.

[0016] In the above first aspect, the processor may classify element information contained in a partial sentence into whether it is about a focused region of interest, which is a region of interest that is the focus of attention, whether it is about a non-focused region of interest, which is a region of interest other than the focused region of interest, or whether it is common to the focused region of interest and the non-focused region of interest, and may determine that a partial sentence containing element information about a non-focused region of interest is described about multiple regions of interest.

[0017] In the above first aspect, the processor may acquire new element information about one new area of ​​interest for which a new finding statement is to be created, and rewrite the finding statement determined in the first determination to be written about one area of ​​interest in accordance with the new element information and present it.

[0018] In the above first aspect, the processor may acquire multiple pieces of new element information for each of multiple new areas of interest for which a new finding statement is to be created, and may rewrite the finding statement determined in the first determination to have been written for one area of ​​interest in accordance with the multiple pieces of new element information and present it.

[0019] In the above first aspect, the processor may acquire a plurality of new element information for each of a plurality of new regions of interest for which a new finding statement is to be created, and may rewrite, in accordance with the plurality of new element information, a finding statement described for a number and type of region of interest that matches the number and type of the plurality of new regions of interest, among the finding statements determined in the first determination to be described for a plurality of regions of interest.

[0020] In the first aspect, the new element information used for rewriting may indicate at least one of a position, a measurement value, and a number regarding the new region of interest.

[0021] In the first aspect, the processor may accept an input of new element information by a user.

[0022] In the above first aspect, the processor may obtain an image including at least one new region of interest of a subject for which a new finding statement is to be created, and generate new element information based on the image.

[0023] In the first aspect, the element information may include at least one of a property, a position, a measurement value, a number, and a diagnosis of the region of interest, and a term that indicates a change in the region of interest.

[0024] A second aspect of the present disclosure is an information processing method in which a computer executes a process of acquiring an observation statement including multiple element information for at least one region of interest, making a first determination based on the element information as to whether the observation statement is described for multiple regions of interest or for a single region of interest, and determining a method for presenting the observation statement based on a result of the first determination.

[0025] A third aspect of the present disclosure is an information processing program for causing a computer to execute a process of acquiring an observation statement including multiple element information for at least one region of interest, making a first determination based on the element information as to whether the observation statement is described for multiple regions of interest or for a single region of interest, and determining a method for presenting the observation statement based on a result of the first determination. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing system. [Diagram 2] FIG. 1 is a diagram illustrating an example of a medical image. [Diagram 3] FIG. 1 is a diagram illustrating an example of a medical image. [Figure 4] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device. [Diagram 5] FIG. 2 is a block diagram showing an example of a functional configuration of an information processing device. [Figure 6] This is an example of a finding statement describing the area of ​​concern 1. [Figure 7] 1 is an example of a finding statement that describes multiple regions of interest. [Figure 8] 1 is an example of a finding statement that describes multiple regions of interest. [Figure 9] FIG. 13 is a diagram showing an example of a screen displayed on a display. [Figure 10] FIG. 13 is a diagram for explaining rewriting of a finding sentence. [Figure 11] FIG. 13 is a diagram for explaining rewriting of a finding sentence. [Figure 12] FIG. 13 is a diagram for explaining rewriting of a finding sentence. [Figure 13] FIG. 13 is a diagram for explaining rewriting of a finding sentence. [Figure 14] FIG. 13 is a diagram showing an example of a screen displayed on a display. [Figure 15] 11 is a flowchart illustrating an example of information processing. [Figure 16] 13 is an example of a finding sentence consisting of a plurality of sub-sentences. [Figure 17] 13 is an example of a finding sentence consisting of a plurality of sub-sentences. [Figure 18] 13 is an example of a finding sentence consisting of a plurality of sub-sentences. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0028] [First embodiment] First, the configuration of an information processing system 1 to which an information processing device 10 of the present disclosure is applied will be described with reference to Fig. 1. Fig. 1 is a diagram showing a schematic configuration of the information processing system 1. The information processing system 1 photographs an examination target part of a subject and stores the photographed medical images based on an examination order from a doctor of a medical department using a known ordering system. It also enables an image interpretation doctor to interpret the medical images and create an image interpretation report, and enables a doctor of the requesting medical department to view the image interpretation report.

[0029] 1, the information processing system 1 includes an imaging device 2, an interpretation WS (WorkStation) 3 which is an interpretation terminal, a medical treatment WS 4, an image server 5, an image DB (DataBase) 6, a report server 7, and a report DB 8. The imaging device 2, the interpretation WS 3, the medical treatment WS 4, the image server 5, the image DB 6, the report server 7, and the report DB 8 are connected to each other via a wired or wireless network 9 in a state in which they can communicate with each other.

[0030] Each device is a computer in which an application program is installed to cause the device to function as a component of the information processing system 1. The application program may be recorded on a recording medium such as a DVD-ROM (Digital Versatile Disc Read Only Memory) or a CD-ROM (Compact Disc Read Only Memory) and distributed, and installed on the computer from the recording medium. Alternatively, the application program may be stored in a storage device or network storage of a server computer connected to the network 9 in an externally accessible state, and downloaded to the computer upon request and installed.

[0031] The imaging device 2 is a device (modality) that captures an image of a part of a subject to be diagnosed, thereby generating a medical image T representing the part to be diagnosed. Examples of the imaging device 2 include a plain X-ray device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, an ultrasonic diagnostic device, an endoscope, and a fundus camera. The medical image generated by the imaging device 2 is transmitted to an image server 5 and stored in an image DB 6.

[0032] Fig. 2 is a diagram showing a schematic diagram of an example of a medical image acquired by the imaging device 2. The medical image T shown in Fig. 2 is, for example, a CT image consisting of a plurality of tomographic images T1 to Tm (m is 2 or more), each of which represents a cross-sectional plane from the head to the waist of a single subject (human body). The medical image T is an example of an image disclosed herein.

[0033] FIG. 3 is a diagram showing a schematic example of one of the tomographic images Tx among the multiple tomographic images T1 to Tm. The tomographic image Tx shown in FIG. 3 shows a cross-sectional plane including the lungs. Each of the tomographic images T1 to Tm may include a structure region SA showing various organs and organs of the human body (e.g., lungs and kidneys, etc.), and various organs and tissues constituting the organs (e.g., blood vessels, nerves, muscles, etc.). Each of the tomographic images may include a lesion region AA such as a nodule, a tumor, damage, a defect, and inflammation. In the tomographic image Tx shown in FIG. 3, the lung region is the structure region SA, and the nodule region is the lesion region AA. Note that one tomographic image may include multiple structure regions SA and / or lesion regions AA. The lesion region AA is an example of a region of interest in the present disclosure.

[0034] The image interpretation WS3 is a computer used by a medical professional such as a radiologist to interpret medical images and prepare image interpretation reports, and includes the information processing device 10 according to this embodiment. The image interpretation WS3 issues a request to view a medical image to the image server 5, performs various image processing on the medical image received from the image server 5, displays the medical image, and accepts input of text related to the medical image. The image interpretation WS3 also performs analysis processing on the medical image, supports the preparation of an image interpretation report based on the analysis result, requests the report server 7 to register and view the image interpretation report, and displays the image interpretation report received from the report server 7. These processes are performed by the image interpretation WS3 executing software programs for each process.

[0035] The medical treatment WS4 is a computer used by medical personnel such as doctors in a medical department to observe medical images in detail, view interpretation reports, and create electronic medical records, and is configured to include a processing device, a display device such as a display, and an input device such as a keyboard and a mouse. The medical treatment WS4 issues requests to the image server 5 to view medical images, displays medical images received from the image server 5, issues requests to the report server 7 to view interpretation reports, and displays interpretation reports received from the report server 7. These processes are performed by the medical treatment WS4 executing software programs for each process.

[0036] The image server 5 is a general-purpose computer installed with a software program that provides the functions of a database management system (DBMS). The image server 5 is connected to an image DB 6. The connection between the image server 5 and the image DB 6 is not particularly limited, and may be a connection via a data bus or a connection via a network such as a NAS (Network Attached Storage) or a SAN (Storage Area Network).

[0037] The image DB 6 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. In the image DB 6, medical images acquired by the imaging device 2 and associated information attached to the medical images are registered in association with each other.

[0038] The incidental information may include, for example, identification information such as an image ID (identification) for identifying a medical image, a tomographic ID assigned to each tomographic image included in the medical image, a subject ID for identifying the subject, and an examination ID for identifying an examination. The incidental information may also include, for example, information related to imaging of a medical image, such as an imaging method, imaging conditions, imaging purpose, imaging date and time, and imaging location. The "imaging method" and "imaging conditions" are, for example, the type of imaging device 2, the imaging site, imaging protocol, imaging sequence, imaging technique, whether or not a contrast agent is used, and slice thickness in tomography. The incidental information may also include information related to the subject, such as the subject's name, date of birth, age, and sex.

[0039] Furthermore, when the image server 5 receives a registration request for a medical image from the imaging device 2, it converts the medical image into a format for a database and registers it in the image DB 6. Furthermore, when the image server 5 receives a viewing request from the image interpretation WS 3 and the medical care WS 4, it searches for medical images registered in the image DB 6 and transmits the searched medical image to the image interpretation WS 3 and the medical care WS 4 that have sent the viewing request.

[0040] The report server 7 is a general-purpose computer installed with a software program that provides the functions of a database management system. The report server 7 is connected to a report DB 8. The connection between the report server 7 and the report DB 8 is not particularly limited, and may be a connection via a data bus or a connection via a network such as a NAS or a SAN.

[0041] The report DB 8 is realized by a storage medium such as a HDD, an SSD, a flash memory, etc. The report DB 8 registers the image interpretation reports created in the image interpretation WS 3 (details will be described later).

[0042] Furthermore, when the report server 7 receives a request to register an interpretation report from the image interpretation WS3, it converts the interpretation report into a format for a database and registers it in the report DB 8. Furthermore, when the report server 7 receives a request to view an interpretation report from the image interpretation WS3 and the medical care WS4, it searches for the interpretation report registered in the report DB8 and transmits the searched interpretation report to the image interpretation WS3 and the medical care WS4 that have made the request to view the interpretation report.

[0043] The network 9 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network). The imaging device 2, the interpretation WS3, the medical care WS4, the image server 5, the image DB6, the report server 7, and the report DB8 included in the information processing system 1 may be located in the same medical institution, or in different medical institutions. The number of each device of the imaging device 2, the interpretation WS3, the medical care WS4, the image server 5, the image DB6, the report server 7, and the report DB8 is not limited to the number shown in FIG. 1, and each device may be composed of a plurality of devices having similar functions.

[0044] Incidentally, when the interpretation WS3 interprets a medical image and creates an interpretation report, past interpretation reports (findings) registered in the report DB8 may be referred to for reference. In this case, a search query according to the medical image to be interpreted is used to search for past findings describing findings and / or diagnoses that are identical or similar to those of the medical image to be interpreted. For example, if the medical image to be interpreted contains a lung nodule, a search is performed using "lung" and "nodule" as the search query, and findings containing "lung" and "nodule" are presented as the search results.

[0045] In addition, new findings may be created by rewriting a part of a past finding obtained as a search result. For example, a new finding may be completed by simply modifying the location, measurement value, number, etc. of the lesion among the descriptions of the lesion contained in the past finding. On the other hand, if the past finding includes descriptions of multiple lesions or multiple lesions of different types, and the dependencies are complex, the effort required for modification increases. Therefore, in order to improve the efficiency of creating findings, it is desirable to present past findings that require fewer rewritings as search results.

[0046] Therefore, the information processing device 10 according to the present embodiment supports the creation of an image interpretation report by presenting past findings suitable for creating a new image interpretation (i.e., past findings suitable for repurposing). The information processing device 10 will be described below. As described above, the information processing device 10 is included in the image interpretation WS3.

[0047] First, an example of a hardware configuration of the information processing device 10 according to the present embodiment will be described with reference to Fig. 4. As shown in Fig. 4, the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I / F (Interface) 26. The network I / F 26 is connected to a network 9 and performs wired and / or wireless communication. The CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I / F 26 are connected to each other via a bus 28 such as a system bus and a control bus so that various information can be exchanged between them.

[0048] The storage unit 22 is realized by a storage medium such as an HDD, an SSD, or a flash memory. An information processing program 27 for the information processing device 10 is stored in the storage unit 22. The CPU 21 reads the information processing program 27 from the storage unit 22, expands it in the memory 23, and executes the expanded information processing program 27. The CPU 21 is an example of a processor of the present disclosure. As the information processing device 10, for example, a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, or the like can be appropriately applied.

[0049] Next, an example of a functional configuration of the information processing device 10 according to the present embodiment will be described with reference to Fig. 5. As shown in Fig. 5, the information processing device 10 includes a registration unit 30, an acquisition unit 32, a search unit 34, a determination unit 36, and a display control unit 38. The CPU 21 executes the information processing program 27, causing the CPU 21 to function as each of the functional units of the registration unit 30, the acquisition unit 32, the search unit 34, the determination unit 36, and the display control unit 38.

[0050] (Structuring of radiology reports) First, a structuring process of an image reading report will be described as a pre-processing executed by the information processing device 10 prior to a search process of a finding sentence. As described above, an image reading report that has already been created is registered in the report DB8. Specifically, a plurality of finding sentences and diagnosis information and finding information related to each finding sentence are registered in the report DB8 in association with each other. The diagnosis information and finding information are each an example of element information of the present disclosure.

[0051] The finding statements are registered in the report DB 8 by the report server 7 when each image interpretation report is created. The registration unit 30 specifies element information related to each of a plurality of finding statements (hereinafter, referred to as a group of findings) registered in the report DB 8, and assigns the element information to the finding statement (so-called structuring).

[0052] The diagnostic information is, for example, a diagnosis of a presumed disease name, etc., diagnosed based on a medical image. The presumed disease name, etc., is an evaluation result presumed based on a lesion included in a medical image, and includes, for example, disease names such as "cancer" and "inflammation," as well as evaluation results regarding the disease name and characteristics such as "negative / positive," "benign / malignant," and "mild / severe."

[0053] Furthermore, for example, one finding may include multiple suspected disease names, such as "renal cell carcinoma is suspected, but angiomyolipoma with a small amount of fat may also be present." In this case, the registration unit 30 may select one suspected disease name as the diagnosis information, and assign other suspected disease names, etc. that were not selected to the finding as the finding information.

[0054] The finding information is information indicating at least one of various findings such as the type (name), characteristics, position, measurement value and number of the region of interest, a phrase expressing a change in the region of interest, and a type of diagnostic information other than the diagnostic information to be paired, etc. As described above, the type of diagnostic information other than the diagnostic information to be paired is a presumed disease name that was not selected as the diagnostic information.

[0055] Examples of types (names) of a region of interest include names of structures such as "lung" and "liver" and names of lesions such as "nodule" and "ground-glass opacity". The properties of a region of interest mainly refer to the characteristics of the lesion. For example, in the case of a nodule, examples include absorption values ​​such as "solid", "low absorption" and "high absorption", margin shapes such as "clear / unclear", "smooth / irregular", "spicule", "lobulated" and "serrated", and findings indicating the overall shape such as "near-circular" and "irregular". Other examples include the relationship with surrounding tissues such as "protrusion", and findings regarding the presence or absence of contrast and washout.

[0056] The location of a region of interest means an anatomical location, a location in a medical image, and a relative positional relationship with other regions of interest such as "inside", "edge", and "surroundings". The anatomical location may be expressed by organ names such as "lung" and "kidney", or may be expressed by subdividing the lungs such as "right lung", "upper lobe", and apical segment ("S1").

[0057] The measurement value of the region of interest is a value that can be quantitatively measured from a medical image, and is, for example, at least one of the size and signal value of the region of interest. The size is expressed, for example, by the major axis, minor axis, area, and volume of the region of interest. The signal value is expressed, for example, by the pixel value of the region of interest and the CT value in units of HU. The finding information indicating the measurement value may be divided into predetermined classes (i.e., quantized), such as "0 mm or more and less than 5 mm", "5 mm or more and less than 10 mm", and "10 mm or more and less than 15 mm". In this case, for example, if the finding statement contains a statement of "12 mm", the finding information of "10 mm or more and less than 15 mm" is specified. The finding information indicating the measurement value may simply be information indicating whether or not there is a statement in the finding statement. This is because there may be a large variation in the measurement value or the statement may be omitted.

[0058] The number of regions of interest may be expressed as a specific number such as one or two, or may be expressed in relative terms such as "single / multiple" and "few / many." The phrases expressing changes in the regions of interest are phrases expressing changes over time in the characteristics, location, measurement values, number, etc., of the regions of interest when they are observed over time, and examples of such phrases include "appearance / disappearance," "increase / reduction," "worsening / improvement," and "metastasis."

[0059] Specifically, the registration unit 30 extracts named entities (words) from the findings registered in the report DB 8, and identifies element information corresponding to the extracted named entities. As a method for extracting named entities from findings, for example, a known named entity extraction method using a natural language processing model such as BERT (Bidirectional Encoder Representations from Transformers) can be appropriately applied.

[0060] In addition, the group of findings may contain different words with the same meaning (synonyms), such as "angiomyolipoma" and "AML" (Angiomyolipoma). In order to deal with spelling variations due to such synonyms, when the group of findings contains synonyms, it is preferable that the registration unit 30 assigns the same element information to each of the finding sentences containing the synonyms (so-called normalization).

[0061] Specifically, a dictionary that defines the correspondence between named entities that may be included in a finding sentence and element information, in which synonymous named entities are associated with the same element information, may be stored in advance in the storage unit 22. For example, in the dictionary, the synonymous words "angiomyolipoma" and "AML" may each be associated with the same diagnostic information, "angiomyolipoma." The registration unit 30 may extract named entities from the finding sentence registered in the report DB8 and identify the element information included in the finding sentence by referring to the dictionary.

[0062] In addition, it is preferable that the registration unit 30 also specifies the factuality of the specified element information. "Factuality" is information indicating whether a finding is observed or not, and the degree of certainty, etc. This is because the radiology report may include not only findings that are clearly observed in medical images, but also findings that are not observed in medical images, and findings that are suspected but have a low degree of certainty. For example, since the presence or absence and the degree of fat components are used to differentiate between AML and renal cell carcinoma (RCC), the radiology report may include the statement that "fat components are not observed."

[0063] Furthermore, element information may be such that it modifies other element information, and in this case, it is preferable that the registration unit 30 also specifies the modifying relationship between element information. For example, "calcification", which is an example of the characteristics of a pulmonary nodule, may be described in detail, such as "microcalcification is observed in the center". In this case, the registration unit 30 may specify the finding information "center" and "micro" as other finding information that modifies the finding information "calcification". Examples of finding information that modifies "calcification" include "micro", "coarse", "scattered", "center", "ring-shaped", and "complete".

[0064] For example, in the case of "renal cell carcinoma," which is an example of diagnostic information based on kidney nodules, the histological types such as "clear cell," "papillary," "chromophobe," and "multilocular cystic" may also be described in the findings. The registration unit 30 may specify element information indicating such a histological type as other element information modifying the diagnostic information "renal cell carcinoma."

[0065] Furthermore, when one finding statement includes element information related to a plurality of regions of interest (lesions), the registration unit 30 preferably identifies element information for each region of interest.

[0066] In addition, when the diagnostic information cannot be identified from the finding statement, the registration unit 30 may assign the diagnostic information of "undiagnosed" to the finding statement. For example, when a lesion (region of interest) included in a medical image is small and the findings can be interpreted but are below the diagnostic standard (i.e., not important) that is predetermined in the guidelines, only the finding information may be recorded and not recorded as diagnostic information. For example, when the region of interest is obviously benign, only the finding information may be recorded and not recorded as diagnostic information. For example, the diagnostic information may not be recorded due to the creator of the finding statement forgetting to write it. For example, the registration unit 30 may not be able to identify the diagnostic information from the finding statement due to a complex sentence structure of the finding statement or other reasons.

[0067] The registration unit 30 may also specify element information based on a medical image related to the finding text, such as a medical image registered in the image DB 6 or the like. Specifically, the registration unit 30 may specify element information not included in the finding text based on the medical image. Note that the registration unit 30 does not need to specify all of the above-mentioned various types of element information, and may specify only a predetermined type of element information (for example, element information indicating the size of the region of interest) based on the medical image.

[0068] For example, if the size of the region of interest is small, the description is less important or cannot be measured from the medical image, whereas if it is large, it can be seen from the medical image, so description in the finding statement may be omitted regardless of the size. On the other hand, the size of the region of interest may be related to characteristics, etc., and even if description in the finding statement is omitted, it may implicitly affect the content of the finding statement. For example, the larger the lung cancer tumor, the more likely it is that internal necrosis will occur. Therefore, it is preferable that the registration unit 30 assigns element information that is not included in the finding statement but can be identified based on the medical image to the finding statement and registers it in the report DB8.

[0069] An example of a method for identifying element information based on a medical image will be described. The registration unit 30 acquires a medical image related to the findings, and extracts at least one region of interest (e.g., a nodule region) included in the medical image. As a method for extracting a region of interest, a method using a known CAD (Computer Aided Detection / Diagnosis) technology and an AI (Artificial Intelligence) technology can be appropriately applied. For example, the registration unit 30 may extract a region of interest from a medical image using a learning model such as a CNN (Convolutional Neural Network) that is trained to input a medical image, extract and output a region of interest included in the medical image.

[0070] The registration unit 30 then generates element information of the extracted region of interest. For example, the registration unit 30 may generate element information of the region of interest using a learning model such as CNN that is trained in advance to receive the region of interest extracted from the medical image and output element information of the region of interest. The registration unit 30 assigns at least one of the element information thus generated to the finding text and registers it in the report DB8.

[0071] The registration unit 30 may also generate element information based on additional information attached to a medical image registered in, for example, the image DB 6. As described above, each medical image is attached with additional information at the time of registration in the image DB 6. In addition, the registration unit 30 may obtain, for example, information included in test orders and electronic medical records, information indicating various test results such as blood tests and infectious disease tests, and information indicating the results of health checkups from an external device such as the medical treatment WS 4, and generate the element information.

[0072] (Search for findings) Next, a process of searching for a finding sentence executed by the information processing device 10 will be described.

[0073] The acquisition unit 32 acquires a search query. For example, the acquisition unit 32 may accept a search query input by the user via the input unit 25 (see FIG. 9). For example, the acquisition unit 32 may acquire a medical image T0 to be interpreted from the image server 5, and generate element information about the medical image T0 as a search query based on the medical image T0. The search query input by the user and the search query generated based on the medical image may be used in combination.

[0074] An example of a method for generating element information (search query) based on a medical image will be described. For example, the acquisition unit 32 extracts at least one region of interest (e.g., a nodule region) included in the acquired medical image T0. As a method for extracting a region of interest, a method using known CAD technology and AI technology, etc., can be appropriately applied. For example, the acquisition unit 32 may extract a region of interest from a medical image using a learning model such as CNN that is trained to input a medical image and extract and output a region of interest included in the medical image.

[0075] The acquisition unit 32 then generates element information of the extracted region of interest. For example, the acquisition unit 32 may generate element information of the region of interest using a learning model such as CNN that is pre-trained to receive the region of interest extracted from the medical image and output element information of the region of interest. The acquisition unit 32 may use at least one of the element information thus generated as a search query. Note that the acquisition unit 32 does not need to identify all of the various types of element information (e.g., organ, area name, findings, changes over time, diagnosis, etc.), and may identify only a predetermined type of element information (e.g., element information indicating the size of the region of interest) based on the medical image and use it as a search query.

[0076] The search unit 34 searches for a plurality of candidate finding sentences related to the search query acquired by the acquisition unit 32 from a group of finding sentences including a plurality of finding sentences registered in the report DB8. The search unit 34 may search by giving priority to a predetermined type of search query. For example, the search unit 34 may search by giving priority to a search query indicating an organ, a type of lesion, characteristics, and a diagnosis. On the other hand, the search unit 34 may not use a search query indicating the position and measurement value of a lesion. This is because, since the position and measurement value vary greatly from subject to subject, using them in a search reduces the number of search results, whereas when a candidate finding sentence is repurposed, the effort required for correcting them is small, and therefore, it may be more efficient not to use them in a search.

[0077] In addition, when searching for a finding sentence candidate, the search unit 34 may allow ambiguity in the search query used for the search. For example, for a search query indicating a position, a finding sentence candidate to which finding information indicating a position within a predetermined range (for example, the position of an anatomically adjacent area) is added may be output as a search result. For example, for a search query indicating a measurement value, a finding sentence candidate to which finding information indicating a measurement value including a difference in a predetermined amount or percentage is added may be output as a search result. For example, for a search query indicating a diagnosis, a finding sentence candidate to which diagnosis information indicating different diagnoses belonging to the same classification (for example, "primary lung cancer" and "lung cancer") is added may be output as a search result. Note that the determination of whether different diagnoses belong to the same classification may be performed using, for example, an ICD (International Statistical Classification of Diseases and Related Health Problems) code, a dictionary, an ontology, or the like.

[0078] (Presentation of findings) The above search process acquires a finding sentence candidate according to the search query. Fig. 6 to Fig. 8 show examples of a plurality of different finding sentence candidates P1 to P3 acquired by the acquisition unit 32. Hereinafter, a presentation method for presenting at least one of the plurality of finding sentence candidates acquired by the acquisition unit 32 will be described. The finding sentence candidate is an example of a finding sentence of the present disclosure.

[0079] Specifically, the acquiring unit 32 acquires, as a search result, a finding sentence candidate including a plurality of element information for at least one region of interest. The determining unit 36 ​​performs a first determination as to whether the finding sentence candidate describes a plurality of regions of interest or a single region of interest, based on the element information included in the finding sentence candidate acquired by the acquiring unit 32. That is, the determining unit 36 ​​performs the first determination by referring to the element information registered in association with each finding sentence candidate by the above-mentioned structuring process.

[0080] As an example, Fig. 6 shows a finding sentence candidate P1 described for one region of interest, and Figs. 7 and 8 show finding sentence candidates P2 and P3 described for multiple regions of interest. For the sake of explanation, Figs. 6 to 8 also show a graph structure in which element information is enclosed in a rectangle as a node and element information (nodes) related to the same region of interest are connected by edges. In Figs. 7 and 8, the structure of one subgraph when element information is divided into subgraphs for each region of interest is shown by a solid line, the structures of other subgraphs are shown by dotted lines, and element information common to each subgraph is enclosed in a rectangle with a thick solid line.

[0081] Specifically, the determination unit 36 ​​may perform the first determination based on the number of element information pieces indicating the type of the region of interest included in the finding sentence candidate. For example, the finding sentence candidate P1 includes only "nodule" as element information indicating the type of the region of interest, and is therefore determined to describe one region of interest. On the other hand, the finding sentence candidate P2 includes two element information pieces indicating the types of the region of interest, "nodule" and "bronchial dilation," and is therefore determined to describe multiple regions of interest.

[0082] More specifically, the determination unit 36 ​​may perform the first determination based on the number of groups when element information included in the finding sentence candidate is divided into groups for each region of interest. The groups for each region of interest are the subgraphs shown in Figs. 6 to 8. For example, the finding sentence candidate P1 includes only element information related to "nodule" as a region of interest, so the number of groups (number of subgraphs) is 1, and it is determined that the finding sentence candidate P1 describes one region of interest.

[0083] On the other hand, for example, finding candidate P2 includes element information on each of the regions of interest, "nodule" and "bronchiectasis," so the number of groups (number of subgraphs) is 2, and it is determined that it describes multiple regions of interest. Also, for example, finding candidate P3 includes element information on each of "nodule" in "right lung S1" and "nodule" in "(right lung) S2," so the number of groups (number of subgraphs) is 2, and it is determined that it describes multiple regions of interest.

[0084] The display control unit 38 determines a presentation method for the finding sentence candidates based on the result of the first determination by the determination unit 36, and presents at least one finding sentence candidate according to the determined presentation method. Specifically, the display control unit 38 may lower the presentation priority of an observation sentence candidate determined in the first determination to describe multiple regions of interest. Then, the display control unit 38 may control the display 24 to display a predetermined number of finding sentence candidates in descending order of priority, or one finding sentence candidate with the highest priority, etc. For example, the display control unit 38 may lower the presentation priority of the finding sentence candidates P2 and P3 among the finding sentence candidates P1 to P3.

[0085] When creating a new finding sentence by rewriting a part of a finding sentence candidate, it is considered that a finding sentence candidate that requires fewer rewriting parts, i.e., a finding sentence candidate that describes one region of interest, is suitable. In other words, when creating a new finding sentence by rewriting a part of a finding sentence candidate that is determined to describe multiple regions of interest, it is considered that the rewriting is often time-consuming. Therefore, the efficiency of creating a new finding sentence can be improved by lowering the priority of presenting a finding sentence candidate that is determined to describe multiple regions of interest.

[0086] 9 shows an example of a screen D1 displayed on the display 24 by the display control unit 38. At the top of the screen D1, text boxes 90 for each item for inputting a search query and a slider bar 92 for specifying the size of the region of interest are displayed. The user inputs at least one search query using these text boxes 90 and slider bar 92. In the example of FIG. 9, "lung," "left lung S1+2," "nodule," and "9mm" have been input as search queries.

[0087] The lower part of the screen D1 displays the medical image T0 to be interpreted and three finding candidates as a result of the search performed by the search unit 34 using the input search query. The user checks the presented finding candidates, and if the user wishes to check other finding candidates, selects the "other candidates" button 94. In this case, the display control unit 38 re-presents the other finding candidates (i.e., finding candidates with lower priority).

[0088] On the other hand, if the user determines that the presented finding sentence candidates include a finding sentence candidate that can be used as the basis for a new finding sentence, the user selects the “Rewrite” button 96. In this case, the display control unit 38 rewrites the finding sentence candidate and presents it again.

[0089] (Rewriting of findings) The process of rewriting a candidate finding sentence will be described below.

[0090] The acquiring unit 32 acquires new element information for a new region of interest for which a new finding sentence is to be created. When there are multiple new regions of interest for which a new finding sentence is to be created, the acquiring unit 32 acquires multiple pieces of new element information for each of them. Here, the new element information is information used to rewrite the finding sentence. In general, when a finding sentence candidate is rewritten, information that is the basis of the content of the finding sentence, such as the organ, the type of lesion, the characteristics, and the diagnosis, is not rewritten and is often diverted as it is. Therefore, the new element information used for rewriting may be, for example, information indicating at least one of the position, the measurement value, and the number for the new region of interest.

[0091] Specifically, the acquisition unit 32 may set the search query as the new element information. Alternatively, for example, the acquisition unit 32 may accept an input of new element information by a user. Alternatively, for example, the acquisition unit 32 may acquire a medical image T0 including at least one new region of interest for which a new finding statement is to be created, and generate new element information based on the medical image T0. A method for generating new element information based on the medical image T0 is similar to, for example, the method for generating element information (search query) based on a medical image described above.

[0092] The display control unit 38 rewrites and presents the finding sentence candidate according to the new element information acquired by the acquisition unit 32. An example of rewriting the finding sentence candidate according to the new element information will be described with reference to Figs. 10 to 13. Note that the finding sentence candidate to be rewritten in the example of Figs. 10 to 13 is the finding sentence candidate P1 that has been determined in the first determination to be a description of one region of interest.

[0093] 10 is an example of rewriting a finding sentence candidate described for one region of interest according to new element information for one new region of interest for which a new finding sentence is to be created. As the new element information, three items are acquired: "nodule" indicating the type of the new region of interest, "left lung S1+2" indicating its position, and "9mm" indicating its measurement value (size). In this case, the display control unit 38 rewrites the "right lung S1" portion indicating the position of the region of interest to "left lung S1+2" in the finding sentence candidate, and rewrites the "10mm" portion indicating the measurement value (size) of the region of interest to "9mm".

[0094] 11 to 13 are examples of rewriting a finding sentence candidate described for one region of interest according to multiple new element information for each of multiple new regions of interest for which a new finding sentence is to be created. In the example of FIG. 11, the type of the multiple new regions of interest is common to "nodule" (first nodule, second nodule), their position is common to "left lung S1+2", and their measurement values ​​(sizes) are different, "10mm" and "8mm", respectively. In this case, the display control unit 38 rewrites the part of the finding sentence candidate "right lung S1" indicating the position of the region of interest to the common "left lung S1+2", and rewrites the part of "10mm" indicating the measurement value (size) of the region of interest to "10mm and 8mm".

[0095] 12, the types of the multiple new regions of interest are common to "nodule" (first nodule, second nodule), their locations are different to "left lung S1+2" and "left lung S3", and their measurement value (size) is common to "9 mm." In this case, the display control unit 38 rewrites, among the candidate findings, the portion "right lung S1" indicating the location of the region of interest to "left lung S1+2 and S3," and rewrites the portion "10 mm" indicating the measurement value (size) of the region of interest to the common "9 mm."

[0096] 13, the type of the multiple new regions of interest is a common "nodule" (first nodule, second nodule), their positions are different, "left lung S1+2" and "left lung S3", and their measurement values ​​(sizes) are different, "10 mm" and "8 mm". In this case, the display control unit 38 rewrites the part of the finding sentence candidate, "right lung S1", which indicates the position of the region of interest, to "left lung S1+2 and S3", and rewrites the part of "10 mm", which indicates the measurement value (size) of the region of interest, to "10 mm and 8 mm".

[0097] The same finding sentence can be generated by performing the same rewriting based on the finding sentence candidates described for multiple regions of interest shown in Fig. 8. However, since there are many rewriting parts in such finding sentence candidates described for multiple regions of interest, the processing time required for rewriting may increase.

[0098] As shown in the examples of Figures 10 to 13, a finding candidate determined in the first determination to be described for one region of interest can be easily rewritten, regardless of whether the new region of interest for which a new finding is to be created is singular or multiple. In other words, a finding candidate described for one region of interest has high versatility when repurposed, and is preferably presented preferentially.

[0099] The display control unit 38 presents at least one finding sentence candidate rewritten according to the new element information. For example, the display control unit 38 may perform control so that each of the finding sentence candidates presented when the “Rewrite” button 96 on the screen D1 is selected is rewritten according to the new element information and then displayed on the display 24.

[0100] 14 shows an example of screen D2 displayed on the display 24 by the display control unit 38. Screen D2 is a screen to which the user transitions when the "Rewrite" button 96 on screen D1 is selected. At the bottom of screen D2, each of the multiple finding sentence candidates displayed on screen D1 is displayed after being rewritten in accordance with the search query (new element information).

[0101] The user checks the presented rewritten finding sentence candidates and selects an appropriate one as a finding sentence for the medical image T0 to be interpreted with the mouse pointer 98. The display control unit 38 registers the selected rewritten finding sentence candidate in the report DB 8 as a finding sentence for the medical image T0 to be interpreted. Note that the display control unit 38 may accept corrections by the user before registering the rewritten finding sentence candidate in the report DB 8.

[0102] Next, the operation of the information processing device 10 according to the embodiment will be described with reference to Fig. 15. In the information processing device 10, the CPU 21 executes the information processing program 27, thereby executing the information processing shown in Fig. 15. The information processing is executed, for example, when a command to start execution is given by a user via the input unit 25. It is assumed that the structuring process of the image interpretation report described above has been executed in advance.

[0103] In step S10, the acquisition unit 32 acquires a search query. In step S12, the search unit 34 searches a group of findings including a plurality of findings registered in the report DB 8 for a plurality of finding sentence candidates related to the search query acquired in step S10.

[0104] In step S14, the determination unit 36 ​​determines whether the finding sentence candidate describes multiple regions of interest or one region of interest based on the element information included in the finding sentence candidate searched (acquired) in step S12 (first determination). In step S16, the display control unit 38 determines a presentation method for the finding sentence candidate based on the result of the first determination in step S14, and performs control to display on the display 24 a screen presenting at least one finding sentence candidate according to the determined presentation method. When step S16 is completed, this information processing ends.

[0105] As described above, the information processing device 10 according to this embodiment includes at least one processor, which acquires an observation statement including multiple element information for at least one region of interest, makes a first determination based on the element information as to whether the observation statement is described for multiple regions of interest or for one region of interest, and determines a method for presenting the observation statement based on the result of the first determination.

[0106] That is, according to the information processing device 10 of this embodiment, when a new finding sentence is to be created by rewriting a part of a finding sentence candidate acquired as a search result, a finding sentence candidate describing one region of interest can be preferentially presented. Since a finding sentence candidate describing one region of interest requires fewer parts to be rewritten, the efficiency of creating a finding sentence can be improved by preferentially presenting such a finding sentence candidate. Therefore, the creation of a radiology report can be supported.

[0107] [Second embodiment] For example, for a lesion with a low occurrence frequency or a lesion that frequently occurs together with other lesions, there may be few past findings written only about that lesion. In this case, even if it is attempted to present findings written about one region of interest preferentially as in the first embodiment, there may be no suitable findings or the number of findings may be small.

[0108] Therefore, the information processing device 10 according to this embodiment supports the creation of radiology reports by presenting parts of the candidate findings acquired as search results that are suitable for creating new findings. That is, in this embodiment, even if a finding describes multiple regions of interest, parts that are suitable for reuse are presented preferentially. Below, the functions of the information processing device 10 according to this embodiment will be described, but descriptions that overlap with those of the first embodiment will be omitted.

[0109] The acquiring unit 32 acquires the finding sentence candidates searched by the searching unit 34 in response to the search query. Figures 16 to 18 show examples of a plurality of different finding sentence candidates P4 to P6 acquired by the acquiring unit 32. For the sake of explanation, Figures 16 to 18 also show graph structures similar to Figure 6 and the like.

[0110] As shown in Figs. 16 to 18, a finding sentence candidate may be composed of a plurality of partial sentences. A partial sentence here is one sentence separated by a period, for example, partial sentence P41 in Fig. 16 ("A 10 mm nodule was found in the right lung S1."). A finding sentence candidate is a set of partial sentences, for example, a sentence consisting of three sentences, partial sentence P41 ("A 10 mm nodule was found in the right lung S1."), partial sentence P42 ("An 8 mm nodule was also found in S2."), and partial sentence P43 ("Inflammatory changes are suspected.") in Fig. 16. Finding sentence candidate P4 consists of partial sentences P41 to P43. Finding sentence candidate P5 consists of partial sentences P51 to P53. Finding sentence candidate P6 consists of partial sentences P61 to P63.

[0111] The determination unit 36 ​​divides the finding sentence candidates acquired by the acquisition unit 32 into partial sentences, and performs a first determination for each partial sentence. That is, the determination unit 36 ​​performs a first determination as to whether a partial sentence describes multiple regions of interest or a single region of interest, based on element information included in the partial sentence.

[0112] Specifically, the determination unit 36 ​​classifies the element information included in the partial sentence into whether it is about a focused region of interest, which is a region of interest to be focused on, whether it is about a non-focused region of interest, which is a region of interest other than the focused region of interest, or whether it is common to the focused region of interest and the non-focused region of interest. Then, the determination unit 36 ​​determines that the partial sentence including the element information about the non-focused region of interest is described for multiple regions of interest.

[0113] The display control unit 38 determines a presentation method for dividing the finding sentence candidate into partial sentences and presenting them based on the result of the first determination by the determination unit 36, and presents at least one partial sentence according to the determined presentation method. Specifically, the display control unit 38 may control the display 24 to display the other partial sentences, excluding the partial sentences determined in the first determination to be described for a plurality of regions of interest (i.e., partial sentences including element information for a non-attention region of interest). The other partial sentences are, in other words, partial sentences including element information for a attention region of interest and partial sentences including element information common to the attention region of interest and the non-attention region of interest (hereinafter, simply referred to as "common partial sentences").

[0114] For example, in the finding sentence candidate P4, the first "nodules" are set as the focused region of interest, and the second "nodules" are set as the non-focused region of interest. The partial sentence P41 includes only element information related to the first "nodules", and is therefore determined to be a partial sentence related to the focused region of interest. The partial sentence P42 includes only element information related to the second "nodules", and is therefore determined to be a partial sentence related to the non-focused region of interest. The partial sentence P43 includes only element information common to the first "nodules" and the second "nodules", and is therefore determined to be a common partial sentence. The display control unit 38 excludes the partial sentence P42 and presents a sentence including the partial sentences P41 and P43 as the finding sentence candidate P4A.

[0115] On the other hand, if the first "nodule" is set as a non-attention region of interest and the second "nodule" is set as a attention region of interest, the determination of the partial sentences P41 and P42 is reversed. The display control unit 38 excludes the partial sentence P41 and presents a sentence including the partial sentences P42 and P43 as a finding sentence candidate P4B.

[0116] For example, in the finding sentence candidate P5, "partial solid nodule" is set as the focused region of interest, and "ground glass opacity" is set as the non-focused region of interest. Since partial sentence P51 contains only element information related to "partial solid nodule", it is determined to be a partial sentence related to the focused region of interest. Since partial sentence P52 contains only element information related to "ground glass opacity", it is determined to be a partial sentence related to the non-focused region of interest. Since partial sentence P53 contains only element information common to "partial solid nodule" and "ground glass opacity", it is determined to be a common partial sentence. The display control unit 38 excludes partial sentence P52 and presents a sentence including partial sentences P51 and P53 as the finding sentence candidate P5A.

[0117] On the other hand, if the "partial solid nodule" is set as the non-attention region of interest and the "ground glass opacity" is set as the attention region of interest, the determination of the partial sentences P51 and P52 is reversed. The display control unit 38 excludes the partial sentence P51 and presents a sentence including the partial sentences P52 and P53 as the finding sentence candidate P5B.

[0118] For example, in the finding sentence candidate P6, the first "nodule" is set as the focused region of interest, and the second "nodule" is set as the non-focused region of interest. The partial sentence P61 includes element information on the first "nodule" and element information common to the first and second "nodule" ("right lung S1"), and is therefore determined to be a common partial sentence. The partial sentence P62 includes only element information on the second "nodule", and is therefore determined to be a partial sentence on a non-focused region of interest. The partial sentence P63 includes element information common to the first and second "nodule", and is therefore determined to be a common partial sentence. The display control unit 38 excludes the partial sentence P62 and presents a sentence including the partial sentence P61 and the partial sentence P63 as the finding sentence candidate P6A.

[0119] On the other hand, if the first "nodule" is set as a non-attention region of interest and the second "nodule" is set as a attention region of interest, there will be no partial sentence that can be divided and presented from the finding sentence candidate P6. In this case, the partial sentence P61 contains element information related to the first "nodule", so it is determined to be a partial sentence related to the non-attention region of interest. Therefore, the partial sentence P61 is required to be excluded when presented as a finding sentence candidate, but since it also contains element information ("right lung S1") common to the first "nodule" and the second "nodule", the meaning of the partial sentence P62 will not be understood if it is excluded. For example, if the partial sentence P61 is excluded, it will be unclear what the "periphery" at the beginning of the partial sentence P62 refers to. Also, for example, if the partial sentence P61 is left, element information related only to the non-attention region of interest ("middle layer", "10 mm", "nodule") will remain, which will require a lot of effort to correct when rewriting. Therefore, the display control unit 38 does not present such a finding sentence candidate P6N.

[0120] As described above, the information processing device 10 according to the present embodiment performs a first judgment for each of a plurality of partial sentences included in a finding sentence, and determines a presentation method for dividing the finding sentence into partial sentences and presenting them based on the result of the first judgment. That is, according to the information processing device 10 according to the present embodiment, since it is possible to present more finding sentence candidates describing one region of interest, it is possible to improve the efficiency of creating a finding sentence. Therefore, it is possible to support the creation of a radiology report.

[0121] [Third embodiment] In the above-mentioned embodiments, the efficiency of creating a finding statement is emphasized, and a finding statement described in one region of interest (i.e., a finding statement with high versatility) is preferentially presented. However, the present invention is not limited to this. For example, when there are a plurality of new regions of interest for which a new finding statement is to be created, it may be possible to create a finding statement with higher accuracy by presenting past finding statements described in the same number and types of regions of interest as the new regions of interest.

[0122] Therefore, the information processing device 10 according to this embodiment supports the creation of radiology reports by presenting appropriate finding sentence candidates according to the new region of interest for which a new finding sentence is to be created. That is, in this embodiment, even if a finding sentence is described for a plurality of regions of interest, one suitable for the new region of interest is preferentially presented. The functions of the information processing device 10 according to this embodiment will be described below, but descriptions that overlap with the above embodiments will be omitted.

[0123] The acquiring unit 32 acquires the finding sentence candidates searched by the searching unit 34 in response to the search query. The acquiring unit 32 also acquires new element information for a new region of interest for which a finding sentence is to be created. When there are multiple new regions of interest for which a finding sentence is to be created, the acquiring unit 32 acquires multiple pieces of new element information for each of the new regions of interest.

[0124] The determination unit 36 ​​performs a first determination on the finding sentence candidate acquired by the acquisition unit 32. This determines whether the finding sentence candidate describes a plurality of regions of interest or a single region of interest. For example, as described above, the finding sentence candidates P2 and P3 shown in Figs. 7 and 8 are determined to describe a plurality of regions of interest.

[0125] Thereafter, the determination unit 36 ​​performs a second determination as to whether the finding sentence candidate determined in the first determination to have described multiple regions of interest has described multiple types of regions of interest, or has described multiple regions of interest of one type. In this case, it is preferable that the determination unit 36 ​​specifies how many types of regions of interest the finding sentence candidate determined to have described multiple types of regions of interest has described.

[0126] For example, finding candidate P2 is determined to describe two types of regions of interest because it contains element information indicating two types of regions of interest, "nodule" and "bronchiectasis."Furthermore, finding candidate P3 is determined to describe multiple regions of interest of one type because it contains element information indicating two types of regions of interest of the same type, "nodule" in "right lung S1" and "nodule" in "(right lung) S2."

[0127] The display control unit 38 determines a presentation method for the finding sentence candidates based on the result of the second determination by the determination unit 36, and presents at least one finding sentence candidate according to the determined presentation method. Specifically, the display control unit 38 may increase the presentation priority of an observation sentence candidate described for a number and type of regions of interest that match the number and type of the new regions of interest. For example, when the new regions of interest are two regions of interest of two types, the display control unit 38 may increase the priority of an observation sentence candidate P2 having the same number and type. Also, for example, when the new regions of interest are two regions of interest of the same type, the display control unit 38 may increase the priority of an observation sentence candidate P3 having the same number and type.

[0128] Furthermore, the display control unit 38 may rewrite and present the finding sentence candidates according to the new element information acquired by the acquisition unit 32. That is, the display control unit 38 may rewrite and present the finding sentence candidates, among the finding sentence candidates determined in the first determination to have described multiple regions of interest, that have a number and type of regions of interest matching the number and type of the multiple new regions of interest, according to the multiple new element information.

[0129] As described above, the information processing device 10 according to the present embodiment performs a second judgment on a finding sentence determined in the first judgment to describe multiple regions of interest, as to whether the finding sentence describes multiple types of regions of interest or one type of multiple regions of interest, and determines a method of presenting the finding sentence based on the result of the second judgment. That is, the information processing device 10 according to the present embodiment can present finding sentence candidates describing the same number and types of regions of interest as the new region of interest, so that it is possible to create a highly accurate finding sentence and to support the creation of an image interpretation report.

[0130] In the third embodiment, a mode in which the finding sentence candidates described for the same number and types of regions of interest as the new region of interest are preferentially presented has been described, but the present invention is not limited to this. For example, the display control unit 38 may lower the presentation priority of the finding sentence candidates determined in the second determination to be described for multiple types of regions of interest. That is, regardless of the number and types of new regions of interest, the priority of the finding sentence candidates described for multiple types of regions of interest may be lowered. This is because, in general, finding sentences described for multiple types of regions of interest have more complex dependencies and may be more difficult to rewrite than finding sentences described for multiple regions of interest of one type.

[0131] In the above-mentioned embodiments, the interpretation of medical images is assumed, but the present disclosure is not limited thereto. The information processing device 10 of the present disclosure is applicable to various images obtained by photographing a subject and including a region of interest. For example, the information processing device 10 may be applied to images obtained from equipment, buildings, pipes, welds, etc. as subjects in non-destructive inspections such as radiographic inspections and ultrasonic flaw detection inspections. In this case, the region of interest may indicate, for example, cracks, flaws, bubbles, foreign objects, etc.

[0132] In each of the above embodiments, the following various processors can be used as the hardware structure of the processing units that execute various processes, such as the registration unit 30, the acquisition unit 32, the search unit 34, the determination unit 36, and the display control unit 38. As described above, the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0133] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0134] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System on Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0135] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0136] In the above embodiment, the various programs are pre-stored (installed) in the storage unit, but the present invention is not limited to this. The various programs may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The various programs may also be downloaded from an external device via a network. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores a program in addition to the program.

[0137] The technology of the present disclosure can also be appropriately combined with the above-mentioned embodiment examples and examples. The above-mentioned description and illustrated contents are detailed descriptions of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustrated contents shown above, within the scope of the gist of the technology of the present disclosure.

[0138] The following supplementary notes are further disclosed regarding the above embodiment. [Appendix 1] at least one processor; The processor, Obtaining a statement of findings including a plurality of element information for at least one region of interest; making a first determination as to whether the finding statement describes a plurality of the regions of interest or a single region of interest based on the element information; determining a method for presenting the findings based on a result of the first determination; Information processing device. [Appendix 2] The processor, When presenting at least one of the plurality of the finding sentences, a presentation priority is lowered for the finding sentence determined to describe the plurality of the regions of interest. 2. An information processing device according to claim 1. [Appendix 3] The processor, A second determination is made as to whether the finding statement determined in the first determination to describe a plurality of the regions of interest is described with respect to a plurality of types of the regions of interest or is described with respect to a single type of a plurality of the regions of interest; determining a method for presenting the findings based on a result of the second determination; 3. An information processing device according to claim 1 or 2. [Appendix 4] The processor, When presenting at least one of the plurality of finding sentences, a presentation priority is lowered for the finding sentence determined to be described for the plurality of types of the plurality of regions of interest. 4. The information processing device according to claim 3. [Appendix 5] The processor, When presenting at least one of the plurality of the finding statements, a presentation priority is raised for the finding statements that describe the number and type of the areas of interest that match the number and type of new areas of interest for which a new finding statement is to be created. 5. The information processing device according to claim 3 or 4. [Appendix 6] The processor, The first determination is made based on the number of element information pieces indicating the type of the region of interest included in the finding text. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 5. [Appendix 7] The processor, The first determination is made based on the number of groups when the element information is divided into groups for each of the regions of interest. 7. An information processing device according to any one of claims 1 to 6. [Appendix 8] The observation sentence is composed of a plurality of sub-sentences, The processor, performing the first determination for each of the partial sentences; Based on a result of the first determination, a presentation method is determined when the finding sentence is divided into the partial sentences and presented. 8. An information processing device according to any one of claims 1 to 7. [Appendix 9] The processor, The element information included in the partial sentence is classified into an attention region of interest, which is the region of interest to be focused on, an unattention region of interest, which is the region of interest other than the attention region of interest, or a common region between the attention region of interest and the unattention region of interest; The partial sentence including the element information about the non-attention region of interest is determined to be described about a plurality of the regions of interest. 9. The information processing device according to claim 8. [Appendix 10] The processor, Obtain new element information about one new area of ​​interest of a subject for which a new finding statement is to be created; The finding sentence determined in the first determination to be a description of one of the regions of interest is rewritten in accordance with the new element information and presented. 10. An information processing device according to any one of claims 1 to 9. [Appendix 11] The processor, Obtaining a plurality of new element information for each of a plurality of new areas of interest of a subject for which a new finding statement is to be created; The finding sentence determined in the first determination to be a description of one of the regions of interest is rewritten in accordance with the plurality of new element information and presented. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 10. [Appendix 12] The processor, Obtaining a plurality of new element information for each of a plurality of new areas of interest of a subject for which a new finding statement is to be created; Among the finding sentences determined in the first determination to be described for the plurality of regions of interest, the finding sentences described for the number and type of regions of interest that match the number and type of the plurality of new regions of interest are rewritten in accordance with the plurality of new element information and presented. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 11. [Appendix 13] The new element information used for rewriting indicates at least one of a position, a measurement value, and a number for the new region of interest. 13. An information processing device according to any one of claims 10 to 12. [Appendix 14] The processor, Accepting input of the new element information by the user 14. An information processing device according to any one of claims 10 to 13. [Appendix 15] The processor, obtaining an image including at least one new region of interest for which a new finding is to be generated; The new element information is generated based on the image. 15. An information processing device according to any one of claims 10 to 14. [Appendix 16] The element information includes at least one of the following: a property, a position, a measurement, a number, and a diagnosis of the region of interest, and a phrase that represents a change in the region of interest. 16. An information processing device according to any one of claims 1 to 15. [Appendix 17] Obtaining a statement of findings including a plurality of element information for at least one region of interest; making a first determination as to whether the finding statement describes a plurality of the regions of interest or a single region of interest based on the element information; determining a method for presenting the findings based on a result of the first determination; An information processing method in which processing is performed by a computer. [Appendix 18] Obtaining a statement of findings including a plurality of element information for at least one region of interest; making a first determination as to whether the finding statement describes a plurality of the regions of interest or a single region of interest based on the element information; determining a method for presenting the findings based on a result of the first determination; An information processing program that enables a computer to execute processing. [Explanation of symbols]

[0139] 1. Information Processing Systems 2. Imaging Device 3. Image Reading Workshop 4. Clinical Workshop 5 Image Server 6. Image Database 7 Report Server 8 Report DB 9 Network 10. Information processing device 21 CPU 22 Memory section 23 Memory 24 Display 25 Input section 26 Network Interface 27 Information Processing Programs 28 Bus 30 Registration Department 32 Acquisition Department 34 Search section 38 Display control section P1~P6, P4A, P4B, P5A, P5B, P6A, P6N Candidates for observations P41~P63 Partial sentence 90 Text Box 92 Slider bar 94, 96 Buttons 98 Mouse Pointer AA Lesion Area D1~D2 screen SA Structure Area T, T0 Medical Imaging T1~Tm, Tx Tomographic images

Claims

1. At least one processor; The processor, Obtaining a statement of findings including a plurality of element information regarding at least one region of interest; making a first determination as to whether the finding statement describes a plurality of the regions of interest or a single region of interest based on the element information; A method for presenting the findings is determined based on a result of the first determination. Information processing device.

2. The processor, When presenting at least one of the plurality of the finding sentences, a presentation priority is lowered for the finding sentence determined to describe the plurality of the regions of interest. The information processing device according to claim 1 .

3. The processor, A second determination is made as to whether the finding sentence determined in the first determination to have described a plurality of the regions of interest has described a plurality of types of the regions of interest or a single type of a plurality of the regions of interest, A method of presenting the finding sentence is determined based on a result of the second judgment. The information processing device according to claim 1 .

4. The processor, When presenting at least one of the plurality of finding sentences, a presentation priority is lowered for the finding sentence determined to describe a plurality of types of the plurality of regions of interest. The information processing device according to claim 3 .

5. The processor, When presenting at least one of the plurality of finding sentences, a priority is raised for presenting the finding sentences that describe the regions of interest whose number and type match the number and type of new regions of interest for which a new finding sentence is to be created. The information processing device according to claim 3 .

6. The processor, The first determination is made based on the number of pieces of element information indicating the type of the region of interest included in the finding text. The information processing device according to claim 1 .

7. The processor, The first determination is made based on the number of groups when the element information is divided into groups for each of the regions of interest. The information processing device according to claim 1 .

8. The observation sentence is composed of a plurality of sub-sentences, The processor, performing the first determination for each of the partial sentences; A presentation method for dividing the finding sentence into the partial sentences and presenting the divided partial sentences is determined based on a result of the first determination. The information processing device according to claim 1 .

9. The processor, The element information included in the partial sentence is classified into an attention region of interest, which is the region of interest to be focused on, an unattention region of interest, which is the region of interest other than the attention region of interest, or a common region between the attention region of interest and the unattention region of interest; The partial sentence including the element information about the non-attention region of interest is determined to be described about a plurality of the regions of interest. The information processing device according to claim 8.

10. The processor, Obtain new element information about one new area of ​​interest of a subject for which a new finding statement is to be created; The finding sentence determined in the first determination to be a description of one of the regions of interest is rewritten in accordance with the new element information and presented. The information processing device according to claim 1 .

11. The processor, Obtaining a plurality of new element information for each of a plurality of new areas of interest of a subject for which a new finding statement is to be created; The finding sentence determined in the first determination to be a description of one of the regions of interest is rewritten in accordance with the plurality of new element information and presented. The information processing device according to claim 1 .

12. The processor, Obtaining a plurality of new element information for each of a plurality of new areas of interest of a subject for which a new finding statement is to be created; Among the finding sentences determined in the first determination to be described for the plurality of regions of interest, the finding sentences described for the number and type of regions of interest that match the number and type of the plurality of new regions of interest are rewritten in accordance with the plurality of new element information and presented. The information processing device according to claim 1 .

13. The new element information used for rewriting indicates at least one of a position, a measurement value, and a number for the new region of interest. The information processing device according to claim 10.

14. The processor, Accepting input of the new element information by the user The information processing device according to claim 10.

15. The processor, acquiring an image including at least one new region of interest for which a new finding is to be generated; The new element information is generated based on the image. The information processing device according to claim 10.

16. The element information includes at least one of the following: a property, a position, a measurement, a number, and a diagnosis of the region of interest, and a phrase that represents a change in the region of interest. The information processing device according to claim 1 .

17. Obtaining a statement of findings including a plurality of element information regarding at least one region of interest; making a first determination as to whether the finding statement describes a plurality of the regions of interest or a single region of interest based on the element information; A method for presenting the findings is determined based on a result of the first determination. An information processing method in which processing is performed by a computer.

18. Obtaining a statement of findings including a plurality of element information regarding at least one region of interest; making a first determination as to whether the finding statement describes a plurality of the regions of interest or a single region of interest based on the element information; A method for presenting the findings is determined based on a result of the first determination. An information processing program that enables a computer to execute processing.

Images