Information processing apparatus, information processing method, and information processing program

Abstract

To provide an information processing apparatus, an information process method, and an information process program capable of supporting creation of a diagnostic reading report.SOLUTION: An information processing apparatus 10 includes at least one processor, and the processor retrieves a plurality of opinion sentence candidates related to a retrieval query from an opinion sentence group including a plurality of opinion sentences, and presents at least one piece of elemental information based on importance level predetermined for each combination of two pieces of elemental information associated with each of the plurality of opinion sentence candidates.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, image diagnosis is performed using medical images obtained by imaging devices such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices. A person who creates an image interpretation report, such as a radiologist, interprets the medical images and creates an image interpretation report including a statement of findings.

[0003] As a method for supporting the creation of radiology reports, a method using statistical information of previously created radiology reports has been proposed. For example, Patent Document 1 discloses a system for suggesting autocomplete terms when inputting text for a report, which provides terms that frequently co-occur with a term being input based on co-occurrence statistics between terms. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2013-541786 Summary of the Invention [Problem to be solved by the invention]

[0005] As a technology for supporting the creation of radiology reports, a technology that presents other information related to information that has already been input is desired. For example, if a wide variety of information can be presented as a possible diagnosis from a certain finding and as other findings that are often written together with a certain finding, it becomes easier to search for radiology reports that have been created in the past and it is possible to prevent information that should be written in the radiology report from being missing. On the other hand, the more the variation of information presented increases, the more difficult it becomes for a user to quickly find desired information. Therefore, a technology that allows a user to quickly find desired information while presenting a wide variety of information is desired.

[0006] The present disclosure provides an information processing device, an information processing method, and an information processing program capable of supporting the creation of an image interpretation report. [Means for solving the problem]

[0007] A first aspect of the present disclosure is an information processing device including at least one processor, which searches for a plurality of candidate finding sentences related to a search query from a group of finding sentences including a plurality of finding sentences, and presents at least one element information based on a predetermined importance for each combination of two element information corresponding to each of the plurality of candidate finding sentences.

[0008] In the first aspect, the processor may present at least one finding sentence candidate associated with a combination having a relatively high degree of importance.

[0009] In the first aspect, the processor may display a character string indicating element information associated with the observation sentence candidate in a manner distinguished from other character strings.

[0010] In the first aspect, the processor may present a diagram according to the importance of each combination.

[0011] In the first aspect, the combination may be a combination of first element information and second element information indicating information of different attributes.

[0012] In the above first aspect, the first element information may indicate diagnosis information, and the second element information may indicate findings information.

[0013] In the first aspect, the processor may present a diagram showing the frequency of the first element information associated with a plurality of observation sentence candidates.

[0014] In the first aspect, the processor may present a diagram showing the frequency of the first element information included in the multiple observation sentence candidates.

[0015] In the first aspect, the processor may present a diagram showing frequencies of first element information associated with a plurality of finding sentence candidates, and then accept a selection of at least one piece of first element information, and present a diagram according to the importance of each combination including the selected first element information.

[0016] In the first aspect, the processor may present a diagram showing the frequency of first element information included in a plurality of finding sentence candidates, and then accept a selection of at least one piece of first element information, and present a diagram according to the importance of each combination including the selected first element information.

[0017] In the above first aspect, the processor may accept a selection of at least one of the presented element information, narrow down a plurality of finding sentence candidates using the selected element information as an additional search query, and present at least one element information based on the importance of each combination of two element information included in each of the plurality of finding sentence candidates after narrowing down.

[0018] In the first aspect, the importance level may be a value according to the frequency with which the combination is associated with the group of observation sentences.

[0019] In the first aspect above, the importance may be a value according to the degree of uniqueness of one piece of element information included in the combination not included in combinations with other element information other than the other piece of element information.

[0020] In the first aspect, the importance may be a value according to the frequency with which the combination is associated with a group of findings and the degree of uniqueness of one piece of element information included in the combination not included in combinations with other piece of element information other than the other piece of element information.

[0021] In the first aspect, the processor may accept an input of a search query by a user.

[0022] In the first aspect above, the processor may obtain an image, and generate element information as a search query based on the image.

[0023] In the above first aspect, the element information may include at least one of the characteristics, position, measurement values ​​and numbers of the region of interest, words or phrases expressing changes in the region of interest, and types of element information other than the element information that constitutes the group.

[0024] A second aspect of the present disclosure is an information processing method in which a computer executes a process of searching for a plurality of candidate finding sentences related to a search query from a group of finding sentences including a plurality of finding sentences, and presenting at least one element information based on a predetermined importance for each combination of two element information corresponding to each of the plurality of candidate finding sentences.

[0025] A third aspect of the present disclosure is an information processing program for causing a computer to execute a process of searching for a plurality of finding sentence candidates related to a search query from a group of finding sentences including a plurality of finding sentences, and presenting at least one element information based on a predetermined importance for each combination of two element information corresponding to each of the plurality of finding sentence candidates.

[0026] According to the above aspects, the information processing device, the information processing method, and the information processing program of the present disclosure can assist in creating an interpretation report. [Brief description of the drawings]

[0027] [Figure 1] FIG. 1 is a diagram illustrating an example of a schematic configuration of an information processing system. [Diagram 2] FIG. 1 is a diagram illustrating an example of a medical image. [Diagram 3] FIG. 1 is a diagram illustrating an example of a medical image. [Figure 4] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device. [Diagram 5] FIG. 2 is a block diagram showing an example of a functional configuration of an information processing device. [Figure 6] FIG. 13 is a diagram illustrating an example of information registered in a report DB. [Figure 7] FIG. 13 illustrates an example of an importance table. [Figure 8] FIG. 13 is a diagram showing an example of a screen displayed on a display. [Figure 9] FIG. 13 is a diagram showing an example of a screen displayed on a display. [Figure 10] FIG. 13 is a diagram showing an example of a screen displayed on a display. [Figure 11] 11 is a flowchart illustrating an example of information processing. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0029] First, the configuration of an information processing system 1 to which an information processing device 10 of the present disclosure is applied will be described with reference to Fig. 1. Fig. 1 is a diagram showing a schematic configuration of the information processing system 1. The information processing system 1 photographs an examination target part of a subject and stores the photographed medical images based on an examination order from a doctor of a medical department using a known ordering system. It also enables an image interpretation doctor to interpret the medical images and create an image interpretation report, and enables a doctor of the requesting medical department to view the image interpretation report.

[0030] 1, the information processing system 1 includes an imaging device 2, an interpretation WS (WorkStation) 3 which is an interpretation terminal, a medical treatment WS 4, an image server 5, an image DB (DataBase) 6, a report server 7, and a report DB 8. The imaging device 2, the interpretation WS 3, the medical treatment WS 4, the image server 5, the image DB 6, the report server 7, and the report DB 8 are connected to each other via a wired or wireless network 9 in a state in which they can communicate with each other.

[0031] Each device is a computer in which an application program is installed to cause the device to function as a component of the information processing system 1. The application program may be recorded on a recording medium such as a DVD-ROM (Digital Versatile Disc Read Only Memory) or a CD-ROM (Compact Disc Read Only Memory) and distributed, and installed on the computer from the recording medium. Alternatively, the application program may be stored in a storage device or network storage of a server computer connected to the network 9 in an externally accessible state, and downloaded to the computer upon request and installed.

[0032] The imaging device 2 is a device (modality) that captures an image of a part of a subject to be diagnosed, thereby generating a medical image T representing the part to be diagnosed. Examples of the imaging device 2 include a plain X-ray device, a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, a PET (Positron Emission Tomography) device, an ultrasonic diagnostic device, an endoscope, and a fundus camera. The medical image generated by the imaging device 2 is transmitted to an image server 5 and stored in an image DB 6.

[0033] Fig. 2 is a diagram showing a schematic diagram of an example of a medical image acquired by the imaging device 2. The medical image T shown in Fig. 2 is, for example, a CT image consisting of a plurality of tomographic images T1 to Tm (m is 2 or more), each of which represents a cross-sectional plane from the head to the waist of a single subject (human body). The medical image T is an example of an image disclosed herein.

[0034] FIG. 3 is a diagram showing a schematic example of one of the tomographic images Tx among the multiple tomographic images T1 to Tm. The tomographic image Tx shown in FIG. 3 shows a tomographic plane including the lungs. Each of the tomographic images T1 to Tm may include a structure area SA showing various organs and organs of the human body (e.g., lungs and kidneys, etc.), and various organs and tissues constituting the organs (e.g., blood vessels, nerves, muscles, etc.). Each tomographic image may also include a lesion area AA such as a nodule, tumor, injury, defect, and inflammation. In the tomographic image Tx shown in FIG. 3, the lung area is the structure area SA, and the nodule area is the lesion area AA. Note that one tomographic image may include multiple structure areas SA and / or lesion areas AA. Hereinafter, at least one of the structure area SA included in the medical image and the lesion area AA included in the medical image is referred to as a "region of interest."

[0035] The image interpretation WS3 is a computer used by a medical professional such as a radiologist to interpret medical images and prepare image interpretation reports, and includes the information processing device 10 according to this embodiment. The image interpretation WS3 issues a request to view a medical image to the image server 5, performs various image processing on the medical image received from the image server 5, displays the medical image, and accepts input of text related to the medical image. The image interpretation WS3 also performs analysis processing on the medical image, supports the preparation of an image interpretation report based on the analysis result, requests the report server 7 to register and view the image interpretation report, and displays the image interpretation report received from the report server 7. These processes are performed by the image interpretation WS3 executing software programs for each process.

[0036] The medical treatment WS4 is a computer used by medical personnel such as doctors in a medical department to observe medical images in detail, view interpretation reports, and create electronic medical records, and is configured to include a processing device, a display device such as a display, and an input device such as a keyboard and a mouse. The medical treatment WS4 issues requests to the image server 5 to view medical images, displays medical images received from the image server 5, issues requests to the report server 7 to view interpretation reports, and displays interpretation reports received from the report server 7. These processes are performed by the medical treatment WS4 executing software programs for each process.

[0037] The image server 5 is a general-purpose computer installed with a software program that provides the functions of a database management system (DBMS). The image server 5 is connected to an image DB 6. The connection between the image server 5 and the image DB 6 is not particularly limited, and may be a connection via a data bus or a connection via a network such as a NAS (Network Attached Storage) or a SAN (Storage Area Network).

[0038] The image DB 6 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), a flash memory, etc. In the image DB 6, medical images acquired by the imaging device 2 and associated information attached to the medical images are registered in association with each other.

[0039] The incidental information may include, for example, identification information such as an image ID (identification) for identifying a medical image, a tomographic ID assigned to each tomographic image included in the medical image, a subject ID for identifying the subject, and an examination ID for identifying an examination. The incidental information may also include, for example, information related to imaging of a medical image, such as an imaging method, imaging conditions, imaging purpose, imaging date and time, and imaging location. The "imaging method" and "imaging conditions" are, for example, the type of imaging device 2, the imaging site, imaging protocol, imaging sequence, imaging technique, whether or not a contrast agent is used, and slice thickness in tomography. The incidental information may also include information related to the subject, such as the subject's name, date of birth, age, and sex.

[0040] Furthermore, when the image server 5 receives a registration request for a medical image from the imaging device 2, it converts the medical image into a format for a database and registers it in the image DB 6. Furthermore, when the image server 5 receives a viewing request from the image interpretation WS 3 and the medical care WS 4, it searches for medical images registered in the image DB 6 and transmits the searched medical image to the image interpretation WS 3 and the medical care WS 4 that have sent the viewing request.

[0041] The report server 7 is a general-purpose computer installed with a software program that provides the functions of a database management system. The report server 7 is connected to a report DB 8. The connection between the report server 7 and the report DB 8 is not particularly limited, and may be a connection via a data bus or a connection via a network such as a NAS or a SAN.

[0042] The report DB 8 is realized by a storage medium such as a HDD, an SSD, a flash memory, etc. The report DB 8 registers the image interpretation reports created in the image interpretation WS 3 (details will be described later).

[0043] Furthermore, when the report server 7 receives a request to register an interpretation report from the image interpretation WS3, it converts the interpretation report into a format for a database and registers it in the report DB 8. Furthermore, when the report server 7 receives a request to view an interpretation report from the image interpretation WS3 and the medical care WS4, it searches for the interpretation report registered in the report DB8 and transmits the searched interpretation report to the image interpretation WS3 and the medical care WS4 that have made the request to view the interpretation report.

[0044] The network 9 is, for example, a network such as a LAN (Local Area Network) or a WAN (Wide Area Network). The imaging device 2, the interpretation WS3, the medical care WS4, the image server 5, the image DB6, the report server 7, and the report DB8 included in the information processing system 1 may be located in the same medical institution, or in different medical institutions. The number of each device of the imaging device 2, the interpretation WS3, the medical care WS4, the image server 5, the image DB6, the report server 7, and the report DB8 is not limited to the number shown in FIG. 1, and each device may be composed of a plurality of devices having similar functions.

[0045] Incidentally, when the interpretation WS3 interprets a medical image and creates an interpretation report, past interpretation reports (findings) registered in the report DB8 may be referred to for reference. In this case, a search query according to the medical image to be interpreted is used to search for past findings describing findings and / or diagnoses that are identical or similar to those of the medical image to be interpreted. For example, if the medical image to be interpreted contains a kidney nodule, a search is performed using "kidney" and "nodule" as the search query, and findings containing "kidney" and "nodule" are presented as the search results.

[0046] In addition, a user who has checked the search results may further narrow down the search results by inputting an additional search query. For example, the additional search query may be a possible diagnosis based on the already input search query (findings), or other findings that are often written together with the already input search query (findings). Therefore, if a wide variety of information can be presented as other information (additional search queries) related to the already input information (search query), it is possible to easily narrow down the search results. Furthermore, presenting a wide variety of information can also contribute to preventing information that should be written in the radiology report from being missing. On the other hand, the more the variation of information presented increases, the more difficult it becomes for a user to quickly find the desired information (additional search query).

[0047] Therefore, the information processing device 10 according to the present embodiment supports the creation of an interpretation report by providing a function that allows the user to quickly find desired information while presenting a wide variety of information. The information processing device 10 will be described below. As described above, the information processing device 10 is included in the interpretation WS3.

[0048] First, an example of a hardware configuration of the information processing device 10 according to the present embodiment will be described with reference to Fig. 4. As shown in Fig. 4, the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I / F (Interface) 26. The network I / F 26 is connected to a network 9 and performs wired and / or wireless communication. The CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I / F 26 are connected to each other via a bus 28 such as a system bus and a control bus so that various information can be exchanged between them.

[0049] The storage unit 22 is realized by a storage medium such as an HDD, an SSD, or a flash memory. An information processing program 27 for the information processing device 10 is stored in the storage unit 22. The CPU 21 reads the information processing program 27 from the storage unit 22, expands it in the memory 23, and executes the expanded information processing program 27. The CPU 21 is an example of a processor of the present disclosure. An importance table 29 is also stored in the storage unit 22 (described in detail later). As the information processing device 10, for example, a personal computer, a server computer, a smartphone, a tablet terminal, a wearable terminal, or the like can be appropriately applied.

[0050] Next, an example of a functional configuration of the information processing device 10 according to the present embodiment will be described with reference to Fig. 5 to Fig. 9. As shown in Fig. 5, the information processing device 10 includes a registration unit 30, an acquisition unit 32, a search unit 34, and a display control unit 36. The CPU 21 executes the information processing program 27, causing the CPU 21 to function as each of the functional units of the registration unit 30, the acquisition unit 32, the search unit 34, and the display control unit 36.

[0051] (Structuring of radiology reports) First, a structuring process of an image reading report will be described as a pre-processing executed by the information processing device 10 prior to a search process of a finding sentence. As described above, an image reading report that has already been created is registered in the report DB8. FIG. 6 shows an example of information included in an image reading report registered in the report DB8. As shown in FIG. 6, a plurality of finding sentences are registered in the report DB8 in association with diagnosis information and finding information related to each finding sentence. The diagnosis information and finding information are each an example of element information of the present disclosure.

[0052] The finding statements are registered in the report DB 8 by the report server 7 when each image interpretation report is created. The registration unit 30 specifies element information related to each of a plurality of finding statements (hereinafter, referred to as a group of findings) registered in the report DB 8, and assigns the element information to the finding statement (so-called structuring).

[0053] The diagnostic information is, for example, a presumed disease name diagnosed based on a medical image. The presumed disease name is an evaluation result presumed based on a lesion included in a medical image, and includes, for example, disease names such as "cancer" and "inflammation," as well as evaluation results regarding the disease name and characteristics such as "negative / positive," "benign / malignant," and "mild / severe." In FIG. 6, "angiomyolipoma," "renal cancer," "tumor," and "cyst" are exemplified as diagnostic information.

[0054] Furthermore, for example, one finding may include multiple suspected disease names, such as "renal cell carcinoma is suspected, but angiomyolipoma with a small amount of fat may also be present." In this case, the registration unit 30 may select one suspected disease name as the diagnosis information, and assign other suspected disease names, etc. that were not selected to the finding as the finding information.

[0055] The finding information is information indicating at least one of various findings such as the type (name), characteristics, position, measurement value and number of the region of interest, a phrase expressing a change in the region of interest, and a type of diagnostic information other than the diagnostic information to be paired, etc. As described above, the type of diagnostic information other than the diagnostic information to be paired is a presumed disease name that was not selected as the diagnostic information.

[0056] Examples of types (names) of a region of interest include names of structures such as "lung" and "liver" and names of lesions such as "nodule" and "ground-glass opacity". The properties of a region of interest mainly refer to the characteristics of the lesion. For example, in the case of a nodule, examples include absorption values ​​such as "solid", "low absorption" and "high absorption", margin shapes such as "clear / unclear", "smooth / irregular", "spicule", "lobulated" and "serrated", and findings indicating the overall shape such as "near-circular" and "irregular". Other examples include the relationship with surrounding tissues such as "protrusion", and findings regarding the presence or absence of contrast and washout.

[0057] The position of a region of interest means an anatomical position, a position in a medical image, and a relative positional relationship with other regions of interest such as "inside", "edge", and "surroundings". The anatomical position may be expressed by organ names such as "lung" and "kidney", or may be expressed by subdividing the kidney into "left kidney" and "superior segment".

[0058] The measurement value of the region of interest is a value that can be quantitatively measured from a medical image, and is, for example, at least one of the size and signal value of the region of interest. The size is expressed, for example, by the major axis, minor axis, area, and volume of the region of interest. The signal value is expressed, for example, by the pixel value of the region of interest and the CT value in units of HU. The finding information indicating the measurement value may be divided into predetermined classes (i.e., quantized), such as "0 mm or more and less than 5 mm", "5 mm or more and less than 10 mm", and "10 mm or more and less than 15 mm". In this case, for example, if the finding statement contains a statement of "12 mm", the finding information of "10 mm or more and less than 15 mm" is specified. The finding information indicating the measurement value may simply be information indicating whether or not there is a statement in the finding statement. This is because there may be a large variation in the measurement value or the statement may be omitted.

[0059] The number of regions of interest may be expressed as a specific number such as one or two, or may be expressed in relative terms such as "single / multiple" and "few / many." The phrases expressing changes in the regions of interest are phrases expressing changes over time in the characteristics, location, measurement values, number, etc., of the regions of interest when they are observed over time, and examples of such phrases include "appearance / disappearance," "increase / reduction," "worsening / improvement," and "metastasis."

[0060] Specifically, the registration unit 30 extracts named entities (words) from the findings registered in the report DB 8, and identifies element information corresponding to the extracted named entities. As a method for extracting named entities from findings, for example, a known named entity extraction method using a natural language processing model such as BERT (Bidirectional Encoder Representations from Transformers) can be appropriately applied.

[0061] In addition, the group of findings may contain different words with the same meaning (synonyms), such as "angiomyolipoma" and "AML" (Angiomyolipoma). In order to deal with spelling variations due to such synonyms, when the group of findings contains synonyms, it is preferable that the registration unit 30 assigns the same element information to each of the finding sentences containing the synonyms (so-called normalization).

[0062] Specifically, a dictionary that defines the correspondence between named entities that may be included in a finding sentence and element information, in which synonymous named entities are associated with the same element information, may be stored in advance in the storage unit 22. For example, in the dictionary, the synonymous words "angiomyolipoma" and "AML" may each be associated with the same diagnostic information, "angiomyolipoma." The registration unit 30 may extract named entities from the finding sentence registered in the report DB8 and identify the element information included in the finding sentence by referring to the dictionary.

[0063] In addition, it is preferable that the registration unit 30 also specifies the factuality of the specified element information. "Factuality" is information indicating whether a finding is observed or not, and the degree of certainty, etc. This is because the radiology report may include not only findings that are clearly observed in medical images, but also findings that are not observed in medical images, and findings that are suspected but have a low degree of certainty. For example, since the presence or absence and the degree of fat components are used to differentiate between AML and renal cell carcinoma (RCC), the radiology report may include the statement that "fat components are not observed."

[0064] Furthermore, element information may be such that it modifies other element information, and in this case, it is preferable that the registration unit 30 also specifies the modifying relationship between element information. For example, "calcification", which is an example of the characteristics of a pulmonary nodule, may be described in detail, such as "microcalcification is observed in the center". In this case, the registration unit 30 may specify the finding information "center" and "micro" as other finding information that modifies the finding information "calcification". Examples of finding information that modifies "calcification" include "micro", "coarse", "scattered", "center", "ring-shaped", and "complete".

[0065] For example, in the case of "renal cell carcinoma," which is an example of diagnostic information based on kidney nodules, the histological types such as "clear cell," "papillary," "chromophobe," and "multilocular cystic" may also be described in the findings. The registration unit 30 may specify element information indicating such a histological type as other element information modifying the diagnostic information "renal cell carcinoma."

[0066] Furthermore, when one finding statement includes element information related to a plurality of regions of interest (lesions), the registration unit 30 preferably identifies element information for each region of interest.

[0067] In addition, when the diagnostic information cannot be identified from the finding statement, the registration unit 30 may assign the diagnostic information of "undiagnosed" to the finding statement. For example, when a lesion (region of interest) included in a medical image is small and the findings can be interpreted but are below the diagnostic standard (i.e., not important) that is predetermined in the guidelines, only the finding information may be recorded and not recorded as diagnostic information. For example, when the region of interest is obviously benign, only the finding information may be recorded and not recorded as diagnostic information. For example, the diagnostic information may not be recorded due to the creator of the finding statement forgetting to write it. For example, the registration unit 30 may not be able to identify the diagnostic information from the finding statement due to a complex sentence structure of the finding statement or other reasons.

[0068] The registration unit 30 may also specify element information based on a medical image related to the finding text, such as a medical image registered in the image DB 6 or the like. Specifically, the registration unit 30 may specify element information not included in the finding text based on the medical image. Note that the registration unit 30 does not need to specify all of the above-mentioned various types of element information, and may specify only a predetermined type of element information (for example, element information indicating the size of the region of interest) based on the medical image.

[0069] For example, if the size of the region of interest is small, the description is less important or cannot be measured from the medical image, whereas if it is large, it can be seen from the medical image, so description in the finding statement may be omitted regardless of the size. On the other hand, the size of the region of interest may be related to characteristics, etc., and even if description in the finding statement is omitted, it may implicitly affect the content of the finding statement. For example, the larger the lung cancer tumor, the more likely it is that internal necrosis will occur. Therefore, it is preferable that the registration unit 30 assigns element information that is not included in the finding statement but can be identified based on the medical image to the finding statement and registers it in the report DB8.

[0070] An example of a method for identifying element information based on a medical image will be described. The registration unit 30 acquires a medical image related to the findings, and extracts at least one region of interest (e.g., a nodule region) included in the medical image. As a method for extracting a region of interest, a method using a known CAD (Computer Aided Detection / Diagnosis) technology and an AI (Artificial Intelligence) technology can be appropriately applied. For example, the registration unit 30 may extract a region of interest from a medical image using a learning model such as a CNN (Convolutional Neural Network) that is trained to input a medical image, extract and output a region of interest included in the medical image.

[0071] The registration unit 30 then generates element information of the extracted region of interest. For example, the registration unit 30 may generate element information of the region of interest using a learning model such as CNN that is trained in advance to receive the region of interest extracted from the medical image and output element information of the region of interest. The registration unit 30 assigns at least one of the element information thus generated to the finding text and registers it in the report DB8.

[0072] The registration unit 30 may also generate element information based on additional information attached to a medical image registered in, for example, the image DB 6. As described above, each medical image is attached with additional information at the time of registration in the image DB 6. In addition, the registration unit 30 may acquire information included in the test order and the electronic medical record, information indicating various test results such as blood tests and infectious disease tests, and information indicating the results of a health check from an external device such as the medical WS 4, and generate the element information. For example, a diagnosis estimated by a radiologist based on a medical image (i.e., a diagnosis included in the findings) may differ from a diagnosis confirmed by a doctor in a medical department taking into account the results of various other tests (i.e., a confirmed diagnosis recorded in an electronic medical record, etc.). In this case, at least one of element information indicating a diagnosis estimated from a medical image and element information indicating a confirmed diagnosis may be added to the findings, or both may be added.

[0073] (Derivation of Importance) Next, a process of deriving importance using structured data registered in the report DB 8 will be described as a pre-processing executed by the information processing device 10 prior to the search process of the finding sentences. By the structuring process of the radiology report described above, a plurality of finding sentences and element information related to each finding sentence are registered in the report DB 8 in association with each other.

[0074] Past radiology reports (findings) registered in the report DB8 may include reports with the same diagnosis but different findings, or the same findings but different diagnoses. For example, when a nodule is seen in a medical image of the kidney, if there is a fatty component, it is typically diagnosed as AML, and if there is no fatty component, it is diagnosed as RCC, but there are also cases where a nodule with a small amount of fatty component is diagnosed as AML. In order to improve the quality of diagnosis in radiology reports, it is desirable to present a wide variety of element information, not just typical examples.

[0075] On the other hand, simply listing a wide variety of element information makes it difficult for the user to quickly find the element information desired. Therefore, the registration unit 30 derives the importance for each combination of two pieces of element information as an index for presenting element information that is likely to be desired by the user.

[0076] 7 shows an example of the importance table 29 in which the importance is determined for each combination of diagnostic information and finding information. The importance table 29 also shows the frequency of diagnostic information associated with the finding sentence group, and the frequency of combinations of diagnostic information and finding information associated with the finding sentence group. The combinations are also classified according to the factuality of the finding information.

[0077] From the example of FIG. 7, it can be seen that the group of findings contains only 1000 findings associated with "angiomyolipoma" and only 517 findings associated with "renal cell carcinoma". It can also be seen that the group of findings contains only 719 findings associated with the combination of "angiomyolipoma" and "fat concentration" being "present". It can also be seen that the group of findings contains only 25 findings associated with the combination of "angiomyolipoma" and "fat concentration" being "absent". It can also be seen that the group of findings contains only 102 findings associated with the combination of "renal cell carcinoma" and "fat concentration" being "absent".

[0078] Specifically, the registration unit 30 may derive the importance based on a value according to the frequency with which a combination of diagnosis information and finding information is associated with a group of findings. The more frequently a combination of diagnosis information and finding information is associated with a group of findings, the more typical the combination is, and the more likely it is that the user performing the search will desire it.

[0079] For example, the registration unit 30 may derive the importance for each combination of diagnostic information d and finding information t based on the following formula (1). f(d) is the frequency of diagnostic information d associated with a group of findings, and f(t, d) is the frequency of the combination of diagnostic information d and finding information t associated with a group of findings. According to formula (1), the more frequently the finding information t is described in a finding text in combination with certain diagnostic information d, the higher the importance of the finding information t.

number

[0080] Furthermore, the registration unit 30 may derive the importance based on a value according to the degree of uniqueness of the finding information included in a combination of certain diagnostic information and finding information, which is not included in combinations with other diagnostic information. The degree of uniqueness indicates whether a certain finding information is rare and is used in combination with only some diagnostic information, or is universal and is used in combination with many diagnostic information. The more unique the finding information is, the more valuable it is as a basis for diagnosis, and the more likely it is that the user performing the search will desire it.

[0081] For example, the registration unit 30 may derive the importance level for each combination of diagnostic information d and finding information t based on the following formula (2): N is the number of types of all diagnostic information D associated with the finding sentence group, and n t is the number of types of diagnostic information that are associated with the finding sentence group in combination with the finding information t among all the diagnostic information D. According to formula (2), the fewer the number of types of diagnostic information described in combination with a certain finding information t (i.e., the higher the degree of specificity), the higher the importance is derived.

number

[0082] The registration unit 30 may also derive the importance based on a value corresponding to the frequency with which a combination of diagnostic information and finding information is associated with a group of findings, and the degree of uniqueness of the finding information included in the combination that is not included in combinations with other diagnostic information. For example, the registration unit 30 may derive the importance based on the following formula (3) that combines two indexes expressed by formulas (1) and (2). According to formula (3), when both the frequency and the degree of uniqueness are high, a high importance is derived.

number

[0083] In importance table 29 in Figure 7, the corresponding value of equation (1) for the combination of "angiomyolipoma" and "fat concentration" being "present" is derived as 0.719, since f(t, d) is 719 and f(d) is 1000.

[0084] In addition, in the importance table 29 of FIG. 7, the number of types N of all the diagnostic information D is 100, and the diagnostic information that can be combined with "fat concentration" as "present" is only one type, "angiomyolipoma" (i.e., n t In this case, the corresponding value of formula (2) for the combination of "angiomyolipoma" and "fat concentration" being "present" is derived as log(100 / (1+1))+1=2.70.

[0085] In the above case, the corresponding value of formula (3) is derived as 0.719×2.70=1.94. As an example, in FIG 7, the corresponding value of formula (3) is multiplied by 10 and shown as the importance.

[0086] The registration unit 30 may regard a combination of diagnostic information and finding information whose derived importance is equal to or lower than a predetermined threshold as noise and exclude it from the importance table 29. The registration unit 30 may also regard a combination of diagnostic information and finding information whose derived importance is relatively low (for example, a lower predetermined number or percentage) as noise and exclude it from the importance table 29.

[0087] In the importance table 29 of FIG. 7, an example has been described in which the importance is predefined for each combination of diagnosis information and finding information indicating information of different attributes, but the present invention is not limited to this. For example, the importance may be predefined for each combination of two element information indicating information of the same type of attribute (i.e., a combination of two pieces of finding information and a combination of two pieces of diagnosis information). In this case, the degree of uniqueness can be said to be a value according to the degree to which one piece of element information included in the combination is not included in combinations with other element information than the other piece of element information. Diagnosis information is an example of the first element information of the present disclosure, and finding information is an example of the second element information of the present disclosure.

[0088] (Search for findings) Next, a process of searching for a finding sentence executed by the information processing device 10 will be described.

[0089] The acquisition unit 32 acquires a search query. For example, the acquisition unit 32 may accept a search query input by the user via the input unit 25. FIG. 8 shows an example of a screen D0 for inputting a search query, which is displayed on the display 24 by the display control unit 36. The screen D0 displays a medical image T0 to be interpreted, a text box 90 for each item for inputting a search query, and a slider bar 92 for specifying the major axis and minor axis of the region of interest. The user checks the medical image T0 and inputs at least one search query. In the example of FIG. 8, "kidney", "left kidney", "nodule", and "18×25 mm" are input as search queries.

[0090] For example, the acquisition unit 32 may acquire a medical image T0 to be interpreted from the image server 5, and generate element information about the medical image T0 as a search query based on the medical image T0. The search query input by the user and the search query generated based on the medical image may be used in combination.

[0091] An example of a method for generating element information (search query) based on a medical image will be described. For example, the acquisition unit 32 extracts at least one region of interest (e.g., a nodule region) included in the acquired medical image T0. As a method for extracting a region of interest, a method using known CAD technology and AI technology, etc., can be appropriately applied. For example, the acquisition unit 32 may extract a region of interest from a medical image using a learning model such as CNN that is trained to input a medical image and extract and output a region of interest included in the medical image.

[0092] The acquisition unit 32 then generates element information of the extracted region of interest. For example, the acquisition unit 32 may generate element information of the region of interest using a learning model such as CNN that is pre-trained to receive the region of interest extracted from the medical image and output element information of the region of interest. The acquisition unit 32 may use at least one of the element information thus generated as a search query. Note that the acquisition unit 32 does not need to identify all of the various types of element information (e.g., organ, area name, findings, changes over time, diagnosis, etc.), and may identify only a predetermined type of element information (e.g., element information indicating the size of the region of interest) based on the medical image and use it as a search query.

[0093] The search unit 34 searches for a plurality of candidate finding sentences related to the search query acquired by the acquisition unit 32 from a group of finding sentences including a plurality of finding sentences registered in the report DB8. The search unit 34 may search by giving priority to a predetermined type of search query. For example, the search unit 34 may search by giving priority to a search query indicating an organ, a type of lesion, characteristics, and a diagnosis. On the other hand, the search unit 34 may not use a search query indicating the position and measurement value of a lesion. This is because, since the position and measurement value vary greatly from subject to subject, using them in a search reduces the number of search results, whereas, when a candidate finding sentence is reused, the effort required for correcting them is small, and therefore, it may be more efficient not to use them in a search.

[0094] In addition, when searching for a finding sentence candidate, the search unit 34 may allow ambiguity in the search query used for the search. For example, for a search query indicating a position, a finding sentence candidate to which finding information indicating a position within a predetermined range (for example, the position of an anatomically adjacent area) is added may be output as a search result. For example, for a search query indicating a measurement value, a finding sentence candidate to which finding information indicating a measurement value including a difference in a predetermined amount or percentage is added may be output as a search result. For example, for a search query indicating a diagnosis, a finding sentence candidate to which diagnosis information indicating different diagnoses belonging to the same classification (for example, "primary lung cancer" and "lung cancer") is added may be output as a search result. Note that the determination of whether different diagnoses belong to the same classification may be performed using, for example, an ICD (International Statistical Classification of Diseases and Related Health Problems) code, a dictionary, an ontology, or the like.

[0095] The display control unit 36 ​​presents at least one piece of element information based on the predetermined importance for each combination of two pieces of element information associated with each of the multiple finding sentence candidates searched for by the search unit 34. Fig. 9 shows an example of a screen D1 displayed on the display 24 by the display control unit 36, showing a search result when "kidney", "left kidney", "nodule" and "18 x 25 mm" are used as a search query.

[0096] Specifically, the display control unit 36 ​​may present at least one finding sentence candidate associated with a combination of two element information pieces with a relatively high level of importance. For example, the display control unit 36 ​​may identify two element information pieces associated with the finding sentence candidate searched for by the search unit 34, and acquire the importance of the identified combination of the two element information pieces by referring to the importance table 29. Then, the display control unit 36 ​​may present the finding sentence candidates associated with combinations with a relatively high level of importance (for example, a predetermined number or percentage of combinations with a high level of importance).

[0097] For example, the finding sentence candidate 86A in FIG. 9 includes a combination of "angiomyolipoma (AML)" and "fat concentration" being "present". In this case, the display control unit 36 ​​obtains the importance of "19.4" corresponding to the combination of "angiomyolipoma" and "fat concentration" being "present" from the importance table 29. The display control unit 36 ​​performs such processing on all finding sentence candidates searched by the search unit 34, and determines the finding sentence candidates to be presented based on the importance. The finding sentence candidates presented in this manner are associated with element information with a relatively high importance, so that the possibility of including information desired by the user is increased, which can contribute to the user quickly finding the desired information.

[0098] The display control unit 36 ​​may divide the multiple finding sentence candidates searched by the search unit 34 into multiple groups based on the type of diagnostic information associated with the finding sentence candidates, and determine at least one finding sentence candidate to be presented for each group. In Fig. 9, one finding sentence candidate 86A to 86C is presented for each of the three groups of "angiomyolipoma," "renal cell carcinoma," and "hemorrhagic renal cyst." This embodiment can increase the variety of the presented finding sentence candidates.

[0099] In addition, when multiple combinations of two pieces of element information are associated with a finding sentence candidate, the display control unit 36 ​​may determine the finding sentence candidate to present using a representative value such as the sum, average, weighted average, and multiplication value of the importance for each combination.

[0100] The display control unit 36 ​​may display the character string indicating the element information associated with the finding sentence candidate 86 in a manner that distinguishes it from other character strings. In FIG. 9, in each of the finding sentence candidates 86A to 86C, the character string indicating the element information is underlined 88, so that it is displayed in a distinguishable manner. For example, in the finding sentence candidate 86A, the character strings "fat concentration" and "AML" are underlined 88. Examples of a method for displaying the character string in a distinguishable manner include changing the thickness, size, italics, font, character color, and background color of the character, adding an underline, a strikethrough, or a box line, or displaying the character string in a different column. According to such a form, it is visually easy to know which element information is associated with the finding sentence candidate. Therefore, the user can quickly find the desired information.

[0101] The display control unit 36 ​​may also present a diagram according to the importance of each combination of two pieces of element information associated with each of a plurality of finding sentence candidates. Such diagrams visually display information and include, for example, various graphs (charts) and tables. FIG. 9 shows, as an example of such a diagram, a bar graph 82 showing the frequency of each piece of finding information (see tf in the above formula (1)) and a curve graph 84 showing a value corresponding to the frequency and degree of uniqueness of each piece of finding information (see tfidf in the above formula (3)). In FIG. 9, (+) means that there is fact (the finding can be interpreted from the medical image), and (-) means that there is no fact (the finding cannot be interpreted from the medical image).

[0102] For example, the horizontal bar graph 82 shows representative values ​​such as the total value of the frequency (see tf in formula (1) above) for each piece of finding information. In other words, the horizontal bar graph 82 makes it possible to know the frequently occurring finding information associated with the candidate finding sentence, regardless of the diagnosis information. For example, in FIG. 7, the frequency of the finding information with "fat concentration" set to "none" is 0.025 (25 / 1000) when combined with angiomyolipoma, and is 0.197 (102 / 517) when combined with renal cell carcinoma. In this case, the value of the "Fat (-)" item in the horizontal bar graph 82 is 0.222 (the sum of 0.025 and 0.197).

[0103] On the other hand, curve graph 84 shows representative values ​​such as the average value of the value corresponding to the frequency and degree of specificity of each finding information (see TFIDF in formula (3) above). In other words, curve graph 84 reflects the degree of specificity that can serve as the basis for diagnosis. For example, in FIG. 7, the importance of the finding information with "fat concentration" set to "none" is 1.2 when combined with angiomyolipoma, and 5.5 when combined with renal cell carcinoma. In this case, the value of the "Fat (-)" item in curve graph 84 is 3.35 (the average of 1.2 and 5.5).

[0104] The display control unit 36 ​​may determine the finding information (items) to be displayed in the graph in the order of frequency or importance in at least one of the horizontal bar graph 82 and the curve graph 84. The display control unit 36 ​​may also categorize and organize the finding information (items) to be displayed in the graph in at least one of the horizontal bar graph 82 and the curve graph 84, as shown in FIG.

[0105] The display control unit 36 ​​may also present a diagram showing the frequency of diagnostic information associated with a plurality of finding sentence candidates. Fig. 9 shows a hierarchical structure graph 80 (a so-called sunburst graph) as an example of such a diagram. The hierarchical structure graph 80 is a graph showing the frequency of diagnostic information assigned to each of a plurality of finding sentence candidates as search results based on a search query ("kidney", "left kidney", "nodule" and "18 x 25 mm"), as a percentage.

[0106] The hierarchical structure graph 80 also shows the hierarchical relationship of diagnostic information from higher classifications (inside the pie chart) to lower classifications (outside the pie chart). The hierarchical structure graph 80 makes it easy to visually understand, for example, the percentage of "angiomyolipoma" among the candidate findings as a search result, and further the percentage of "angiomyolipoma with little fat component (fat poor AML)" among them. Therefore, the user can quickly find the desired information.

[0107] The hierarchical relationship of the diagnosis classification may be determined using, for example, ICD (International Statistical Classification of Diseases and Related Health Problems) codes, dictionaries, ontologies, etc. For example, based on the overlap of terms in dictionaries and ontologies, the higher classification of ["renal" cell "cancer"] may be specified as ["renal" "cancer"]. Also, the higher classification of [clear cell "renal" "cell" "cell" cancer] may be specified as ["renal" "cell" "cell" cancer].

[0108] Also, as described above, the diagnosis estimated by the radiologist based on the medical image (i.e., the estimated diagnosis included in the finding text) may differ from the diagnosis confirmed by the doctor of the medical department considering the results of various other tests (i.e., the definitive diagnosis recorded in the electronic medical record, etc.). In this case, the diagnostic information used in the diagram presented by the display control unit 36 ​​may indicate at least one of the estimated diagnosis and the definitive diagnosis. Specifically, the display control unit 36 ​​may present a diagram in which the estimated diagnosis and the definitive diagnosis are mixed, may present two diagrams in which the estimated diagnosis and the definitive diagnosis are separated, or may present one diagram in which only either the estimated diagnosis or the definitive diagnosis is shown. For example, the frequency may be basically calculated using the definitive diagnosis, and the frequency may be calculated using the estimated diagnosis only for the finding sentence candidates to which the definitive diagnosis has not been assigned. For example, the frequency may be calculated using both the definitive diagnosis and the estimated diagnosis after different weights are assigned to them. For example, only the finding sentence candidates in which the estimated diagnosis and the definitive diagnosis match may be the subject of the frequency calculation. The information indicating a definite diagnosis and the information indicating a presumptive diagnosis are examples of first element information associated with the multiple finding sentence candidates of the present disclosure. The information indicating a presumptive diagnosis is an example of first element information included in the multiple finding sentence candidates of the present disclosure.

[0109] The display control unit 36 ​​may also receive an additional search query for further narrowing down the candidate findings as a search result, and update the contents of the screen D1 based on the received search query. Fig. 10 shows an example of a screen D2 that is updated when "angiomyolipoma" is specified as an additional search query on the screen D1, and is displayed on the display 24 by the display control unit 36. As shown in a hierarchical structure graph 80 in Fig. 10, the display control unit 36 ​​may change the color of the portion corresponding to the received search query ("angiomyolipoma") to make it easier to understand.

[0110] Specifically, the display control unit 36 ​​may accept the selection of at least one of the element information presented on the screen D1. In this case, in order to facilitate the user's narrowing down, it is preferable to present a diagram (e.g., a hierarchical structure graph 80) showing the frequency of diagnostic information associated with or included in a plurality of finding sentence candidates, and then accept the selection of at least one of the diagnostic information. For example, the user uses the pointer 94 on the screen D1 to select the diagnostic information of "angiomyolipoma" written in the hierarchical structure graph 80. Note that the selectable parts on the screen D1 are not limited to the items in the hierarchical structure graph 80, and the user may select the item name of the finding sentence candidate, or the item name of the horizontal bar graph 82 and the curve graph 84.

[0111] The search unit 34 narrows down the multiple finding sentence candidates using the selected element information as an additional search query. For example, when "angiomyolipoma" is selected as the additional search query, the search unit 34 outputs only the finding sentence candidates associated with "angiomyolipoma" as the search result.

[0112] The display control unit 36 ​​presents at least one element information based on the importance of each combination of two element information associated with each of the multiple finding sentence candidates after narrowing down by the search unit 34. For example, the display control unit 36 ​​presents at least one finding sentence candidate associated with the selected element information. In addition, for example, the display control unit 36 ​​presents a diagram (a hierarchical structure graph 80) according to the importance of each combination including the selected diagnostic information. In addition, for example, the display control unit 36 ​​presents a diagram (a bar graph 82 and a curve graph 84) according to the importance of each combination.

[0113] In this manner, element information related to the "angiomyolipoma" selected by the user is presented, allowing the user to consider the basis for the diagnosis and to increase the confidence in the diagnosis or to cast doubt on it.

[0114] The display control unit 36 ​​may further receive selection of additional element information on the screen D2 after narrowing down. In this case, based on the selected element information, similarly narrowing down of the finding sentence candidates and updating of the screen are performed.

[0115] Furthermore, the display control unit 36 ​​may accept cancellation of element information once accepted on the screen D2 after narrowing down. In this case, narrowing down of the observation sentence candidates is cancelled, and the screen returns to the state of the screen D1 again.

[0116] The above-mentioned hierarchical structure graph 80, bar graph 82, and curve graph 84 are examples of diagrams in the present disclosure, and similar information may be displayed using charts and graphs in other forms. Also, for example, the number, percentage, numerical value, etc. may be displayed in a table format without being limited to a graph.

[0117] Next, the operation of the information processing device 10 according to the embodiment will be described with reference to Fig. 11. In the information processing device 10, the CPU 21 executes the information processing program 27, thereby executing the information processing shown in Fig. 11. The information processing is executed, for example, when a command to start execution is given by a user via the input unit 25. It is assumed that the structuring process of the image interpretation report and the derivation process of importance described above have been executed in advance.

[0118] In step S10, the acquisition unit 32 acquires a search query. In step S12, the search unit 34 searches a group of findings including a plurality of findings registered in the report DB 8 for a plurality of finding sentence candidates related to the search query acquired in step S10.

[0119] In step S14, the display control unit 36 ​​acquires a predetermined importance (e.g., an importance registered in the importance table 29) for each combination of two pieces of element information associated with the finding sentence candidate. In step S16, the display control unit 36 ​​performs control to display a screen presenting the element information on the display 24 based on the importance acquired in step S14, and ends this information processing.

[0120] As described above, the information processing device 10 according to one embodiment of the present disclosure includes at least one processor, and the processor searches for a plurality of finding sentence candidates related to a search query from a group of findings including a plurality of finding sentences, and presents at least one element information based on a predetermined importance for each combination of two element information items associated with each of the plurality of finding sentence candidates.

[0121] That is, according to the information processing device 10 of the present embodiment, it is possible to present element information that is relatively important to the search query. Relatively high importance means that the element information is likely to be desired by the user. Therefore, even when a wide variety of information is presented, the user can quickly find the desired information, which can support the creation of the radiology report.

[0122] In the above embodiment, the description has been given assuming the interpretation of medical images, but the present disclosure is not limited thereto. The information processing device 10 of the present disclosure is applicable to various images obtained by photographing a subject and including a region of interest. For example, the information processing device 10 may be applied to images obtained from equipment, buildings, pipes, welds, etc. as subjects in non-destructive inspections such as radiographic inspections and ultrasonic flaw detection inspections. In this case, the region of interest may indicate, for example, cracks, flaws, bubbles, foreign objects, etc.

[0123] In the above embodiment, the following various processors can be used as the hardware structure of the processing units that execute various processes, such as the registration unit 30, the acquisition unit 32, the search unit 34, and the display control unit 36. As described above, the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0124] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0125] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System on Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0126] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0127] In the above embodiment, the various programs are pre-stored (installed) in the storage unit, but the present invention is not limited to this. The various programs may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The various programs may also be downloaded from an external device via a network. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores a program in addition to the program.

[0128] The technology of the present disclosure can also be appropriately combined with the above-mentioned embodiment examples and examples. The above-mentioned description and illustrated contents are detailed descriptions of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustrated contents shown above, within the scope of the gist of the technology of the present disclosure.

[0129] The following supplementary notes are further disclosed regarding the above embodiment. [Appendix 1] at least one processor; The processor, Searching for multiple candidate finding sentences related to the search query from a group of finding sentences including multiple finding sentences; At least one of the element information is presented based on a predetermined importance for each combination of two pieces of element information associated with each of the plurality of the finding sentence candidates. Information processing device. [Appendix 2] The processor, presenting at least one of the finding sentence candidates associated with the combination having a relatively high degree of importance; 2. An information processing device according to claim 1. [Appendix 3] The processor, A character string indicating the element information associated with the finding sentence candidate is displayed in a manner distinguished from other character strings. 3. An information processing device according to claim 2. [Appendix 4] The processor, Present a diagram according to the importance of each of the combinations. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 3. [Appendix 5] The combination is a combination of first element information and second element information indicating information of different attributes. 5. An information processing device according to any one of claims 1 to 4. [Appendix 6] The first element information indicates diagnostic information, The second element information indicates finding information. 6. The information processing device according to claim 5. [Appendix 7] The processor, A diagram showing the frequency of the first element information associated with a plurality of the finding sentence candidates is presented. 7. The information processing device according to claim 5 or 6. [Appendix 8] The processor, A diagram showing the frequency of the first element information contained in a plurality of the finding sentence candidates is presented. 8. The information processing device according to claim 7. [Appendix 9] The processor, presenting a diagram showing the frequency of the first element information associated with a plurality of the finding sentence candidates, and then accepting selection of at least one of the first element information; A diagram according to the importance of each of the combinations including the selected first element information is presented. The information processing device according to claim 5. [Appendix 10] The processor, presenting a diagram showing the frequency of the first element information included in the plurality of the finding sentence candidates, and then accepting selection of at least one of the first element information; A diagram according to the importance of each of the combinations including the selected first element information is presented. 10. The information processing device according to claim 9. [Appendix 11] The processor, Accepting a selection of at least one of the presented element information; Narrowing down the plurality of finding sentence candidates using the selected element information as an additional search query; At least one of the element information is presented based on the importance of each combination of two pieces of the element information included in each of the plurality of the narrowed-down finding sentence candidates. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 10. [Appendix 12] The importance level is a value according to the frequency with which the combination is associated with the group of observation sentences. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 11. [Appendix 13] The importance is a value according to the degree of uniqueness of one of the element information included in the combination, that is, the element information is not included in combinations with other element information other than the other element information. 13. An information processing device according to any one of claims 1 to 12. [Appendix 14] The importance is a value according to the frequency with which the combination is associated with the group of findings and the degree of uniqueness of one of the element information included in the combination not being included in combinations with other element information other than the other element information. 14. An information processing device according to any one of claims 1 to 13. [Appendix 15] The processor, Accepting input of the search query by a user 15. An information processing device according to any one of claims 1 to 14. [Appendix 16] The processor, Acquire an image, The element information is generated as the search query based on the image. 16. An information processing device according to any one of claims 1 to 15. [Appendix 17] The element information includes at least one of the following: a property, a position, a measurement value, and a number of the region of interest; a phrase expressing a change in the region of interest; and a type of element information other than the element information that is paired with the element information. 16. An information processing device according to any one of claims 1 to 15. [Appendix 18] Searching for multiple candidate finding sentences related to the search query from a group of finding sentences including multiple finding sentences; At least one of the element information is presented based on a predetermined importance for each combination of two pieces of element information associated with each of the plurality of the finding sentence candidates. An information processing method in which processing is performed by a computer. [Appendix 19] Searching for multiple candidate finding sentences related to the search query from a group of finding sentences including multiple finding sentences; At least one of the element information is presented based on a predetermined importance for each combination of two pieces of element information associated with each of the plurality of the finding sentence candidates. An information processing program that enables a computer to execute processing. [Explanation of symbols]

[0130] 1. Information Processing Systems 2. Imaging Device 3. Image Reading Workshop 4. Clinical Workshop 5 Image Server 6. Image Database 7 Report Server 8 Report DB 9 Network 10. Information processing device 21 CPU 22 Memory section 23 Memory 24 Display 25 Input section 26 Network Interface 27 Information Processing Programs 28 Bus 29 Importance Table 30 Registration Department 32 Acquisition Department 34 Search section 36 Display control section 80 Hierarchical Graph 82 Bar Chart 84 Curve Graph 86A~86C Candidates for comments 88 Underline 90 Text Box 92 Slider bar 94 Button AA Lesion Area D0~D2 screen SA Structure Area T, T0 Medical Imaging T1~Tm, Tx Tomographic images

Claims

1. At least one processor; The processor, Searching for multiple candidate finding sentences related to the search query from a group of finding sentences including multiple finding sentences; At least one of the element information is presented based on a predetermined importance for each combination of two pieces of element information associated with each of the plurality of the finding sentence candidates. Information processing device.

2. The processor, presenting at least one of the finding sentence candidates associated with the combination having a relatively high degree of importance; The information processing device according to claim 1 .

3. The processor, A character string indicating the element information associated with the finding sentence candidate is displayed in a manner distinguished from other character strings. The information processing device according to claim 2 .

4. The processor, Present a diagram according to the importance of each of the combinations. The information processing device according to claim 1 .

5. The combination is a combination of first element information and second element information indicating information of different attributes. The information processing device according to claim 1 .

6. The first element information indicates diagnostic information, The second element information indicates finding information. The information processing device according to claim 5 .

7. The processor, A diagram showing the frequency of the first element information associated with a plurality of the observation sentence candidates is presented. The information processing device according to claim 5 .

8. The processor, A diagram showing the frequency of the first element information included in a plurality of the finding sentence candidates is presented. The information processing device according to claim 7.

9. The processor, presenting a diagram showing the frequency of the first element information associated with a plurality of the observation sentence candidates, and then accepting selection of at least one of the first element information; A diagram according to the importance of each of the combinations including the selected first element information is presented. The information processing device according to claim 5 .

10. The processor, presenting a diagram showing the frequency of the first element information included in the plurality of observation sentence candidates, and then accepting selection of at least one of the first element information; A diagram according to the importance of each of the combinations including the selected first element information is presented. The information processing device according to claim 9.

11. The processor, Accepting a selection of at least one of the presented element information; Narrowing down the plurality of finding sentence candidates using the selected element information as an additional search query; At least one of the element information is presented based on the importance of each combination of two pieces of the element information associated with each of the plurality of the narrowed-down finding sentence candidates. The information processing device according to claim 1 .

12. The importance level is a value according to the frequency with which the combination is associated with the group of observation sentences. The information processing device according to claim 1 .

13. The importance is a value according to the degree of uniqueness of one of the element information included in the combination, that is, the element information is not included in combinations with other element information other than the other element information. The information processing device according to claim 1 .

14. The importance is a value according to the frequency with which the combination is associated with the group of findings and the degree of uniqueness of one of the element information included in the combination not being included in combinations with other element information other than the other element information. The information processing device according to claim 1 .

15. The processor, Accepting input of the search query by a user The information processing device according to claim 1 .

16. The processor, Acquire an image, The element information is generated as the search query based on the image. The information processing device according to claim 1 .

17. The element information includes at least one of the following: a property, a position, a measurement value, and a number of the region of interest; a phrase expressing a change in the region of interest; and a type of element information other than the element information that is paired with the element information. The information processing device according to claim 1 .

18. Searching for multiple candidate finding sentences related to the search query from a group of finding sentences including multiple finding sentences; At least one of the element information is presented based on a predetermined importance for each combination of two pieces of element information associated with each of the plurality of the finding sentence candidates. An information processing method in which processing is performed by a computer.

19. Searching for multiple candidate finding sentences related to the search query from a group of finding sentences including multiple finding sentences; At least one of the element information is presented based on a predetermined importance for each combination of two pieces of element information associated with each of the plurality of the finding sentence candidates. An information processing program that enables a computer to execute processing.

Images