Information processing device, information processing method, and information processing program

Abstract

An information processing device that assists in recalling disease names using the results of analysis of examination images. [Solution] An information processing device is provided, comprising: an analysis unit that obtains the results of analyzing an examination image; a first output unit that outputs a search term using a string of words that combines at least one of the location of the abnormal finding and the content or size of the abnormal finding obtained from the analysis results of the analysis unit; a search unit that searches a database for information on disease names or disease names using the search term output by the first output unit; and a presentation unit that presents disease names or disease names based on the search results from the search unit.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Doctors are required to recall the disease names of patients who visit a medical institution for examination. Medical knowledge in the medical field is constantly increasing, and it is impossible for doctors to memorize all of it. Therefore, technologies for assisting in recalling disease names from the content of a patient's findings have been disclosed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In addition to interviews by doctors, examinations for patients include X-ray examinations, CT (Computed Tomography) examinations, MRI (Magnetic Resonance Imaging) examinations, electrocardiograms, ultrasonic echograms, endoscopes, pathological images, infrared thermographs, angiograms, etc., which capture the internal conditions of patients. It is considered that using not only the symptoms of the patient known from the interview but also the examination images of the patient's internal body can help in recalling the disease name. However, it may be difficult for a doctor to recall the disease name just by looking at the examination images.

[0005] The present disclosure has been made in view of the above points, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program that assist in recalling a disease name using the analysis result of an examination image.

Means for Solving the Problems

[0006] According to one aspect of this disclosure, an information processing device is provided, comprising: an analysis unit that obtains analysis results of an examination image; a first output unit that outputs a search term using a string of words that combines at least one of the location of an abnormal finding and the content or size of the abnormal finding obtained from the analysis results of the analysis unit; a search unit that searches a database for information regarding a disease name or disease name using the search term output by the first output unit; and a presentation unit that presents a disease name or disease name based on the search results from the search unit.

[0007] The above-described information processing device may further include a second output unit that outputs a search term by combining the search term output by the first output unit with one or more strings of symptom findings and disease name or disease name provided from one or more of the patient information provided by user input or stored in the medical record system.

[0008] The first output unit may output the search terms by classifying them into the location of the abnormal finding, the content of the abnormal finding, or the size of the abnormal finding.

[0009] The first output unit may further output the search term with the name of the inspection from which the inspection image was obtained.

[0010] The first output unit may output the search term with the size expressed as time-series data.

[0011] The analysis unit may obtain analysis results for multiple examination images obtained from the same subject at different times, and the first output unit may output information on the time progression of each examination image as the search term.

[0012] The analysis unit may obtain the analysis results of the inspection images by providing the inspection images to a large-scale multimodal model and obtaining the output from the large-scale multimodal model.

[0013] The display unit may also present a user interface that allows the user to select an area in the examination image that is considered to be an abnormal finding.

[0014] In another aspect of this disclosure, an information processing method is provided in which a processor obtains the results of analyzing an examination image, outputs a search term using a string of words that combines at least one of the location of the abnormal finding obtained from the analysis results and the content or size of the abnormal finding, searches a database for information on disease names or disease names using the output search term, and presents disease names or disease names based on the search results.

[0015] In another aspect of this disclosure, an information processing program is provided that causes a computer to obtain the results of analyzing an examination image, output a search term using a string of words that combines at least one of the location of the abnormal finding obtained from the analysis results and the content or size of the abnormal finding, use the output search term to search a database for information on the disease name or disease name, and present the disease name or disease name based on the search results. [Effects of the Invention]

[0016] According to this disclosure, it is possible to provide an information processing device, an information processing method, and an information processing program that support the recall of disease names using the results of analysis of examination images. [Brief explanation of the drawing]

[0017] [Figure 1] This is a diagram illustrating an information processing device according to an embodiment of the disclosed technology. [Figure 2] This is a block diagram showing the hardware configuration of an information processing device. [Figure 3] This is a block diagram showing an example of the functional configuration of an information processing device. [Figure 4] This diagram shows an example of a database structure. [Figure 5]It is a diagram showing an example of a user interface displayed on a user terminal by an information processing device. [Figure 6] It is a diagram showing an example of a user interface displayed on a user terminal by an information processing device. [Figure 7] It is a diagram showing an example of a time-series inspection image displayed on a user terminal by an information processing device. [Figure 8] It is a flowchart showing the flow of information processing by an information processing device.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the present disclosure will be described while referring to the drawings. In each of the drawings, the same or equivalent components and parts are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for convenience of explanation and may be different from the actual ratios.

[0019] FIG. 1 is a diagram for explaining an information processing device according to the present embodiment. The information processing device 10 shown in FIG. 1 receives an input of a patient's symptoms and a disease name from a user terminal 20 connected via a network 1, and outputs a search term using the symptoms and the disease name. The user terminal 20 is a terminal used by a doctor or other medical staff as a user. The information processing device 10 is a device that searches a database 30 storing information related to diseases with the output search term and presents a disease name to the user terminal 20 based on the search results. The database 30 stores information related to diseases, such as case information, suspected case information, and information of textbooks with descriptions related to diseases or pathological conditions. The information processing device 10 according to the present embodiment outputs a search term using the analysis result of the patient's inspection image in addition to the patient's symptoms and disease name when searching the database 30.

[0020] Furthermore, the term "pathological condition name" is an expanded concept of "disease name" (disease name), referring to conditions where the pathology of the disease is consistent. Therefore, in this disclosure, "pathological condition name" may be considered equivalent to "disease name." For example, "immune state" is a pathological condition name, but it is an example of a term that is not a disease name. Moreover, as one aspect of this, when registering case information or suspected case information in database 30, considering that multiple pathological conditions may exist in a single case, the information is registered separately for each pathological condition. This prevents pathological condition names unrelated to symptoms from being displayed in the search results. In addition, it is also possible to register by combining multiple pathological conditions. This makes it possible to explain sets of symptoms that can only be explained when two or more pathological conditions are combined. For example, wheezing is seen in COPD but rarely in lung cancer, and hemoptysis is often seen in lung cancer but rarely in COPD. However, when searching for the three words "hemoptysis + wheezing + CT = lung tumor," the information processing device 10 can combine the two conditions, COPD and lung cancer, and display the search results on the user terminal 20.

[0021] The information processing device 10 may be configured, for example, as a web server, and the acceptance of input from the user terminal 20 and the presentation of search results from the database 30 may be implemented in the form of a web page. Therefore, the user terminal 20 has a browser installed for viewing web pages, but it may also have a dedicated application installed for using the search service provided by the information processing device 10.

[0022] In this embodiment, when searching the database 30, the information processing device 10 outputs search terms using the results of the analysis of the patient's examination images in addition to the patient's symptom findings and disease name. This makes it possible to better assist physicians in recalling disease names compared to cases where the results of the examination image analysis are not used.

[0023] Network 1 can be the Internet, an intranet, or any other network, and the communication protocol, type of communication, and scale of communication may be anything. Furthermore, database 30 may be built within the information processing device 10 as shown in Figure 1, or it may be built on a device different from the information processing device 10.

[0024] Figure 2 is a block diagram showing the hardware configuration of the information processing device 10.

[0025] As shown in Figure 2, the information processing device 10 includes a CPU (Central Processing Unit) 11, ROM (Read Only Memory) 12, RAM (Random Access Memory) 13, storage 14, input unit 15, display unit 16, and communication interface (I / F) 17. Each component is connected to the others via a bus 19 so that they can communicate with each other.

[0026] The CPU 11 is a central processing unit that executes various programs and controls various parts. Specifically, the CPU 11 reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculations according to the program recorded in the ROM 12 or storage 14. In this embodiment, the ROM 12 or storage 14 stores an information processing program that outputs a search term based on user input and the analysis results of the examination image, searches the database 30 which stores information about the pathological condition using the output search term, and presents information about the disease to the user terminal 20 based on the search results.

[0027] ROM12 stores various programs and data. RAM13 temporarily stores programs or data as a working area. Storage14 consists of a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs, including the operating system, and various data.

[0028] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used for various types of input.

[0029] The display unit 16 is, for example, a liquid crystal display and displays various information. The display unit 16 may also function as an input unit 15 by employing a touch panel system.

[0030] The communication interface 17 is an interface for communicating with other devices such as the user terminal 20, and standards such as Ethernet®, FDDI, and Wi-Fi® are used.

[0031] When executing the above information processing program, the information processing device 10 uses the above hardware resources to implement various functions. The functional configuration implemented by the information processing device 10 will now be described.

[0032] Figure 3 is a block diagram showing an example of the functional configuration of the information processing device 10.

[0033] As shown in Figure 3, the information processing device 10 has the following functional configuration: acquisition unit 101, analysis unit 102, first output unit 103, second output unit 104, standardization unit 105, search unit 106, and presentation unit 107. Each functional configuration is realized by the CPU 11 reading and executing an information processing program stored in the ROM 12 or storage 14.

[0034] The acquisition unit 101 acquires user input from the user terminal 20. The user input acquired by the acquisition unit 101 includes the patient's symptom findings and disease name. The acquisition unit 101 also acquires examination images of the same patient as the patient with the symptom findings and disease name. The examination images are, for example, images obtained from examinations that photograph the inside of the patient's body, such as X-ray examinations, CT scans, and MRI scans.

[0035] The analysis unit 102 analyzes the patient's examination images acquired by the acquisition unit 101 to obtain the analysis results of the examination images. The analysis unit 102 may use any pre-trained model for the analysis of the examination images. This pre-trained model is trained to output information about any abnormal findings when an examination image is input. In other words, the analysis unit 102 provides the examination image to the pre-trained model and obtains the analysis results of the examination images by obtaining the output from the pre-trained model. The analysis unit 102 can obtain findings from the examination images by analyzing them.

[0036] In this embodiment, the analysis unit 102 may use a Large Multimodal Model (LMM) for analyzing the examination images. A Large Multimodal Model is a model that can respond in natural language text to inputs of multiple modalities, such as images, in addition to natural language text. This Large Multimodal Model is trained to output any abnormal findings in natural language text when an examination image is input. That is, the analysis unit 102 obtains the analysis results of the examination images by providing the examination images to the Large Multimodal Model and obtaining the output from the Large Multimodal Model.

[0037] The first output unit 103 outputs search terms related to abnormal findings based on the analysis results of the examination images obtained by the analysis unit 102. Specifically, the first output unit 103 outputs search terms using strings that include a combination of the location of the abnormal finding and at least one of the content or size of the abnormal finding obtained from the analysis results. The first output unit 103 may also output search terms classified by the name of the examination from which the examination images were obtained, the location of the abnormal finding, the content of the abnormal finding, or the size of the abnormal finding. In other words, the first output unit 103 outputs search terms that combine a phrase indicating the name of the examination from which the examination images were obtained with a phrase relating to the location of the abnormal finding, the content of the abnormal finding, or the size of the abnormal finding. The first output unit 103 may also output search terms according to the shape, density, or contrast of the abnormal finding with surrounding tissue. For example, suppose the analysis unit 102 found a cavity in the lung as a result of analyzing the examination images of a CT scan. In this case, the first output unit 103 outputs the search term "CT=cavity@lung" as a search term related to the abnormal finding. In this case, "CT" is an example of a phrase indicating the name of the examination from which the examination image was obtained, and "Cavity@Lung" is an example of a phrase relating to the location of the abnormal finding and the content of the abnormal finding. When the first output unit 103 outputs a search term by combining phrases relating to the size of the abnormal finding, it may output a search term that expresses the size as time-series data. Furthermore, the analysis unit 102 may use machine learning, deep learning, or image processing algorithms for analysis. Moreover, as one configuration, the analysis unit 102 and the first generation unit 103 may be integrated so that the examination image is passed directly to the LMM and the search term is output. In addition, the symptom findings and disease name used as search terms may be received directly from the user via voice or keyboard input.

[0038] The second output unit 104 outputs multiple search terms by combining the search term output by the first output unit 103 with the symptom findings and disease name acquired by the acquisition unit 101. For example, if the first output unit 103 outputs the search term "CT=cavity@lung" as described above, and the acquisition unit 101 acquires the symptom findings and disease name "fever" and "body temperature 38 degrees", the second output unit 104 outputs the search terms "fever body temperature 38 degrees CT=cavity@lung".

[0039] The second output unit 104 generates search terms by combining the string of abnormal findings obtained from the analysis results of the analysis unit 102 with the symptom findings and disease name (disease name) input or provided via the user terminal 20. This includes treating the string as a logical AND search or a phrase search. For example, the second output unit 104 tags the abnormal location "lung" and the abnormal content "cavity" with the user input "fever". This allows the database 30 to be searched using the phrase "cavity@lung fever" as the search term.

[0040] This embodiment is not limited to cases where symptoms, findings, and disease names are manually entered on the input screen displayed on the user terminal 20. It also includes forms in which patient information stored in medical record systems such as electronic medical records (EMR) and picture archiving and communication systems (PACS) is referenced and used as part of the search string. For example, the user terminal 20 automatically or semi-automatically acquires symptoms and findings such as "fever," "body temperature 38.5°C," and "cough" recorded in the electronic medical record into the input area, and the second output unit 104 can generate search terms by combining the information acquired from the electronic medical record by the user terminal 20 with abnormal findings (location, content, size) obtained by the analysis unit 102. In this case, the user may select necessary items from the medical information on the electronic medical record and have them imported into the acquisition unit 101 as search terms via the user terminal 20. By using this form, the input workload of medical staff can be reduced, and searches using more accurate data can be performed.

[0041] Another example of this embodiment is the implementation of multiple search filters. In the example above, the search terms are separated by spaces, etc., but for example, one per line, Location: Right lower lung field, Findings: Cavity present, mass Size: about the size of a small bean, Location: upper right lung field Findings: Cavity-filled mass, size: about the size of a chicken egg, location: right upper lung field It is also possible to apply multiple filters, as shown above.

[0042] The standardization unit 105 standardizes the terminology for symptom findings and disease names in order to unify the search terms output by the second output unit 104. In unifying the terminology, the standardization unit 105 may obtain words by means of natural language processing, rule-based modification, vector search on the word list, or word search by calculating the edit distance on the word list.

[0043] As another example of this embodiment, the first output unit 103 may generate search terms that express the change in the size of abnormal findings as time-series data from the analysis results of multiple examination images obtained from the same patient at different imaging timings by the analysis unit 102. For example, the first output unit 103 may assign the increase or decrease in tumor size obtained from CT images taken every three months to the search terms in the format of "tumor 10mm (January 2024) → 12mm (April 2024) → 15mm (July 2024)". By generating search terms that express the change in the size of abnormal findings as time-series data, the first output unit 103 can be used to search for the progression over time or the effectiveness of treatment. Furthermore, the standardization unit 105 may convert the representation of the time-series information generated by the first output unit 103 into a unified standard for search terms. For example, the standardization unit 105 may convert changes in the size of abnormal findings into notations such as unchanged, increased, decreased, disappeared, CR (complete response), PR (partial response), NC (no change), and PD (progressive disease). These notations represent changes in size over time and changes in treatment interventions.

[0044] The search unit 106 searches the database 30 for information related to the pathology using the search term output by the first output unit 103 or multiple search terms output by the second output unit 104. For example, the search unit 106 outputs a query to search the database 30 using the search term output by the first output unit 103 or multiple search terms output by the second output unit 104, and searches the database 30 using the output query. Information related to the pathology includes at least one of a case or a suspected case, or a textbook. That is, the information stored in the database 30 includes at least one of a case or a suspected case, or a textbook, as information related to the pathology.

[0045] Here is an example of the data structure of database 30. Figure 4 is a diagram showing an example of the data structure of database 30. In this embodiment, database 30 has a data type column, a disease state column, a disease state findings column, an ID column, a case ID column, and a case number column.

[0046] The data type column is used to identify whether a record is a collection of multiple cases or a record of a single case. A data type of 1 indicates a record of multiple cases, while a data type of 2 indicates a record of a single case.

[0047] The "Pathology" column stores the name of the pathology. The "Pathology Findings" column stores information about the findings of the pathology. The "ID" column stores the ID that identifies the pathology. The "Case ID" column stores the case ID that identifies the case data related to the pathology. If the data type is 1, the Case ID column stores multiple case IDs, corresponding to the ID column of the case information held in a separate table in the same configuration as in Figure 4. The "Number of Cases" column stores the number of cases of the pathology; if the data type is 2, the Number of Cases column stores 1.

[0048] The presentation unit 107 presents a user interface to the user terminal 20 for searching for disease names. The presentation unit 107 then presents information about the disease to the user terminal 20 based on the search results from the search unit 106. The presentation unit 107 presents the information about the disease to the user terminal 20, for example, in the form of a web page.

[0049] Here, an example of a user interface displayed by the presentation unit 107 on the user terminal 20 is shown. Figure 5 is a diagram showing an example of a user interface displayed on the user terminal 20 by the information processing device 10. The user interface 200 shown in Figure 5 is implemented, for example, in the form of a web page.

[0050] The user interface 200 shown in Figure 5 includes an input area 201 for the user of the user terminal 20 to input the name of the symptom and pathological condition they wish to search for, an image upload button 202 for sending the examination image to the information processing device 10, and a search button 203 for causing the information processing device 10 to perform a search for information related to the pathological condition from the database 30. When the user of the user terminal 20 enters the name of the symptom and pathological condition into the input area 201, uploads the examination image to the information processing device 10 using the image upload button 202, and selects the search button 203, the information processing device 10 performs a search for information related to the pathological condition from the database 30 and presents the search results to the user interface 200.

[0051] In another embodiment, the presentation unit 107 may present a graphical user interface that allows the user to select an abnormal area highlighted on the examination image. When the user selects a candidate such as "lung field" or "frontal lobe," the area information is overwritten or added to the analysis results of the analysis unit 102 and reflected in the final generated search term. This makes it possible to improve search accuracy, including preventing misrecognition and selection from multiple candidates.

[0052] Figure 6 shows an example of a user interface displayed on a user terminal 20 by an information processing device 10. The user interface 200 shown in Figure 6 is implemented, for example, in the form of a web page. The user interface 200 shown in Figure 6 is an example of the presentation of search results for information about a disease.

[0053] The user interface 200 shown in Figure 6 includes an input area 201, an image upload button 202, a search button 203, as well as a display area 211 for a summary of search results and a display area 212 for case report search results.

[0054] The display area 211 of the search results summary shows the diseases that can be thought of based on the symptom findings and disease name entered in the input area 201 and the examination images uploaded by the image upload button 202 and analyzed by the information processing device 10, separated by medical department. The presentation unit 107 may also present the diseases to the user interface 200 without separating them by medical department.

[0055] The case report search results display area 212 displays the number of cases of the disease that can be inferred from the symptom findings and disease name entered in the input area 201 and the examination images uploaded by the image upload button 202 and analyzed by the information processing device 10. When the user of the user terminal 20 selects a disease name presented in the case report search results display area 212, the presentation unit 107 presents detailed case information of that disease to the user terminal 20.

[0056] The information processing device 10 presents the user interface 200 shown in Figure 6 to the user terminal 20, thereby enabling the user of the user terminal 20 (a medical professional such as a doctor) to recall the name of the disease using the results of the analysis of the examination images.

[0057] When the presentation unit 107 presents detailed case information of a disease state to the user terminal 20, if corresponding examination images are registered in the database 30, it may also present those examination images to the user terminal 20. In this case, if the examination images are registered in the database 30 in chronological order, the presentation unit 107 may also present the examination images to the user terminal 20 along with the time when the examination images were taken. Figure 7 shows an example of chronological examination images displayed on the user terminal 20 by the information processing device 10. The chronological examination images 221 shown in Figure 7 consist of, for example, three examination images 221a, 221b, and 221c. By presenting the chronological examination images 221 together with detailed case information of the disease state, the presentation unit 107 can assist the user of the user terminal 20 in recalling the name of the disease by comparing it with the examination images uploaded to the information processing device 10. Furthermore, the display unit 107 can, in one example, overlay two or more images as a time-series image, assign different colors or symbols to the lesion area, or process the boundaries to show how the size of the lesion has changed over time, making it easier for machine learning to understand the concept. When overlaying images, the display unit 107 may determine the color of the most recent image and the image before the most recent to indicate whether the change in size has increased or decreased.

[0058] When using a large-scale multimodal model, the analysis unit 102 can obtain output expressed in natural language, such as "suspected tumor in the left lung" or "possible pneumonia with cavity formation," by inputting examination images. The first output unit 103 performs text analysis on the output from the large-scale multimodal model to extract phrases such as "lung tumor" and "cavity @ left lung," and combines these as search terms to streamline the search of the database 30.

[0059] Another example of this embodiment is that when the analysis unit 102 extracts the location, content, and size of abnormal findings using artificial intelligence (AI), particularly a large-scale multimodal model, it may also have a function to show the user the basis on which the search terms were generated. For example, the analysis unit 102 may visualize a weighted heatmap or area of ​​interest on the examination image, enclose it in an arbitrary shape such as a rectangle, and show which part was used as the basis for selecting words such as "pneumonia" or "cavity." This allows the user (doctor or medical staff) to intuitively understand why the AI ​​selected "cavity@left lung" as a search term.

[0060] In this case, the display unit 107 may also display the "site of AI judgment" and "confidence score" for each candidate search term. For example, if the abnormal finding is in the left lung field, the display unit 107 may display an explanation such as, "The image pixel values ​​in the left lung field fall within the range of XX, and it was diagnosed as a severe cavitary lesion, so 'cavitary@left lung' was generated." By displaying the information in this way, the display unit 107 makes it easier for the user to understand the AI's reasoning process, adds transparency to the generation of search terms by the AI ​​(Explainable AI), and increases the reliability of the search results.

[0061] In this embodiment, the first output unit 103 may generate text that combines at least two of the following: the location where the abnormality was observed (lungs, liver, bone, etc.), the content of the abnormality (cavity, tumor, inflammation, etc.), and the size of the abnormality (long diameter, short diameter, volume, etc.). For example, the first output unit 103 may generate text that simultaneously indicates the location and content or size, such as "tumor in the left lung, 10 mm" or "cavity in the right lung, 15 mm". By generating text in this manner, it is possible to improve the search accuracy of the database 30.

[0062] In this embodiment, the size of abnormal findings may not only be expressed using specific numerical values ​​such as "10mm" or "15mm," but may also include everyday or analog size expressions such as "the size of a chicken egg" or "the size of a red bean." When these everyday or analog size expressions are received, the standardization unit 105 may create a dictionary of correspondences such as "the size of a chicken egg = 40mm" and "the size of a red bean = 10mm," and convert them into a unified standard (numerical values, etc.) as search terms. For example, if the input is "tumor in the right lung, the size of a chicken egg," the standardization unit 105 may convert it into a phrase such as "tumor in the right lung, 40mm" before generating a search term. This makes it possible to search the database 30 more efficiently compared to not converting to a unified standard as a search term, even when various expressions are mixed together, thereby improving search accuracy and versatility.

[0063] The information processing device 10, having the configuration shown in Figure 3, can present disease names obtained from the database 30 to the user terminal 20. By presenting disease names obtained from the database 30 to the user terminal 20, the information processing device 10 can use the results of the analysis of the examination images to support the user of the user terminal 20 (a medical professional such as a doctor) in recalling the disease name.

[0064] Next, the operation of the information processing device 10 will be explained.

[0065] Figure 8 is a flowchart showing the flow of information processing by the information processing device 10. Information processing is performed when the CPU 11 reads an information processing program from the ROM 12 or storage 14, loads it into the RAM 13, and executes it.

[0066] In step S101, the CPU 11 acquires user input from the user terminal 20. The user input acquired by the CPU 11 includes the patient's symptom findings and disease name. The CPU 11 also acquires examination images of the same patient as the patient with the symptom findings and disease name. The examination images are, for example, images obtained from examinations that capture the inside of the patient's body, such as X-rays, CT scans, and MRI scans.

[0067] Following step S101, in step S102, the CPU 11 analyzes the examination image acquired in step S101. The CPU 11 may use any pre-trained model to analyze the examination image. This pre-trained model is trained to output information about any abnormal findings when an examination image is input. In other words, the CPU 11 provides the examination image to the pre-trained model and obtains the output from the pre-trained model to obtain the analysis result of the examination image. The CPU 11 can obtain findings from the examination image through its analysis.

[0068] In this embodiment, the CPU 11 may use a large-scale multimodal model (LMM) for analyzing the inspection images.

[0069] Following step S102, in step S103, the CPU 11 outputs a search term using the analysis of the examination image. For example, suppose the CPU 11 analyzes the examination image of a CT scan and finds a cavity in the lung. In this case, the CPU 11 outputs the search term "CT=cavity@lung" as a search term related to the abnormal finding.

[0070] Following step S103, in step S104, the CPU 11 outputs multiple search terms by combining the search term output in step S103 with the symptom findings and disease name obtained in step S101. For example, if the CPU 11 outputs the search term "CT=cavity@lung" as described above, and obtains the symptom findings and disease name "fever" and "body temperature 38 degrees", the CPU 11 outputs the search terms "fever body temperature 38 degrees CT=cavity@lung".

[0071] Following step S104, in step S105, the CPU 11 searches the database 30 using the search term output in step S104.

[0072] Following step S105, in step S106, the CPU 11 presents the search results to the user terminal 20 based on the search results from step S105. The CPU 11 presents the disease names obtained from the database 30 to the user terminal 20 as search results, for example in the form of a web page.

[0073] The information processing device 10 can present the disease name obtained from the database 30 to the user terminal 20 by executing the series of processes shown in Figure 8. By presenting the disease name obtained from the database 30 to the user terminal 20, the information processing device 10 can use the results of the analysis of the examination images to support the user of the user terminal 20 (a medical professional such as a doctor) in recalling the disease name.

[0074] Although embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary skill in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims, and these modifications or alterations are also understood to naturally fall within the technical scope of the present disclosure. For example, in one aspect of the present disclosure, the information processing device may generate text that will become a radiation report from inspection images of a radiation inspection using an LMM, and output search terms from the text generated by the LMM after natural language processing or generation processing by an LLM.

[0075] Furthermore, the effects described in the above embodiments are descriptive or illustrative, and are not limited to those described in the above embodiments. In other words, the technology relating to this disclosure may produce other effects that would be obvious to a person of ordinary skill in the art of this disclosure from the descriptions in the above embodiments, in addition to or in lieu of the effects described in the above embodiments.

[0076] Furthermore, the information processing that the CPU reads and executes in each of the above embodiments may be executed by various processors other than the CPU. Examples of such processors include PLDs (Programmable Logic Devices) such as FPGAs (Field-Programmable Gate Arrays) whose circuit configuration can be changed after manufacturing, and dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processing, such as ASICs (Application Specific Integrated Circuits). In addition, the information processing may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, and a combination of a CPU and an FPGA). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor elements.

[0077] Furthermore, while the above embodiments describe a configuration in which the information processing program is pre-stored (installed) in ROM or storage, the invention is not limited thereto. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), DVD-ROM (Digital Versatile Disk Read Only Memory), or USB (Universal Serial Bus) memory. The program may also be provided in a form that can be downloaded from an external device via a network. This disclosure may also be applied to program products. [Explanation of Symbols]

[0078] 1 Network 10 Information Processing Devices 20 User Terminals 30 databases 101 Acquisition Department 102 Analysis Department 103 First Output Section 104 Second Output Section 105 Standardization Department 106 Search Section 107 Presentation section

Claims

1. An analysis unit that obtains the results of the analysis of the examination images, A first output unit outputs a search term using a string of words that combines the location of the abnormal finding obtained from the analysis results of the aforementioned analysis unit and at least one of the content or size of the abnormal finding. A search unit that uses the search term output by the first output unit to search a database for information regarding disease names or disease names, A display unit that presents a disease name or disease name based on the search results from the aforementioned search unit, An information processing device equipped with the following features.

2. The information processing apparatus according to claim 1, further comprising a second output unit that outputs a search term by combining the search term output by the first output unit with one or more strings of symptom findings and disease name or disease name provided from one or more of the patient information provided from user input or stored in the medical record system.

3. The information processing apparatus according to claim 1, wherein the first output unit outputs the search term by classifying it into a part with an abnormal finding, the content of the abnormal finding, or the size of the abnormal finding.

4. The information processing apparatus according to claim 1, wherein the first output unit further outputs the search term with the inspection name from which the inspection image was obtained.

5. The information processing apparatus according to claim 1, wherein the first output unit outputs the search term with the size expressed as time-series data.

6. The analysis unit obtains the analysis results of multiple examination images obtained from the same subject at different time points, The information processing apparatus according to claim 1, wherein the first output unit outputs information on the time progression of each of the inspection images as the search term.

7. The information processing apparatus according to claim 1, wherein the analysis unit provides the inspection image to a large-scale multimodal model and obtains the output from the large-scale multimodal model to obtain the analysis result of the inspection image.

8. The information processing apparatus according to claim 1, wherein the display unit presents a user interface for the user to select a region considered to be an abnormal area in the examination image.

9. The processor, After obtaining the results of the analysis of the examination images, A search term is output using a string containing a combination of at least one of the locations of abnormal findings obtained from the analysis results and the content or size of the abnormal findings. Using the outputted search terms, search the database for information related to the disease name or condition name. Based on the search results, present the name of the condition or disease. An information processing method that performs a process.

10. On the computer, After obtaining the results of the analysis of the examination images, A search term is output using a string containing a combination of at least one of the locations of abnormal findings obtained from the analysis results and the content or size of the abnormal findings. Using the outputted search terms, search the database for information related to the disease name or condition name. Based on the search results, present the name of the condition or disease. An information processing program that executes a process.

Images