Information processing device, information processing method, and information processing program

Abstract

An information processing device that uses the results of analysis of medical images to help recall disease names [Solution] An information processing device is provided that includes an analysis unit that obtains analysis results of an examination image, a first output unit that outputs a search term using a character string including a phrase that combines at least one of the location of an abnormal finding obtained from the analysis results of the analysis unit and the content or size of the abnormal finding, a search unit that searches a database for information related to the name of a pathological condition or disease name using the search term output by the first output unit, and a presentation unit that presents the name of a pathological condition or disease name based on the search results by the search unit.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Doctors are required to recall the names of illnesses that patients who visit medical institutions for examinations have. Medical knowledge in the medical field is constantly increasing, and it is impossible for doctors to remember all of this medical knowledge. Therefore, a technology has been disclosed that supports recalling possible illness names from the contents of a patient's findings (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-17137 Summary of the Invention [Problem to be solved by the invention]

[0004] In addition to the doctor's interview, a patient's examination may involve tests that capture images of the patient's internal state, such as X-rays, CT (Computed Tomography), MRI (Magnetic Resonance Imaging), electrocardiograms, ultrasound echoes, endoscopies, pathology images, infrared thermography, and angiograms. It is thought that using test images of the inside of the patient's body, in addition to the patient's symptoms identified through the interview, would be more useful in recalling the name of the disease, but it is sometimes difficult for a doctor to recall the name of the disease just by looking at the test images.

[0005] The present disclosure has been made in consideration of the above points, and aims to provide an information processing device, an information processing method, and an information processing program that support recall of disease names using the analysis results of test images. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, there is provided an information processing device comprising: an analysis unit that obtains analysis results of an examination image; a first output unit that outputs a search term using a character string including a phrase that combines at least one of the location of an abnormal finding obtained from the analysis results of the analysis unit and the content or size of the abnormal finding; a search unit that searches a database for information related to a pathological condition name or disease name using the search term output by the first output unit; and a presentation unit that presents the pathological condition name or disease name based on the search results by the search unit.

[0007] The information processing device may further include a second output unit that outputs a search term by combining the search term output by the first output unit with one or more character strings of symptoms and pathological conditions or disease names provided from one or more of user input or patient information stored in a medical record system.

[0008] The first output unit may output the search terms by classifying them into a site where an abnormal finding is present, a content of the abnormal finding, or a size of the abnormal finding.

[0009] The first output unit may further output the search term together with the name of the examination from which the examination image was obtained.

[0010] The first output unit may output the search term by expressing the size as time-series data.

[0011] The analysis unit may obtain analysis results of multiple examination images obtained at different times from the same subject, and the first output unit may output information on the time course of each of the examination images as the search term.

[0012] The analysis unit may obtain an analysis result of the inspection image by providing the inspection image to a large-scale multimodal model and obtaining an output from the large-scale multimodal model.

[0013] The presentation unit may present a user interface for allowing a user to select a region of the test image that is considered to be an abnormal finding region.

[0014] According to another aspect of the present disclosure, there is provided an information processing method in which a processor obtains an analysis result of an examination image, outputs a search term using a character string including a phrase combining at least one of the location of an abnormal finding obtained from the analysis result and the content or size of the abnormal finding, searches a database for information regarding the name of a pathological condition or disease name using the output search term, and performs a process of presenting the name of the pathological condition or disease name based on the search result.

[0015] According to another aspect of the present disclosure, there is provided an information processing program that causes a computer to obtain analysis results of an examination image, output a search term using a character string including a phrase that combines at least one of the location of an abnormal finding obtained from the analysis result and the content or size of the abnormal finding, search a database for information related to the name of a pathological condition or disease name using the output search term, and present the name of the pathological condition or disease name based on the search results. [Effects of the Invention]

[0016] According to the present disclosure, it is possible to provide an information processing device, an information processing method, and an information processing program that support recall of disease names using the analysis results of test images. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram illustrating an information processing apparatus according to an embodiment of the disclosed technology. [Figure 2] FIG. 2 is a block diagram showing a hardware configuration of the information processing device. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing device. [Figure 4] FIG. 10 is a diagram illustrating an example of a database structure. [Figure 5]FIG. 10 is a diagram illustrating an example of a user interface displayed on a user terminal by an information processing device. [Figure 6] FIG. 10 is a diagram illustrating an example of a user interface displayed on a user terminal by an information processing device. [Figure 7] 10A and 10B are diagrams illustrating examples of time-series inspection images displayed on a user terminal by an information processing device [Figure 8] 10 is a flowchart showing a flow of information processing by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of the present disclosure will be described below with reference to the drawings. The same reference numerals are used throughout the drawings to designate identical or equivalent components and parts. The dimensional proportions of the drawings are exaggerated for illustrative purposes and may differ from the actual proportions.

[0019] FIG. 1 is a diagram illustrating an information processing device according to this embodiment. The information processing device 10 shown in FIG. 1 accepts input of a patient's symptoms and pathology name from a user terminal 20 connected via a network 1, and outputs a search term using the symptoms and pathology name. The user terminal 20 is a terminal used by a doctor or other medical professional as a user. The information processing device 10 searches a database 30 storing information about pathologies using the output search term, and presents a disease name to the user terminal 20 based on the search results. The database 30 stores, as information about pathologies, information about cases, information about suspected cases, and information from textbooks that describe diseases or pathologies. When searching the database 30, the information processing device 10 according to this embodiment outputs a search term using the patient's symptoms and pathology name as well as the analysis results of the patient's examination image.

[0020] Note that a pathology name is an expanded concept of a disease name (disease name). It refers to a disease pathology that matches. Therefore, in the present disclosure, a pathology name may be considered to be a disease name. As an example, an immunocompromised state is a pathology name, and is considered to be an example of a word that is not a disease name. Furthermore, as one aspect of this, when registering case information or suspected case information in the database 30, the information is divided by pathology, taking into account that a single case may have multiple pathologies. This makes it possible to avoid pathology names that are not related to symptoms from appearing in search results. It is also possible to register multiple pathologies in combination. This makes it possible to explain a set of symptoms that can only be explained when two or more pathologies are combined. In one example, wheezing is seen in COPD but is rarely seen in lung cancer, and hemoptysis is often seen in lung cancer but is rarely seen in COPD. However, when searching for the three words hemoptysis + wheezing + CT = lung @ mass, the information processing device 10 can display the search results on the user terminal 20 as a combination of the two pathological conditions COPD + lung cancer.

[0021] The information processing device 10 may be constructed as, for example, a web server, and the reception of input from the user terminal 20 and the presentation of search results from the database 30 may be realized in the form of a web page. Therefore, a browser for viewing web pages is installed on the user terminal 20, but a dedicated application for using the search service provided by the information processing device 10 may also be installed.

[0022] When searching the database 30, the information processing device 10 of this embodiment outputs search terms using the analysis results of the patient's examination images in addition to the patient's symptoms and pathological condition, thereby providing better support to doctors in recalling the name of the disease compared to when the analysis results of the examination images are not used.

[0023] The network 1 may be the Internet, an intranet, or any other network, and may use any communication protocol, type of communication, or scale of communication. The database 30 may be built in the information processing device 10 as shown in FIG. 1, or may be built in a device different from the information processing device 10.

[0024] FIG. 2 is a block diagram showing the hardware configuration of the information processing device 10. As shown in FIG.

[0025] 2, the information processing device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0026] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads programs from the ROM 12 or storage 14 and executes the programs using the RAM 13 as a work area. The CPU 11 controls the above-mentioned components and performs various arithmetic processing in accordance with the programs recorded in the ROM 12 or storage 14. In this embodiment, the ROM 12 or storage 14 stores an information processing program that outputs search terms based on user input and the results of analysis of the examination image, searches a database 30 that stores information about pathological conditions using the output search terms, and presents information about the disease to the user terminal 20 based on the search results.

[0027] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs including the operating system and various data.

[0028] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0029] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may also function as the input unit 15 by adopting a touch panel system.

[0030] The communication interface 17 is an interface for communicating with other devices such as the user terminal 20, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0031] When executing the above information processing program, the information processing device 10 uses the above hardware resources to realize various functions. The functional configuration realized by the information processing device 10 will be described.

[0032] FIG. 3 is a block diagram showing an example of the functional configuration of the information processing device 10. As shown in FIG.

[0033] 3, the information processing device 10 has, as functional components, an acquisition unit 101, an analysis unit 102, a first output unit 103, a second output unit 104, a standardization unit 105, a search unit 106, and a presentation unit 107. Each functional component is realized by the CPU 11 reading and executing an information processing program stored in the ROM 12 or the storage 14.

[0034] The acquisition unit 101 acquires user input via the user terminal 20. The user input acquired by the acquisition unit 101 includes the symptom findings and pathological condition name of the patient. The acquisition unit 101 also acquires examination images of the same patient as the patient with the symptom findings and pathological condition name. The examination images are, for example, images obtained by an examination such as an X-ray examination, a CT examination, or an MRI examination, which captures images of the inside of the patient's body.

[0035] The analysis unit 102 analyzes the patient's examination image acquired by the acquisition unit 101 to obtain an analysis result of the examination image. The analysis unit 102 may use any trained model to analyze the examination image. The trained model is a model that has been trained to output an abnormal finding if an examination image is input. That is, the analysis unit 102 provides the examination image to the trained model and obtains an output from the trained model to obtain an analysis result of the examination image. The analysis unit 102 can obtain findings from the examination image by analyzing the examination image.

[0036] In this embodiment, the analysis unit 102 may use a large multimodal model (LMM) to analyze the examination image. The large multimodal model is a model that can respond to inputs of multiple modalities, such as images, in natural language text in addition to natural language text. The large multimodal model is a model that is trained to output abnormal findings in natural language text if an examination image is input. In other words, the analysis unit 102 provides the examination image to the large multimodal model and obtains output from the large multimodal model to obtain an analysis result of the examination image.

[0037] The first output unit 103 outputs search terms related to abnormal findings based on the analysis results of the examination images obtained by the analysis unit 102. Specifically, the first output unit 103 outputs search terms using a character string including a phrase combining at least one of the location of the abnormal findings and the content or size of the abnormal findings obtained from the analysis results. The first output unit 103 may output search terms classified into the name of the examination from which the examination image was obtained, the location of the abnormal findings, the content of the abnormal findings, or the size of the abnormal findings. In other words, the first output unit 103 outputs search terms by combining a phrase indicating the name of the examination from which the examination image was obtained and a phrase related to the location of the abnormal findings, the content of the abnormal findings, or the size of the abnormal findings. The first output unit 103 may output search terms based on the shape, density, or contrast of the abnormal findings with surrounding tissues. For example, suppose that the analysis unit 102 analyzes the examination images from a CT scan and finds a cavity in the lung. In this case, the first output unit 103 outputs the search term "CT=cavity@lung" as a search term related to abnormal findings. In this case, "CT" is an example of a phrase indicating the name of the test from which the test image was obtained, and "cavity@lung" is an example of a phrase regarding the location of the abnormal finding and the content of the abnormal finding. When outputting search terms by combining phrases regarding the size of the abnormal finding, the first output unit 103 may output search terms that represent the size as time-series data. Furthermore, the analysis unit 102 may use machine learning, deep learning, or image processing algorithms for the analysis. Furthermore, in one embodiment, the analysis unit 102 and the first generation unit 103 may be integrated so that the test image is directly passed to the LMM and the search term is output. Furthermore, the symptom findings and pathological condition names used as search terms may be received directly from the user via voice or keyboard input.

[0038] The second output unit 104 outputs a plurality of search terms by combining the search terms output by the first output unit 103 with the symptom findings and pathology names acquired by the acquisition unit 101. For example, if the first output unit 103 outputs the search term "CT=cavity@lungs" as described above, and the acquisition unit 101 acquires the symptom findings and pathology names "fever" and "body temperature 38 degrees," the second output unit 104 outputs the search terms "fever body temperature 38 degrees CT=cavity@lungs."

[0039] When the second output unit 104 generates a search term by combining the character string of the abnormal finding obtained from the analysis result of the analysis unit 102 with the symptom finding or pathological condition name (disease name) input or provided via the user terminal 20, the second output unit 104 may treat the character string as a logical product (AND) search or a phrase search. For example, the second output unit 104 adds the user-inputted "fever" to a character string tagged with the site of the abnormal finding "lung" and the content of the abnormal finding "cavity." This allows the database 30 to be searched using the phrase "cavity@lung fever" as a search term.

[0040] This embodiment is not limited to manual input of symptoms and findings and pathological conditions on an input screen displayed on the user terminal 20. It also includes a configuration in which patient information stored in a medical record system, such as an electronic medical record (EMR) or a picture archiving and communication system (PACS), is referenced and used as part of a search string. For example, the user terminal 20 automatically or semi-automatically acquires symptoms and findings, such as "fever," "body temperature 38.5°C," and "cough" recorded in the electronic medical record into an input field. The second output unit 104 can generate search terms by combining the information acquired by the user terminal 20 from the electronic medical record with abnormal findings (location, content, size) obtained by the analysis unit 102. In this case, the user may select necessary items from the medical information in the electronic medical record and import them into the acquisition unit 101 as search terms via the user terminal 20. Using this configuration reduces the input workload of medical staff and enables searches using more accurate data.

[0041] As another example of this embodiment, multiple search filters may be implemented. In the above example, search terms are separated by spaces, but for example, each line can be separated by: Location: Right lower lung field, Findings: Cavity tumor Size: about the size of a red bean, Location: right upper lung field Findings: Cavity tumor, size: chicken egg size, location: right upper lung field It is also possible to add multiple filters like this:

[0042] The standardization unit 105 standardizes the terms of symptoms and pathological conditions to unify them for the search terms output by the second output unit 104. When unifying the terms, the standardization unit 105 may obtain words by natural language processing, rule-based modification, vector search for a word list, word search by calculating edit distance for a word list, or the like.

[0043] As another example of this embodiment, the first output unit 103 may generate search terms that express the change in size of abnormal findings as time-series data based on the analysis results of multiple test images obtained from the same patient at different times by the analysis unit 102. For example, the first output unit 103 may assign the increase or decrease in tumor size obtained from CT images taken every three months to the search terms in a format such as "tumor 10 mm (January 2024) → 12 mm (April 2024) → 15 mm (July 2024)." By generating search terms that express the change in size of abnormal findings as time-series data, the first output unit 103 can be useful in searching for the degree of progression over time and the effectiveness of treatment. Furthermore, the standardization unit 105 may convert the representation of the time-series information generated by the first output unit 103 into a unified standard for the search terms. For example, the standardization unit 105 may convert the change in size of an abnormal finding into notation such as unchanged, increased, decreased, disappeared, CR (complete response), PR (partial response), NC (no change), PD (progressive disease), etc. These notations are expressions that mean the change in size over time and the change in response to therapeutic intervention.

[0044] The search unit 106 searches the database 30 for information related to the pathological condition using the search term output by the first output unit 103 or the multiple search terms output by the second output unit 104. The search unit 106 outputs a query for searching the database 30 using, for example, the search term output by the first output unit 103 or the multiple search terms output by the second output unit 104, and searches the database 30 with the output query. The information related to the pathological condition includes at least one of cases, suspected cases, and textbooks. That is, the information stored in the database 30 includes at least one of cases, suspected cases, and textbooks as information related to the pathological condition.

[0045] Here, we will show an example of the data structure of the database 30. Fig. 4 is a diagram showing an example of the data structure of the database 30. In this embodiment, the database 30 has a data type column, a pathological condition column, a pathological finding column, an ID column, a case ID column, and a case number column.

[0046] The data type column is a column for identifying whether the record is a collection of multiple cases or a single case record; if the data type is 1, it is a collection of multiple cases, and if the data type is 2, it is a single case record.

[0047] The pathology column is a column in which the pathology name is stored. The pathology findings column is a column in which information on the findings of the pathology is stored. The ID column is a column in which an ID that identifies the pathology is stored. The case ID column is a column in which a case ID that identifies case data related to the pathology is stored. If the data type is 1, multiple case IDs are stored in the case ID column, which corresponds to the ID column of case information held in a separate table with a configuration similar to that of Figure 4. The number of cases column is a column in which the number of cases of the pathology is stored, and if the data type is 2, 1 is stored in the number of cases column.

[0048] The presentation unit 107 presents a user interface for searching for pathological condition names to the user terminal 20. The presentation unit 107 then presents information about diseases to the user terminal 20 based on the search results by the search unit 106. The presentation unit 107 presents the information about diseases to the user terminal 20 in the form of, for example, a web page.

[0049] Here, an example of a user interface that the presentation unit 107 displays on the user terminal 20 is shown. Fig. 5 is a diagram showing an example of a user interface that is displayed on the user terminal 20 by the information processing device 10. The user interface 200 shown in Fig. 5 is realized in the form of, for example, a web page.

[0050] 5 includes an input area 201 in which the user of the user terminal 20 inputs the symptom findings and pathology name to be searched for, an image upload button 202 for transmitting the examination image to the information processing device 10, and a search button 203 for causing the information processing device 10 to execute a process of searching for information related to the pathology from the database 30. When the user of the user terminal 20 inputs the symptom findings and pathology name in the input area 201, uploads the examination image to the information processing device 10 using the image upload button 202, and selects the search button 203, the information processing device 10 executes a process of searching for information related to the pathology from the database 30 and presents the search results on the user interface 200.

[0051] In another embodiment, the presentation unit 107 may present a graphical user interface that allows the user to select an abnormal finding region highlighted on the examination image. When the user selects a candidate such as "lung field" or "frontal lobe," the region information is overwritten or added to the analysis results of the analysis unit 102 and is reflected in the ultimately generated search term. This allows for improved search accuracy, including preventing misrecognition and selection from multiple candidates.

[0052] Fig. 6 is a diagram showing an example of a user interface displayed on the user terminal 20 by the information processing device 10. The user interface 200 shown in Fig. 6 is realized in the form of, for example, a web page. The user interface 200 shown in Fig. 6 is an example of a presentation of search results for information related to diseases.

[0053] The user interface 200 shown in FIG. 6 includes an input area 201, an image upload button 202, a search button 203, as well as a display area 211 for a summary of search results and a display area 212 for case reports.

[0054] In the display area 211 of the summary of the search results, diseases that can be considered based on the symptom findings and pathological condition names entered in the input area 201 and the examination images uploaded using the image upload button 202 and analyzed by the information processing device 10 are displayed, sorted by medical department. Note that the presentation unit 107 may present the diseases on the user interface 200 without sorting them by medical department.

[0055] The case report search result display area 212 displays the number of cases of the pathology that can be considered based on the symptom findings and pathology name entered in the input area 201 and the test image uploaded with the image upload button 202 and analyzed by the information processing device 10. When the user of the user terminal 20 selects a disease name presented in the case report search result display area 212, the presentation unit 107 presents detailed case information of the disease on the user terminal 20.

[0056] By presenting the user interface 200 shown in Figure 6 to the user terminal 20, the information processing device 10 can use the analysis results of the examination image to assist the user of the user terminal 20 (a medical professional such as a doctor) in recalling the name of the disease.

[0057] When presenting detailed case information of a pathological condition to the user terminal 20, if an examination image corresponding to the pathological condition is registered in the database 30, the presentation unit 107 may also present the examination image to the user terminal 20. In this case, if the examination images are registered in chronological order in the database 30, the presentation unit 107 may present the examination images to the user terminal 20 along with the dates when the examination images were taken. FIG. 7 is a diagram showing an example of time-series examination images displayed on the user terminal 20 by the information processing device 10. The time-series examination images 221 shown in FIG. 7 include, for example, three examination images 221a, 221b, and 221c. By presenting the time-series examination images 221 along with detailed case information of a pathological condition, the presentation unit 107 can assist the user of the user terminal 20 in recalling the name of the disease by comparing them with the examination images uploaded to the information processing device 10. In addition, the presentation unit 107 may, for example, overlay two or more images as time-series images, apply different colors or symbols to lesion sites, process boundaries, or the like to show how the size of the lesion has changed over time, making it easier for machine learning to understand the concept. When overlaying the images, the presentation unit 107 may use different colors for the latest image and the image before the latest, to indicate whether the change in size has increased or decreased.

[0058] When using a large-scale multimodal model, the analysis unit 102 can input an examination image and obtain output expressed in natural language, such as "There is a suspected tumor in the left lung" or "There is a possibility of pneumonia due to the formation of a cavity." The first output unit 103 performs text analysis on the output from the large-scale multimodal model to extract phrases such as "lung tumor" and "cavity @ left lung," and combines them as search terms to make searches of the database 30 more efficient.

[0059] As another example, this embodiment may also include a function for presenting to the user the basis on which search terms were generated when the analysis unit 102 extracts the location, content, and size of abnormal findings using artificial intelligence (AI), particularly a large-scale multimodal model. For example, the analysis unit 102 may visualize a weighted heat map or region of interest on the examination image, enclosing it in an arbitrary shape such as a rectangle, and indicate which region was used to select terms such as "pneumonia" or "cavity." This allows the user (doctor or medical staff) to intuitively understand why the AI ​​selected "cavity@left lung" as a search term.

[0060] In this case, the presentation unit 107 may also display the "basis site of AI judgment" and the "reliability score" for each search term candidate. For example, if the site of the abnormal finding is the left lung field, the presentation unit 107 may display an explanation such as "The image pixel value of the left lung field falls within the XX range, and it was diagnosed as a severe cavitary lesion, so 'cavitary @ left lung' was generated." Displaying the information in this way by the presentation unit 107 makes it easier for the user to confirm the AI's inference process, adds transparency (explainable AI) to the generation of search terms by AI, and can increase the reliability of search results.

[0061] In this embodiment, the first output unit 103 may generate a statement that combines at least two of the location where the abnormal finding was found (lung, liver, bone, etc.), the content of the abnormal finding (cavity, tumor, inflammation, etc.), and the size of the abnormal finding (major axis, minor axis, volume, etc.). For example, the first output unit 103 may generate a statement that simultaneously indicates the location and the content or size, such as "tumor in left lung 10 mm" or "cavity in right lung 15 mm." Generating statements in this manner can improve the accuracy of searching the database 30.

[0062] In this embodiment, the size of abnormal findings may not only be expressed using specific numerical values ​​such as "10 mm" or "15 mm," but may also include everyday or analog size expressions such as "the size of a chicken egg" or "the size of a red bean." When these everyday or analog size expressions are received, the standardization unit 105 may create a dictionary of correspondences such as "the size of a chicken egg = 40 mm" and "the size of a red bean = 10 mm" and convert them into a unified standard (numerical value, etc.) as search terms. For example, if "tumor in right lung, the size of a chicken egg" is input, the standardization unit 105 may convert it into a phrase such as "tumor in right lung, 40 mm" and then generate a search term. This allows for more efficient searching of the database 30, even when various expressions are mixed, compared to when search terms are not converted into a unified standard, thereby improving search accuracy and versatility.

[0063] 3, the information processing device 10 can present the disease name obtained from the database 30 to the user terminal 20. By presenting the disease name obtained from the database 30 to the user terminal 20, the information processing device 10 can use the analysis results of the examination image to assist the user of the user terminal 20 (a medical professional such as a doctor) in recalling the disease name.

[0064] Next, the operation of the information processing device 10 will be described.

[0065] 8 is a flowchart showing the flow of information processing by the information processing device 10. The CPU 11 reads out an information processing program from the ROM 12 or the storage 14, loads it into the RAM 13, and executes it, thereby performing information processing.

[0066] In step S101, the CPU 11 acquires a user input from the user terminal 20. The user input acquired by the CPU 11 includes the symptom findings and pathological condition name of the patient. The CPU 11 also acquires an examination image of the same patient as the patient with the symptom findings and pathological condition name. The examination image is, for example, an image obtained by an examination such as an X-ray examination, a CT examination, or an MRI examination, which captures the state of the patient's body.

[0067] Following step S101, in step S102, CPU 11 analyzes the inspection image acquired in step S101. CPU 11 may use any trained model to analyze the inspection image. The trained model is a model that has been trained to output an abnormal finding if an inspection image is input. That is, CPU 11 provides the inspection image to the trained model and obtains an output from the trained model, thereby obtaining an analysis result of the inspection image. CPU 11 can obtain findings from the inspection image by analyzing the inspection image.

[0068] In this embodiment, the CPU 11 may use a large scale multimodal model (LMM) to analyze the inspection image.

[0069] Following step S102, in step S103, the CPU 11 outputs a search term using the analysis of the examination image. For example, assume that the CPU 11 analyzes the examination image of a CT examination and finds a cavity in the lung. In this case, the CPU 11 outputs the search term "CT=cavity@lung" as a search term related to abnormal findings.

[0070] Following step S103, in step S104, the CPU 11 outputs a plurality of search terms by combining the search terms output in step S103 with the symptom findings and pathology names acquired in step S101. For example, if the CPU 11 outputs the search term "CT=cavity@lungs" as described above and acquires the symptom findings and pathology names "fever" and "body temperature 38 degrees," the CPU 11 outputs the search terms "fever body temperature 38 degrees CT=cavity@lungs."

[0071] Following step S104, in step S105, the CPU 11 searches the database 30 using the search term output in step S104.

[0072] Following step S105, in step S106, the CPU 11 presents the search results based on the search results of step S105 to the user terminal 20. The CPU 11 presents the disease names obtained from the database 30 as the search results to the user terminal 20 in the form of, for example, a web page.

[0073] 8, the information processing device 10 can present the disease name obtained from the database 30 to the user terminal 20. By presenting the disease name obtained from the database 30 to the user terminal 20, the information processing device 10 can support the user of the user terminal 20 (a medical professional such as a doctor) in recalling the disease name using the analysis results of the examination image.

[0074] Although the embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical ideas described in the claims, and it is understood that these modifications or alterations naturally fall within the technical scope of the present disclosure. For example, in one aspect of the present disclosure, an information processing device may generate text that will become a radiology report from an examination image of a radiological examination using an LMM, and then output search terms from the text generated by the LMM through a generation process using natural language processing or an LLM.

[0075] Furthermore, the effects described in the above embodiments are explanatory or exemplary and are not limited to those described in the above embodiments. In other words, the technology according to the present disclosure may achieve other effects that are obvious to a person skilled in the art of the present disclosure from the description in the above embodiments, in addition to or instead of the effects described in the above embodiments.

[0076] In the above embodiments, the information processing performed by the CPU after reading the software (program) may be performed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) whose circuit configuration can be changed after fabrication, such as field-programmable gate arrays (FPGAs), and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to perform specific processing. The information processing may be performed by one of these processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0077] In addition, in each of the above embodiments, the information processing program is described as being pre-stored (installed) in a ROM or storage, but this is not limiting. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network. The present disclosure may also be applied to a program product. [Explanation of symbols]

[0078] 1 Network 10. Information processing equipment 20 User terminal 30 databases 101 Acquisition Department 102 Analysis Department 103 First output section 104 Second output section 105 Standardization Department 106 Search Department 107 Presentation section

Claims

1. an analysis unit that obtains an analysis result of the inspection image; a first output unit that outputs a search term using a character string including a phrase that combines at least one of the location of the abnormal finding obtained from the analysis result of the analysis unit and the content or size of the abnormal finding; a search unit that searches a database for information related to a pathological condition name or a disease name using the search term output by the first output unit; An information processing device comprising:

2. 2. The information processing device according to claim 1, further comprising a second output unit that outputs a search term by combining the search term output by the first output unit with one or more character strings of symptoms and pathological conditions or disease names provided from one or more of user input or patient information stored in a medical record system.

3. The information processing device according to claim 1 , wherein the first output unit outputs the search terms by classifying them into a region where an abnormal finding is present, a content of the abnormal finding, or a size of the abnormal finding.

4. The information processing apparatus according to claim 1 , wherein the first output unit further outputs the search term together with the name of the examination from which the examination image was obtained.

5. The information processing device according to claim 1 , wherein the first output unit outputs the search term by expressing the size as time-series data.

6. The analysis unit obtains analysis results of a plurality of test images obtained from the same subject at different times, The information processing apparatus according to claim 1 , wherein the first output unit outputs information on the time course of each of the inspection images as the search term.

7. The information processing apparatus according to claim 1 , wherein the analysis unit obtains an analysis result of the inspection image by providing the inspection image to a large-scale multimodal model and obtaining an output from the large-scale multimodal model.

8. The processor: After obtaining the results of the inspection image analysis, outputting a search term using a character string including a phrase combining at least one of the site of the abnormal finding obtained from the analysis result and the content or size of the abnormal finding; Use the output search terms to search the database for information on the pathology or disease name. An information processing method that performs processing.

9. On the computer, After obtaining the results of the inspection image analysis, outputting a search term using a character string including a phrase combining at least one of the site of the abnormal finding obtained from the analysis result and the content or size of the abnormal finding; Use the output search terms to search the database for information on the pathology or disease name. An information processing program that executes processing.

Images