Information processing device, information processing system, information processing method, and program

Abstract

To evaluate the reliability of generative information generated by a generative model. [Solution] The information processing device according to the embodiment comprises a prompt acquisition unit, a generated information acquisition unit, and an evaluation unit. The prompt acquisition unit acquires a first prompt containing medical information and a second prompt similar to the first prompt. The generated information acquisition unit inputs the first prompt into a generation model and acquires first generated information corresponding to the first prompt from the generation model, and inputs the second prompt into the generation model and acquires second generated information corresponding to the second prompt from the generation model. The evaluation unit evaluates the relationship between the first generated information and the second generated information.

Description

Technical Field

[0001] The embodiments disclosed in this specification and the drawings relate to an information processing apparatus, an information processing system, an information processing method, and a program.

Background Art

[0002] In recent years, the use of generated information generated by generative AI (Artificial Intelligence) has also advanced in the medical field. In such a technology, by inputting a prompt into a generative model, it is possible to easily perform extraction of abnormal regions, generation of image data, etc. However, the reliability of the generated information by such a generative model is not always high.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to evaluate the reliability of the generated information generated by the generative model. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problems. The problems corresponding to the respective effects of each configuration shown in the embodiments described later can also be positioned as other problems.

Means for Solving the Problems

[0005] The information processing device according to the embodiment comprises a prompt acquisition unit, a generated information acquisition unit, and an evaluation unit. The prompt acquisition unit acquires a first prompt containing medical information and a second prompt similar to the first prompt. The generated information acquisition unit inputs the first prompt into a generation model and acquires first generated information corresponding to the first prompt from the generation model, and inputs the second prompt into the generation model and acquires second generated information corresponding to the second prompt from the generation model. The evaluation unit evaluates the relationship between the first generated information and the second generated information. [Brief explanation of the drawing]

[0006] [Figure 1] Figure 1 shows an example of the overall configuration of the information processing system according to the first embodiment. [Figure 2] Figure 2 shows an example of the first prompt and the first generated information according to the first embodiment. [Figure 3] Figure 3 shows an example of a second prompt and second generated information according to the first embodiment. [Figure 4] Figure 4 shows an example of the first generated information according to the first embodiment. [Figure 5] Figure 5 shows an example of a comparison between the first generated information and the second generated information according to the first embodiment. [Figure 6] Figure 6 shows an example where the location and area of ​​the lesion are the same in the first generated information and the second generated information according to the first embodiment. [Figure 7] Figure 7 shows an example of a case in which the location of the lesion differs between the first generated information and the second generated information according to the first embodiment, and the lesion area overlaps. [Figure 8] Figure 8 shows an example in which the location of the lesion differs between the first generated information and the second generated information according to the first embodiment, and the lesion regions do not overlap. [Figure 9]Figure 9 shows an example of a case in which a lesion is detected in the first generated information according to the first embodiment, but not in the second generated information. [Figure 10] Figure 10 shows an example of how the evaluation results are displayed according to the first embodiment. [Figure 11] Figure 11 is a flowchart showing an example of the evaluation process flow according to the first embodiment. [Figure 12] Figure 12 is a flowchart showing an example of the evaluation process flow according to the third embodiment. [Figure 13] Figure 13 shows an example of the first prompt and the first generated information according to the fifth embodiment. [Figure 14] Figure 14 shows an example of a second prompt and second generated information according to the fifth embodiment. [Figure 15] Figure 15 shows an example of the first prompt and the first generated information according to the sixth embodiment. [Figure 16] Figure 16 shows an example of a second prompt and second generated information according to the sixth embodiment. [Figure 17] Figure 17 shows an example of the first prompt and the first generated information according to the seventh embodiment. [Figure 18] Figure 18 shows an example of a second prompt and second generated information according to the seventh embodiment. [Figure 19] Figure 19 shows another example of the first prompt and first generated information according to the seventh embodiment. [Figure 20] Figure 20 shows another example of the second prompt and second generated information according to the seventh embodiment. [Modes for carrying out the invention]

[0007] The following describes in detail the embodiments of the information processing device, information processing system, information processing method, and program, with reference to the drawings.

[0008] (First Embodiment) FIG. 1 is a diagram showing an example of the overall configuration of an information processing system S according to the first embodiment. As shown in FIG. 1, the information processing system S includes, as an example, a first information processing device 100, a second information processing device 200, and a medical information system 300. The information processing system S is provided, for example, in a medical institution such as a hospital. The first information processing device 100, the second information processing device 200, and the medical information system 300 are communicably connected via a network 400 such as an in-hospital LAN (Local Area Network).

[0009] The medical information system 300 is a system for storing and managing medical information, and includes, for example, one or more servers or PCs (Personal Computers). The medical information stored in the medical information system 300 is, for example, a patient's electronic medical record, medical treatment information, interview results, examination data, medical image data, and diagnosis results. The medical information system 300 may be, for example, an electronic medical record system, a hospital information system (HIS), a clinical laboratory information system (LIS), a radiology information system (RIS), or a medical image storage device. Further, the medical information system 300 may include a plurality of the above systems and devices.

[0010] The first information processing device 100 and the second information processing device 200 are, for example, computers such as servers or PCs.

[0011] The second information processing device 200 includes a learned model of a generative AI (Artificial Intelligence) (hereinafter referred to as a generative model 40). The second information processing device 200 includes, for example, a memory circuit and a processor. The generative model 40 is stored, for example, in the memory circuit in the second information processing device 200 and operates by the processor. Note that the configuration of the second information processing device 200 is not particularly limited, and a known configuration can be adopted.

[0012] When the generation model 40 receives an input of a prompt, it generates information corresponding to the prompt and outputs the generated information. The generation model 40 is, for example, an MMM (Multimodal Model), an LLM (Large Language Model) that can handle data of review types such as image data, etc., and is an integrated AI model that can process multiple types of modalities (data types) such as text, images, voices, numerical values, etc. at once.

[0013] It is assumed that the generation model 40 of this embodiment has been learned by associating, for example, medical image data with abnormal regions included in the medical image data. The abnormal region is, for example, a lesion. The medical image data is, for example, medical image data in a format compliant with DICOM (Digital Imaging and Communications IN Medicine).

[0014] The information generated by the generation model 40 is referred to as generated information. The generated information includes, for example, image data. Note that the generated information may include not only image data but also text. Also, depending on the content of the prompt, the generated information may not include image data. In this embodiment, since the information processing system S is used in the medical field, the prompt and the generated information include, for example, information about a patient. The information about the patient is, for example, medical information such as examination results, interview results, findings by a doctor, etc. Also, the information about the patient may be information regarding the patient's gender, age, and build. Also, the prompt and the generated information may include medical image data. Note that the medical image data is also referred to as medical imaging data.

[0015] The first information processing device 100 is operated by a user and provides the user with the output results of the generation model 40 stored in the second information processing device 200. The user of the first information processing device 100 is, for example, a physician or other medical professional. For example, a physician may use the generation model 40 via the first information processing device 100 to obtain information to refer to for diagnosing a patient. The first information processing device 100 is an example of an information processing device in this embodiment.

[0016] The first information processing device 100 includes a network interface 110, a memory circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0017] The NW interface 110 is connected to the processing circuit 150 and controls the transmission and communication of various data between the first information processing device 100, the second information processing device 200, and the medical information system 300. The NW interface 110 is implemented by a network card, network adapter, NIC (Network Interface Controller), etc.

[0018] The memory circuit 120 stores various types of information used by the processing circuit 150 in advance. The memory circuit 120 also stores various programs. The memory circuit 120 is, for example, a non-volatile storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or integrated circuit memory device that stores various types of information. In addition to HDDs and SSDs, the memory circuit 120 may also be a drive device that reads and writes various types of information to portable storage media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), flash memory, or semiconductor memory elements such as RAM (Random Access Memory). The memory circuit 120 is an example of a storage unit.

[0019] The input interface 130 is implemented by a mouse, keyboard, pen tablet (combining a stylus and tablet that accept user input), trackball, switch buttons, touchpad (for input operations by touching the operating surface), touchscreen (integrating a display screen and touchpad), non-contact input circuit using an optical sensor, and audio input circuit, etc. The input interface 130 may include multiple devices that accept user operations. The input interface 130 is connected to a processing circuit 150 and converts the input operations received from the user into electrical signals and outputs them to the processing circuit 150. In this specification, the input interface 130 is not limited to those equipped with physical operating components such as a mouse or keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the device and outputs these electrical signals to the processing circuit 150 is also included as an example of the input interface 130.

[0020] The display 140 displays various types of information under the control of the processing circuit 150. For example, the display 140 outputs a screen containing generated information generated by the generation model 40, or a GUI (Graphical User Interface) for accepting various operations from the user. Specifically, the display 140 is an LCD display or a CRT (Cathode Ray Tube) display, etc. The input interface 130 and the display 140 may be integrated. For example, the input interface 130 and the display 140 may be implemented by a touch panel. The display 140 is an example of a display unit.

[0021] The processing circuit 150 is a processor that reads programs from the memory circuit 120 and executes them to realize functions corresponding to each program. The processing circuit 150 in this embodiment includes a reception function 151, a prompt acquisition function 152, a generated information acquisition function 153, an evaluation function 154, and a display control function 155. The reception function 151 is an example of a reception unit. The prompt acquisition function 152 is an example of a prompt acquisition unit. The generated information acquisition function 153 is an example of a generated information acquisition unit. The evaluation function 154 is an example of an evaluation unit. The display control function 155 is an example of an output unit or a display control unit.

[0022] Here, for example, the processing functions of the processing circuit 150, namely the reception function 151, the prompt acquisition function 152, the generated information acquisition function 153, the evaluation function 154, and the display control function 155, are stored in the memory circuit 120 in the form of programs that can be executed by a computer. The processing circuit 150 is a processor. For example, the processing circuit 150 reads the programs from the memory circuit 120 and executes them to realize the functions corresponding to each program. In other words, the processing circuit 150 in the state in which each program has been read will have the functions shown in the processing circuit 150 of Figure 1. In Figure 1, the processing functions performed by the reception function 151, the prompt acquisition function 152, the generated information acquisition function 153, the evaluation function 154, and the display control function 155 are realized by a single processor, but it is also possible to configure the processing circuit 150 by combining multiple independent processors, and each processor realizes the functions by executing a program. Furthermore, although Figure 1 describes a single memory circuit 120 that stores programs corresponding to each processing function, it is also possible to have multiple memory circuits distributed and the processing circuit 150 read the corresponding programs from individual memory circuits.

[0023] The above description illustrates an example in which the "processor" reads and executes programs corresponding to each function from the memory circuit 120, but the embodiments are not limited to this. The term "processor" refers to circuits such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), Application Specific Integrated Circuit (ASIC), and Programmable Logic Device (e.g., Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). If the processor is a CPU, for example, it realizes its functions by reading and executing programs stored in the memory circuit 120. On the other hand, if the processor is an ASIC, instead of storing programs in the memory circuit 120, the functions are directly incorporated as logic circuits within the processor's circuitry. In this embodiment, each processor is not limited to being configured as a single circuit; multiple independent circuits may be combined to form a single processor and realize its functions. Furthermore, the multiple components shown in Figure 1 may be integrated into a single processor to realize their functions.

[0024] The reception function 151 accepts various user operations via the input interface 130. For example, the reception function 151 accepts a user operation to instruct the start of processing.

[0025] The prompt acquisition function 152 acquires the prompts that are input to the generation model 40. More specifically, the prompt acquisition function 152 acquires the first prompt and the second prompt entered by the user.

[0026] The first prompt is a prompt that allows the user to obtain desired information. For example, the user enters the first prompt to obtain information necessary for diagnosing a patient from the generation model 40. The second prompt is used to evaluate the reliability of the first generated information produced by the first prompt 501. Therefore, the second prompt itself does not necessarily have to be what the user wants for diagnosing a patient. Also, the second prompt is similar to the first prompt.

[0027] Since the information processing system S of this embodiment is used, for example, in the medical field, the first and second prompts input to the generation model 40 include medical information. Medical information includes, but is not limited to, the electronic medical record of the subject (patient), medical information, interview results, test data, medical image data, and diagnostic results.

[0028] More specifically, the first and second prompts input to the generation model 40 include at least one of medical image data and medical text data.

[0029] Figure 2 shows an example of a first prompt 501 and first generated information 401 according to the first embodiment. The first prompt 501 includes an instruction that causes the generation model 40 to generate information about a first abnormal region in a first cross-section of medical image data 301 as first generated information 401. The instruction is included in the first prompt 501 as text data tagged with, for example, "#task".

[0030] As a specific example, as shown in Figure 2, the first prompt 501 includes, for example, text data such as "Generate and output information regarding lesions in the axial section based on the subject's medical image data," and medical image data 301 corresponding to "subject's medical image data" in the text data. This text data is entered, for example, by the user. The axial section is an example of the first section. Note that the description of "lesion" in the first prompt 501 may be replaced with a description such as "abnormal area."

[0031] Medical image data 301 is, for example, three-dimensional volume data of X-ray CT (Computed Tomography) image data or magnetic resonance image data. Medical image data 301 may also be X-ray image data or ultrasound image data, etc. Medical image data 301 is, for example, image data acquired from a medical information system 300.

[0032] The way in which the prompt text data includes medical image data 301 is not particularly limited, but for example, the path to the storage location of the medical image data 301 may be specified within the prompt text data. For example, the medical image data 301 can be included in the prompt in a manner that allows the generation model 40 to search, such as RAG (Retrieval-Augmented Generation). In addition to medical image data 301, the storage locations of various medical information stored in the medical information system 300 may also be described in the text data of the first prompt 501.

[0033] The location where medical information referenced from the prompt's text data is stored may be a path indicating a storage location within the medical information system 300, or a path indicating a storage location within the memory circuit 120 of the first information processing device 100. Alternatively, the medical information may be directly written as the text data of the first prompt 501.

[0034] Note that the content of the first prompt 501 is not limited to the example shown in Figure 2. For example, a lesion is an example of an abnormal area, and "lesion" in the text data may be replaced with the more comprehensive concept of "abnormal area." Alternatively, other specific targets such as "tumor" or "polyp" may be specified in the text data. In addition, other instructions for the generation model 40 may be included in the text data depending on the user's application. In this embodiment, a lesion will be used below as an example of an abnormal area.

[0035] Figure 3 shows an example of the second prompt 502 and the second generated information 402 according to the first embodiment.

[0036] As described above, the second prompt 502 is similar to the first prompt 501. More specifically, the first prompt 501 and the second prompt 502 differ in at least some respects, but are otherwise identical. Furthermore, the differences between the first prompt 501 and the second prompt 502 are in the wording or presentation, and the first prompt 501 and the second prompt 502 are medically synonymous.

[0037] The second prompt 502 includes a command to cause the generation model 40 to generate information about a second abnormal region in a second cross-section of the medical image data 301 as second generated information 402. The second cross-section is different from the first cross-section. In the example shown in Figure 3, the second prompt 502 includes, for example, text data that reads, "Based on the patient's medical image data, generate and output information about lesions in the sagittal and coronal sections," and medical image data 301 corresponding to "the patient's medical image data" in the text data. Note that the sagittal and coronal sections are examples of the second cross-section. In the example shown in Figure 3, the second prompt 502 includes the specification of two types of cross-sections, the sagittal and coronal sections, but it is also acceptable to specify only one type.

[0038] The medical image data 301 included in the second prompt 502 is the same three-dimensional volume data as the medical image data 301 included in the first prompt 501. The text data of the second prompt 502 differs from that of the first prompt 501 in that the type of cross-section used to generate information about the lesion is different. Specifically, the first prompt 501 specifies an axial cross-section, while the second prompt 502 specifies sagittal and coronal cross-sections. In other words, the first prompt 501 and the second prompt 502 differ in the type of cross-section, but are otherwise identical prompts.

[0039] Furthermore, since the medical image data 301 is three-dimensional volume data, differences in the display configuration occur depending on the direction in which the cross-section is cut and displayed. However, since these differences relate to the display configuration and the medical image data 301, which is the target of lesion identification, is the same, the first prompt 501 and the second prompt 502 are medically synonymous.

[0040] Note that the content of the first and second prompts 501 and 502 shown in Figures 2 and 3 are examples only and are not limited to these.

[0041] Returning to Figure 1, the generation information acquisition function 153 inputs the first and second prompts 501 and 502 acquired by the prompt acquisition function 152 to the generation model 40 of the second information processing device 200 via the network 400 and NW interface 110, and acquires the first and second generation information from the generation model 40. The first generation information corresponds to the first prompt 501, and the second generation information corresponds to the second prompt 502.

[0042] In the example shown in Figure 2 above, when the generation model 40 receives the input of the first prompt 501, it outputs first generated information 401 based on the instructions of the first prompt 501. Specifically, when the generation model 40 receives the input of the first prompt 501, it detects lesions in the axial section of the medical image data 301 and generates and outputs information about the detected lesions as first generated information 401. The information about the lesions includes location information of the lesions. Location information of the lesions includes, for example, information indicating the location, area range, and number of lesions. The generated information acquisition function 153 stores the first generated information 401 and the second generated information 402, described later, in the memory circuit 120.

[0043] Figure 4 shows an example of the first generated information 401 according to the first embodiment. The first generated information 401 is image data in which, for example, as shown in Figure 4, the first abnormal region information 601 indicating the location, region range, and number of lesions (abnormal regions) is superimposed on two-dimensional image data 31 showing an axial cross-section of the medical image data 301 included in the first prompt 501. More specifically, the first abnormal region information 601 indicates the location and region range of the lesions (abnormal regions). In Figure 4, the number of identified lesions (abnormal regions) is one, but if multiple lesions (abnormal regions) are identified, the number of first abnormal region information 601 on the two-dimensional image data 31 will be multiple. In this embodiment, the first generated information 401 is data in which the first abnormal region information 601 is added to the medical image data 301 input to the generation model 40 as the target of analysis. Therefore, among the first generated information 401 output from the generation model 40, the medical image data 301 is the same as the medical image data 301 input to the generation model 40 as the target of analysis. In this case, the first abnormal region information 601 added to the medical image data 301 by the generation model 40 may be referred to as the first generated information in the medical image data 301.

[0044] The generation model 40 identifies a lesion from the medical image data 301 and generates two-dimensional image data 31 from the axial section specified by the first prompt 501, at the cross-sectional position where the lesion is displayed most prominently.

[0045] The first abnormal region information 601 is an example of information regarding the first abnormal region included in the first generated information 401 generated by the generation model 40. Furthermore, the display method of the first abnormal region information 601 is not limited to the example shown in Figure 4.

[0046] Furthermore, based on the instructions included in the second prompt 502, the generation model 40 generates and outputs lesion image data as second generation information 402, which shows the location and area range of the lesion in the sagittal and coronal sections of the medical image data 301. In this embodiment, the second generation information 402 is data to which second abnormal area information has been added to the medical image data 301 input to the generation model 40 as the target of analysis. Therefore, the medical image data 301 in the second generation information 402 output from the generation model 40 is the same as the medical image data 301 input to the generation model 40 as the target of analysis. In this case, the second abnormal area information added to the medical image data 301 by the generation model 40 may be referred to as the second generation information in the medical image data 301. The second abnormal area information, like the first abnormal area information 601, indicates the location and area range of the lesion (abnormal area).

[0047] Returning to Figure 1, the evaluation function 154 evaluates the relationship between the first generated information 401 and the second generated information 402. In this embodiment, the relationship between the first generated information 401 and the second generated information 402 can be expressed, for example, by the degree of similarity between the first generated information 401 and the second generated information 402. Furthermore, the evaluation function 154 evaluates the reliability of the generation model 40 higher the higher the degree of similarity between the first generated information 401 and the second generated information 402. In other words, the evaluation function 154 indirectly evaluates the reliability of the generation model 40 by evaluating the relationship between the first generated information 401 and the second generated information 402. Evaluating the reliability of the generation model 40 also means evaluating the reliability of the generated information generated by the generation model 40.

[0048] More specifically, the evaluation function 154 evaluates the relationship between the first generated information 401 and the second generated information 402 based on the location, region range, and number of the first anomaly region contained in the first generated information 401 and the second anomaly region contained in the second generated information 402.

[0049] Figure 5 shows an example of evaluation criteria for the relationship between the first generated information 401 and the second generated information 402 according to the first embodiment. As shown in Figure 5, the evaluation function 154 evaluates that the relationship between the first generated information 401 and the second generated information 402 is high if the location and area range of the lesion in the axial section of the first generated information 401 and the sagittal and coronal sections of the second generated information 402 are as close to identical as possible. In the comparison of the first generated information 401 and the second generated information 402, the location of the lesion is based on, for example, the center position or centroid position of the lesion. Since the medical image data 301 to be analyzed in this embodiment is three-dimensional volume data, the location of the lesion is a three-dimensional position within the entire medical image data 301. For this reason, the location and area range of the lesion are indicated by three-dimensional coordinates in the medical image data 301.

[0050] As described above, the first prompt 501 and the second prompt 502 differ only in the type of cross-section they specify, but the instructions regarding the medical image data 301 to be analyzed and the target to be identified from the medical image data 301 are identical. For this reason, it is desirable that the location, region range, and number of lesions identified in the first generated information 401 and the second generated information 402 be the same.

[0051] However, depending on the amount and content of the prior training of the generative model 40, the accuracy of lesion identification may differ depending on the type of cross-section being analyzed. For example, if the training data in the prior training of the generative model 40 is biased towards lesion identification results in the axial cross-section, the accuracy of lesion identification in the sagittal and coronal cross-sections may be lower than that in the axial cross-section. In such cases, fluctuations may occur in the location, region range, and number of lesions in the generative model 40's generated information depending on the type of cross-section specified in the prompt. Furthermore, even if the generative model 40 is sufficiently prior trained, the accuracy of the first generated information 401 may be low if there is insufficient medical information or specific instructions included in the first prompt 501 and the second prompt 502. In this case, fluctuations are more likely to occur in the output of the generative model 40, and the difference between the first generated information 401 and the second generated information 402 may become larger.

[0052] The evaluation function 154 evaluates the relationship between the first generated information 401 and the second generated information 402, thereby assigning a high reliability rating to generative models 40 with small fluctuations due to the type of cross-section specified by such prompts, and a low reliability rating to generative models 40 with large fluctuations.

[0053] In the example shown in Figure 5, the evaluation function 154 gives the highest rating to the relationship between the first generated information 401 and the second generated information 402 when the location and extent of the lesion (abnormal area) are the same in the axial section of the first generated information 401 and the sagittal and coronal sections of the second generated information 402. In other words, in this case, the evaluation function 154 gives the highest rating to the reliability of the generated model 40.

[0054] Furthermore, as shown in the example in Figure 3, if the second generation information 402 instructs the generation of information regarding lesions in two types of cross-sections, sagittal and coronal, then the first pattern shown in Figure 5 occurs when the location and geographical extent of both the lesions in the sagittal and coronal sections in the second generation information 402 are the same as the location and geographical extent of the lesions in the axial section in the first generation information 401.

[0055] Figure 6 shows an example of a case in which the location and area range of the lesions in the first generated information 401 and the second generated information 402 according to the first embodiment are the same. When the location, area range, and number of lesions indicated by the first abnormal area information 601 in the first generated information 401 are the same as the location, area range, and number of lesions indicated by the second abnormal area information 602a and 602b in the second generated information 402, it matches the first pattern shown in Figure 5, and therefore the evaluation function 154 rates the reliability of the generated model 40 as the highest.

[0056] Furthermore, in the example shown in Figure 5, the evaluation function 154 evaluates the relationship between the first generated information 401 and the second generated information 402 as the second highest if the location of the lesion (abnormal area) is the same in the axial section of the first generated information 401 and in the sagittal and coronal sections of the second generated information 402, and the area range of the lesion (abnormal area) overlaps.

[0057] Furthermore, in the example shown in Figure 5, the evaluation function 154 evaluates the relationship between the first generated information 401 and the second generated information 402 as the third highest if the location of the lesion (abnormal area) differs between the axial section in the first generated information 401 and the sagittal and coronal sections in the second generated information 402, and the area ranges of the lesions (abnormal areas) overlap.

[0058] Note that the number of lesions (abnormal regions) detected in the first generation information 401 may differ from the number of lesions (abnormal regions) detected in the second generation information 402. In this case, if there is an overlap in the area range of multiple lesions (abnormal regions) detected in the first generation information 401 and the area range of multiple lesions (abnormal regions) detected in the second generation information 402, it corresponds to the third pattern shown in Figure 5.

[0059] Furthermore, in cases where the second and third patterns shown in Figure 5 are applicable, the evaluation function 154 may further identify the degree of correlation by evaluating the correlation more highly the greater the overlap in the area range of the lesion (abnormal region).

[0060] Figure 7 shows an example of a case in which the location of the lesion differs between the first generated information 401 and the second generated information 402 according to the first embodiment, and the area range of the lesion overlaps.

[0061] In the example shown in Figure 7, the centroid position of the lesion indicated by the first abnormal region information 601 in the first generated information 401 is different from the centroid position of the lesion indicated by the second abnormal region information 602a and 602b in the second generated information 402. Also, in the example shown in Figure 7, the area range of the lesion indicated by the first abnormal region information 601 in the first generated information 401 encompasses the area range of the lesion indicated by the second abnormal region information 602a and 602b in the second generated information 402. Therefore, since there is an overlapping area between the area range of the lesion detected in the first generated information 401 and the area range of the lesion detected in the second generated information 402, it corresponds to the third pattern shown in Figure 5.

[0062] Furthermore, in the example shown in Figure 5, the evaluation function 154 evaluates the relationship between the first generated information 401 and the second generated information 402 as the fourth highest if the location of the lesion (abnormal area) differs between the axial section in the first generated information 401 and the sagittal and coronal sections in the second generated information 402, and there is no overlap in the area range of the lesion (abnormal area).

[0063] Figure 8 shows an example in which the location of the lesion in the first generated information 401 and the second generated information 402 according to the first embodiment is different, and the area range of the lesion does not overlap.

[0064] In the example shown in Figure 8, the centroid position of the lesion indicated by the first abnormal region information 601 in the first generated information 401 is different from the centroid position of the lesion indicated by the second abnormal region information 602a and 602b in the second generated information 402. Also, in the example shown in Figure 8, the area range of the lesion indicated by the first abnormal region information 601 in the first generated information 401 does not overlap with the area range of the lesion indicated by the second abnormal region information 602a and 602b in the second generated information 402. Therefore, the example shown in Figure 8 corresponds to the fourth pattern shown in Figure 5.

[0065] Furthermore, in the fifth example shown in Figure 5, the evaluation function 154 evaluates the correlation to be the lowest if a lesion (abnormal area) is detected in either the first generated information 401 or the second generated information 402, but not in the other.

[0066] Figure 9 shows an example where a lesion is detected in the first generated information 401 according to the first embodiment, but not in the second generated information 402. The second generated information 402 shown in Figure 9 does not include the second abnormal region information 602a, 602b. Therefore, the example shown in Figure 8 corresponds to the fifth pattern shown in Figure 5.

[0067] The relevance evaluation criteria shown in Figure 5 are merely examples and are not limited to them. For example, the evaluation function 154 may dynamically determine the evaluation criteria according to the content of the first and second prompts 501 and 502. Also, for example, if the first and second prompts 501 and 502 include commands to search for or generate similar images, the evaluation function 154 may evaluate that the relevance between the first generated information 401 and the second generated information 402 is higher if there are many identical or similar image data among the image data included in the first generated information 401 and the image data included in the second generated information 402.

[0068] Furthermore, if the first and second prompts 501 and 502 include commands instructing the output of a diagnostic name, the evaluation function 154 may evaluate that the relationship between the first generated information 401 and the second generated information 402 is high if the diagnostic names included in the first generated information 401 and the diagnostic names included in the second generated information 402 are the same or similar. Also, if the first generated information 401 and the second generated information 402 each contain multiple diagnostic names or image data, the evaluation function 154 may evaluate that the relationship between the first generated information 401 and the second generated information 402 is high if there are many identical diagnostic names or image data in the first generated information 401 and the second generated information 402.

[0069] Furthermore, the number of stages for evaluating relevance is not limited to the example shown in Figure 5. For example, the evaluation function 154 may evaluate the relationship between the first generated information 401 and the second generated information 402 in two stages: relevance present or non-relevance.

[0070] Returning to Figure 1, the display control function 155 controls the display 140 to display various information. For example, the display control function 155 displays the evaluation result of the relationship between the first generated information 401 and the second generated information 402, as determined by the evaluation function 154, on the display 140.

[0071] Figure 10 shows an example of how the evaluation results are displayed according to the first embodiment. As shown in Figure 10, the display control function 155 displays, for example, the evaluation results of the relationship between the first generated information 401 and the second generated information 402, along with the first generated information 401 and the second generated information 402, on the display 140. In the example shown in Figure 10, the relationship evaluation results are shown as text such as, "The information regarding each lesion is highly related." By displaying such evaluation results, the user can recognize that the lesions detected by the generation model 40 in the axial section based on the first prompt 501 are highly reliable.

[0072] Furthermore, if the correlation evaluation result displayed on the display 140 by the display control function 155 indicates that the correlation evaluation between the first generated information 401 and the second generated information 402 is low, the user can recognize that the lesion detected by the generation model 40 in the axial section based on the first prompt 501 is unreliable. Also, as shown in Figure 10, by displaying the first generated information 401 and the second generated information 402 together with the evaluation result, the user can grasp the location, area range, and number of lesions that can be estimated taking into account fluctuations in the generation model 40.

[0073] The method of representing the correlation evaluation results is not limited to the example shown in Figure 10. For example, the evaluation results may be shown using a graded level indicating the degree of correlation, or by numerical values.

[0074] Furthermore, the display control function 155 may display the relevance evaluation result as the reliability evaluation result of the first generated information 401. For example, the display control function 155 may display text such as "The reliability of the output generated information is high" or "The reliability of the output generated information is low" on the display 140. Also, in Figure 10, both the first generated information 401 and the second generated information 402 are displayed on the display 140, but if the information the user originally wanted was only the first generated information 401, and the second generated information 402 was only used to evaluate the first generated information 401, the display control function 155 may display only the first generated information 401 and the evaluation result on the display 140.

[0075] Furthermore, the display control function 155 may display the correlation evaluation results together with the medical image data 301 to be analyzed, which are included in the first prompt 501 and the second prompt 502, and the first generated information 401 and the second generated information 402.

[0076] Displaying on the display 140 is an example of output in this embodiment. The first information processing device 100 may employ output methods other than displaying on the display 140. For example, the first information processing device 100 may include a transmission function, as an example of an output unit, for transmitting evaluation results to other information processing devices via the NW interface 110.

[0077] Furthermore, the display control function 156 displays a prompt input screen on the display 140 in which the user can input the first and second prompts 501 and 502. The prompt input screen may include a function to restrict the input of the first and second prompts 501 and 502 to a predetermined format so that the first and second prompts 501 and 502 include medical image data 301 when the user generates the first and second prompts 501 and 502. The function to restrict the input of the first and second prompts 501 and 502 to a predetermined format may include, but is not limited to, an input check function that requires the user to specify the path to the memory area where the medical image data 301 is stored as a reference on the prompt input screen, or templating the input format of the first and second prompts 501 and 502.

[0078] Next, we will explain the flow of the evaluation process performed by the first information processing device 100 configured as described above.

[0079] Figure 11 is a flowchart showing an example of the evaluation process flow according to the first embodiment. The process in this flowchart is executed, for example, when the reception function 151 receives an operation from the user to open the prompt input screen.

[0080] First, the display control function 155 displays a prompt input screen on the display 140 (S1).

[0081] Then, the prompt acquisition function 152 acquires the first prompt 501 entered by the user (S2).

[0082] Then, the generation information acquisition function 153 inputs the first prompt 501 to the generation model 40 (S3). In this case, based on the input first prompt 501, the generation model 40 generates the first generation information 401.

[0083] Then, the generation information acquisition function 153 acquires the first generation information 401 output from the generation model 40 (S4).

[0084] Next, the prompt acquisition function 152 acquires the second prompt 502 entered by the user (S5).

[0085] Then, the generation information acquisition function 153 inputs a second prompt 502 to the generation model 40 (S6). In this case, based on the input second prompt 502, the generation model 40 generates second generation information 402.

[0086] Then, the generation information acquisition function 153 acquires the second generation information 402 output from the generation model 40 (S7).

[0087] Then, the evaluation function 154 compares the first generated information 401 and the second generated information 402 output from the generative model 40 and evaluates their relationship (S8).

[0088] Then, the evaluation function 154 confirms the evaluation result if the correlation in the evaluation result is above the standard (S9 "Yes"). Once the correlation evaluation result is confirmed, the display control function 155 displays the correlation evaluation result of the first generated information 401 and the second generated information 402 by the evaluation function 154 on the display 140 (S10).

[0089] The criteria for confirming the correlation evaluation may be, for example, whether or not it is at least as high as the third pattern from the top in Figure 5, "different locations, overlapping areas." In other words, the evaluation function 154 confirms the evaluation result obtained in S8 if the correlation between the first generated information 401 and the second generated information 402 is higher than the case where "the location of the lesion in the first generated information 401 and the second generated information 402 are different, and the area range of the lesions overlaps." Note that the criteria for confirming the correlation are not limited to this example.

[0090] Furthermore, if the correlation in the evaluation result is below the standard (S9 "No"), the evaluation function 154 regenerates the first generated information 401 a specified number of times and re-evaluates the first generated information 401 and the second generated information 402. This is because, for example, if the output result of the generation model 40 fluctuates greatly, the output result may differ even if the same prompt is entered.

[0091] In this case, the generation information acquisition function 153, while the number of loops has not reached the specified number (S11 "No"), inputs the first prompt 501 to the generation model 40 again (S12) and acquires the first generation information 401 output from the generation model 40 (S13). The specified number, which is the upper limit of the number of loops, is, for example, 3 times, but is not limited to this.

[0092] Then, returning to S8, the evaluation function 154 compares the regenerated first generated information 401 with the second generated information 402 and evaluates the relationship.

[0093] Then, the process proceeds to S9, where the evaluation function 154 confirms the evaluation result if the relationship between the regenerated first generated information 401 and the second generated information 402 in the evaluation result is above a certain threshold. The evaluation function 154 may also take into account the number of loops required until the relationship is above a certain threshold in its evaluation. For example, the evaluation function 154 may subtract from the evaluation of the relationship if the number of loops required until the relationship is above a certain threshold is large.

[0094] Furthermore, if the number of loops reaches a predetermined number (S11 "Yes"), the evaluation function 154 confirms the latest evaluation result. Then, the process proceeds to S10, where the display control function 155 displays the evaluation result of the relationship between the first generated information 401 and the second generated information 402, as determined by the evaluation function 154, on the display 140. At this point, the processing of this flowchart ends.

[0095] Note that the processes in S9 and S11-13 are not mandatory, and the evaluation function 154 may determine the relationship evaluation with a single relationship evaluation. Also, in the retry process in S11-13, not only the first generated information 401 but also the second generated information 402 may be regenerated.

[0096] Thus, the first information processing device 100 of this embodiment evaluates the relationship between the first generated information 401 obtained by inputting the first prompt 501 into the generation model 40 and the second generated information 402 obtained by inputting the second prompt 502, which is similar to the first prompt 501, into the generation model 40. In other words, according to the first information processing device 100 of this embodiment, the reliability of the first generated information 401 obtained by the first prompt 501 can be verified by comparing it with the second generated information 402 obtained by the second prompt 502, which is similar to the first prompt 501. For this reason, according to the first information processing device 100 of this embodiment, the reliability of the first generated information 401 generated by the generation model 40 can be evaluated.

[0097] Furthermore, the first information processing device 100 of this embodiment outputs an evaluation result of the relationship between the first generated information 401 and the second generated information 402. Therefore, the first information processing device 100 of this embodiment allows the user to understand the reliability of the first generated information 401.

[0098] Furthermore, the first information processing device 100 of this embodiment outputs the evaluation result of the relationship between the first generated information 401 and the second generated information 402 together with the first generated information 401 and the second generated information 402. Therefore, according to the first information processing device 100 of this embodiment, the user can confirm the difference between the first generated information 401 and the second generated information 402 along with the evaluation result.

[0099] Furthermore, the first information processing device 100 of this embodiment may output the evaluation result of the relationship between the first generated information 401 and the second generated information 402 together with the medical image data 301 to be analyzed included in the first prompt 501 and the second prompt 502, and the first generated information 401 and the second generated information 402 in the medical image data 301. With this configuration, when the first prompt 501 and the second prompt 502 include a command to analyze the medical image data 301, as in this embodiment, the user can easily confirm the medical image data 301 before it is processed by the generation model 40 together with the first generated information 401 and the second generated information 402, thereby improving the convenience of the user's confirmation work.

[0100] Furthermore, in this embodiment, the first prompt 501 and the second prompt 502 include at least one of medical image data and medical text data. Also, the first prompt 501 and the second prompt 502 have at least some differences, but are identical except for the differences. The first information processing device 100 of this embodiment evaluates the degree of relevance between the first generated information 401 and the second generated information 402 by the degree of similarity between the first generated information 401 and the second generated information 402. That is, the first information processing device 100 of this embodiment evaluates the degree of relevance between the first generated information 401 and the second generated information 402 by the degree of similarity between the first generated information 401 and the second generated information 402 obtained from similar first prompts 501 and second prompts 502. For this reason, the first information processing device 100 of this embodiment can verify whether the generation model 40 is in a stable state with little fluctuation due to superficial differences in prompts, achieved through appropriate prior learning.

[0101] (Second embodiment) In the first embodiment described above, both the first prompt 501 and the second prompt 502 were entered by the user. In contrast, in this second embodiment, the first information processing device 100 automatically generates part or all of the second prompt 502.

[0102] The information processing system S of this embodiment includes, for example, a first information processing device 100, a second information processing device 200, and a medical information system 300, similar to the first embodiment described in Figure 1. Furthermore, the first information processing device 100 of this embodiment includes, similar to the first embodiment, an NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0103] The processing circuit 150 of this embodiment includes a reception function 151, a prompt acquisition function 152, a generated information acquisition function 153, an evaluation function 154, and a display control function 155, similar to the first embodiment.

[0104] The reception function 151, the generated information acquisition function 153, the evaluation function 154, and the display control function 155 have the same functions as in the first embodiment.

[0105] In addition to the functions of the first embodiment, the prompt acquisition function 152 of this embodiment generates a second prompt 502 similar to the first prompt 501 based on the first prompt 501 entered by the user.

[0106] For example, if the first prompt 501 includes an instruction specifying a particular type of cross-section, such as "axial section," as shown in the example in Figure 2, the prompt acquisition function 152 generates the second prompt 502 by replacing the type of cross-section with another type of cross-section. Alternatively, the prompt acquisition function 152 may generate the second prompt 502 by replacing the wording of the text data included in the first prompt 501 with other medically equivalent wording, regardless of the type of cross-section.

[0107] The prompt acquisition function 152 may, for example, replace a part of the first prompt 501 based on predetermined replacement rules. The predetermined replacement rules may be, for example, data that associates the word to be replaced with the word to be replaced, and may be stored in the memory circuit 120. If the word to be replaced (for example, "axial section") is included in the first prompt 501, the prompt acquisition function 152 replaces the word with the word to be replaced as defined in the rules (for example, "sagittal section" or "coronatal section").

[0108] Alternatively, the prompt acquisition function 152 may acquire a second prompt 502 by inputting the first prompt 501 to a different generative model than the generative model 40 being evaluated, thereby replacing part of the first prompt 501 with other words. The other generative model may be, for example, an LLM capable of processing text data. The other generative model may also be stored in the memory circuit 120 of the first information processing device 100 or the second information processing device 200, or it may be stored in another information processing device.

[0109] Except for the process of generating the second prompt 502 by the prompt acquisition function 152, the processing performed by the first information processing device 100 in this embodiment is the same as in the first embodiment.

[0110] Thus, the first information processing device 100 of this embodiment can reduce the effort required for the user to create the second prompt 502 for evaluation processing by automatically generating a second prompt 502 that is similar to the first prompt 501. Furthermore, by having the first information processing device 100 automatically generate the second prompt 502, it is possible to reduce the occurrence of unintended errors caused by the user manually generating the second prompt 502, as well as the occurrence of differences between the second prompt 502 and the first prompt 501 that exceed the range of similarity.

[0111] Furthermore, the prompt acquisition function 152 does not necessarily have to automatically generate all of the second prompt 502, and may have a function to check (proofread) the content of the second prompt 502 entered by the user, or a recommendation function for when the second prompt 502 is entered.

[0112] For example, the prompt acquisition function 152 may determine whether the second prompt 502 entered by the user has a difference that exceeds the range of similarity with the first prompt 501. A difference that exceeds the range of similarity with the first prompt 502 means that the second prompt 502 and the first prompt 501 are not medically equivalent. For example, if the first prompt 501 instructs the system to output "information about lesions," and the second prompt 502 instructs the system to output "information about thrombosis," then these are not medically equivalent. If the second prompt 502 has a difference that exceeds the range of similarity with the first prompt 501, it is natural that the first generated information 401 and the second generated information 402 output by the generation model 40 will not be similar; therefore, such a second prompt 502 is not suitable for evaluation purposes. Therefore, if the prompt acquisition function 152 determines that the second prompt 502 has a difference from the first prompt 501 that exceeds the range of similarity, the display control function 155 may display the determination result on the display 140.

[0113] Furthermore, the prompt acquisition function 152 may generate candidates for the second prompt 502 before the user inputs the second prompt 502. In this case, the display control function 155 may display the candidates for the second prompt 502 generated by the prompt acquisition function 152 as recommendations on the display 140.

[0114] (Third embodiment) In the first embodiment described above, the first information processing device 100 would regenerate the first generated information 401 and repeat the evaluation if the relationship between the first generated information 401 and the second generated information 402 was below a certain standard. In contrast, in this third embodiment, the first information processing device 100 generates a new prompt if the relationship between the first generated information 401 and the second generated information 402 is below a certain standard, and performs the evaluation again based on the new prompt.

[0115] The information processing system S of this embodiment includes, for example, a first information processing device 100, a second information processing device 200, and a medical information system 300, similar to the first embodiment described in Figure 1. Furthermore, the first information processing device 100 of this embodiment includes, similar to the first embodiment, an NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0116] The processing circuit 150 of this embodiment includes a reception function 151, a prompt acquisition function 152, a generated information acquisition function 153, an evaluation function 154, and a display control function 155, similar to the first embodiment.

[0117] The reception function 151, the generated information acquisition function 153, and the display control function 155 have the same functions as in the first embodiment.

[0118] Figure 12 is a flowchart showing an example of the evaluation process flow according to the third embodiment. From the display of the prompt input screen in S1 of Figure 12 to the determination in S9 whether the correlation is above the standard, and the display of the evaluation result in S10, it is the same as in the first embodiment.

[0119] Furthermore, if the correlation in the evaluation result is below a certain threshold, the evaluation function 154 of this embodiment repeatedly performs the evaluation of the first generated information 401 with the new generated information a predetermined number of times. The predetermined number of loops in Figure 12 is, for example, 10 times, but is not limited to this.

[0120] The prompt acquisition function 152 of this embodiment acquires a new prompt (S102) as long as the number of loops has not reached a predetermined number (S101 "No"). The method for acquiring a new prompt may be, for example, automatically generated by the prompt acquisition function 152, similar to the method for generating the second prompt 502 in the second embodiment described above. Alternatively, the display control function 155 may display an instruction on the display 140 to prompt the user to input a new prompt, and the prompt acquisition function 152 may acquire the new prompt entered by the user.

[0121] A new prompt is, for example, a third prompt, similar to the first prompt 501. More specifically, the new prompt is medically synonymous with the first prompt 501. Furthermore, the new prompt is different from other prompts created before it (e.g., the second prompt). As a concrete example, if the first and second prompts 501 and 502 include "axial section," "sagittal section," and "cornal section," the prompt acquisition function 152 generates a third prompt specifying a section in a different direction from the cutting direction of these three sections (e.g., "a section tilted n degrees in the Z-axis direction of the subject").

[0122] Then, the generation information acquisition function 153 inputs the new prompt acquired in S102 to the generation model 40 (S103).

[0123] Then, the generation information acquisition function 153 acquires the new generation information output from the generation model 40 (S104).

[0124] The evaluation function 154 then evaluates the relationship between the first generated information 401 and the new generated information generated in S104 (S105). If the relationship does not meet the criteria, the processes in S9, S101 to S105 are repeatedly executed until the number of loops reaches a predetermined number. In this case, the prompt acquisition function 152 acquires a new prompt each time the loop is completed.

[0125] Furthermore, if the relevance exceeds the threshold during the evaluation process using the newly generated information, the process proceeds to S10. The evaluation function 154 may also take into account the number of loops required to reach the threshold for relevance. For example, the evaluation function 154 may subtract from the relevance evaluation if the number of loops required to reach the threshold is high.

[0126] Furthermore, if the number of loops reaches a predetermined number (S101 "Yes"), the evaluation function 154 confirms the latest evaluation result. Then, the process proceeds to S10, where the display control function 155 displays the evaluation result of the relationship between the first generated information 401 and the second generated information 402, as determined by the evaluation function 154, on the display 140. At this point, the processing of this flowchart ends.

[0127] Thus, if the correlation in the evaluation result is below a certain threshold, the first information processing device 100 of this embodiment obtains a new prompt and evaluates the correlation between the new generated information based on the new prompt and the first generated information 401. As a result, the first information processing device 100 of this embodiment can improve the accuracy of the evaluation result by performing multiple evaluations using diverse generated information.

[0128] (Fourth embodiment) In the first to third embodiments described above, the first information processing device 100 used a first prompt 501 and a second prompt 502 similar to the first prompt 501 to evaluate the generated information. In this third embodiment, however, the second prompt 502, which is identical to the first prompt 501, is used to evaluate the generated information.

[0129] The information processing system S of this embodiment includes, for example, a first information processing device 100, a second information processing device 200, and a medical information system 300, similar to the first embodiment described in Figure 1. Furthermore, the first information processing device 100 of this embodiment includes, similar to the first embodiment, an NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0130] The processing circuit 150 of this embodiment includes a reception function 151, a prompt acquisition function 152, a generated information acquisition function 153, an evaluation function 154, and a display control function 155, similar to the first embodiment.

[0131] The reception function 151, the generated information acquisition function 153, the evaluation function 154, and the display control function 155 have the same functions as in the first embodiment.

[0132] The prompt acquisition function 152 of this embodiment acquires a first prompt 501 and a second prompt 502 that is identical to the first prompt 501. For example, the first prompt 501 is entered by the user. The prompt acquisition function 152 acquires the second prompt 502 by duplicating the first prompt 501.

[0133] Except for the process of obtaining a copy of the second prompt 502 by the prompt acquisition function 152, the processing performed by the first information processing device 100 in this embodiment is the same as in the first embodiment.

[0134] Thus, the first information processing device 100 of this embodiment evaluates the relationship between the first generated information 401 obtained by inputting the first prompt 501 to the generation model 40 and the second generated information 402 obtained by inputting the same second prompt 502 as the first prompt 501 to the generation model 40, and outputs the evaluation result. In other words, according to the first information processing device 100 of this embodiment, in addition to the same effects as the first embodiment, it is possible to verify fluctuations in the output of the generation model 40 when the same prompt is input.

[0135] (Fifth embodiment) In the first to fourth embodiments described above, the first information processing device 100 had the same medical image data 301 included in the first prompt 501 and the second prompt 502. In this fifth embodiment, the first prompt 501 and the second prompt 502 include three-dimensional first medical image data composed of different stack images.

[0136] The information processing system S of this embodiment includes, for example, a first information processing device 100, a second information processing device 200, and a medical information system 300, similar to the first embodiment described in Figure 1. Furthermore, the first information processing device 100 of this embodiment includes, similar to the first embodiment, an NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0137] The processing circuit 150 of this embodiment includes a reception function 151, a prompt acquisition function 152, a generated information acquisition function 153, an evaluation function 154, and a display control function 155, similar to the first embodiment.

[0138] Figure 13 shows an example of a first prompt 501a and first generated information 401a according to a fifth embodiment. As shown in Figure 13, the first prompt 501a includes a three-dimensional first medical image data 301a composed of a plurality of first stacked image data. The plurality of first stacked image data are, for example, two-dimensional image data of a plurality of axial cross-sections. An axial cross-section is an example of a cross-section in a first direction.

[0139] Furthermore, the first prompt 501a includes a command to cause the generation model 40 to generate information about the first abnormal region (lesion) from the first medical image data 301a as the first generated information 401a, such as "Generate and output information about the lesion based on the subject's medical image data."

[0140] Figure 14 also shows an example of a second prompt 502a and second generated information 402a according to the fifth embodiment. As shown in Figure 14, the second prompt 502a includes a three-dimensional second medical image data 302a composed of a plurality of second stacked image data. The plurality of second stacked image data are two-dimensional image data of a different type of cross-section than the plurality of first stacked image data, for example, two-dimensional image data of a plurality of coronal cross-sections. A coronal cross-section is an example of a cross-section in a second direction.

[0141] The three-dimensional object captured in the first medical image data 301a and the three-dimensional object captured in the second medical image data 302a are identical. In other words, the first medical image data 301a and the second medical image data 302a are medically equivalent image data, except that the cross-sectional direction of the stacked image data that constitutes them is different.

[0142] Furthermore, as shown in Figure 14, the second prompt 502a contains the same text data as the first prompt 501a, for example, "Generate and output information about the lesion based on the subject's medical image data." Therefore, the first prompt 501a and the second prompt 502a are essentially synonymous from a medical standpoint.

[0143] The evaluation process flow in this embodiment is the same as that of the first embodiment described in Figure 11.

[0144] Thus, the first information processing device 100 of this embodiment evaluates the relationship between first generated information 401a obtained by inputting a first prompt 501a, which includes a three-dimensional first medical image data 301a composed of a plurality of first stacked image data, into the generation model 40, and second generated information 402a obtained by inputting a second prompt 502a, which includes a three-dimensional second medical image data 302a composed of a plurality of second stacked image data different from the plurality of first stacked image data, into the generation model 40, and outputs the evaluation result. In other words, according to the first information processing device 100 of this embodiment, in addition to the same effects as the first embodiment, it is possible to verify fluctuations in the first and second generated information 401a and 402a due to differences in the types of stacked image data.

[0145] (Sixth embodiment) In this sixth embodiment, the image conditions of the medical image data included in the first prompt and the second prompt are different.

[0146] The information processing system S of this embodiment includes, for example, a first information processing device 100, a second information processing device 200, and a medical information system 300, similar to the first embodiment described in Figure 1. Furthermore, the first information processing device 100 of this embodiment includes, similar to the first embodiment, an NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0147] The processing circuit 150 of this embodiment includes a reception function 151, a prompt acquisition function 152, a generated information acquisition function 153, an evaluation function 154, and a display control function 155, similar to the first embodiment.

[0148] Figure 15 shows an example of a first prompt 501b and first generated information 401b according to the sixth embodiment. As shown in Figure 15, the first prompt 501b includes first medical image data 301b generated by first image conditions. The first prompt 501b also includes text data that instructs the generation model 40 to generate and output first generated information based on the first medical image data 301b, for example, "Generate and output information about lesions based on the subject's medical image data." When distinguishing the first medical image data 301b from the first medical image data in other embodiments, it may be referred to as the third medical image data.

[0149] Figure 16 shows an example of the second prompt 502b and the second generated information 402b according to the sixth embodiment. As shown in Figure 16, the second prompt 502b includes the second medical image data 302b generated by the second image conditions. The second prompt 502b also includes the same text data as the first prompt 501b. When distinguishing the second medical image data 302b from the second medical image data in other embodiments, it may be referred to as the fourth medical image data.

[0150] The second image condition is different from the first image condition. Furthermore, both the first and second image conditions include at least one of the imaging conditions and the image processing conditions. Specifically, the first and second image conditions include, for example, reconstruction conditions, back projection, and iterative approximation conditions. Also, the first and second image conditions include, for example, contrast-enhanced / non-contrast conditions during imaging. Furthermore, the first and second image conditions may define, for example, the sequence during acquisition of magnetic resonance image data.

[0151] Specifically, if the first medical image data 301b and the second medical image data 302b are CT image data, for example, the first image conditions may be "contrast-enhanced, slice thickness 5 mm, kernel FC51, phase 30%", and the second image conditions may be "non-contrast-enhanced, slice thickness 0.5 mm, kernel FC30, phase 70%", etc. Note that this example is provided for illustrative purposes only, and the first and second image conditions are not limited to this example.

[0152] Furthermore, the first medical image data 301b and the second medical image data 302b are image data obtained from the same subject and the same imaging area. In other words, the first medical image data 301b and the second medical image data 302b are identical except for the image conditions during generation.

[0153] Note that the text data included in the first and second prompts 501b and 502b are not limited to the examples shown in Figures 15 and 16. For example, the text data may instruct the generation of at least one of the following: an abnormal region (including a lesion), a similar image, or a diagnostic name. In this case, the first generation information 401b and the second generation information 402b in this embodiment include at least one of the following: an abnormal region in the first medical image data 301b and the second medical image data 302b, similar image data for the first medical image data 301b and the second medical image data 302b, or a diagnostic name based on the first medical image data 301b and the second medical image data 302b.

[0154] The evaluation process flow in this embodiment is the same as that of the first embodiment described in Figure 11.

[0155] Thus, the first information processing device 100 of this embodiment evaluates the relationship between the first generated information 401b obtained by inputting a first prompt 501b containing first medical image data 301b generated under first image conditions into the generation model 40, and the second generated information 402b obtained by inputting a second prompt 502b containing second medical image data 302b generated under second image conditions different from the first image conditions into the generation model 40, and outputs the evaluation result. Therefore, according to the first information processing device 100 of this embodiment, in addition to the same effects as the first embodiment, it is possible to verify fluctuations in the output of the generation model 40 due to differences in the image conditions of the first and second medical image data 301b and 302b contained in the first and second prompts 501b and 502b.

[0156] For example, if the training data used in the prior training of the generative model 40 is biased towards image data generated under specific image conditions, the accuracy of the generated information for prompts containing image data generated under conditions other than those specific may be low. In such cases, the bias in the accuracy of the generative model 40 can be evaluated by applying the first information processing device 100 of this embodiment.

[0157] Furthermore, the content and scope of the image conditions are not limited to the examples described above. For example, the image conditions may be imposed on medical image data generated by the generative model 40 based on the first and second prompts 501b and 502b.

[0158] For example, the first prompt 501b includes a command to generate and output ES (systolic) phase image data from three-dimensional medical image data 301 obtained by imaging the subject's chest for one cardiac cycle. The second prompt 502b includes a command to generate and output ED (diastolic) phase image data from three-dimensional medical image data 301 obtained by imaging the subject's chest for one cardiac cycle. In this case, ES phase and ED phase are examples of image conditions. In this case, since the only difference between the first generated information 401b and the second generated information 402b is the phase, if the fluctuations in the generation model 40 are small, the image data included in the first generated information 401b and the image data included in the second generated information 402b will be identical or very similar in areas other than the heart. For example, the evaluation function 154 may distinguish the heart or lungs, which change shape significantly depending on the phase, as the area of ​​focus, and other organs or bones, which change shape less depending on the phase, as the area of ​​non-focus. The evaluation function 154 may evaluate the relationship between the first generated information 401b and the second generated information 402b more highly if the similarity between the non-focus regions in the image data contained in the first generated information 401b and the image data contained in the second generated information 402b is high.

[0159] Furthermore, the first prompt 501b may include a command to generate and output an ED phase image based on ES phase medical image data. In this case, the corresponding second prompt 502b may include a command to generate and output an ES phase image based on ED phase medical image data. In this way, the evaluation function 154 may compare the first generated information 401b and the second generated information 402b, which are generated based on the first and second prompts 501b and 502b with other conditions equalized, treating the difference in image conditions as a difference in phase.

[0160] (Seventh Embodiment) In the first to sixth embodiments described above, the first and second prompts 501, 501a, 501b, 502, 502a, and 502b included medical image data, but each prompt does not necessarily have to include medical image data. In this seventh embodiment, the first and second prompts do not include medical image data, but instead include medical information in text data format.

[0161] The information processing system S of this embodiment includes, for example, a first information processing device 100, a second information processing device 200, and a medical information system 300, similar to the first embodiment described in Figure 1. Furthermore, the first information processing device 100 of this embodiment includes, similar to the first embodiment, an NW interface 110, a storage circuit 120, an input interface 130, a display 140, and a processing circuit 150.

[0162] The processing circuit 150 of this embodiment includes a reception function 151, a prompt acquisition function 152, a generated information acquisition function 153, an evaluation function 154, and a display control function 155, similar to the first embodiment. The reception function 151 and the display control function 155 have the same functions as in the first embodiment.

[0163] Figure 17 shows an example of a first prompt 501c and first generated information 401c according to the seventh embodiment. As shown in Figure 17, the first prompt 501c does not include image data. The first prompt 501c also includes text data indicating an instruction to the generation model 40, "Estimate possible diagnostic names based on the following medical information," and medical information in text data format, "There is an abnormal shadow in the left lower lobe of the lung on the CT image, which is circular and includes a high-attenuation area. The left lung is normal." This medical information is obtained, for example, from a medical information system 300. Alternatively, this medical information may be directly written by the user in the body of the first prompt. The medical information included in the first prompt 501c is an example of first medical information.

[0164] In the example shown in Figure 17, the generation model 40 receives the input of the first prompt 501c and outputs the diagnosis name "lung cancer" as the first generated information 401c.

[0165] Figure 18 shows an example of the second prompt 502c and the second generated information 402c according to the seventh embodiment. Similar to the first prompt 501c, the second prompt 502c does not include image data. The second prompt 502c also includes text data of the same instruction as the instruction included in the first prompt 501c, which is "Estimate possible diagnostic names based on the following medical information."

[0166] Furthermore, the second prompt 502c contains medical information in text format, which reads, "A shadow containing an oval-shaped calcification-like area is present in the left lung S10 region on the CT image. No findings in the left lung." This medical information is obtained by replacing some of the wording of the first medical information contained in the first prompt 501c with other similar wording. The medical information contained in the second prompt 502c is an example of the second medical information.

[0167] Similarities in wording between the first and second medical information include, for example, cases where the original wording is a subordinate or superordinate concept of the replacement wording, or where the relationship between the original and replacement wording is an expression of degree (high / low, large / small) or a correspondence between an adjective and a numerical value. The first and second medical information are considered to be medically synonymous despite differences in expression.

[0168] For example, in the examples shown in Figures 17 and 18, the words "left lower lobe," "circular," and "high-attenuation area" in the first medical information included in the first prompt 501c are replaced in the second medical information included in the second prompt 502c with more specific sub-conceptual words such as "left lung S10 area," "elliptical," and "calcification-like area."

[0169] Figure 19 also shows another example of the generation of first generated information 401d based on the first prompt 501d according to the seventh embodiment. In the example shown in Figure 19, the first prompt 501d includes text data indicating an instruction to the generating model 40, "Estimate candidate drugs to be prescribed based on the following medical information. Also, estimate candidate additional tests to be performed," and first medical information in text data format, "Cough has persisted for a week, slight fever present, high CRP level, high γGTP level, no abnormalities on X-ray imaging."

[0170] Figure 20 also shows another example of the generation of second generated information 402d based on the second prompt 502d according to the seventh embodiment. The second prompt 502d shown in Figure 20 is similar to the first prompt 501d shown in Figure 19. Specifically, the second prompt 502d shown in Figure 20 includes text data indicating an instruction to the same generation model 40 as the first prompt 501d, and second medical information in text data format, such as "I have had a cough for a week, and have also had a fever of 37.2 to 38.0 degrees, CRP: 1.0, γGTP: 120, and no abnormalities on chest X-ray." The second medical information shown in Figure 20 replaces the expressions representing the degree in the first medical information shown in Figure 19 with specific numerical values.

[0171] The prompt acquisition function 152 of this embodiment acquires the first prompts 501c, 501d and the second prompts 502c, 502d. The method for acquiring the first prompts 501c, 501d and the second prompts 502c, 502d may be by user input, as in the first embodiment. Alternatively, as in the second embodiment, only the first prompts 501c, 501d may be generated by user input, and the prompt acquisition function 152 may generate the second prompts 502c, 502d based on the first prompts 501c, 501d.

[0172] Furthermore, if both the first prompts 501c, 501d and the second prompts 502c, 502d are generated by user input, the prompt acquisition function 152 may generate the parts of the first medical information text included in the first prompts 501c, 501d that are to be replaced, and candidate replacement texts. In this case, the display control function 155 may display the parts to be replaced and candidate replacement texts on the display 140 to suggest the replacement to the user.

[0173] Furthermore, the generation information acquisition function 153 of this embodiment, similar to the first embodiment, inputs the first prompts 501c, 501d and the second prompts 502c, 502d acquired by the prompt acquisition function 152 to the generation model 40 of the second information processing device 200 via the network 400 and the NW interface 110, and acquires the first generation information 401c, 401d and the second generation information 402c, 402d from the generation model 40.

[0174] Since the first prompts 501c, 501d and the second prompts 502c, 502d are medically equivalent, if the generative model 40 has performed appropriate training, the first generated information 401c, 401d and the second generated information 402c, 402d will be identical or similar. In the example shown in Figure 18, the second generated information 402c is the same diagnosis, "lung cancer," as the first generated information 401c.

[0175] However, if there is a bias in the prior training of the generative model 40, for example, differences may arise between the first generated information 401c, 401d and the second generated information 402c, 402d due to superficial differences in wording between the first prompts 501c, 501d and the second prompts 502c, 502d. In the example shown in Figures 19 and 20, the first generated information 401d is "Prescription: AA, Test: BB", while the second generated information 402d is "Prescription: AA, Test: CC", showing some differences.

[0176] The evaluation function 154 of this embodiment evaluates the degree of relevance between the first generated information 401c,401d and the second generated information 402c,402d based on the degree of similarity between the first generated information 401c,401d and the second generated information 402c,402d, similar to the first embodiment. In other words, the higher the similarity between the first generated information 401c,401d and the second generated information 402c,402d, the higher the reliability of the generation model 40 that the evaluation function 154 evaluates.

[0177] For example, as shown in Figures 17-20, if the first and second prompts 501c, 501d, 502c, and 502d include commands that instruct the output of a diagnosis, prescription, or test item, the evaluation function 154 may evaluate that the relationship between the first generated information 401c, 401d and the second generated information 402c, 402d is high if the diagnosis contained in the first generated information 401c, 401d and the diagnosis contained in the second generated information 402c, 402d are the same or similar. Furthermore, if the first generated information 401c,401d and the second generated information 402c,402d each contain multiple diagnostic names, the evaluation function 154 may evaluate that the greater the number of identical diagnostic names in the first generated information 401c,401d and the second generated information 402c,402d, the higher the correlation between the first generated information 401c,401d and the second generated information 402c,402d.

[0178] Thus, in addition to the same effects as the first embodiment, the first information processing device 100 of this embodiment can evaluate the relationship between the first generated information 401c, 401d and the second generated information 402c, 402d, even when the first and second prompts 501c, 501d, 502c, 502d include medical information in text data format.

[0179] (Variation 1) In the embodiments described above, the information processing system S is provided, for example, in a medical institution such as a hospital, but it may also be provided in a company or other organization other than a medical institution. Furthermore, although the embodiments described above describe examples of using the information processing system S in the medical field, the fields of use of the information processing system S are not limited to the medical field, and it can be used in various businesses and research and development projects that utilize generational AI. Note that when the information processing system S is used in a field other than the medical field, it is not necessary to include the medical information system 300. For example, the information processing system S may include the first information processing device 100 and the generation model 40. In addition, the information processing system S may include other database management devices, etc., instead of the medical information system 300, or in addition to the medical information system 300.

[0180] (Modification 2) In the embodiments described above, as shown in Figure 1 and other figures, an example was described in which the first information processing device 100, the second information processing device 200, and the medical information system 300 are connected via a network 400 such as an in-hospital LAN. However, the configuration of the information processing system S is not limited to this. For example, the second information processing device 200 may be located outside the medical institution. As a specific example, the second information processing device 200 may be a server of a business operator that provides generated AI as a service. In this case, the second information processing device 200 may be connected to the first information processing device 100 and the medical information system 300 via the internet or the like, instead of the in-hospital LAN.

[0181] (Variation 3) The generation model 40 may be provided in the first information processing device 100, rather than the second information processing device 200. For example, the generation model 40 may be stored in the memory circuit 120 of the first information processing device 100 and used by the generation information acquisition function 153 described above. Alternatively, the generation information acquisition function 153 may be configured to include the generation model 40.

[0182] (Modification 4) In the embodiments described above, the generative model 40 was assumed to be a multimodal model, but the technology constituting the generative model 40 is not limited to this, and various AI models can be employed.

[0183] (Variation 5) In the embodiments described above, the first information processing device 100 outputs the evaluation results by displaying them on the display 140 or by other means. However, the first information processing device 100 does not necessarily have to output the evaluation results externally. For example, the storage of the evaluation results by the evaluation function 154 of the first information processing device 100 in the memory circuit 120 or the like may also be referred to as "output."

[0184] Alternatively, the first information processing device 100 may use the evaluation results only for internal processing. For example, if the evaluation result is below a specified standard, the first information processing device 100 may delete the first generated information 401. In this case, the evaluation result does not necessarily have to be output.

[0185] The various types of data discussed in this specification are typically digital data.

[0186] According to at least one embodiment described above, the reliability of the generated information generated by the generative model can be evaluated.

[0187] While several embodiments have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These embodiments can be implemented in a variety of other forms, and various omissions, substitutions, modifications, and combinations of embodiments are possible without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0188] 31 2D image data 40 Generative Models 100 First Information Processing Device 110 NW Interfaces 120 Memory circuit 130 Input Interfaces 140 displays 150 Processing Circuits 151 Reception function 152 Prompt acquisition function 153 Generation information acquisition function 154 Rating function 155 Display control function 200 Second Information Processing Device 300 Medical Information Systems 301 Medical image data 301a, 301b First medical image data 302a, 302b Second medical image data 400 Networks 401, 401a, 401b, 401c, 401d First generation information 402, 402a, 402b, 402c, 402d Second generation information 501, 501a, 501b, 501c, 501d First prompt 502, 502a, 502b, 502c, 502d Second prompt 601 First Anomaly Area Information 602a, 602b Second abnormal region information S Information Processing System

Claims

1. A prompt acquisition unit that acquires a first prompt containing medical information and a second prompt similar to the first prompt, A generation information acquisition unit inputs the first prompt into a generation model and obtains first generation information corresponding to the first prompt from the generation model, and inputs the second prompt into the generation model and obtains second generation information corresponding to the second prompt from the generation model, An evaluation unit that evaluates the relationship between the first generated information and the second generated information, An information processing device equipped with the following features.

2. The system further includes an output unit that outputs the evaluation result of the relationship between the first generated information and the second generated information by the evaluation unit, The information processing apparatus according to claim 1.

3. The output unit outputs the evaluation result of the relationship between the first generated information and the second generated information together with the first generated information and the second generated information. The information processing apparatus according to claim 2.

4. The first prompt and the second prompt include at least one of medical image data and medical text data. The output unit outputs the evaluation result of the relationship between the first generated information and the second generated information together with the medical image data, the first generated information in the medical image data, and the second generated information in the medical image data. The information processing apparatus according to claim 2.

5. The first prompt and the second prompt include at least one of medical image data and medical text data. The first prompt and the second prompt differ in at least part, and are identical except for the differences. The evaluation unit evaluates the degree of relationship between the first generated information and the second generated information based on the degree of similarity between the first generated information and the second generated information. The information processing apparatus according to claim 1.

6. The first prompt includes an instruction to cause the generation model to generate information regarding a first abnormal region in a first cross-section of the medical image data as the first generated information, The second prompt includes an instruction to cause the generation model to generate information regarding a second abnormal region in a second cross-section of the medical image data that is different from the first cross-section, as the second generated information. The evaluation unit evaluates the relationship between the first generated information and the second generated information based on at least one of the location, region range, and number of the first abnormal region included in the first generated information and the second abnormal region included in the second generated information. The information processing apparatus according to claim 5.

7. The first prompt includes a three-dimensional first medical image data composed of a plurality of first stacked image data, The second prompt includes a three-dimensional second medical image data composed of a plurality of second stack image data sets that are different from the plurality of first stack image data sets. The three-dimensional object captured in the first medical image data and the three-dimensional object captured in the second medical image data are identical. The plurality of first stacked image data are two-dimensional image data in which the three-dimensional object is represented in a cross-section in a first direction. The plurality of second stacked image data are two-dimensional image data in which the three-dimensional object is represented in a cross-section in a second direction different from the first direction. The information processing apparatus according to claim 1.

8. The first prompt includes an instruction to cause the generation model to generate the first generation information based on the third medical image data generated under the first image conditions, The second prompt includes an instruction to cause the generation model to generate the second generation information based on a fourth medical image data generated under a second image condition different from the first image condition, The first image conditions and the second image conditions include at least one of the imaging conditions and the image processing conditions. The information processing apparatus according to claim 1.

9. The first prompt includes first medical information in text data format, The second prompt includes second medical information in text data format in which some of the wording in the first medical information is replaced with other words similar to said wording. The evaluation unit evaluates the degree of relationship between the first generated information and the second generated information based on the degree of similarity between the first generated information and the second generated information. The information processing apparatus according to claim 1.

10. A prompt acquisition unit that acquires a first prompt containing medical information and a second prompt identical to the first prompt, A generation information acquisition unit inputs the first prompt into a generation model and obtains first generation information corresponding to the first prompt from the generation model, and inputs the second prompt into the generation model and obtains second generation information corresponding to the second prompt from the generation model, An evaluation unit that evaluates the relationship between the first generated information and the second generated information, An information processing device equipped with the following features.

11. A generative model that receives prompt input and outputs information corresponding to the prompt, Equipped with an information processing device, The aforementioned information processing device is A prompt acquisition unit that acquires a first prompt containing medical information and a second prompt similar to the first prompt, A generation information acquisition unit inputs the first prompt to the generation model and obtains first generation information corresponding to the first prompt from the generation model, and inputs the second prompt to the generation model and obtains second generation information corresponding to the second prompt from the generation model, An evaluation unit that evaluates the relationship between the first generated information and the second generated information, An information processing system equipped with the following features.

12. A prompt acquisition step of acquiring a first prompt containing medical information and a second prompt similar to the first prompt, A generation information acquisition step involves inputting the first prompt into a generation model to obtain first generation information corresponding to the first prompt from the generation model, and inputting the second prompt into the generation model to obtain second generation information corresponding to the second prompt from the generation model. An evaluation step to evaluate the relationship between the first generated information and the second generated information, Information processing methods including

13. A prompt acquisition step of acquiring a first prompt containing medical information and a second prompt similar to the first prompt, A generation information acquisition step involves inputting the first prompt into a generation model to obtain first generation information corresponding to the first prompt from the generation model, and inputting the second prompt into the generation model to obtain second generation information corresponding to the second prompt from the generation model. An evaluation step to evaluate the relationship between the first generated information and the second generated information, A program that causes a computer to execute something.

Images