Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program with which preparation of a report can be supported with a sentence generation model, while time and effort for a user in checking are reduced.SOLUTION: An information processing system described in the present invention has: an input information acquisition unit that acquires input information to be input to a sentence generation model; an output information acquisition unit that acquires output information that is a result of inputting the input information to the sentence generation model; a specification unit that specifies a portion of the output information recommended to be checked by a user through comparison between the input information and the output information; and a display control unit that displays, on a display, the output information and the portion recommended to be checked by the user in association with each other.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The disclosure of this specification relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Recently, various applications of sentences generated by sentence generation models, which are models that generate sentences using artificial intelligence, have been considered.

[0003] Non-Patent Document 1 discloses ChatGPT, an interactive interface that uses a GPT (Generative Pre-trained Transformer) text generation model to perform text-related tasks such as summarization, proofreading, and text generation with high accuracy. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] OpenAI,“GPT-4 Technical Report,”arXiv:2303.08774v3,2023. Summary of the Invention [Problem to be solved by the invention]

[0005] It is known that sentence generation models such as ChatGPT output a large number of sentences in response to input information such as text. However, the output information from a sentence generation model may contain errors due to hallucination, and it is necessary to check the output information, but checking the entire output information is time-consuming. [Means for solving the problem]

[0006] In order to solve the above problems, the information processing system of the present invention includes an input information acquisition unit that acquires input information to be input to a sentence generation model, an output information acquisition unit that acquires output information that is a result of inputting the input information to the sentence generation model, an identification unit that identifies a portion of the output information that is recommended for user confirmation by comparing the input information with the output information, and a display control unit that causes the output information and the portion that is recommended for user confirmation to be displayed on a display unit in association with each other; The present invention is characterized by having the following. [Effects of the Invention]

[0007] The present invention can assist in creating reports using a sentence generation model while reducing the effort required for user confirmation. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a functional configuration of an information processing system according to a first embodiment. [Figure 2] 4 is a flowchart illustrating a flow of an example of processing by the information processing system according to the first embodiment. [Figure 3] FIG. 4 is a diagram showing an example of a diagnostic report input by a user according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a medical image (chest X-ray image) according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing an example of an output diagnostic report according to the first embodiment. [Figure 6] FIG. 10 is a diagram showing an example of a display in which an outputted diagnostic report and recommended check points are associated with each other according to the first embodiment. [Figure 7] 10 is a flowchart illustrating a flow of an example of processing by an information processing system according to a second embodiment. [Figure 8] FIG. 10 is a diagram showing an example of an output diagnostic report according to the second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a display in which an outputted diagnostic report and recommended check points are associated with each other according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Each embodiment and each modified example of an information processing system will be described in detail with reference to the drawings. Note that the embodiments can be combined with conventional technology, other embodiments, or modified examples to the extent that no contradiction occurs. Similarly, the modified examples can be combined with conventional technology, embodiments, or other modified examples to the extent that no contradiction occurs. In the following description, similar components will be assigned common reference numerals, and redundant description may be omitted. The configuration of the information processing system is not limited to the embodiments described below, and may be composed of multiple information processing devices each having each unit, or a small number of information processing devices each having multiple units.

[0010] [First embodiment] FIG. 1 is a diagram showing an example of the configuration of an information processing system 100 according to the first embodiment.

[0011] The information processing system 100 is a system configured to be connectable to a sentence generation model 300, which is represented by a large-scale language model such as GPT, via a network 200. In this embodiment, the information processing system 100 will be described as being configured to acquire information output by the sentence generation model 300 by linking with an external sentence generation model 300; however, the sentence generation model 300 may also be realized as a component of the information processing system 100.

[0012] The information processing system 100 may have at least one of the following functions: displaying information processing results, saving the information processing results in association with text data or image data, and outputting the information processing results to an external device. Here, the input information is, for example, a diagnostic report for a subject. The information processing system 100 in this embodiment identifies portions recommended for user confirmation by comparing output information, such as a diagnostic report output by the sentence generation model 300, with input information acquired by the information processing system 100. The information processing system 100 also performs display control to display the portions recommended for user confirmation on a display unit such as a display, in association with the output information from the sentence generation model 300. Furthermore, the information processing system 100 reflects the user's input for the portions recommended for user confirmation displayed on the display unit in the diagnostic report. In other words, the information processing system 100 generates a final diagnostic report in which the user selects whether to accept or reject the report. The information processing system 100 may also display the generated diagnostic report on a connected display or a display device (not shown). The information processing system 100 is realized by computer devices such as a server and a workstation, for example.

[0013] The information processing system 100 may be communicatively connected to a data management device (not shown) to acquire a diagnostic report to be processed and store a completed diagnostic report. Communication is performed, for example, via a network 200 or a communication cable or communication circuit (not shown). The data management device stores various data such as text data of the diagnostic report and medical image data, and is capable of transmitting and receiving various data to other devices, such as the information processing device 100, that can communicate with the data management device. The data management device may be integrated with the information processing system 100 as one of the components constituting the information processing system 100.

[0014] The image data input to the sentence generation model 300 is not limited in terms of the number of dimensions or type of image. Examples include three-dimensional image data (i.e., volume data) such as medical image data captured by optical coherence tomography (OCT) imaging devices, computed tomography (CT) devices, ultrasound diagnostic devices, magnetic resonance imaging (MRI) devices, positron emission tomography (PET) devices, and single photon emission computed tomography (SPECT) devices; two-dimensional image data such as medical image data captured by simple X-ray devices, digital microscopes, fundus cameras, and slice image data of the three-dimensional image data; three-dimensional data of people, animals, and man-made objects captured by a three-dimensional scanner; and three-dimensional image data generated by stacking two-dimensional image data. Such image data may also be multidimensional image data with four or more coordinate axes.

[0015] The functional configuration of the information processing system 100 will now be described with reference to Fig. 1. The information processing system 100 includes a communication interface 101, a storage circuit 102, and a processing circuit 103. The information processing system 100 may also include an input interface 104 and a display 105 depending on the situation in which the information processing system 100 is used.

[0016] In this embodiment, the information processing system 100 is communicably connected to a network 200 via a communication interface 101 .

[0017] The information processing system 100 communicates with the writing generation model 300 via the network 200, transmits information to be input to the writing generation model 300, and receives information output by the writing generation model 300. Note that the writing generation model 300 may be integrated with the information processing system 100 as one of the components constituting the information processing system 100.

[0018] The communication interface 101 is an interface for communicating diagnostic reports, medical image data, processing results, etc. with other devices. The communication interface 101 is realized by, for example, a network communication interface such as a network adapter or a NIC (Network Interface Controller), or a device connection interface such as a USB (Universal Serial Bus), PCI Express, SATA (Serial ATA), or M.2.

[0019] The memory circuitry 102 stores various data and various programs. Specifically, the memory circuitry 102 is connected to the processing circuitry 103 and stores various data under the control of the processing circuitry 103. For example, the memory circuitry 102 stores diagnostic reports and medical images under the control of the processing circuitry 103. The various data stored in the memory circuitry 102 is read and used by the processing circuitry 103. The memory circuitry 102 also functions as a work memory that temporarily stores various data used in processing executed by the processing circuitry 103. The memory circuitry 102 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, a hard disk, an optical disk, or the like.

[0020] The processing circuitry 103 controls the entire information processing device 100. For example, the processing circuitry 103 performs various processes in response to a processing start instruction operation received from a user via an input interface 104 connected to the information processing device 100. Alternatively, for example, the processing circuitry 103 may perform various processes in response to an analysis instruction operation received from a user via the communication interface 101. Alternatively, for example, the processing circuitry 103 may perform various processes upon detecting that a diagnostic report has been stored in the memory circuitry 102. The processing circuitry 103 is realized, for example, by a processor.

[0021] The processing circuitry 103 in the information processing system 100 includes, for example, an input information acquisition unit 103a, an output information acquisition unit 103b, a specification unit 103c, and a display control unit 103d.

[0022] 1, each processing unit, which is a component of the processing circuitry 103, is stored in the form of a computer-executable program in the storage circuitry 102. Each processing unit is a functional unit including an input information acquisition unit 103a, an analysis data acquisition unit 103b, an analysis performance acquisition unit 103c, and an analysis data integration unit 103d.

[0023] The processing circuitry 103 realizes the function corresponding to each program by the processor reading each program from the storage circuitry 102 and executing the read program. In other words, the processing circuitry 103 in a state in which each program has been read has the function of each unit shown in the processing circuitry 103 in FIG.

[0024] The information processing system 100 includes an acquisition unit 103a for input information to be input to the sentence generation model, and an acquisition unit 103b for output information that acquires output information resulting from inputting the input information to the sentence generation model 300. The information processing system 100 also includes an identification unit 103c that identifies portions of the output information that are recommended for user confirmation by comparing the input information with the output information. The information processing system 100 further includes a display control unit 103d that associates the output information with the portions that are recommended for user confirmation and displays them on a display unit such as a display 105. This configuration of the information processing system makes it possible to present portions of the output information that require user confirmation, thereby reducing the user's effort in checking the entire output information.

[0025] The input interface 104 accepts various instructions and input operations of various information from a user of the information processing system 100. Specifically, the input interface 104 is connected to the processing circuit 103 and converts the input operations received from the user into electrical signals and transmits them to the processing circuit 103. For example, the input interface 104 may be realized by a trackball, a switch button, a mouse, a keyboard, a touchpad that performs input operations by touching the operation surface, or the like. Alternatively, the input interface 104 may be realized by a touchscreen in which a display screen and a touchpad are integrated, a non-contact input interface using an optical sensor, a voice input interface, or the like. Note that the input interface 104 is not limited to those that include physical operation components such as a mouse and a keyboard. For example, an electrical signal processing circuit that receives electrical signals corresponding to input operations from an external input device provided separately from the information processing system 100 and transmits these electrical signals to the processing circuit 103 is also included as an example of the input interface 104.

[0026] The display 105 is an example of a display unit that displays various data such as diagnostic reports and data based on processing results. Specifically, the display 105 is connected to the processing circuit 103 and displays various data received from the processing circuit 103, specifically the display control unit 103d. For example, the display 105 displays diagnostic reports based on text data and medical image data based on image data. The display 105 is realized by, for example, a liquid crystal monitor, a CRT (Cathode Ray Tube) monitor, a touch panel, or the like.

[0027] The sentence generation model 300 is a system equipped with a large-scale language model implemented by machine learning techniques such as Transformer, which generates sentences from input information. For example, the sentence generation model 300 outputs output information in response to input information such as a diagnostic report input from the processing circuit 103. The output information results from correcting sentences or supplementing additional information with respect to the input information, i.e., the diagnostic report. The sentence generation model 300 may also be a multimodal model, and may, for example, accept input of medical image data or the like as input information. Furthermore, sentences may be generated by inputting test results of a subject, such as medical image data or blood test data, as input information to the sentence generation model 300.

[0028] The sentence generation model 300 may output a diagnostic report supplemented with information such as the position and size of a tumor detected from a chest X-ray image and the presence or absence of inflammation estimated from the CRP value of a blood test. The sentence generation model 300 may be integrated with the information processing system 100 as one of the components constituting the information processing system 100. In this case, the sentence generation model 300 may be stored in the storage circuitry 102 and read out and used by the processing circuitry 103. The sentence generation model 300 is realized by, for example, ChatGPT or BERT (Bidirectional Encoder Representations from Transformers).

[0029] An example of the configuration of the information processing system 100 according to this embodiment has been described above. In this embodiment, the information processing system 100 executes various processes described below so as to create a report of a sentence generation model while reducing the user's confirmation effort. Below, an example of the procedure by which the information processing system 100 executes various processes on a diagnostic report will be described.

[0030] An example of a report creation process for a diagnostic report input by a user, which is executed by the information processing system 100 according to this embodiment, using ChatGPT as the text generation model 300, will be described below. Note that the same process as this embodiment can be easily extended to text data other than diagnostic reports, and may be rephrased as necessary.

[0031] 2 is a flowchart showing an example of the flow of processing executed by the information processing system 100 according to the first embodiment. In the description of this embodiment, the processing shown in FIG. 2 is executed when the data to be processed is stored in the storage circuitry 102 and the user issues an instruction to start the processing by operating the input interface 104. The data to be processed is, for example, text data of a diagnostic report 400 as shown in FIG. 3 and image data of a chest X-ray image 500 as shown in FIG. 4. The following description will be given according to the steps of the flowchart shown in FIG. 2. Note that the order of the steps may be changed within the scope of not causing any contradiction in the content, and certain steps may be skipped.

[0032] In step S101, the input information acquisition unit 103a acquires input information to be input to the sentence generation model 300. For example, a diagnostic report 400 and a chest X-ray image 500 as the examination result are acquired as input information, and the process proceeds to the next step. Note that only sentence information may be acquired as input information.

[0033] In step S102, the output information acquisition unit 103b sends the diagnostic report 400 and the chest X-ray image 500 to the sentence generation model 300, and proceeds to the next step after acquiring the output information, which is the output result. The output information is, for example, a diagnostic report 600 as shown in FIG. 5.

[0034] The diagnostic report, which is the output information, includes additional information, for example, based on the inference that "contrast imaging is required for detailed evaluation" with respect to the swelling of the left hilum that was described in the diagnostic report 400, which is the input information.

[0035] Furthermore, for example, the input information may include information in which "information about the shadow 502 in the right lung field" that was not written in the diagnostic report 400, which is the input information, is added by the sentence generation model performing inference on a chest X-ray image 500, which is medical image data. The output information acquiring unit 103b may further output the sentence added based on the inference and the likelihood of the inference. Alternatively, the input information acquiring unit 103a may add a constraint that requests the output of the likelihood as input information. That is, the input information acquiring unit 103a may acquire input information including a constraint to be input to the sentence generation model 300.

[0036] In step S103, the identification unit 103c compares the input information with the output information to identify a portion of the output information recommended for user confirmation, and then proceeds to the next step. For example, the input information and the output information are vector-converted and a similarity comparison is performed for each sentence vector. Note that the similarity comparison method is merely an example, and the difference between the input information and the output information may be calculated using an existing document comparison method. Alternatively, the identification unit 103c may identify a portion to be recommended based on the difference obtained by inputting the input information, the output information, and a prompt to output the difference between the two pieces of information to the sentence generation model 300. Alternatively, the difference may be identified by outputting the input information and the output information to an existing sentence comparison tool. Alternatively, the identification unit 103c may input the input information, the output information, and a constraint to the sentence generation model 300 that causes the sentence generation model 300 to output a portion with a large semantic difference between the two pieces of information. The identification unit 103c may then identify a portion to be recommended based on the portion with a large semantic difference obtained by the sentence generation model 300. Alternatively, the portions with large semantic differences may be obtained from the results of inputting input information and output information into a language model different from the sentence generation model 300 and executing a comparison process.

[0037] Furthermore, the identification unit 103c may compare the inference results obtained by inputting the input information and the output information into an inference model that infers specific items from text, and if the inference results differ, identify the recommended part based on the part that served as the basis for the inference. Furthermore, if the input information and the output information are structured documents, the identification unit 103c may identify the recommended part by comparing the input information with the text written in each item of the output information.

[0038] In step S104, the display control unit 103d causes the display 105, which is the display unit, to display 700, as shown in FIG. 6, in which the diagnostic report 600, which is output information, is associated with the portion recommended for user confirmation. For example, the display control unit 103d causes dialogs 701, 702, and 703 to be displayed for the portion recommended for user confirmation. After this display process, the information processing system 100 proceeds to the next step. The display control unit 103d displays that the sentence in the portion recommended for user confirmation is a corrected or added sentence, and requests the user to select whether or not to allow the insertion. Alternatively, in step S102, the output information acquisition unit 103b causes the sentence generation model 300 to output output information, which is a sentence inferred based on the result of inference on the chest X-ray image 500, and the likelihood of the inference. In this case, the display control unit 103d may display, for example, a message indicating that the sentence was added based on the result of the inference, or the likelihood of the inference, for a portion that matches the sentence added based on the result of the inference, as in a dialog 701. Also, the diagnostic report 400, which is output information, may be displayed in parallel.

[0039] In step S105, the display control unit 103d accepts a selection of whether or not to permit insertion of text at locations where confirmation of the dialogs 701, 702, and 703 is recommended. The user makes this selection using the input interface 104 or the like. For text at locations where insertion of text has been prohibited as a result of this selection, processing proceeds to step S106. If all insertions are permitted, the input of the permitted text into the diagnostic report is confirmed.

[0040] In step S106, the processing unit 103 deletes or modifies the text at the location where insertion is not permitted, and returns it to the text written in the diagnostic report 400, which is the input information. If the text has been newly added by inference, the processing unit 103 deletes the text and finalizes the diagnostic report.

[0041] By executing the above steps, the information processing system 100 can create a report of the sentence generation model while reducing the user's confirmation effort.

[0042] The diagnostic report after the user has performed the checking work may be output as management target data to a data management device (not shown) via the network 200 or a communication cable or communication circuit (not shown).

[0043] (Modification 1 of the first embodiment) In step S101, the input information acquisition unit 103a may acquire not only text information such as a diagnostic report, but also words, voice data, etc. If voice data is acquired, in step S102, the output information acquisition unit 103b inputs the text information converted from the voice by analysis processing into ChatGPT.

[0044] This reduces the time and effort required for the user to input text.

[0045] (Modification 2 of the first embodiment) In step S101, the input information acquiring unit 103a may acquire measurement data such as blood test data or urine test data as the test results, instead of medical image data such as chest X-ray image data. Also, multiple test result data may be acquired. In this case, in step S102, the output information acquiring unit 103b can acquire a diagnostic report whose output information is the test results output by the sentence generation model 300 or information inferred by the sentence generation model 300 from the test result data.

[0046] For example, the output information acquisition unit 103b may acquire the position and size of a tumor detected from a chest X-ray image, or may acquire a CRP (C-reactive protein) value from a blood test, or the presence or absence of inflammation inferred from the CRP value, etc. The information acquired here is not included in the input information, but is information that is ultimately supplemented by inference by the sentence generation model 300.

[0047] In step S104, the display control unit 103d also displays the parts recommended for checking and the corresponding inspection results in association with each other as the generating subjects.

[0048] This allows the diagnostic report to be supplemented based on various tests, reducing the effort required for the user to view the results of each test and preventing the user from overlooking information.

[0049] (Modification 3 of the first embodiment) In step S103, the identification unit 103c acquires the diagnostic report 400, which is input information input by the user to the sentence generation model 300 such as ChatGPT, and the diagnostic report 600, which is output information output by the sentence generation model 300. Next, the identification unit 103c may identify portions recommended for user confirmation by performing a comparison process using the diagnostic report 400 and the diagnostic report 600 as inputs to a language model. Furthermore, the identification unit 103c may input a constraint to the sentence generation model 300 to cause the sentence generation model 300 to output portions with large semantic differences in an identifiable manner. In this case, the identification unit 103c identifies the output portions inferred to have large semantic differences as portions recommended for confirmation. Furthermore, the inference of portions with large semantic differences between the diagnostic report 400 and the diagnostic report 600 may be performed using a language model different from the sentence generation model 300. Also, for example, a portion determined not to be a conjunction or particle among the differences between the diagnostic report 400 and the diagnostic report 600 may be identified as a portion recommended for confirmation. With this modification, the identifying unit 103c can identify a portion inferred to be important among the differences between the diagnostic report 400 and the diagnostic report 600 as a portion recommended for confirmation by the user, thereby further reducing the effort required for the user to confirm.

[0050] (Modification 4 of the first embodiment) The processing circuit 103 may further include a learning unit. Then, in step S104, the display control unit 103d extracts sentences that the user has not permitted to be inserted into the portion displayed as a portion recommended for confirmation, or sentences that the user has further selected as undesirable expressions. The sentences extracted by the learning unit may be used to perform additional learning, such as fine tuning, so that the sentence generation model 300 does not produce similar outputs. In other words, the information processing system 100 may further include a learning unit that performs additional learning of the sentence generation model based on a user's input for the portion recommended for user confirmation. Alternatively, if the identification unit 103c is configured using an inference device that infers the importance of the difference between input information and output information, the learning unit may train the portion extracted by the above process as training data for the inference device.

[0051] In step S104, the display control unit 103d may prompt the user to input an appropriate sentence for the portion displayed as a portion recommended for confirmation. In this case, the learning unit uses the input appropriate sentence as a correct answer output and performs at least one of fine-tuning the sentence generation model 300 and training of the above-mentioned inference unit using the API.

[0052] Furthermore, as one of the functions constituting the information processing system 100, fine tuning of the sentence generation model 300 integrated with the information processing system 100 may be performed.

[0053] This makes it easier for the sentence generation model to output appropriate sentences, further reducing the effort required for users to check.

[0054] (Modification 5 of the first embodiment) In step S104, the display control unit 103d may further allow the user to select whether or not confirmation is not required for the portion displayed as a portion recommended for confirmation. If confirmation is determined to be unnecessary, the identification unit 103c updates the identification criteria so that subsequent portions where similar sentences are generated are not identified as portions recommended for confirmation. For example, the user may determine that confirmation is not required for conjunctions such as "also" and "however" or particles such as "wo" and "ga." In this case, the identification unit 103c calculates the degree of agreement between the portions that are the difference between the diagnosis results of the input diagnostic report 400 and the diagnostic report 600 output by ChatGPT and the portions that are determined to be unnecessary for confirmation. Here, the identification unit 103c calculates the agreement rate of "also," "however," "wo," and "ga," and excludes portions with a high agreement rate from the portions recommended for confirmation.

[0055] In addition, a process will be described in which, in step S103, the identification unit 103c determines a portion for which confirmation is recommended using a language model, as in Modification 3 of the first embodiment. In this case, in step S104, the identification unit 103c may fine-tune the language model so that a portion that the user has determined not to require confirmation is not determined as a portion for which confirmation is recommended thereafter.

[0056] This makes it difficult to recommend checking unimportant parts of the differences between the diagnostic report 400 and the diagnostic report 600, which are input information, and further reduces the effort required for the user to check.

[0057] (Modification 6 of the first embodiment) In step S101, the diagnostic report 400 acquired by the input information acquiring unit 103a and the diagnostic report 600 output by the sentence generation model 300 acquired by the output information acquiring unit 103b in step S102 may be structured reports. A structured report refers to a report in which sentences are written for at least one or more clinical items. For example, a structured report may be a report in which sentences are written for each of the items of medical history, findings, and diagnosis. In this case, in step S103, the identifying unit 103c compares, for example, the sentence written in the findings field of the diagnostic report 400, which is the input information, with the sentence written in the findings field of the diagnostic report 600, which is the output information output by the sentence generation model 300. If there is a difference in the clinical items, the identifying unit 103c may identify a portion of the diagnostic report 600 output by the sentence generation model 300 containing the difference as a portion recommended for user confirmation.

[0058] In addition, the text written in the items to be compared in both reports may be text written in items such as medical history, diagnostic results, treatment plan, prognosis prediction, similar cases, etc., and the identification unit 103c may compare text written in multiple items.

[0059] Furthermore, the user may use the interface 104 to select in advance the items to be compared between the diagnostic report 400, which is the input information, and the diagnostic report 600, which is the output information output by the sentence generation model 300. In this case, the identifying unit 103c compares the sentences only for the selected items, and identifies the parts containing differences as parts recommended for user confirmation.

[0060] This makes it easier for the user to be encouraged to check changes related to items that the user wants to check, further reducing the effort required for the user to check.

[0061] (Seventh modification of the first embodiment) In step S103, the identification unit 103c may send the diagnostic report as input information to the sentence generation model 300 and further infer a disease name. Specifically, the identification unit 103c acquires the diagnostic report 400, which is input information, and the diagnostic report 600 output by the sentence generation model 300. The identification unit 103c infers a disease name by inputting the diagnostic report 400 and the diagnostic report 600 to the sentence generation model 300. When the inferred disease name is different, the identification unit 103c further acquires a portion that served as the basis for the inference in the diagnostic report 600 using the sentence generation model 300. Then, the identification unit 103c may identify the portion that served as the basis for the inference as a portion recommended for user confirmation. That is, the identification unit 103c acquires a first inference result inferred using an inference model that infers clinical items for the diagnostic report that constitutes the input information. Furthermore, the identification unit 103c acquires a second inference result inferred using the inference model for the diagnostic report that constitutes the output information. If the two inference results are different, the identifying unit 103c identifies a portion of the output information that contributes to the second inference result as a portion recommended for user confirmation.

[0062] Furthermore, for example, the disease name may be inferred using an inference model configured using Transformer and trained on the task of inferring the disease name from a diagnostic report. When making the inference for the diagnostic report 600, the identification unit 103c acquires a portion with a high score calculated from values ​​obtained from the attention layer of the inference model. The identification unit 103c may then identify the portion with the high score as a portion recommended for confirmation by a user who contributed to the inference.

[0063] Furthermore, items to be inferred from the diagnostic report may be, for example, whether or not re-examination is necessary, a treatment policy, a prognosis prediction, similar cases, or multiple items may be inferred.

[0064] This makes it easier for the user to check changes related to the items they want to check, further reducing the effort required for the user to check. Also, by using a model specialized for inferring the items the user wants to check and performing more accurate inference, the possibility of incorrectly recommending confirmation can be reduced.

[0065] [Second embodiment] The information processing system 100 according to the second embodiment inputs a diagnostic report, which is input information input by a user such as a doctor, together with constraints for controlling the output of a sentence generation model 300 such as ChatGPT, into sentence generation. The information processing system 100 identifies portions recommended for user confirmation based on output information based on the constraints, and displays the portions on a display unit in association with information output by the sentence generation model. The constraints are conditions for outputting newly generated information by the sentence generation model 300 in a manner distinguishable from the input information to the sentence generation model 300. In this embodiment, the information processing system 100 includes an input information acquisition unit 103a that acquires input information to be input to the sentence generation model and constraints for outputting newly generated information by the sentence generation model 300 in a manner distinguishable from the input information. The information processing system 100 also includes an output information acquisition unit 103b that acquires output information resulting from inputting the input information to the sentence generation model 300, and an identification unit 103c that identifies portions recommended for user confirmation in the output information. The information processing system 100 according to the second embodiment further includes a display control unit that associates the output information with the portion recommended for user confirmation and displays them on the display unit. Note that the functional configuration of the information processing system 100 according to the second embodiment is the same as the functional configuration of the information processing system according to the first embodiment, and therefore will be described with reference to FIG.

[0066] 7 is a flowchart showing an example of the flow of processing executed by the information processing system 100 according to the second embodiment. In the description of this embodiment, the processing shown in FIG. 7 is assumed to be executed when the user operates the input interface 104 to issue an instruction to start processing while the data to be processed is stored in the storage circuitry 102. The data to be processed is text data of the diagnostic report 400 as shown in FIG. The following description will be given in accordance with the steps of the flowchart shown in Fig. 7. Note that the order of the steps may be changed as long as no contradiction occurs in the content.

[0067] In step S201, the input information acquisition unit 103a acquires the diagnostic report 400 to be processed, which is input information, and constraints that control the output of the sentence generation model 300. For example, a prompt saying "Please do not change the original sentence, just add." is acquired as a constraint. The processing flow will be described below assuming that this constraint has been acquired. Note that the content of the constraint is not limited by this, and any constraint may be acquired. For example, a constraint for outputting the added portion in an identifiable manner may be input as a prompt. For example, by inputting the above constraint, the sentence generation model 300 outputs the added portion in bold, with a different font size, in italics, or with underlining.

[0068] In step S202, the output information acquisition unit 103b acquires output information that is the result of transmitting the diagnostic report 400 and the constraint conditions to the sentence generation model 300. The output information is, for example, a diagnostic report 800 as shown in Fig. 8. For example, regarding the swelling of the left hilum described in the diagnostic report 400, information such as "contrast imaging is required for detailed evaluation" is added based on the constraint conditions without modifying the surrounding sentences, and is output by underlining it so that it can be distinguished from the input information.

[0069] In step S203, the identifying unit 103c identifies a portion of the diagnostic report 800, which is output information, that is recommended for user confirmation based on the constraint. For example, based on the constraint "Please do not change the original sentence, but only add to it," the identifying unit 103c identifies the difference between the diagnostic report 400, which is input information, and the diagnostic report 800, which is output information, as the portion that is recommended for user confirmation. In other words, the input information acquiring unit 103a is characterized by acquiring a constraint that controls the sentence generation model 300 so that newly created information is only added. Furthermore, the identifying unit 103c identifies the difference between the input information and the output information as the portion that is recommended for user confirmation.

[0070] In step S204, the display control unit 103d causes the display 105, which is the display unit, to display 900, which associates the diagnostic report 800, which is output information, with the portion identified by the identification unit 103c, which is recommended for user confirmation, as shown in Fig. 9. For example, the display control unit 103d displays, as in dialogs 901 and 902, that the text in the portion recommended for user confirmation is text that has been corrected or added, and requests the user to select whether or not to allow insertion.

[0071] In step S205, the display control unit 103d selects whether or not to permit insertion of text at the locations recommended for confirmation in the dialogs 901 and 902, in response to a selection instruction from the user using the input interface 104. The user makes this selection using the input interface 104 or the like. For text at locations where insertion of text has been selected not to be permitted, the process proceeds to step S106. If all insertions are permitted, the input of the permitted insertions into the diagnostic report is confirmed.

[0072] In step S206, the processing unit 103 deletes or modifies the text at the location where insertion is not permitted, and returns it to the text written in the diagnostic report 400, which is the input information. If the text has been newly added by inference, the processing unit 103 deletes the text and finalizes the diagnostic report.

[0073] By executing the above steps, the information processing system 100 can create a report of the sentence generation model while reducing the user's confirmation effort.

[0074] The diagnostic report after the user has performed the checking work may be output as management target data to a data management device (not shown) via the network 200 or a communication cable or communication circuit (not shown).

[0075] (Modification 1 of the second embodiment) In step S201, the input information acquiring unit 103a may acquire a constraint that controls the output of the sentence generation model 300 so that the beginning and end of newly created information are surrounded by **, which indicates bold font decoration, in Markdown notation and output. In this case, in step S203, the identifying unit 103c may extract, by pattern matching, the portions surrounded by ** at the beginning and end of the diagnostic report 800 output by the sentence generation model 300, and identify them as portions recommended for user confirmation. Furthermore, the display control unit 103d may display the portions recommended for confirmation in bold font according to Markdown notation.

[0076] Furthermore, the constraints may be any as long as the newly created information can be distinguished from other parts by the sentence generation model 300. For example, the beginning and end of the text may be surrounded by *, which indicates italicization in Markdown notation, or may be surrounded by other specific symbols.

[0077] This allows the part recommended for checking to be displayed in a more visually understandable and decorative way. Furthermore, the identifying unit 103c can identify the part recommended for checking from the diagnostic report 800, which is the output information output by the sentence generation model 300, without comparing it with the input information, thereby reducing the processing load of the identifying unit 103c.

[0078] (Modification 2 of the second embodiment) In step S201, the input information acquiring unit 103a may acquire a constraint condition that controls the output of the sentence generation model 300 so as not to change the parts or items including the diagnostic result in the input information, the diagnostic report 400. In other words, the input information acquiring unit 103a acquires a constraint condition that is a condition that causes the sentence generation model 300 to output without changing the parts including the clinical items.

[0079] Furthermore, the portion where the output of the sentence generation model 300 is controlled by the constraints may be a portion including a treatment policy, a prognosis prediction, a similar case, or a portion including multiple items.

[0080] This stabilizes the output of output information for parts that include items that the user does not want changed in the sentence generation model 300, further reducing the user's confirmation effort.

[0081] (Modification 3 of the second embodiment) In step S201, the input information acquiring unit 103a causes the sentence generation model 300 to output the diagnostic result inferred from the diagnostic report 400, which is input information, and the part that is the basis of the inference, in a manner that makes them identifiable. The input information acquiring unit 103a acquires the acquired diagnostic result and the part that is the basis of the inference, and may generate a diagnostic report using a constraint that controls the output of the sentence generation model 300 so as not to change the part that is the basis of the inference.

[0082] Furthermore, the input information acquiring unit 103a may input the diagnostic report 400, which is input information, and the diagnostic report 800 output by the sentence generation model 300, to the sentence generation model 300. The input information acquiring unit 103a may acquire the diagnostic result inferred from the diagnostic report and the part that is the basis of the inference, in other words, the part that contributes to the inference.

[0083] Furthermore, if the diagnostic results inferred by the sentence generation model 300 from the diagnostic report 400 and the diagnostic report 800 are different, the output information acquisition unit 103b executes the following process: In addition to the diagnostic report 400, the output information acquisition unit 103b may input a constraint condition that controls the output of the sentence generation model 300 so as not to change the part that served as the basis for the inference for the diagnostic report 400, and output the diagnostic report 800 again.

[0084] Also, for example, the input information acquiring unit 103a may infer a diagnostic result using an inference model configured using Transformer and trained to infer a diagnostic result from a diagnostic report. In this case, a constraint condition for controlling the output of the sentence generation model 300 may be acquired so as not to change the part with a high score calculated from the value obtained from the attention layer of the inference model.

[0085] This makes it difficult for clinically important points that form the basis of diagnosis to be changed, further reducing the effort required for user confirmation.

[0086] (Modification 4 of the second embodiment) In step S201, the input information acquisition unit 103a may further acquire medical image data and an inference model that infers the location of a tumor from the medical image. The input information acquisition unit 103a may apply the inference model to the medical image data to further acquire an inference result, and may set constraints that control the output of the sentence generation model 300 so as to output output information based on the inference result. In this case, in step S204, the display control unit 103d may display the name of the inference model as the generator of the input information, in association with the part recommended for confirmation, and may also display the likelihood of the inference in association.

[0087] Furthermore, the input information acquired by the acquisition unit 103a and used as the subject of inference by the inference model may be measurement values ​​from blood tests, urine tests, etc., or may be multiple test results.

[0088] Furthermore, the inference model may infer multiple items from a single test result, or may make inferences based on multiple test results.

[0089] This allows the user to check the output information in consideration of the subject of output of the information and the likelihood, thereby enabling efficient report creation.

[0090] (Modification 5 of the second embodiment) In step S201, the input information acquiring unit 103a may further acquire a past diagnostic report and further acquire constraints for controlling the output of the sentence generation model 300 so that output is performed based on the past diagnostic report. In this case, in step S202, the output information acquiring unit 103b acquires a diagnostic report 800, which is the input information of the diagnostic report 400, and the diagnostic report 800 is an inference result based on the past diagnostic report, such as "The size of the tumor is larger than it was three years ago," or an inference indicating a contradiction by comparing with the past diagnostic report, such as "The cough has continued not for two years but for three years."

[0091] Furthermore, in step S204, the display control unit 103d may display, in addition to the output information, information about the input information, such as the fact that the diagnostic report was generated from a past diagnostic report, the date on which the past diagnostic report was created, etc. Alternatively, the display control unit 103d may display information about the input information in association with the portion recommended for confirmation identified by the identification unit 103c. Note that input of constraint conditions is not essential, and a diagnostic report may be obtained by inputting a past diagnostic report and a newer diagnostic report 400 into the sentence generation model.

[0092] This configuration makes it possible to obtain highly accurate output information based on information from past reports, and also reduces the user's effort in checking the output information by displaying information related to the input information that was used as input.

[0093] (Modification 6 of the second embodiment) In step S201, the input information acquiring unit 103a further acquires clinical guidelines as input information, and acquires information as constraints to control the output of the sentence generation model 300 so that output is based on the clinical guidelines. Specifically, in step S202, information based on the clinical guidelines is added to the diagnostic report 400, which is the input information, to generate the diagnostic report 800. The added information is, for example, sentence information such as "The treatment method for lung carcinoma newly listed in the guidelines should also be considered in the future."

[0094] Also, in step S204, the display control unit 103d may display, as information regarding the input information, information that the input information was generated from the medical treatment guideline and the date the medical treatment guideline was issued, or may display the information in association with the part that is recommended to be checked.

[0095] This allows the user to obtain output information that takes into account information outside the diagnostic report created by the user, and by displaying the input source information together with the output information, the user can efficiently check the points being checked, taking into account the basis for the points being checked, etc.

[0096] (Other embodiments) In addition to the above-described embodiments, the present invention may be implemented in various different forms.

[0097] For example, the components of each device shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of each device can be configured by partial or physical distribution and integration in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of each processing unit performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0098] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0099] The methods described in the embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. The control program can also be recorded on a non-transitory computer-readable recording medium and executed by being read from the recording medium by a computer. Non-transitory storage media include hard disks, floppy disks (FDs), CD-ROMs, magneto-optical disks (MOs), DVDs, etc.

[0100] According to at least one of the embodiments described above, it is possible to support the creation of a report using a sentence generation model while reducing the effort required for confirmation by the user.

[0101] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0102] 100 Information Processing Systems 101 Communication Interface 102 Memory circuit 103 Processing Circuit 103a Input information acquisition unit 103b Output information acquisition section 103c Specific part 103d Display control unit 200 Network 300 Sentence Generation Model

Claims

1. an input information acquisition unit that acquires input information to be input to the sentence generation model; an output information acquisition unit that acquires output information that is a result of inputting the input information into the sentence generation model; an identifying unit that identifies a portion of the output information that is recommended for user confirmation by comparing the input information with the output information; a display control unit that causes a display unit to display the output information and the portion recommended for user confirmation in association with each other; An information processing system comprising:

2. 2. The information processing system according to claim 1, wherein the input information acquisition unit acquires information including a sentence as the input information.

3. 2. The information processing system according to claim 1, wherein the input information acquisition unit acquires input information including constraints to be input to the sentence generation model.

4. 2. The information processing system according to claim 1, wherein the specification unit performs a comparison process by inputting the input information and the output information to the sentence generation model or a language model different from the sentence generation model.

5. The information processing system according to claim 4, characterized in that the identification unit identifies a portion where the comparison process determines that there is a large semantic difference between the input information and the output information as a portion recommended for the user to check.

6. 2. The information processing system according to claim 1, wherein the display control unit displays the output information in association with a generator of the input information.

7. 2. The information processing system according to claim 1, further comprising a learning unit that performs additional learning of the sentence generation model based on a user's input for the portion recommended for user confirmation.

8. The information processing system according to claim 1 , wherein the specifying unit updates criteria for specifying the areas recommended for the user to check based on an input by the user regarding the areas recommended for the user to check.

9. 2. The information processing system according to claim 1, wherein the input information is a diagnostic report for a subject.

10. 10. The information processing system according to claim 9, wherein the input information acquisition unit further acquires information on test results for the subject as the input information.

11. The information processing system according to claim 10, characterized in that the input information acquisition unit further acquires, as the input information, the results of inference made using an inference model on the information on the test results for the subject.

12. 12. The information processing system according to claim 10, wherein the information on the test results for the subject is information including medical image data acquired by the test on the subject.

13. 12. The information processing system according to claim 10, wherein the information on the test results for the subject includes measurement values ​​obtained by testing the subject.

14. 10. The information processing system according to claim 9, wherein the input information acquisition unit further acquires a past diagnostic report for the subject as the input information.

15. 10. The information processing system according to claim 9, wherein the input information acquisition unit further acquires information on clinical practice guidelines as the input information.

16. The information processing system according to claim 9, characterized in that the identification unit compares the input information with the output information, and if there is a difference in a clinical item, identifies the part containing the difference as a part recommended for the user to check.

17. The identification unit generates a first inference result inferred from the diagnostic report constituting the input information using an inference model that infers clinical items; When the second inference result inferred using the inference model differs from the diagnostic report constituting the output information, 10. The information processing system according to claim 9, wherein a portion of the output information that contributes to the second inference result is identified as a portion recommended for the user to check.

18. The information processing system according to claim 1, characterized in that the input information acquisition unit further acquires constraint conditions that cause the sentence generation model to output without changing parts including clinical items.

19. The information processing system described in claim 9, characterized in that the input information acquisition unit further acquires constraints that are output using an inference model that infers clinical items from the diagnostic report so as not to change the parts that contribute to the inference results inferred from the input information.

20. an input information acquisition step for acquiring input information to be input to the sentence generation model; an output information acquisition step of acquiring output information that is a result of inputting the input information into the sentence generation model; a specifying step of specifying a portion of the output information that is recommended for user confirmation by comparing the input information with the output information; a display control step of displaying the output information and the portion recommended for user confirmation in association with each other on a display unit; An information processing method comprising:

21. A program for executing the information processing method according to claim 20 on a computer.

Images