Information processing device, method for supporting verification of medical documents, and program for supporting verification of medical documents.

Abstract

Simplifying the process of reviewing medical documents. [Solution] The information processing device comprises a source data showing the patient's medical history, an identification unit that identifies corresponding descriptions in medical documents generated using the source data, and a detection unit that detects descriptions for which the identification unit could not identify corresponding descriptions. Furthermore, this information processing device makes it possible to optimize the process of verifying medical documents.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, a method for supporting the verification of medical documents, and a program for supporting the verification of medical documents.

Background Art

[0002] Many doctors perform tasks such as creating medical documents such as progress reports, referral letters, and insurance diagnosis certificates in addition to medical practices such as examinations and treatments. As a technology for reducing the burden on doctors in such tasks, for example, there is an information processing apparatus described in Patent Document 1 below. This information processing apparatus derives recording information to be recorded in the recording items of a medical document based on patient information, and records the derived recording information in the recording items to generate medical document data. By using this information processing apparatus, it is possible to reduce the burden of creating medical documents.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the medical document data generated by the information processing apparatus does not always have appropriate content. Therefore, even when using the information processing apparatus described in Patent Document 1, doctors still need to check the generated medical document data against the original medical records, etc., to ensure that there are no inconsistencies in the content, no omissions, etc.

[0005] Thus, the information processing device described in Patent Document 1 has room for improvement in that it cannot support the verification of generated medical documents. Furthermore, the burden of verifying medical documents is not limited to cases where the information processing device generates the documents, but is also true when doctors or other medical professionals create the documents themselves. The exemplary purpose of this disclosure is to provide a technology that facilitates the verification of medical documents. [Means for solving the problem]

[0006] An information processing device relating to an illustrative aspect of this disclosure includes, with respect to raw data showing a patient's medical history and a medical document of the patient generated using the raw data, an identification means for identifying descriptions of corresponding content in the raw data and the medical document, and a detection means for detecting descriptions for which the identification means did not identify corresponding content.

[0007] Another information processing device relating to an illustrative aspect of this disclosure includes a consistency determination means that determines whether the content of the description of a generated medical document is consistent with the content of the description of the source data that is the data from which the medical document was generated, using a language model that has been trained on natural language, and an output control means that outputs the determination result of the consistency determination means.

[0008] In the example-like aspect of the medical document verification support method relating to this disclosure, at least one processor performs an identification process that identifies corresponding descriptions in the original data and the medical document of the patient, targeting the original data showing the patient's medical treatment history and the medical document of the patient generated using the original data, and a detection process that detects descriptions for which corresponding descriptions were not identified in the identification process.

[0009] The medical document verification support program relating to an illustrative aspect of this disclosure causes a computer to function as an identification means for identifying corresponding descriptions in the original data showing a patient's medical treatment history and the patient's medical document generated using the original data, and a detection means for detecting descriptions for which the corresponding descriptions were not identified by the identification means. [Effects of the Invention]

[0010] One illustrative aspect of this disclosure is that it can facilitate the verification process of medical documents. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of the information processing device related to this disclosure. [Figure 2] This flowchart shows the flow of the verification support method related to this disclosure. [Figure 3] This is a block diagram showing the configuration of other information processing devices related to this disclosure. [Figure 4] Figure 3 shows an example of the processing performed by the information processing device. [Figure 5] This diagram illustrates a method for identifying descriptions of corresponding content using a feature generation model and a similarity estimation model. [Figure 6] This diagram illustrates a method for determining consistency using a language model. [Figure 7] This figure shows an example of a display screen shown by the output control unit. [Figure 8] This figure shows another example of a display screen shown by the output control unit. [Figure 9] Figure 3 is a flowchart showing the processing flow executed by the information processing device. [Figure 10] This is a flowchart showing the details of the process in S13 of Figure 9. [Figure 11] This is a block diagram showing the configuration of an information processing device related to a reference example. [Figure 12]This is a block diagram showing the configuration of a computer that functions as an information processing device related to this disclosure. [Modes for carrying out the invention]

[0012] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining some or all of the technologies (things or methods) employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in each of the exemplary embodiments shown below may also be included in the scope of the present invention. In addition, the effects mentioned in each of the exemplary embodiments shown below are examples of effects that can be expected in that exemplary embodiment and do not define the scope of the present invention. That is, embodiments that do not produce the effects mentioned in each of the exemplary embodiments shown below may also be included in the scope of the present invention.

[0013] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form for each of the exemplary embodiments described later. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems occur.

[0014] (Configuration of Information Processing Device 1) The configuration of the information processing device 1 according to this exemplary embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing the configuration of the information processing device 1. As shown in Figure 1, the information processing device 1 includes a specific unit 101 and a detection unit 102.

[0015] The specific part 101 specifies the description of the corresponding content in the original data and the medical document by targeting the original data indicating the medical history of the patient and the medical document of the patient generated using the original data.

[0016] The above "original data" indicates the medical history of the patient and may include any information necessary for generating a desired medical document. For example, the above medical record of the patient (which may be an electronic medical record or an image of a paper medical record with character recognition), data indicating the results of examinations and diagnoses received by the patient, and data indicating pharmaceuticals prescribed to the patient, etc., may be used as the above original data. The above original data may be data indicating the medical history of the patient in natural language.

[0017] Also, the above "medical document" indicates the content of the medical treatment performed on the target patient in a predetermined format. For example, the above medical document may be various diagnostic documents such as insurance diagnosis certificates, reports on the progress of treatment, or referral letters, etc. Note that the medical document may be automatically generated by the information processing device 1 or other devices, may be generated manually by a doctor or the like, or may be partially automatically generated and partially manually generated. The specific part 101 performs the above-described process of specifying the description of the corresponding content for the medical document and the original data that are electronic data.

[0018] Also, the above "description of the corresponding content" may be any description that has some relevance to the content. Depending on the specific method applied, what description with what relevance to the content is specified may vary. For example, when specifying a description with similar content as the "description of the corresponding content", if both the original data and the medical document contain a sentence stating that the patient's underlying disease is diabetes, the specific part 101 specifies those sentences as the description of the corresponding content.

[0019] The detection unit 102 detects descriptions for which the identification unit 101 could not identify the corresponding content. By using the detection results of the detection unit 102, the verification process for medical documents can be facilitated.

[0020] For example, suppose the source data contains a sentence stating that the patient has diabetes, but no corresponding description can be found in the medical document. In this case, the detection unit 102 detects the above sentence as a description for which no corresponding description could be found by the identification unit 101. In this case, the detected description may be a description that was omitted from the medical document. In other words, in this case, by using the detection result of the detection unit 102, it becomes possible to easily discover descriptions that were omitted from the medical document.

[0021] Conversely, suppose a medical document contains a sentence stating that the patient has diabetes, but no corresponding description was identified in the original data. In this case as well, the detection unit 102 detects the sentence as a description for which no corresponding description was identified by the identification unit 101. In this case, the detected description may have been mistakenly included in the medical document. In other words, in this case, by using the detection result of the detection unit 102, it becomes possible to easily discover descriptions that have been mistakenly included in the medical document.

[0022] Furthermore, descriptions for which the identification unit 101 did not identify a corresponding content may include not only descriptions that were mistakenly included or that were not reflected, but also descriptions whose content has been altered to the extent that it cannot be identified as a corresponding description, or descriptions that do not need to be included in medical documents, such as greetings and acknowledgments. For this reason, the detection unit 102 may detect descriptions that have been altered or descriptions that do not need to be included in medical documents from the descriptions for which the identification unit 101 did not identify a corresponding content, by removing at least one of them. Descriptions whose content has been altered can be detected, for example, by the method described in Exemplary Embodiment 2 later. Descriptions that do not need to be included in medical documents can be detected, for example, by creating a list of such descriptions in advance.

[0023] As described above, the information processing device 1 according to this exemplary embodiment employs a configuration that includes a specification unit 101 that identifies corresponding descriptions in the original data showing the patient's medical treatment history and the medical document generated using the original data, and a detection unit 102 that detects descriptions for which the specification unit 101 did not identify corresponding descriptions.

[0024] As described above, the detection results from the detection unit 102 can be used to find descriptions that have been omitted from medical documents or descriptions that have been mistakenly included in medical documents. Therefore, the information processing device 1 has the effect of making the verification process of medical documents easier. Furthermore, the information processing device 1 also makes it possible to optimize the verification process of medical documents.

[0025] The method of using the detection results from the detection unit 102 in the medical document verification process is at the user's discretion. For example, the information processing device 1 may present the detection results from the detection unit 102 to the user of the information processing device 1 (e.g., a doctor or other final reviewer of the medical document). This allows the user to efficiently verify the appropriateness of the content of the medical document by referring to the detection results from the detection unit 102 and make corrections as necessary. The detection results from the detection unit 102 can also be used for further analysis of the medical document or for improving the medical document generation process. Applying these methods of use will ultimately lead to an easier verification process for medical documents.

[0026] (Medical document verification support program) The functions of the information processing device 1 described above can also be implemented by a program. The medical document verification support program according to this exemplary embodiment causes the computer to function as an identification means for identifying corresponding descriptions in the original data showing the patient's medical treatment history and the patient's medical document generated using the original data, and as a detection means for detecting descriptions for which corresponding descriptions were not identified by the identification means. This verification support program has the effect of making the verification work of medical documents easier.

[0027] (Flowchart for supporting the verification of medical documents) The flow of the medical document verification support method according to this exemplary embodiment will be explained with reference to Figure 2. Figure 2 is a flowchart showing the flow of the medical document verification support method. Note that the entity executing each step in this verification support method may be a processor provided in the information processing device 1, a processor provided in another device, or the entity executing each step may be a processor provided in a different device.

[0028] In S1 (specific processing), at least one processor identifies corresponding descriptions in the original data showing the patient's medical history and the patient's medical document generated using the original data.

[0029] In S2 (detection process), at least one processor detects descriptions from at least one of the source data and the medical document for which the corresponding content was not identified in the S1 process. This completes the process shown in Figure 2. Note that if the medical document does not contain any omissions or incorrect information, no descriptions will be detected in S2.

[0030] As described above, the medical document verification support method according to this exemplary embodiment employs a configuration in which at least one processor performs an identification process to identify corresponding descriptions in the original data showing the patient's medical treatment history and the patient's medical document generated using the original data, and a detection process to detect descriptions for which corresponding descriptions were not identified in the identification process. This verification support method has the effect of making the medical document verification process easier.

[0031] [Second exemplary embodiment] A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same function as those described in the above-described exemplary embodiment are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The scope of application of each technology adopted in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology adopted in this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise. Furthermore, each technology shown in the drawings referenced to describe this exemplary embodiment can also be adopted in other exemplary embodiments included in this disclosure, to the extent that no particular technical problems arise.

[0032] (Configuration of Information Processing Device 1A) The configuration of the information processing device 1A according to this exemplary embodiment will be described with reference to Figure 3. Figure 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A is a device equipped with a function to support the verification of the contents of medical documents. The information processing device 1A may be a local device used by individual users, or it may be a server that provides medical document verification support services to multiple users.

[0033] As shown in the figure, the information processing device 1A includes a control unit 10A that controls all parts of the information processing device 1A, and a storage unit 11A that stores various data used by the information processing device 1A. The information processing device 1A also includes a communication unit 12A for the information processing device 1A to communicate with other devices, an input unit 13A that receives input to the information processing device 1A, and an output unit 14A for the information processing device 1A to output data. The control unit 10A includes a identification unit 101A, a detection unit 102A, an acquisition unit 103A, a document generation unit 104A, a consistency determination unit 105A, a reception unit 106A, and an output control unit 107A.

[0034] The identification unit 101A, similar to the identification unit 101 in Exemplary Embodiment 1, targets raw data showing the patient's medical treatment history and the patient's medical document generated using said raw data, and identifies corresponding descriptions of content in said raw data and said medical document. As will be described in detail later, two machine learning models, a feature information generation model M1 and a similarity estimation model M2, are used when identifying corresponding descriptions of content.

[0035] The detection unit 102A, similar to the detection unit 102 in the exemplary embodiment 1, detects descriptions for which the corresponding content description was not identified by the identification unit 101A. For example, when the original data is the target of detection, the detection unit 102A only needs to detect the remaining descriptions after removing the descriptions identified by the identification unit 101A from the descriptions included in the original data. This makes it possible to detect descriptions that are suspected to have been omitted from the medical document. Similarly, the detection unit 102A may detect the remaining descriptions after removing the descriptions identified by the identification unit 101A from the descriptions in the medical document. The descriptions detected in this way are descriptions that are suspected to have been mistakenly included in the medical document.

[0036] The acquisition unit 103A acquires various data related to supporting the verification of medical documents. For example, the acquisition unit 103A acquires the source data that forms the basis of the medical document. The method of data acquisition is arbitrary; for example, the acquisition unit 103A may acquire data from an external device (for example, a terminal device used by the user) via the communication unit 12A, or it may acquire data that is input to the information processing device 1A via the input unit 13A.

[0037] The document generation unit 104A generates a medical document from the raw data acquired by the acquisition unit 103A. The method of generating the medical document is arbitrary. For example, the document generation unit 104A may generate the medical document by detecting each item to be entered in the medical document from the raw data and inputting each detected item into a medical document template. Alternatively, a language model that has been trained on natural language may be used to generate the medical document.

[0038] Here, machine learning of natural language means, more specifically, learning the arrangement of its constituent elements (such as words) in natural language sentences, and the arrangement of sentences within a text. Examples of language models that have learned natural language include BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly optimized BERT approach), and ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately).

[0039] The consistency determination unit 105A determines whether the content of the description in the original data and the medical document are consistent using a language model trained on natural language. As mentioned above, a language model trained on natural language is a model that has been trained on the arrangement of constituent elements in natural language sentences and the arrangement of sentences in documents. Hereafter, the language model used by the consistency determination unit 105A will be referred to as language model M3.

[0040] In this exemplary embodiment, we describe an example in which the language model M3 accepts input in the form of a text prompt written in natural language and outputs a response in natural language. However, the language model M3 may also be a model that can accept input in the form of data other than text data, such as images. This makes it possible to facilitate the verification of medical documents that contain data in the form of data other than text data.

[0041] Furthermore, the language model M3 may be a general-purpose language model that can be used for purposes other than inferring the consistency of the content of the description, or it may be a general-purpose language model that has been fine-tuned for inferring the consistency of the content of the description.

[0042] Furthermore, in this exemplary embodiment, we describe an example in which the medical documents generated by the document generation unit 104A are subject to detection by the detection unit 102A and consistency determination by the consistency determination unit 105A. However, the medical documents subject to consistency determination only need to be generated using the original data. In other words, the entity that generates the medical documents subject to detection by the detection unit 102A and consistency determination by the consistency determination unit 105A and the method of generation are arbitrary.

[0043] The reception unit 106A accepts various operations related to supporting the verification of medical documents. For example, the reception unit 106A accepts an operation to specify a part of the description contained in the medical document or the original data. The method of accepting operations is arbitrary. For example, the reception unit 106A may accept operations via the input unit 13A, or it may accept operations from other devices via the communication unit 12A.

[0044] The output control unit 107A displays various information related to supporting the verification of medical documents. For example, as described above, the output control unit 107A displays the medical document and the original data. Also, for example, the output control unit 107A displays the detection results of the detection unit 102A and the judgment results of the consistency determination unit 105A.

[0045] If the output unit 14A has a function to display and output an image, the output control unit 107A may cause the output unit 14A to display the data described above. Alternatively, the output control unit 107A may cause the data described above to be displayed on an external display device of the information processing device 1A (for example, a display device provided on a terminal device used by the user) via the communication unit 12A.

[0046] Furthermore, the method of presenting information is arbitrary and not limited to display. For example, the output control unit 107A can present information in any manner, such as display, printing, audio, or a combination thereof. In other words, the output control unit 107A has the discretion to decide what device to output and in what manner the information, such as the detection results from the detection unit 102A and the judgment results from the matching determination unit 105A, to.

[0047] As described above, the information processing device 1A includes an identification unit 101A that identifies corresponding descriptions in the original data showing the patient's medical treatment history and the medical document generated using the original data, and a detection unit 102A that detects descriptions for which the identification unit 101A did not identify corresponding descriptions. Therefore, the information processing device 1A has the effect of making it possible to easily verify medical documents, or more specifically, to verify whether there are any descriptions that have been omitted from the medical document and / or descriptions that have been mistakenly included in the medical document, similar to the information processing device 1.

[0048] Furthermore, as described above, the information processing device 1A includes a consistency determination unit 105A that uses a language model M3 trained on natural language to determine whether the content of the set of descriptions identified by the identification unit 101A is consistent. This provides the added benefit of simplifying the process of checking for descriptions in medical documents that are inconsistent with the original data, in addition to the effects of the information processing device 1.

[0049] Furthermore, the determination by the consistency determination unit 105A can be omitted. If the determination by the consistency determination unit 105A is omitted, the output control unit 107A only needs to output the detection result from the detection unit 102A. This allows the user of the information processing device 1A to efficiently check for any descriptions that have been omitted from the medical document and / or descriptions that have been mistakenly included in the medical document by referring to the outputted detection result.

[0050] Furthermore, as described above, the information processing device 1A includes a consistency determination unit 105A that uses a language model trained on natural language to determine whether the content of the generated medical document is consistent with the content of the source data from which the medical document was generated, and an output control unit 107A that outputs the determination result of the consistency determination unit 105A. Therefore, it is possible to simplify the process of verifying medical documents, or more specifically, the process of checking whether there are any descriptions in the medical document that are inconsistent with the source data.

[0051] (Specific example of processing) An example of processing performed by the information processing device 1A will be explained based on Figure 4. Figure 4 is a diagram showing an example of processing performed by the information processing device 1A. In the example in Figure 4, the original data d1 is input to the information processing device 1A. The original data d1 is data showing the medical history of a patient whose name is "XXXX" and whose date of birth is "yyyy / mm / dd" in natural language. The input original data d1 is acquired by the acquisition unit 103A provided in the information processing device 1A.

[0052] In the example shown in Figure 4, the document generation unit 104A generates a medical document (specifically, a medical certificate) d2 from the acquired source data d1. The information processing device 1A then assists in verifying the generated medical document d2 for any omissions or errors and for consistency with the source data d1. This assistance is mainly provided by the determination unit 101A, the detection unit 102A, and the consistency determination unit 105A. Furthermore, the feature information generation model M1, the similarity estimation model M2, and the language model M3 shown in Figure 4 are used for this assistance.

[0053] Furthermore, the language model M3 may also be used to generate medical documents d2. These models may be stored in the memory unit 11A of the information processing device 1A, or they may be stored in an external server or the like. In the latter case, the information processing device 1A utilizes each model via the server or the like that stores the models.

[0054] The feature information generation model M1 is a pre-trained model that has been trained to generate feature information that describes the characteristics of the input sentence. For example, a Bi-Encoder, which generates embedding vectors that describe the characteristics of the input sentence, can be used as the feature information generation model M1.

[0055] The similarity estimation model M2 is a pre-trained model that outputs the similarity of sentence content for input pairs of sentences. Such a model can be generated by machine learning using training data that associates similar pairs of sentences with ground truth labels indicating that those sentences are similar, and training data that associates dissimilar pairs of sentences with ground truth labels indicating that those sentences are dissimilar. For example, a Cross-Encoder can be used as the similarity estimation model M2.

[0056] As will be explained in detail later based on Figure 5, the determination unit 101A uses a combination of the feature information generation model M1 and the similarity estimation model M2 to identify descriptions in the original data d1 and medical document d2 that correspond in content, or more specifically, descriptions that are similar in content. In this way, since the feature information generation model M1 and the similarity estimation model M2 are used in combination, they may be configured as a single model.

[0057] As described above, the language model M3 is a pre-trained model that has been trained on natural language. The consistency determination unit 105A uses the language model M3 to determine whether the content of the set of descriptions identified by the identification unit 101A is consistent. The consistency determination using the language model M3 will be explained with reference to Figure 6.

[0058] The output control unit 107A then presents the user with the results of the determination and detection performed by the determination unit 101A, the detection unit 102A, and the matching determination unit 105A. For example, the output control unit 107A may display an image showing these determination and detection results. The images that the output control unit 107A displays will be described later based on Figures 7 and 8.

[0059] (Method for identifying the description of the corresponding content) The method for identifying corresponding content descriptions using the feature information generation model M1 and the similarity estimation model M2 will be explained with reference to Figure 5. Figure 5 is a diagram illustrating the method for identifying corresponding content descriptions using the feature information generation model M1 and the similarity estimation model M2. As mentioned above, the identification of corresponding content descriptions is performed by the identification unit 101A.

[0060] As shown in the figure, the identification unit 101A inputs the original data d1 and the medical document d2 into the feature information generation model M1. At this time, the identification unit 101A divides the original data d1 into chunks of content and inputs each of the resulting divisions into the feature information generation model M1. For example, the identification unit 101A may divide the sentences contained in the original data d1 into multiple sentences by separating them with periods, etc., and input each of the resulting sentences into the feature information generation model M1. The same applies to the medical document d2. As a result, for each coherent description (e.g., a sentence) in the original data d1 and the medical document d2, feature information indicating the characteristics of that description is output from the feature information generation model M1.

[0061] Next, the identification unit 101A calculates the similarity of the feature information generated by the feature information generation model M1. More specifically, the identification unit 101A calculates the similarity between one feature information generated from the original data d1 and one feature information generated from the medical document d2, for each combination of multiple feature information generated from the original data d1 and the medical document d2, respectively. For example, if 100 and 90 feature information are generated from the original data d1 and the medical document d2, respectively, the similarity will be calculated for 100 × 90 = 9000 combinations of feature information. The method for calculating the similarity of the feature information is arbitrary. For example, if the feature information is represented as a vector, the identification unit 101A may calculate the cosine similarity.

[0062] Next, the identification unit 101A pairs corresponding descriptions of content in the original data d1 and medical document d2 based on the calculated similarity. For example, the identification unit 101A may pair one description contained in medical document d2 with a predetermined number of feature information (feature information of original data d1) that have a high similarity to the feature information of that description. If the predetermined number is 3, three pairs will be generated for one description contained in medical document d2. Alternatively, one description contained in original data d1 may pair with a predetermined number of feature information (feature information of medical document d2) that have a high similarity to the feature information of that description. These processes can also be described as processes for selecting pairs of descriptions with high similarity of feature information, or processes for extracting candidate descriptions of corresponding content.

[0063] Next, the identification unit 101A inputs the pairs of descriptions from the source data d1 and the medical document d2, generated as described above, into the similarity estimation model M2. The similarity estimation model M2 then outputs the similarity score of the input descriptions. Based on the output similarity score, the identification unit 101A identifies the corresponding description. For example, if the identification unit 101A generates three pairs for one description in the medical document d2, it may identify the pair whose calculated similarity score is above a predetermined threshold as the corresponding description. In this case, multiple descriptions may be identified as corresponding descriptions. Alternatively, the identification unit 101A may identify the pair with the highest similarity score among the multiple pairs of descriptions generated for one description in the medical document d2 as the corresponding description.

[0064] Alternatively, the identification unit 101A may use the similarity estimation model M2 to identify the corresponding content description without using the feature information generation model M1. In this case, the identification unit 101A can divide the original data d1 and the medical document d2 into chunks of content and input the descriptions of each resulting division into the similarity estimation model M2 as sets. For example, if 10 and 9 descriptions are obtained from the original data d1 and medical document d2 respectively, the identification unit 101A can input 10 × 9 = 90 sets of descriptions into the similarity estimation model M2.

[0065] However, generally speaking, the process of calculating the similarity between individual descriptions using the similarity estimation model M2 is more accurate but time-consuming compared to the process of generating feature information using the feature information generation model M1 and calculating the similarity between the generated feature information. Therefore, as shown in the example in Figure 5, it is preferable to first narrow down the set of descriptions to be input to the similarity estimation model M2 using the feature information generation model M1, and then use the similarity estimation model M2 to identify the descriptions of the corresponding content.

[0066] As described above, the identification unit 101A may use a similarity estimation model M2, which has been trained to output the similarity of sentence content for the input set of sentences, to identify the corresponding content description. This provides the effect of being able to identify the corresponding content description with high accuracy, in addition to the effects performed by the information processing device 1.

[0067] Furthermore, as described above, the identification unit 101A may use a feature information generation model M1, which has been trained by machine learning to generate feature information that indicates the characteristics of the input sentence, to generate feature information for each description contained in the original data d1 and the medical document d2, and input the selected sentence set based on the similarity of each generated feature information into the similarity estimation model M2. This provides the effect of achieving both processing speed and accuracy in identifying descriptions whose content corresponds, in addition to the effects performed by the information processing device 1.

[0068] (Method for determining consistency) The method for determining consistency using the language model M3 will be explained based on Figure 6. Figure 6 is a diagram illustrating the method for determining consistency using the language model M3. Figure 6 shows examples using prompt p1 and prompt p2.

[0069] Prompt p1 is a prompt that instructs the system to determine the consistency between pairs of descriptions identified by the identification unit 101A, that is, between descriptions identified by the identification unit 101A as corresponding in content. Specifically, the "Medical Record Description" and "Medical Certificate Description" fields in prompt p1 are populated with pairs of descriptions identified by the identification unit 101A: the original data description "The cough has persisted for one month" and the medical document description "Chronic cough." Prompt p1 also includes the sentence, "Determine whether the following description in the medical record is consistent with the following description in the medical certificate, and provide your result," which instructs the system to determine the consistency of the content of the two descriptions.

[0070] Furthermore, prompt p1 includes the sentence, "You are a physician and are currently checking a medical certificate created based on a patient's medical record." While including such a sentence is not mandatory, it is expected to improve inference accuracy. The wording in prompt p1 can be modified as appropriate within the range required to obtain the desired inference result. For example, the consistency determination unit 105A may generate prompts with different inference instructions depending on the type of medical document, the type of source data, and the language model M3 used.

[0071] Furthermore, the consistency determination unit 105A may generate a prompt that includes an answer format and instructs the user to respond in that format. By specifying the answer format in this way, it becomes possible to obtain inference results in a desired format. For example, the consistency determination unit 105A may generate a prompt that includes a sentence instructing the user to respond with either "matches" or "does not match". This prevents the language model M3 from outputting any answers other than "matches" or "does not match".

[0072] In addition, in prompt p1, everything except the content of "medical record description" and "medical certificate description" is standard. For this reason, the parts of prompt p1 other than the content of "medical record description" and "medical certificate description" may be stored as a standard template in the storage unit 11A or the like. This allows the consistency determination unit 105A to generate prompt p1 by inputting the set of descriptions identified by the identification unit 101A, i.e., the descriptions corresponding to the content of the original data and the medical document, into the above template.

[0073] The consistency determination unit 105A inputs the prompt p1 generated as described above to the language model M3. As a result, the language model M3 outputs the inference result regarding the consistency of the set of descriptions identified by the identification unit 101A. In the example in Figure 6, output data o1 indicating that the set is inconsistent is output from the language model M3.

[0074] The format in which the inference results are output can be specified by a prompt. For example, the consistency determination unit 105A may generate a prompt instructing the user to answer with one of three options: consistent, neutral, or inconsistent. In this prompt, "neutral" means that it cannot be said to be consistent or inconsistent. The processing to be performed when a neutral answer is output can be predetermined. For example, the output control unit 107A may present the user with a set of descriptions for which a neutral answer was output and ask the user whether those descriptions are consistent or not. Alternatively, for example, the consistency determination unit 105A may generate a prompt instructing the user to output a numerical value (for example, a number between 0 and 1) indicating the likelihood that the contents of the set of descriptions are consistent. In this case, the consistency determination unit 105A should determine that the set of descriptions is consistent if the output numerical value is above a predetermined threshold.

[0075] As described above, the consistency determination unit 105A may generate a prompt instructing the identification unit 101A to determine the consistency of the set of descriptions it identifies, and then determine whether the contents of those descriptions are consistent based on the output obtained by inputting the generated prompt into the language model M3. This provides the effect of obtaining accurate inference results regarding the consistency of the description contents, in addition to the effects performed by the information processing device 1.

[0076] Furthermore, the consistency determination unit 105A can also determine the consistency between the description of the original data and the description of the medical document without using the specific result of the identification unit 101A. The prompt p2 shown in Figure 6 is an example of a prompt for determining the consistency between the description of the original data and the description of the medical document without using the specific result of the identification unit 101A.

[0077] Prompt p2 is generally similar in content to prompt p1, but differs in that it includes the entire text of the medical record and diagnosis report, i.e., the original data and medical documents, rather than just excerpts from them. Prompt p2 also differs from prompt p1 in that it instructs the user to respond with inconsistent statements along with the reasoning behind that inconsistency. Prompt p2 can be generated by inputting the original data and medical documents into a predetermined template. As mentioned above, "neutral" may be included as an output option in the prompt. In that case, the prompt may also include a sentence instructing the user to respond with the reasoning behind the inference that the statement is neutral.

[0078] By inputting the prompt p2 described above into the language model M3, the language model M3 outputs an inference result regarding whether the medical document contains descriptions that are inconsistent with the original data. In the example in Figure 6, output data o2 is output from the language model M3. Output data o2 lists descriptions in the medical certificate that are inconsistent with the descriptions in the medical record. Furthermore, the reasoning behind the inconsistency is given for each listed description. As will be described in detail later, the output control unit 107A may present the above reasoning output by the language model M3 to the user as a basis for judging the validity of the inference result.

[0079] (Regarding iterative reasoning) Since the language model M3 is a probabilistic model, even if the exact same prompt is entered, the inference results in multiple inferences may differ. Furthermore, it has been found that inference results that differ from the facts tend not to be output repeatedly. For this reason, the consistency determination unit 105A may perform the process of inputting a prompt to the language model M3 and outputting the inference results multiple times. In this case, the consistency determination unit 105A may determine that the content is inconsistent for sets of descriptions with large variations in the inference results. For example, the consistency determination unit 105A may calculate a score for each set of descriptions that indicates the magnitude of the variation in the inference results (for example, the proportion of inference results that differ in content from other inference results out of all inference results), and determine that the content is inconsistent for sets of descriptions whose calculated score exceeds a predetermined threshold.

[0080] (Display screen example 1) An example of a display screen shown by the output control unit 107A will be explained with reference to Figure 7. Figure 7 is a diagram showing an example of a display screen shown by the output control unit 107A. The example screen Img1 shown in Figure 7 includes a display area 701 for displaying the original data and a display area 702 for displaying the medical document. In addition, in the example screen Img1, the detection result of the detection unit 102A is shown on the original data shown in the display area 701, and the judgment result of the consistency determination unit 105A is shown on the medical document shown in the display area 702.

[0081] Specifically, in the source data shown in display area 701, the word "bloody sputum" is marked, making it distinguishable from other descriptions. This indicates that the detection unit 102A detected the word "bloody sputum," meaning that no description corresponding to the word "bloody sputum" was identified in the medical document. The word "bloody sputum" is included in the source data but not in the medical document, suggesting that it may have been omitted from the medical document.

[0082] Furthermore, in the medical document shown in display area 702, the phrases "chronic" and "chest CT scan" are underlined, making them distinguishable from other phrases. This indicates that the consistency determination unit 105A has determined that these phrases are inconsistent with the phrases in the original data.

[0083] In this manner, it is preferable that the output control unit 107A displays both the original data and the medical document, and displays the detection results of the detection unit 102A and the judgment results of the consistency determination unit 105A on the displayed original data and medical document. This allows the user to smoothly check the appropriateness of the content of the medical document and revise it while comparing the original data and the medical document.

[0084] The manner in which the detection results of the detection unit 102A and the judgment results of the consistency determination unit 105A are displayed is arbitrary and is not limited to the example in Figure 7. For example, the output control unit 107A may highlight the detection results of the detection unit 102A and the judgment results of the consistency determination unit 105A on the original data and medical document in a manner different from the example in Figure 7 (for example, by changing the display color and / or font of the characters). Thus, the manner in which a description can be distinguished from other descriptions is arbitrary. This is also true in the example in Figure 8, which will be described later. In addition, for example, the output control unit 107A may display the detection results of the detection unit 102A and the judgment results of the consistency determination unit 105A in a display area other than display areas 701 and 702, or on a separate screen, etc.

[0085] (Display screen example 2) Furthermore, the output control unit 107A may display the identification result of the identification unit 101A, or the basis for the determination of the consistency determination unit 105A. This will be explained with reference to Figure 8. Figure 8 is a diagram showing another example of a display screen displayed by the output control unit 107A. The example screen Img2 shown in Figure 8 includes a display area 801 for displaying the original data and a display area 802 for displaying the medical document.

[0086] Furthermore, in the example screen Img2, a portion of the medical document description displayed in display area 802 is specified by the cursor Cur. The specified description is also marked, making it distinguishable from other descriptions. In addition, in display area 801, the description of the source data corresponding to the description specified by the cursor Cur is marked in the same way as the specified description, making it distinguishable from other descriptions.

[0087] Specifically, the description "A chest CT scan was performed on 2024 / 3 / 3." from the medical document displayed in display area 802 is specified by the cursor Cur. The operation to specify the description is received by the reception unit 106A. As a result of specifying the above description, it is highlighted in display area 802 by marking. In addition, as a result of this specification, the descriptions "2024 / 3 / 3" and "A bronchoscopy was performed." from the original data displayed in display area 801 are also highlighted by marking. These highlighted descriptions are those whose content has been identified as corresponding by the identification unit 101A.

[0088] Thus, the output control unit 107A may display both the medical document and the original data, and in response to an operation to specify a part of the displayed medical document, it may display the content corresponding to the specified description in a way that is distinguishable from other descriptions. This provides the added benefit of enabling a smoother process of comparing the description in the medical document with the description in the original data to verify consistency, in addition to the effects of the information processing device 1.

[0089] Furthermore, in the example in Figure 8, the specified description includes the underlined description, that is, the description that the consistency determination unit 105A determined to be inconsistent with the description in the original data (specifically, the description "chest CT scan"). In this case, the output control unit 107A may, as shown in the figure, display the basis for the reasoning that the specified description is inconsistent with the description in the original data, in association with the specified description. As explained with reference to Figure 6, the consistency determination unit 105A can output the basis for the reasoning regarding consistency to the language model M3. Therefore, the output control unit 107A only needs to display the basis output by the language model M3 as the basis information 803.

[0090] Furthermore, the reception unit 106A may also accept operations to specify a portion of the original data. In this case, the output control unit 107A may, in response to the operation to specify a portion of the displayed original data, display the medical document description corresponding to the specified description in a way that makes it distinguishable from other descriptions. Also, if it is determined that the specified description (or a portion thereof) in the original data is inconsistent with the medical document description, the output control unit 107A may display supporting information indicating the basis for this determination.

[0091] Thus, the consistency determination unit 105A may also output the basis for the consistency determination to the language model M3. The output control unit 107A may then display the above basis when an operation is performed to specify a description that the consistency determination unit 105A has determined to be inconsistent. This provides the effect that, in addition to the effects performed by the information processing device 1, the user can refer to the displayed basis to confirm whether the original data and the description in the medical document are consistent. As mentioned above, the consistency determination result may include "neutral," and the basis for determining neutrality can also be output to the language model M3. For this reason, the output control unit 107A may display the basis for determining neutrality when an operation is performed to specify a description that has been determined to be neutral.

[0092] (Process flow: overall) The processing flow performed by the information processing device 1A will be explained with reference to Figure 9. Figure 9 is a flowchart showing the processing flow performed by the information processing device 1A. The flowchart in Figure 9 includes each process of the verification support method according to this exemplary embodiment.

[0093] In S11, the acquisition unit 103A acquires the source data, which is the data from which the medical document is generated. Subsequently, in S12, the document generation unit 104A generates a medical document from the source data acquired in S11.

[0094] In S13 (specific processing), the specific unit 101A identifies the corresponding descriptions in the original data acquired in S11 and the medical document generated in S12. Details of the processing in S13 will be described later based on Figure 10.

[0095] In S14 (detection process), the detection unit 102A detects descriptions for which a corresponding description was not identified in S13. In S14, the detection unit 102A may also detect descriptions in the original data for which a corresponding description was not identified in the medical document, that is, descriptions suspected of being omitted from the medical document. In addition, in S14, the detection unit 102A may also detect descriptions in the medical document for which a corresponding description was not identified in the original data, that is, descriptions suspected of being mistakenly included in the medical document. Furthermore, the detection unit 102A may detect both descriptions suspected of being omitted from the medical document and descriptions suspected of being mistakenly included in the medical document.

[0096] In S15, the consistency determination unit 105A generates a prompt instructing the system to determine the consistency of the set of descriptions (the set of descriptions with corresponding content) identified in S13. For example, the consistency determination unit 105A may generate a prompt like prompt p1 in Figure 6, which includes the set of descriptions identified in S13. Alternatively, the consistency determination unit 105A may generate a prompt like prompt p2 in Figure 6, for example, without using the identification result from S13.

[0097] In S16, the consistency determination unit 105A inputs the prompt generated in S15 to the language model M3 and, based on the output obtained, determines whether the content of the description in the original data and the medical document are consistent.

[0098] In S17, the output control unit 107A outputs the detection results from S14 (descriptions suspected of being omitted from the medical document and / or descriptions suspected of being mistakenly included in the medical document) and the judgment results from S16 (descriptions determined to be inconsistent). For example, the output control unit 107A may display the medical document and the original data, as shown in the example screen Img1 in Figure 7, and may also display the above-mentioned detection results and judgment results on the displayed medical document and original data. The output control unit 107A may also display as a judgment result descriptions that the consistency judgment unit 105A has determined to be neutral.

[0099] In S18, the reception unit 106A determines whether an operation to specify the description of a medical document has been performed. The operation to specify the description may be performed using a cursor with an input device such as a mouse, as shown in the example in Figure 8, or it may be performed using other input devices such as a keyboard or touch panel. If the result in S18 is YES, the process proceeds to S19; if the result in S18 is NO, the process proceeds to S22.

[0100] In S19, the output control unit 107A highlights the description of the source data corresponding to the specified description. The highlighting should be performed in a manner that makes the target description distinguishable from other descriptions.

[0101] In S20, the output control unit 107A determines whether the specified description contains a description that is inconsistent with the description in the original data. Specifically, the output control unit 107A determines whether the specified description contains a description that was determined to be inconsistent in S16. If the result in S20 is YES, the process proceeds to S21; if the result in S20 is NO, the process proceeds to S22.

[0102] In S21, the output control unit 107A displays the reasoning behind its inference that the specified description is inconsistent with the description in the original data. This reasoning can be output to the language model M3 as described above. For example, the output control unit 107A may display the reasoning information in association with the specified description, as shown in the example in Figure 8.

[0103] In S22, the output control unit 107A determines whether or not to terminate the display. In S22, for example, if the reception unit 106A receives a predetermined operation to terminate the display, it is determined to terminate the display. If the result in S22 is YES, the process in Figure 10 ends. On the other hand, if the result in S22 is NO, the process returns to S18.

[0104] As described in Exemplary Embodiment 1, in addition to descriptions that were mistakenly included or omitted, descriptions whose content has been altered to the extent that it is not possible to identify the corresponding description, or descriptions that are unnecessary for medical documents, such as greetings and interjections, may also not be detected as corresponding descriptions. For this reason, the detection unit 102A may be configured to detect descriptions that have been altered or descriptions that are unnecessary for medical documents from the descriptions for which the identification unit 101A could not identify the corresponding content, thereby removing at least one of them.

[0105] For example, to exclude descriptions whose content has been altered, before performing the processing in S14, the consistency determination unit 105A can use the original data acquired in S11 and the medical document generated in S12 to generate a prompt, such as prompt p2 in Figure 6, for determining the consistency between the description in the original data and the description in the medical document. The consistency determination unit 105A then inputs the generated prompt to the language model M3 and, based on the output of the language model M3, detects descriptions in the medical document that do not match the description in the original data. This process makes it possible to detect descriptions whose content has been altered.

[0106] In this case, in S14, the detection unit 102A detects descriptions for which the corresponding content description was not identified in S13, and the consistency determination unit 105A detects the remaining descriptions after removing the descriptions detected as described above from the detected descriptions.

[0107] Thus, the detection unit 102A may detect descriptions from the medical document that have been identified by the identification unit 101A (i.e., descriptions that correspond to the original data) and descriptions detected by the consistency determination unit 105A (i.e., descriptions that do not match the original data), leaving only the remaining descriptions. This makes it possible to focus detection on descriptions that are highly likely to have been omitted from the update.

[0108] (Processing flow: S13) The details of the process in S13 of Figure 9 will be explained based on Figure 10. Figure 10 is a flowchart showing the details of the process in S13 of Figure 9.

[0109] In S131, the identification unit 101A divides the raw data acquired in S11 of Figure 9 into chunks of content, and inputs each resulting division into the feature information generation model M1 to generate feature information for each division. Similarly, the identification unit 101A divides the medical document generated in S12 of Figure 9 into chunks of content, and inputs each resulting division into the feature information generation model M1 to generate feature information for each division.

[0110] In S132, the identification unit 101A calculates the similarity of the feature information generated in S131. More specifically, the identification unit 101A calculates the similarity between one feature information generated from the original data and one feature information generated from the medical document for each combination of multiple feature information generated from the original data and the medical document, respectively.

[0111] In S133, the identification unit 101A selects one feature information of the medical document description from the feature information generated in S131. Next, in S134, the identification unit 101A identifies a predetermined number of feature information from the original data feature information generated in S131 that have a high similarity (calculated in S132) to the feature information selected in S133, and selects a description (description of the original data) corresponding to each identified feature information.

[0112] In S135, the identification unit 101A inputs a pair of descriptions selected in S133 and one of the descriptions selected in S134 into the similarity estimation model M2. This process is performed for each of the descriptions selected in S134. For example, if three descriptions are selected in S134, then in S135, three pairs of descriptions are input into the similarity estimation model M2, and the similarity score for each pair is output from the similarity estimation model M2.

[0113] In S136, the identification unit 101A identifies the description of the source data corresponding to the description of the medical document (the description of the medical document corresponding to the feature information selected in S133) based on the similarity output from the similarity estimation model M2 by the processing in S135. For example, the identification unit 101A may identify a set of descriptions in which the similarity output in S135 is equal to or greater than a predetermined threshold as the description of the corresponding content. However, if the description of the medical document corresponding to the feature information selected in S133 is incorrectly included, the description of the corresponding content will not be identified in S136.

[0114] In S137, the identification unit 101A determines whether to terminate the process of identifying the description of the corresponding content. In S137, the identification unit 101A determines to terminate the process if the processes from S133 to S136 have been executed for all of the characteristic information of the medical document description generated in S131. If the result in S137 is YES, the process in Figure 10 is terminated; if the result in S137 is NO, the process returns to S133. In S133, which is the transitioned step from S137, the identification unit 101A selects one characteristic from the unselected characteristic information among the characteristic information of the medical document description generated in S131.

[0115] Note that in S133 of Figure 10, characteristic information of the medical document description is selected, but characteristic information of the source data may also be selected. In this case, in S134, a predetermined number of medical document descriptions corresponding to the most similar characteristic information of the medical document generated in S131 to the characteristic information selected in S133 are selected. In this case, in S136, if the description of the source data corresponding to the characteristic information selected in S133 is not reflected in the medical document, the description of the corresponding content will not be identified.

[0116] [Reference example 1] Figure 11 is a block diagram showing the configuration of the information processing device 1B according to this reference example. As shown in the figure, the information processing device 1B includes a matching determination unit 105B and an output control unit 107B.

[0117] Similar to the consistency determination unit 105A in Exemplary Embodiment 2, the consistency determination unit 105B uses a language model M3 trained on natural language to determine whether the content of the description in the generated medical document is consistent with the content of the description in the source data that generated the medical document. The medical document subject to consistency determination only needs to be one generated using the source data, and the entity and method of generating the medical document are arbitrary, as in Exemplary Embodiment 2. Furthermore, the consistency determination may be performed for each pair of descriptions extracted from the medical document and descriptions extracted from the source data, or it may be performed on the entire medical document and the entire source data, as in Exemplary Embodiment 2. In the former case, similar to Exemplary Embodiment 2, the identification unit 101A and the detection unit 102A may be used to pair corresponding descriptions in the medical document and the source data. The identification unit 101A and the detection unit 102A may be provided by the information processing device 1B or by other devices. In the latter case, the other device mentioned above can be used to perform the process of pairing descriptions of corresponding content between medical documents and the original data.

[0118] The output control unit 107B outputs the determination result of the matching determination unit 105B, similar to the output control unit 107A in the exemplary embodiment 2. Similar to the exemplary embodiment 2, the device and manner in which the determination result is output are arbitrary.

[0119] As described above, the information processing device 1B includes a consistency determination unit 105B that uses a language model M3 trained on natural language to determine whether the content of the generated medical document is consistent with the content of the source data from which the medical document was generated, and an output control unit 107B that outputs the determination result of the consistency determination unit 105B. This information processing device 1B has the effect of making it possible to simplify the verification process of medical documents.

[0120] (Verification Support Program) The functions of the information processing device 1B described above can also be implemented by a program. The verification support program in this reference example is a medical document verification support program in which a computer functions as a consistency determination means that uses a language model M3 trained on natural language to determine whether the content of the description in the generated medical document is consistent with the content of the description in the source data that generated the medical document, and as an output control means that outputs the determination result of the consistency determination means. This verification support program has the effect of making the verification work of medical documents easier.

[0121] (Verification support method) The verification support method described in this reference example is a medical document verification support method, in which at least one processor performs a consistency determination process that uses a language model M3 trained on natural language to determine whether the content of the generated medical document description is consistent with the content of the source data that generated the medical document, and an output control process that outputs the determination result of the consistency determination process. This verification support method has the effect of making the verification work of medical documents easier.

[0122] [Reference example 2] The above-described exemplary embodiments and Reference Example 1 illustrate an example in which information processing devices 1, 1A, and 1B support the verification of medical documents generated from source data. These information processing devices 1, 1A, and 1B can be used to support the verification of any document, in addition to supporting the verification of medical documents.

[0123] For example, the information processing devices 1, 1A, and 1B can also assist in the verification of design documents generated based on specifications. In this case, the specifications can be applied instead of the original data in the exemplary embodiments and reference example 1 described above, and the design documents generated based on those specifications can be applied instead of the medical documents. This makes it easier to verify that the design documents conform to the specifications.

[0124] Furthermore, for example, the information processing devices 1, 1A, and 1B can also assist in the verification of the summary text. In this case, instead of the original data in the exemplary embodiments and reference example 1 described above, the document to be summarized can be applied, and instead of the medical document, the summary text obtained by summarizing the said document can be applied. This makes the verification of the summary text easier.

[0125] [Variation] The entities executing each process described in the above-described exemplary embodiments and reference examples are arbitrary and not limited to the examples above. For example, a system having the same functions as information processing devices 1, 1A, and 1B can be constructed using multiple devices that can communicate with each other. Also, the entities executing each process shown in the flowcharts of Figures 9 and 10 may be a single device (which can also be called a processor) or multiple devices (which can also be called processors).

[0126] [Examples of implementation using software] Some or all of the functions of the information processing devices 1, 1A, and 1B (hereinafter also referred to as "the above devices") may be implemented by hardware such as integrated circuits (IC chips) or by software.

[0127] In the latter case, each of the above devices is implemented, for example, by a computer that executes instructions for a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 11. Figure 11 is a block diagram showing the hardware configuration of computer C, which functions as each of the above devices.

[0128] Computer C comprises at least one processor C1 and at least one memory C2. Memory C2 stores a program (verification support program) P for operating computer C as each of the above-mentioned devices. In computer C, processor C1 reads program P from memory C2 and executes it, thereby realizing each of the above-mentioned devices.

[0129] For processor C1, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), TPU (Tensor Processing Unit), quantum processor, microcontroller, or a combination thereof can be used. For memory C2, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof can be used.

[0130] Computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Furthermore, computer C may be equipped with communication interfaces for sending and receiving data with other devices. Additionally, computer C may be equipped with input / output interfaces for connecting input / output devices such as keyboards, mice, displays, and printers.

[0131] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0132] Furthermore, each of the above functions of each of the above devices may be implemented by a single processor in a single computer, by multiple processors in a single computer working together, or by multiple processors in each of multiple computers working together. In addition, the programs for implementing each of the above functions in each of the above devices may be stored in a single memory in a single computer, distributed and stored in multiple memories in a single computer, or distributed and stored in multiple memories in each of multiple computers.

[0133] [Additional Notes] This disclosure includes the technologies described in the following appendices. However, the present invention is not limited to the technologies described in the following appendices, and various modifications are possible within the scope of the claims.

[0134] (Note A1) An information processing device comprising: identification means for identifying corresponding descriptions in the original data and the medical document of the patient, with respect to the original data showing the patient's medical history and the medical document of the patient generated using the original data; and detection means for detecting descriptions for which the corresponding descriptions were not identified by the identification means.

[0135] (Appendix A2) The information processing device described in Appendix A1, wherein the identification means identifies a description of corresponding content using a similarity estimation model trained to output the similarity of the content of the input sentences.

[0136] (Note A3) The information processing device described in Appendix A2, wherein the identifying means generates feature information for each description contained in the original data and the medical document, respectively, using a feature information generation model trained to generate feature information indicating the characteristics of the input sentence, and inputs a set of sentences selected based on the similarity of the generated feature information into the similarity estimation model.

[0137] (Note A4) An information processing device according to any one of the appendices A1 to A3, comprising a consistency determination means for determining whether the content of a set of descriptions identified by the aforementioned identification means is consistent, using a language model that has been trained on natural language.

[0138] (Note A5) The information processing device described in Appendix A4, wherein the consistency determination means generates a prompt instructing the identification means to determine the consistency of a set of descriptions identified by the identification means, and determines whether the contents of those descriptions are consistent based on the output obtained by inputting the generated prompt into the language model.

[0139] (Note A6) The information processing device described in Appendix A5, wherein the consistency determination means also causes the language model to output the basis for the consistency determination, and the output control means displays the basis in response to an operation being performed to specify a description that the consistency determination means has determined to be inconsistent.

[0140] (Note A7) An information processing device according to any one of the appendices A1 to A6, comprising output control means for displaying the medical document and the original data together, and for displaying a description of content corresponding to the specified description in a manner that can be distinguished from other descriptions when an operation is performed to specify a part of the description of the displayed medical document or a part of the description of the displayed original data.

[0141] (Note A8) An information processing device comprising: a consistency determination means that determines whether the content of the description in the generated medical document is consistent with the content of the description in the source data from which the medical document was generated, using a language model trained on natural language; and an output control means that outputs the determination result of the consistency determination means.

[0142] A method for supporting the verification of medical documents, wherein at least one processor performs an identification process to identify descriptions of corresponding content in source data showing a patient's medical treatment history and a medical document of the patient generated using the source data, and a detection process to detect descriptions for which no corresponding content description was identified in the identification process.

[0143] (Note B2) The method for supporting the verification of medical documents as described in Appendix B1, wherein in the specified processing, at least one processor identifies a description of the corresponding content using a similarity estimation model trained to output the similarity of the content of the input sentences.

[0144] (Note B3) The method for supporting verification of a medical document as described in Appendix B2, wherein in the specified processing, the at least one processor generates feature information for each description contained in the original data and the medical document using a feature information generation model trained to generate feature information that indicates the characteristics of the input sentence, and inputs a set of sentences selected based on the similarity of the generated feature information into the similarity estimation model.

[0145] (Note B4) A method for supporting the verification of medical documents as described in any of Appendix B1 to B3, which includes a consistency determination process in which at least one processor determines whether the contents of a set of descriptions identified in the specific process are consistent, using a language model trained on natural language.

[0146] (Note B5) The method for supporting verification of medical documents as described in Appendix B4, wherein in the consistency determination process, at least one processor generates a prompt instructing the system to determine the consistency of the set of descriptions identified in the specific process, and determines whether the contents of those descriptions are consistent based on the output obtained by inputting the generated prompt into the language model.

[0147] (Note B6) The method for supporting verification of medical documents as described in Appendix B5, which includes, in the consistency determination process, the at least one processor also causes the language model to output the basis for the consistency determination, and the at least one processor displays the basis in response to an operation being performed to specify a description that was determined to be inconsistent in the consistency determination process.

[0148] (Note B7) A method for supporting verification of a medical document as described in any of Appendix B1 to B6, which includes output control processing that causes at least one processor to display the medical document and the original data together, and, in response to an operation being performed to specify a description of a part of the displayed medical document or a description of a part of the displayed original data, to display a description of the content corresponding to the specified description in a manner that can be distinguished from other descriptions.

[0149] (Note B8) A method for supporting the verification of medical documents, comprising: a consistency determination process in which at least one processor determines whether the content of the description in the generated medical document is consistent with the content of the description in the source data from which the medical document was generated, using a language model trained on natural language; and an output control process in which the at least one processor causes the determination result of the consistency determination process to be output.

[0150] (Note C1) A medical document verification support program that causes a computer to function as an identification means for identifying corresponding descriptions in the original data showing a patient's medical treatment history and the patient's medical documents generated using the original data, and a detection means for detecting descriptions for which corresponding descriptions were not identified by the identification means.

[0151] (Note C2) The aforementioned identification means is a medical document verification support program as described in Appendix C1, which identifies a description of corresponding content using a similarity estimation model trained to output the similarity of sentence content for a given set of sentences.

[0152] (Note C3) The aforementioned identification means is a medical document verification support program as described in Appendix C2, which uses a feature information generation model trained to generate feature information indicating the characteristics of an input sentence to generate feature information for each description contained in the source data and the medical document, and inputs a set of sentences selected based on the similarity of the generated feature information into the similarity estimation model.

[0153] (Note C4) A medical document verification support program as described in any of Appendix C1 to C3, wherein the computer functions as a consistency determination means that determines whether the content of a set of descriptions identified by the specified means is consistent, using a language model trained on natural language.

[0154] (Note C5) The medical document verification support program described in Appendix C4, wherein the consistency determination means generates a prompt instructing the identification means to determine the consistency of a set of descriptions identified by the identification means, and determines whether the contents of those descriptions are consistent based on the output obtained by inputting the generated prompt into the language model.

[0155] (Appendix C6) The medical document verification support program described in Appendix C5, wherein the consistency determination means also causes the language model to output the basis for the consistency determination, and the computer functions as an output control means that displays the basis in response to an operation that specifies a description that the consistency determination means has determined to be inconsistent.

[0156] (Note C7) A medical document verification support program as described in any of the appendices C1 to C6, wherein the computer is configured to function as an output control means for displaying the medical document and the original data together, and, in response to an operation being performed to specify a part of the description of the displayed medical document or a part of the displayed original data, a description of the content corresponding to the specified description is displayed in a manner that can be distinguished from other descriptions.

[0157] (Note C8) A medical document verification support program that enables a computer to function as a consistency determination means that uses a language model trained on natural language to determine whether the content of the description in a generated medical document is consistent with the content of the description in the source data from which the medical document was generated, and as an output control means that outputs the determination result of the consistency determination means.

[0158] (Note D1) An information processing device comprising at least one processor, wherein the at least one processor performs an identification process to identify descriptions of corresponding content in source data showing a patient's medical history and a patient's medical document generated using the source data, and a detection process to detect descriptions for which no corresponding content description was identified by the identification process.

[0159] The information processing device may also include memory. Furthermore, the memory may store a program that causes at least one processor to execute each of the aforementioned processes.

[0160] (Note D2) The information processing apparatus according to Appendix D1, wherein in the specified processing, at least one processor identifies a description of the corresponding content using a similarity estimation model trained to output the similarity of the content of the input sentences.

[0161] (Note D3) The information processing apparatus according to Appendix D2, wherein in the specified processing, at least one processor generates feature information for each description contained in the original data and the medical document, respectively, using a feature information generation model trained to generate feature information that indicates the characteristics of the input sentence, and inputs a set of sentences selected based on the similarity of the generated feature information into the similarity estimation model.

[0162] (Note D4) The information processing apparatus according to any one of the appendices D1 to D3, wherein at least one processor performs a consistency determination process that determines whether the contents of a set of descriptions identified by the specific process are consistent, using a language model trained on natural language.

[0163] (Note D5) The information processing apparatus according to Appendix D4, wherein in the consistency determination process, at least one processor generates a prompt instructing to determine the consistency of the set of descriptions identified in the specific process, and determines whether the contents of those descriptions are consistent based on the output obtained by inputting the generated prompt to the language model.

[0164] (Note D6) The information processing apparatus according to Appendix D5, wherein in the consistency determination process, at least one processor causes the language model to output the basis for the consistency determination, and performs output control processing to display the basis in response to an operation being performed to specify a description that was determined to be inconsistent in the consistency determination process.

[0165] (Note D7) The information processing apparatus according to any one of the appendices D1 to D6, wherein at least one processor displays the medical document and the original data together, and in response to an operation being performed to specify a description of a part of the displayed medical document or a description of a part of the displayed original data, it performs output control processing to display a description of the content corresponding to the specified description in a manner that can be distinguished from other descriptions.

[0166] (Note D8) An information processing device comprising at least one processor, wherein the at least one processor performs a consistency determination process that determines, using a language model trained on natural language, whether the content of the description of a generated medical document is consistent with the content of the description of the source data from which the medical document was generated, and an output control process that outputs the determination result of the consistency determination process.

[0167] (Note E1) A non-temporary recording medium that records a medical document verification support program that causes a computer to perform an identification process to identify corresponding descriptions in the original data and the medical document of the patient generated using the original data, and a detection process to detect descriptions for which no corresponding descriptions were identified by the identification process.

[0168] (Note E2) A non-temporary recording medium that records a medical document verification support program that causes a computer to execute a consistency determination process that determines whether the content of the description in a medical document is consistent with the content of the description in the source data from which the medical document was generated, using a language model trained on natural language, and an output control process that outputs the determination result of the consistency determination process. [Explanation of Symbols]

[0169] 1. Information Processing Device 101 Specification part (specification means) 102 Detection unit (detection means) 1A Information Processing Device 101A Specification part (specification means) 102A Detection unit (detection means) 105A Matching judgment section (matching judgment means) 107A Output control unit (output control means) M1 Feature Information Generation Model M2 Similarity Estimation Model M3 Language Model

Claims

1. A means for identifying descriptions of corresponding content in the original data showing the patient's medical history and the patient's medical document generated using said original data, An information processing apparatus comprising: detection means for detecting descriptions for which the corresponding content description was not identified by the aforementioned identification means.

2. The information processing apparatus according to claim 1, wherein the identifying means identifies a description of corresponding content using a similarity estimation model trained on machine learning to output the similarity of the content of the input sentences.

3. The information processing apparatus according to claim 2, wherein the identifying means generates feature information for each description contained in the original data and the medical document, respectively, using a feature information generation model trained to generate feature information indicating the characteristics of the input sentence, and inputs a set of sentences selected based on the similarity of the generated feature information into the similarity estimation model.

4. The information processing apparatus according to any one of claims 1 to 3, further comprising a consistency determination means for determining whether the contents of a set of descriptions identified by the identification means are consistent, using a language model trained on natural language.

5. The information processing apparatus according to claim 4, wherein the consistency determination means generates a prompt instructing the identification means to determine the consistency of a set of descriptions identified by the identification means, and determines whether the contents of those descriptions are consistent based on the output obtained by inputting the generated prompt into the language model.

6. The consistency determination means also causes the language model to output the basis for the consistency determination. The information processing apparatus according to claim 5, further comprising output control means for displaying the basis in response to an operation being performed to specify a description that the consistency determination means has determined to be inconsistent.

7. An information processing device according to any one of claims 1 to 3, comprising output control means for displaying the medical document and the original data together, and for displaying a description of content corresponding to a specified description in a manner that can be distinguished from other descriptions when an operation is performed to specify a part of the description of the displayed medical document or a part of the description of the displayed original data.

8. A consistency determination means that determines whether the content of the generated medical document is consistent with the content of the source data from which the medical document was generated, using a language model that has been trained on natural language. An information processing apparatus comprising an output control means for outputting the determination result of the aforementioned consistency determination means.

9. At least one processor, A process for identifying descriptions of corresponding content in the original data showing the patient's medical history and the patient's medical document generated using said original data, A method for supporting the verification of medical documents, comprising: a detection process for detecting descriptions for which the corresponding content was not identified in the aforementioned specific processing; and a method for supporting the verification of medical documents.

10. Computers, A means for identifying descriptions of corresponding content in the original data showing the patient's medical history and the patient's medical document generated using said original data, and A medical document verification support program that functions as a detection means for detecting descriptions for which the corresponding content description could not be identified by the aforementioned identification means.

Images