Language model training system, language model training device, language model training method, and program

Abstract

This language model training system comprises: a sentence generation unit that generates a medical sentence by inputting medical information acquired from a terminal of a medical institution in a closed network to a language model which has been machine-trained to generate the medical sentence by using the medical information as an input in order to support decision making by a doctor; a feedback reception unit that receives feedback for the medical sentence from the doctor; a consent acquisition unit that acquires consent information indicating whether there is consent to use the feedback for machine-training the language model; and a training unit that machine-trains the language model by using the feedback when the consent information indicates consent.

Description

Language model learning system, language model learning device, language model learning method, and program

[0001] The present disclosure relates to a language model training system, a language model training device, a language model training method, and a program.

[0002] Techniques are known that use user feedback in machine learning of language models.

[0003] For example, Patent Literature 1 discloses an information providing device that causes a language model to generate an answer to a question. The information providing device acquires an evaluation of the answer from the questioner and uses the evaluation for additional learning of the language model.

[0004] Patent No. 7353695

[0005] Due to the recent effects of the declining birthrate and aging population, as well as work style reforms for doctors, it is certain that there will be a significant shortage of resources in the medical field in the future. Therefore, there is a need to use language models that generate medical documents such as explanations and referral letters written by doctors for patients. In order to improve the accuracy of such language models, configurations that use user feedback, as described above, in the machine learning of language models are being considered.

[0006] In a configuration in which user feedback is used for machine learning of a language model, the greater the amount of user feedback, the higher the accuracy of the language model. Therefore, a configuration that can obtain a large amount of feedback is desirable. However, in large medical institutions, the network at the medical institution is not connected to other medical institutions and is siloed. In this case, there is a problem that the feedback obtained for use in machine learning of a language model at a large medical institution is limited to feedback from doctors within the medical institution. In view of this problem, a configuration that obtains feedback from doctors at many medical institutions is needed.

[0007] However, the configuration described in Patent Document 1 is not designed to obtain feedback from doctors at many medical institutions. In other words, the configuration described in Patent Document 1 cannot obtain feedback from doctors at many medical institutions, which poses a problem in that it is not possible to improve the accuracy of the language model that generates medical sentences.

[0008] The present disclosure has been made in view of the above-mentioned problems, and an exemplary purpose thereof is to provide a technology for improving the accuracy of a language model that generates medical sentences.

[0009] A language model learning system according to an exemplary aspect of the present disclosure includes a text generation means for generating medical text by inputting medical information acquired from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical text using medical information related to medical care as input, a feedback receiving means for receiving feedback from doctors regarding the medical text, a consent acquisition means for acquiring consent information indicating whether or not the feedback is used for machine learning of the language model, and a learning means for, if the consent information indicates consent, using the feedback to train the language model by machine learning.

[0010] A language model learning device according to an exemplary aspect of the present disclosure includes a sentence generation means for generating medical sentences by inputting medical information related to medical care acquired from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical sentences; a feedback receiving means for receiving feedback from doctors regarding the medical sentences; a consent acquisition means for acquiring consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning means for, if the consent information indicates consent, using the feedback to train the language model by machine learning.

[0011] A language model training method according to an exemplary aspect of the present disclosure includes a text generation process in which at least one processor generates medical text by inputting medical information acquired from terminals of one or more medical institutions within a closed network into a language model that has been machine-trained to generate medical text using medical information related to medical care as input; a feedback acceptance process in which the at least one processor accepts feedback from doctors regarding the medical text; a consent acquisition process in which the at least one processor acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a training process in which the at least one processor, if the consent information indicates consent, uses the feedback to train the language model on machine learning.

[0012] A program according to an exemplary aspect of the present disclosure is a program that causes a computer to function as a language model learning system, and causes the computer to function as: a text generation means that generates medical text by inputting medical information related to medical care obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical text; a feedback receiving means that receives feedback from doctors regarding the medical text; a consent acquisition means that acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning means that, if the consent information indicates consent, uses the feedback to train the language model by machine learning.

[0013] According to an exemplary aspect of the present disclosure, an exemplary effect is achieved in that a technology can be provided that improves the accuracy of a language model that generates medical sentences.

[0014] FIG. 1 is a block diagram showing the configuration of a language model learning system according to the present disclosure. FIG. 2 is a block diagram showing the configuration of a language model learning device according to the present disclosure. FIG. 3 is a flow diagram showing the flow of a language model learning method according to the present disclosure. FIG. 4 is a block diagram showing the configuration of a language model learning system according to the present disclosure. FIG. 5 is a flow diagram showing the flow of a language model learning method according to the present disclosure. FIG. 6 is a diagram showing an example of an image displayed on a terminal according to the present disclosure. FIG. 7 is a diagram showing another example of an image displayed on a terminal according to the present disclosure. FIG. 8 is a diagram showing an example of a database managed by a management unit according to the present disclosure. FIG. 9 is a block diagram showing the configuration of a language model learning system according to the present disclosure. FIG. 10 is a flow diagram showing the flow of a language model learning method according to the present disclosure. FIG. 11 is a block diagram showing the configuration of a computer functioning as a language model learning system and a language model learning device according to the present disclosure.

[0015] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0016] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for each of the exemplary embodiments described below. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0017] (Configuration of Language Model Learning System 1) The configuration of the language model learning system 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the language model learning system 1. As shown in Fig. 1, the language model learning system 1 includes a sentence generation unit 11, a feedback reception unit 12, a consent acquisition unit 13, and a learning unit 14. In this exemplary embodiment, the sentence generation unit 11, the feedback reception unit 12, the consent acquisition unit 13, and the learning unit 14 respectively realize sentence generation means, feedback reception means, consent acquisition means, and learning means.

[0018] The sentence generation unit 11 generates medical sentences by inputting medical information related to medical care into a language model that has been machine-learned to generate medical sentences, and inputting medical information obtained from terminals of one or more medical institutions within a closed network.

[0019] The feedback receiving unit 12 receives feedback from doctors regarding the medical text generated by the text generating unit 11. The feedback receiving unit 12 provides the received feedback to the learning unit .

[0020] The consent acquisition unit 13 acquires consent information indicating whether or not the user consents to using the feedback received by the feedback receiving unit 12 for machine learning of the language model. The consent acquisition unit 13 supplies the acquired consent information to the learning unit 14.

[0021] If the consent information acquired by the consent acquisition unit 13 indicates consent, the learning unit 14 uses the feedback to train a language model by machine learning.

[0022] (Effects of Language Model Learning System 1) As described above, the language model learning system 1 employs a configuration including a sentence generation unit 11 that generates medical sentences by inputting medical information related to medical care and medical information acquired from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical sentences; a feedback receiving unit 12 that receives feedback from doctors on the medical sentences generated by the sentence generation unit 11; a consent acquisition unit 13 that acquires consent information indicating whether or not the feedback received by the feedback receiving unit 12 is used for machine learning of the language model; and a learning unit 14 that, when the consent information acquired by the consent acquisition unit 13 indicates consent, trains the language model by machine learning using the feedback.

[0023] Therefore, the language model learning system 1 has the effect of improving the accuracy of the language model that generates medical sentences.

[0024] (Configuration of Language Model Learning Device 2) The configuration of the language model learning device 2 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the language model learning device 2. As shown in Fig. 2, the language model learning device 2 includes a sentence generation unit 11, a feedback reception unit 12, a consent acquisition unit 13, and a learning unit 14. In this exemplary embodiment, the sentence generation unit 11, the feedback reception unit 12, the consent acquisition unit 13, and the learning unit 14 respectively realize sentence generation means, feedback reception means, consent acquisition means, and learning means.

[0025] The sentence generation unit 11 generates medical sentences by inputting medical information related to medical care into a language model that has been machine-learned to generate medical sentences, and inputting medical information obtained from terminals of one or more medical institutions within a closed network.

[0026] The feedback receiving unit 12 receives feedback from doctors regarding the medical text generated by the text generating unit 11. The feedback receiving unit 12 provides the received feedback to the learning unit .

[0027] The consent acquisition unit 13 acquires consent information indicating whether or not the user consents to using the feedback received by the feedback receiving unit 12 for machine learning of the language model. The consent acquisition unit 13 supplies the acquired consent information to the learning unit 14.

[0028] If the consent information acquired by the consent acquisition unit 13 indicates consent, the learning unit 14 uses the feedback to train a language model by machine learning.

[0029] (Effects of Language Model Learning Device 2) As described above, the language model learning device 2 employs a configuration including: a sentence generation unit 11 that generates medical sentences by inputting medical information acquired from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical sentences using medical information related to medical care as input; a feedback receiving unit 12 that receives feedback from doctors on the medical sentences generated by the sentence generation unit 11; a consent acquisition unit 13 that acquires consent information indicating whether or not the feedback received by the feedback receiving unit 12 is used for machine learning of the language model; and a learning unit 14 that, when the consent information acquired by the consent acquisition unit 13 indicates consent, causes the language model to undergo machine learning using the feedback. Therefore, the language model learning device 2 can achieve effects similar to those of the above-described language model learning system 1.

[0030] (Flow of Language Model Training Method S1) The flow of the language model training method S1 will be described with reference to Fig. 3. Fig. 3 is a flow chart showing the flow of the language model training method S1. As shown in Fig. 3, the language model training method S1 includes a sentence generation process S11, a feedback reception process S12, a consent acquisition process S13, and a training process S15.

[0031] (Sentence generation process S11) In sentence generation process S11, the sentence generation unit 11 generates medical sentences by inputting medical information related to medical care into a language model that has been machine-learned to generate medical sentences, and inputting medical information obtained from terminals of one or more medical institutions within a closed network.

[0032] (Feedback Receiving Process S12) In the feedback receiving process S12, the feedback receiving unit 12 receives feedback from a doctor regarding the medical text generated by the text generating unit 11. The feedback receiving unit 12 provides the received feedback to the learning unit 14.

[0033] In the consent acquisition process S13, the consent acquisition unit 13 acquires consent information indicating whether or not the user consents to using the feedback received by the feedback receiving unit 12 for machine learning of the language model. The consent acquisition unit 13 supplies the acquired consent information to the learning unit 14.

[0034] (Step S14) The learning unit 14 determines whether the consent information acquired by the consent acquisition unit 13 indicates consent.

[0035] (Learning Process S15) When the consent information acquired by the consent acquisition unit 13 indicates consent, in the learning process S15, the learning unit 14 uses the feedback to train a language model by machine learning.

[0036] On the other hand, if the consent information acquired by the consent acquisition unit 13 indicates no consent, the processing in the language model training method S1 ends.

[0037] (Effects of Language Model Training Method S1) As described above, the language model training method S1 includes a sentence generation process S11 in which the sentence generation unit 11 generates medical sentences by inputting medical information related to medical care acquired from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical sentences, a feedback reception process S12 in which the feedback reception unit 12 receives feedback from doctors on the medical sentences generated by the sentence generation unit 11, a consent acquisition process S13 in which the consent acquisition unit 13 acquires consent information indicating whether the feedback received by the feedback reception unit 12 is used for machine learning of the language model, and a learning process S15 in which the learning unit 14 uses the feedback to train the language model by machine learning if the consent information acquired by the consent acquisition unit 13 indicates consent. Therefore, the language model training method S1 can achieve effects similar to those of the above-described language model training system 1.

[0038] Second Exemplary Embodiment A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be denoted by the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technical means shown in each drawing referenced to describe this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0039] (Configuration of Language Model Learning System 1A) The configuration of the language model learning system 1A will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the language model learning system 1A. As shown in Fig. 4, the language model learning system 1A includes a language model learning device 2A, a terminal 6_1, and a terminal 6_2.

[0040] Terminal 6_1 and terminal 6_2 are terminals of one or more medical institutions within the closed network. Terminal 6_1 and terminal 6_2 may be terminals of different medical institutions or may be terminals of the same medical institution. Furthermore, although the language model training system 1A shown in FIG. 4 includes two terminals, terminal 6_1 and terminal 6_2, the number of terminals is not limited and may be three or more.

[0041] Hereinafter, when there is no need to particularly distinguish between the terminal 6_1 and the terminal 6_2, they will be simply referred to as the terminal 6. The same applies to the components of the terminal 6_1 and the terminal 6_2, which will be described later.

[0042] Here, "terminals of one or more medical institutions within a closed network" refers to terminals that can communicate within a closed network that is available only to a limited number of terminals within the medical institution. Furthermore, the terminals 6 of multiple medical institutions may or may not be able to communicate with each other. Furthermore, the language model learning device 2A and the terminals 6 may be connected to each other via a virtual dedicated line (VPN (Virtual Private Network)) so that they can communicate with each other.

[0043] In the language model learning system 1A, a language model learning device 2A generates medical text by using a language model. A terminal 6 outputs feedback from a doctor on the generated medical text and consent information indicating whether the doctor agrees to using the feedback in machine learning of the language model to the language model learning device 2A. If consent is obtained from the doctor to using the feedback in learning the language model, the language model learning device 2A uses the feedback to train the language model by machine learning.

[0044] The data acquired by the language model learning device 2A from the terminal 6 may be data based on a predetermined standard. As an example, the data acquired by the language model learning device 2A from the terminal 6 may be in the FHIR (Fast Healthcare Interoperability Resources) (registered trademark) format. Furthermore, the language model that generates medical text may have a filtering function. For example, the language model that generates medical text may have a function for selecting whether to delete hate speech, sexual language, language related to self-harm, and violent language contained in the input medical information and the generated medical text.

[0045] (Configuration of Language Model Learning Device 2A) As shown in FIG. 4, the language model learning device 2A includes a control unit 20A, a storage unit 30A, a communication unit 40A, and an input / output unit 50A.

[0046] Information referenced by the control unit 20 A is stored in the storage unit 30 A. Examples of the storage unit 30 A include, but are not limited to, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0047] As an example, the storage unit 30A stores feedback FB, medical text MS, and medical information MI related to medical care, as shown in FIG. 4. Examples of the medical information MI include patient information (age, gender, etc.), the patient's illness, the type of medical text MS to be generated, the target person to whom the medical text MS is presented, items to be included in the medical text MS (expected outcomes of treatment, side effects, etc.), the intention behind creating the medical text MS, the progress of treatment, and use cases. The feedback FB, medical text MS, and medical information MI may be stored in the storage unit 30A in association with each other, as shown in FIG. 4.

[0048] As another example, the storage unit 30A stores a language model LM that has been machine-learned to generate medical text MS using medical information MI as input. Storing the language model LM in the storage unit 30A means that parameters defining the language model LM are stored in the storage unit 30A. Furthermore, the storage unit 30A may store a plurality of different language models LM1 and LM2, as shown in FIG. 4 . Hereinafter, when there is no need to particularly distinguish between the language model LM1 and the language model LM2, they will simply be referred to as the language model LM.

[0049] Examples of language models LM include, but are not limited to, large language models (LLMs) such as BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), T5 (Text-to-Text Transfer Transformer), RoBERTa (Robustly optimized BERT approach), and ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately), as well as learning models generated by transfer learning or fine-tuning using pre-trained models (e.g., ChatGPT (Chat Generative Pre-trained Transformer)).

[0050] The communication unit 40A is an interface for transmitting and receiving data via a network. Examples of the communication unit 40A include, but are not limited to, communication chips for various communication standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), and wireless communication standards for mobile data communication networks.

[0051] As an example, the communication unit 40A transmits data supplied from the control unit 20A to the terminal 6, and supplies data received from the terminal 6 to the control unit 20A. An example of data that the communication unit 40 transmits to the terminal 6 is medical text MS. Examples of data that the communication unit 40 receives from the terminal 6 are medical information MI, feedback FB, and consent information.

[0052] The input / output unit 50A is an interface that receives input of data and outputs data. Examples of the input / output unit 50A include, but are not limited to, a keyboard, a mouse, a touchpad, a microphone, and a liquid crystal display.

[0053] When the input / output unit 50A receives input of data, it supplies the received data to the control unit 20A. In addition, the input / output unit 50A outputs the data supplied from the control unit 20A.

[0054] (Control Unit 20A) The control unit 20A controls each of the components included in the language model learning device 2A.

[0055] 4, the control unit 20A includes a sentence generation unit 11A, a feedback reception unit 12A, a consent acquisition unit 13A, a learning unit 14A, an acquisition unit 21A, a selection unit 22A, a management unit 23A, and an output unit 24A. In this exemplary embodiment, the sentence generation unit 11A, the feedback reception unit 12A, the consent acquisition unit 13A, the learning unit 14A, the selection unit 22A, and the management unit 23A realize sentence generation means, feedback reception means, consent acquisition means, learning means, selection means, and management means.

[0056] The text generation unit 11A generates medical text MS using a language model LM. As an example, the text generation unit 11A generates medical text MS by inputting medical information MI into a language model LM selected by a selection unit 22A, which will be described later. The text generation unit 11A stores the generated medical text MS in the memory unit 30A.

[0057] The feedback receiving unit 12A receives feedback FB from a doctor regarding the medical text MS, and stores the received feedback FB in the storage unit 30A.

[0058] The consent acquisition unit 13A acquires consent information indicating whether or not the user consents to using the feedback FB received by the feedback receiving unit 12A in machine learning of the language model LM. The consent acquisition unit 13A stores the acquired consent information in the storage unit 30A.

[0059] The learning unit 14A trains the language model LM by machine learning. As an example, when the consent information acquired by the consent acquisition unit 13A indicates consent, the learning unit 14A trains the language model LM by machine learning using the feedback FB received by the feedback receiving unit 12A.

[0060] The acquisition unit 21A acquires data. As an example, the acquisition unit 21A acquires medical information MI from the terminal 6. The acquisition unit 21A stores the acquired data in the storage unit 30A.

[0061] As one example, the acquisition unit 21A acquires medical information MI based on a user's input to an image DP1 (described later). As another example, the acquisition unit 21A acquires medical information MI based on input to an electronic medical record in a predetermined format. As yet another example, the acquisition unit 21A collaborates with a company different from the company that provides the language model training system 1A, and acquires medical information MI via a system provided by the different company.

[0062] The selection unit 22A selects to which of a plurality of different language models LM the medical information MI acquired by the acquisition unit 21A from the terminal 6 is to be input. The selection unit 22A supplies information indicating the selected language model LM to the sentence generation unit 11A. An example of the processing executed by the selection unit 22A will be described later.

[0063] The management unit 23A manages multiple pieces of data by associating them with one another. As an example, the management unit 23A manages medical information MI, medical text MS generated by inputting the medical information MI into a language model LM, and feedback FB from a doctor on the medical text MS by storing them in the storage unit 30A in association with one another. An example of the processing executed by the management unit 23A will be described later.

[0064] The output unit 24A outputs data. For example, the output unit 24A outputs the medical document MS to the terminal 6.

[0065] (Configuration of Terminal 6_1) As shown in Fig. 4, the terminal 6_1 includes a control unit 60_1, a storage unit 70_1, a communication unit 80_1, and an input / output unit 90_1. Note that the terminal 6_2 has the same configuration as the terminal 6_1, and therefore a description thereof will be omitted.

[0066] The storage unit 70_1 stores information referenced by the control unit 60_1. Examples of the storage unit 70_1 include, but are not limited to, a flash memory, an HDD, an SSD, or a combination thereof.

[0067] As an example, the storage unit 70_1 stores a language model LM3 as shown in Fig. 4. The language model LM3 may be the same as either the language model LM1 or the language model LM2 stored in the storage unit 30A of the language model learning device 2A. Alternatively, the language model LM3 may be a different version of either the language model LM1 or the language model LM2 stored in the storage unit 30A of the language model learning device 2A. Furthermore, the storage unit 70_1 may store a plurality of language models LM that are different from each other.

[0068] The communication unit 80_1 is an interface for transmitting and receiving data via a network. Examples of the communication unit 80_1 include, but are not limited to, communication chips for various communication standards such as Ethernet, Wi-Fi, and wireless communication standards for mobile data communication networks.

[0069] For example, the communication unit 80_1 transmits data supplied from the control unit 60_1 to the language model learning device 2A, and supplies data received from the language model learning device 2A to the control unit 60_1. Examples of data that the communication unit 80_1 transmits to the language model learning device 2A include medical information MI, feedback FB, and consent information. Examples of data that the communication unit 80_1 receives from the language model learning device 2A include medical text MS.

[0070] The input / output unit 90_1 is an interface that receives input of data and outputs data. Examples of the input / output unit 90_1 include, but are not limited to, a keyboard, a mouse, a touchpad, a microphone, and a liquid crystal display.

[0071] When the input / output unit 90_1 receives input of data, it supplies the received data to the control unit 60_1. The input / output unit 90_1 also outputs the data supplied from the control unit 60_1.

[0072] (Control Unit 60_1) The control unit 60_1 controls each component included in the terminal 6_1.

[0073] As shown in FIG. 4, the control unit 60_1 includes an acquisition unit 61_1, an output unit 62_1, and a sentence generation unit 63_1.

[0074] The acquisition unit 61_1 acquires data. For example, the acquisition unit 61_1 acquires medical information MI, feedback FB, and consent information input by a user via the input / output unit 90_1. The acquisition unit 61_1 stores the acquired data in the storage unit 70_1.

[0075] The output unit 62_1 outputs data. As one example, the output unit 24A outputs feedback FB and agreement information to the language model training device 2A. As another example, the output unit 62_1 presents medical text MS to the user via the input / output unit 90_1.

[0076] The sentence generation unit 63_1 generates medical text MS. As an example, the sentence generation unit 63_1 generates medical text MS by outputting medical information MI to the language model learning device 2A and requesting that the medical text MS be generated. In this case, the sentence generation unit 63_1 may output the medical information MI to a uniform resource locator (URL) corresponding to the medical information MI. As an example, when the medical information MI includes information indicating a language model LM that the user has selected from multiple language models LM to generate the medical text MS, the sentence generation unit 63_1 may output the medical information MI to a URL including words indicating the selected language model LM.

[0077] As another example, the sentence generator 63_1 generates medical sentences MS by inputting medical information MI into a language model LM3. The sentence generator 63_1 stores the generated medical sentences MS in the storage unit 70_1.

[0078] (Processing Executed in Language Model Learning System 1A) The flow of processing (language model learning method S2) executed in the language model learning system 1A will be described with reference to Fig. 5. Fig. 5 is a flow chart showing the flow of the language model learning method S2.

[0079] (Step S21) In step S21, the acquisition unit 61 of the terminal 6 acquires the medical information MI. The acquisition unit 61 stores the acquired medical information MI in the storage unit 70.

[0080] (Step S22) In step S22, the sentence generating unit 63 outputs the medical information MI acquired by the acquiring unit 61 in step S21 to the language model learning device 2A.

[0081] (Step S23) In step S23, the acquisition unit 21A of the language model learning device 2A acquires the medical information MI output in step S22 from the terminal 6. The acquisition unit 21A stores the acquired medical information MI in the storage unit 30A.

[0082] (Step S24) In step S24, the selection unit 22A selects to which of the multiple language models LM the medical information MI acquired by the acquisition unit 21A in step S23 is to be input. The selection unit 22A supplies information indicating the selected language model ML to the sentence generation unit 11A.

[0083] (Step S25) In step S25, the text generation unit 11A generates medical text MS by inputting the medical information MI into the language model LM selected by the selection unit 22A in step S24. The text generation unit 11A stores the generated medical text MS in the storage unit 30A. Here, the management unit 23A may store the medical text MS in the storage unit 30A in association with the medical information MI.

[0084] (Step S26) In step S26, the output unit 24A outputs the medical text MS generated by the text generation unit 11A in step S25 to the terminal 6 that output the medical information MI in step S22.

[0085] (Step S27) In step S27, the sentence generation unit 63 of the terminal 6 acquires the medical sentence MS output from the language model training device 2A in step S26. The sentence generation unit 63 stores the acquired medical sentence MS in the storage unit 70.

[0086] (Step S28) In step S28, the acquisition unit 61 receives feedback FB from the doctor regarding the medical text MS acquired by the text generation unit 63 in step S27. The acquisition unit 61 stores the received feedback FB in the storage unit 70.

[0087] (Step S29) In step S29, the output unit 62 outputs the feedback FB received by the acquisition unit 61 in step S28 to the language model learning device 2A.

[0088] (Step S30) In step S30, the feedback receiving unit 12A of the language model learning device 2A receives the feedback FB output from the terminal 6 in step S29. The feedback receiving unit 12A stores the received feedback FB in the storage unit 30A. Here, the management unit 23A may associate the feedback FB with the medical text MS and medical information MI associated in step S25 and store them in the storage unit 30A.

[0089] (Step S31) In step S31, the acquisition unit 61 of the terminal 6 acquires consent information indicating whether or not the user consents to using the feedback FB received in step S28 in machine learning of the language model LM. The acquisition unit 61 stores the acquired consent information in the storage unit 70.

[0090] (Step S32) In step S32, the output unit 62 outputs the agreement information received by the acquisition unit 61 in step S31 to the language model learning device 2A.

[0091] (Step S33) In step S33, the consent acquisition unit 13A of the language model learning device 2A acquires the consent information output in step S32 from the terminal 6. The consent acquisition unit 13A stores the acquired consent information in the storage unit 30A.

[0092] (Step S34) In step S34, the learning unit 14A determines whether the consent information acquired by the consent acquisition unit 13A in step S33 indicates consent.

[0093] (Step S35) In step S34, if the consent information indicates consent (step S34: YES), in step S35, the learning unit 14A trains the language model LM by machine learning using the feedback FB received by the feedback receiving unit 12A in step S30.

[0094] The language model LM that is machine-learned in step S35 may be the language model LM that generated the medical text MS in step S25. Furthermore, the language model LM that is machine-learned in step S35 may be at least one language model LM that was not selected by the selection unit 22A in step S24, in addition to the language model LM that generated the medical text MS in step S25.

[0095] Furthermore, instead of step S30, in step S35, the management unit 23A may associate the feedback FB with the medical text MS and medical information MI associated in step S25 and store them in the storage unit 30A. That is, when consent is given to using the feedback FB in machine learning of the language model LM, the medical text MS, medical information MI, and feedback FB may be associated with each other.

[0096] On the other hand, if the consent information indicates no consent at step S34 (step S34: NO), the processing in the language model training method S2 shown in FIG. 5 ends.

[0097] (Example 1 of an image displayed on the terminal 6) An example of an image displayed on the terminal 6 is shown in Fig. 6. Fig. 6 is a diagram showing an example of an image displayed on the terminal 6.

[0098] When the acquisition unit 61 of the terminal 6 receives an instruction from the user via the input / output unit 90 to generate medical text MS, the output unit 62 outputs, as an example, the image DP1 shown in Figure 6 to the input / output unit 90.

[0099] The image DP1 includes an item I_MI for inputting medical information MI. The acquisition unit 61 acquires the medical information MI by having the user input information into the item I_MI. The item I_MI may also include an item I_LM for selecting a language model LM that the user desires to use to create the medical text MS from among multiple language models LM stored in the storage unit 30A of the language model learning device 2A.

[0100] Image DP1 also includes a button I_GMS for receiving instructions to create a document. When the user presses the button I_GMS, the acquisition unit 21A acquires the medical information MI entered in the item I_MI in step S21. Then, in step S22, the sentence generation unit 63 outputs the medical information MI to the language model training device 2A.

[0101] Image DP1 also includes an item D_MS that displays the generated medical text MS. In step S27 described above, when text generation unit 63 of terminal 6 acquires medical text MS output from language model training device 2A, output unit 62 displays medical text MS in item D_MS.

[0102] Image DP1 also includes buttons I_FB1 and I_FB2 for receiving feedback from the user. If the doctor determines that the medical text MS displayed in item D_MS is appropriate, button I_FB1 is pressed. On the other hand, if the doctor determines that the medical text MS displayed in item D_MS is inappropriate, button I_FB2 is pressed. In step S28 described above, the acquisition unit 61 receives feedback FB corresponding to the pressed button.

[0103] When the output unit 62 outputs the feedback FB in step S29 described above, in step S30, the feedback receiving unit 12A of the language model learning device 2A receives the feedback FB output from the terminal 6. That is, the feedback receiving unit 12A receives the feedback FB including information indicating whether or not the doctor has determined that the medical text MS is appropriate.

[0104] With this configuration, the language model learning device 2A can recognize whether or not the language model LM that generated the medical text MS needs to undergo further machine learning.

[0105] Furthermore, the feedback receiving unit 12A may receive feedback FB including comments from a doctor. For example, when a button I_FB2 included in the image DP1 is pressed, an item for inputting a comment indicating what the doctor has determined to be inappropriate may be displayed on the input / output unit 90. With this configuration, the language model learning device 2A can use the feedback FB to train the language model LM by machine learning.

[0106] (Example 2 of image displayed on terminal 6) When the acquisition unit 61 receives the feedback FB, the output unit 62 displays an image for the acquisition unit 61 to acquire consent information. An example of the image for the acquisition unit 61 to acquire consent information is shown in Fig. 7. Fig. 7 is a diagram showing another example of an image displayed on the terminal 6.

[0107] Image DP2 includes a button I_AG1 for accepting the user's consent to using the feedback FB in the machine learning of the language model LM, and a button I_AG2 for accepting the user's disapproval. If the doctor consents to using the feedback FB in the machine learning of the language model LM, the button I_AG1 is pressed. On the other hand, if the doctor does not consent to using the feedback FB in the machine learning of the language model LM, the button I_AG2 is pressed. In step S31 described above, the acquisition unit 61 acquires consent information corresponding to the pressed button.

[0108] (Example of Processing Executed by Management Unit 23A) An example of processing executed by management unit 23A will be described with reference to Fig. 8. Fig. 8 is a diagram showing an example of a database DB managed by management unit 23A.

[0109] As an example, the management unit 23A generates a database DB shown in FIG. 8 , which associates medical information MI, medical text MS generated by inputting the medical information MI into a language model LM, and feedback FB from a doctor on the medical text MS. In FIG. 8 , the items correspond to the following information: Medical information MI: item "ID," item "datetime," item "input," item "usecase" Medical text MS: item "output" Feedback FB: item "evaluation," item "comment" Furthermore, the management unit 23A may further associate the language model LM that generated the medical text MS in addition to the medical information MI, medical text MS, and feedback FB. For example, as shown in FIG. 8 , the management unit 23A may further associate the language model LM that generated the medical text MS and an item "llmVersion" indicating the version with the medical information MI, medical text MS, and feedback FB.

[0110] With this configuration, the language model learning device 2A can manage data such as, for example, the frequency with which a certain doctor generates medical text MS, and the bias in the evaluation of medical text MS by a certain doctor.

[0111] Furthermore, by the management unit 23A further associating the language model LM that generated the medical text MS in addition to the medical information MI, medical text MS, and feedback FB, it is possible to manage data such as, for example, which language model LM can generate appropriate medical text MS in a certain use case.

[0112] Furthermore, the management unit 23A may encrypt and manage the personal information included in the medical information MI. As an example, the management unit 23A may encrypt and manage the personal information included in the medical information MI using a hash function.

[0113] For example, the management unit 23A encrypts the value of the item "ID" using a hash function and then manages the value in association with other information in the database DB shown in Fig. 8. The value of the item "ID" may also be a value including the ID of the medical institution and the user ID.

[0114] Personal information managed by a medical institution (e.g., personal information of patients and personal information of doctors) requires high confidentiality. In this configuration, the management unit 23A encrypts and manages the personal information included in the medical information MI, thereby increasing the confidentiality of the personal information included in the medical information MI.

[0115] (Example 1 of Processing Executed by Selector 22A) An example of processing executed by selector 22A will be described.

[0116] As an example, in step S23 described above, if the acquisition unit 21A acquires medical information MI that includes information indicating which language model LM the user wants to use to create the medical text MS, the selection unit 22A refers to the medical information MI and selects a language model LM.

[0117] For example, in the image DP1 shown in FIG. 6 , if a doctor inputs "language model LM1" into item I_LM, the acquisition unit 21A acquires medical information MI including information indicating that the doctor wishes to have the "language model LM1" create medical text MS. The selection unit 22A references the medical information MI and selects "language model LM1" from among the multiple language models LM. In this case, in step S24 described above, the text generation unit 11A generates medical text MS by inputting the medical information MI into the "language model LM1."

[0118] (Second Example of Processing Executed by Selector 22A) As another example, the selector 22A may select a language model LM by referring to data managed by the manager 23A.

[0119] For example, the selection unit 22A refers to the database DB shown in Fig. 8 and selects a language model LM based on the feedback history. As an example of this configuration, the selection unit 22A extracts the item "llmVersion" that has the most "good" responses in the item "evaluation." Then, the selection unit 22A selects the language model LM corresponding to the extracted item "llmVersion."

[0120] As another example of this configuration, assume that the medical information MI acquired by the acquisition unit 21A in step S23 described above includes a use case. In this case, the selection unit 22A refers to the database DB shown in FIG. 8 and extracts data from the database DB whose item "usecase" is the same as the use case included in the medical information MI. Next, the selection unit 22A extracts, from the extracted data, the item "llmVersion" that has the most "good" in the item "evaluation." Then, the selection unit 22A selects a language model LM corresponding to the extracted item "llmVersion."

[0121] In this way, the selection unit 22A selects to which of multiple different language models LM the acquisition unit 21A will input the medical information MI acquired from the terminal 6, and the language model learning device 2A can generate more accurate medical text MS.

[0122] (Effects of Language Model Learning System 1A) As described above, in the language model learning system 1A, the language model learning device 2A generates medical text MS by inputting medical information MI into the language model LM. The language model learning device 2A also receives feedback FB from a doctor on the medical text MS from a terminal 6 within the closed network, and acquires consent information indicating whether or not the doctor agrees to using the feedback FB in machine learning of the language model LM. If the consent information indicates consent, the language model learning device 2A uses the feedback FB to train the language model LM.

[0123] With this configuration, the language model learning system 1A can obtain consent from doctors using terminals 6 of one or more medical institutions within the closed network to use feedback FB from the doctors for machine learning of the language model LM. In other words, feedback FB can be obtained not from one medical institution within the closed network, but from each of the terminals 6 of one or more medical institutions. Therefore, the language model learning system 1A can obtain a large amount of feedback FB for the medical text MS generated by the language model LM, thereby improving the accuracy of the language model LM that generates the medical text MS.

[0124] (Modification) The language model learning device 2A may be configured such that the medical text MS is generated in the terminal 6, and the language model learning device 2A acquires feedback FB and agreement information for the medical text MS.

[0125] In this case, after executing step S21 described above, the terminal 6 generates the medical text MS by inputting the medical information MI acquired in step S21 into the language model LM (language model LM3 or language model LM4 in FIG. 4). After generating the medical text MS, the terminal 6 executes the processes from step S28 described above.

[0126] In this configuration, machine learning cannot be performed on the language model LM held by the terminal 6. Therefore, the language model learning device 2A may be configured to remotely update the version of the language model LM held by the terminal 6 to the same version as the language model LM held by the language model learning device 2A (language model LM1 or language model LM2 in FIG. 4 ).

[0127] With this configuration, the language model learning system 1A can reduce the amount of medical information MI output from the terminal 6 to the language model learning device 2A. Therefore, the language model learning system 1A can obtain feedback FB from the terminal 6 of a medical institution even if the medical institution does not want to output medical information MI outside the medical institution.

[0128] [Third Exemplary Embodiment] A third exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be denoted by the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technical means shown in each drawing referenced to describe this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0129] (Configuration of Language Model Learning System 1B) The configuration of language model learning system 1B will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of language model learning system 1B. As shown in Fig. 9, language model learning system 1B includes a language model learning device 2B, a terminal 6_1B, and a terminal 6_2B.

[0130] In the language model learning system 1B, a language model learning device 2B generates a plurality of medical sentences MS by using a plurality of language models LM. A terminal 6 outputs feedback FB from a doctor on the generated plurality of medical sentences MS and consent information indicating whether or not the doctor consents to using the feedback FB in machine learning of the language model LM to the language model learning device 2B. When consent is obtained from the doctor to using the feedback FB in learning the language model LM, the language model learning device 2B uses the feedback FB to train the language model LM by machine learning.

[0131] (Configuration of Language Model Learning Device 2B) As shown in FIG. 9, the language model learning device 2B includes a control unit 20B instead of the control unit 20A in the language model learning device 2A.

[0132] The control unit 20B controls each of the components included in the language model learning device 2B.

[0133] Furthermore, the control unit 20B includes a sentence generation unit 11B and a feedback reception unit 12B instead of the sentence generation unit 11A and the feedback reception unit 12A in the control unit 20A. In this exemplary embodiment, the sentence generation unit 11B and the feedback reception unit 12B respectively realize a sentence generation means and a feedback reception means. Furthermore, the control unit 20B does not include the selection unit 22A that the control unit 20A includes.

[0134] The sentence generation unit 11B generates a plurality of medical sentences MS by inputting medical information MI acquired from the terminal 6 into each of a plurality of different language models LM. The sentence generation unit 11B stores the generated medical sentences MS in the memory unit 30A.

[0135] The feedback receiving unit 12B receives feedback FB including information indicating which medical text MS is appropriate among the multiple medical texts MS generated by the text generation unit 11. The feedback receiving unit 12B stores the received feedback FB in the memory unit 30A.

[0136] (Configuration of Terminal 6_1B) As shown in FIG. 9, the terminal 6_1B includes a control unit 60_1B instead of the control unit 60_1 in the terminal 6_1.

[0137] The control unit 60_1B controls each component included in the terminal 6_1B.

[0138] Moreover, the control unit 60_1B includes an acquisition unit 61_1B instead of the acquisition unit 61_1 included in the control unit 60_1.

[0139] The acquisition unit 61_1B acquires feedback FB including information indicating which of the multiple medical documents MS is appropriate, in addition to the medical information MI and the consent information.

[0140] (Processing Executed in Language Model Learning System 1B) The flow of processing executed in the language model learning system 1B (language model learning method S2B) will be described with reference to Fig. 10. Fig. 10 is a flow chart showing the flow of the language model learning method S2B.

[0141] (Steps S21 to S23) The acquisition unit 61B of the terminal 6B acquires the medical information MI, and the acquisition unit 21A of the language model learning device 2B acquires the medical information MI. The processes in steps S21 to S23 are as described above.

[0142] (Step S41) In step S41, the sentence generation unit 11B of the language model learning device 2B generates multiple medical sentences MS by inputting the medical information MI acquired by the acquisition unit 21A in step S23 into each of multiple different language models LM (language model LM1 and language model LM2). The sentence generation unit 11B stores the generated medical sentences MS in the memory unit 30A.

[0143] (Steps S26 to S27) The processing in steps S26 to S27 in which the output unit 24A outputs a plurality of medical documents MS to the terminal 6B and the document generation unit 63 of the terminal 6B acquires a plurality of medical documents MS is as described above.

[0144] (Step S42) In step S42, the acquisition unit 61B receives feedback FB including information indicating which medical text MS is appropriate from among the multiple medical texts MS acquired by the text generation unit 63 in step S27. The acquisition unit 61B stores the received feedback FB in the storage unit 70.

[0145] (Step S29) In step S29, the output unit 62 outputs the feedback FB received by the acquisition unit 61B in step S42 to the language model learning device 2B.

[0146] (Step S43) In step S43, the feedback receiving unit 12B of the language model learning device 2B receives the feedback FB output from the terminal 6B in step S29, which includes information indicating which of the multiple medical sentences MS is appropriate. The feedback receiving unit 12B stores the received feedback FB in the memory unit 30A.

[0147] (Steps S31 to S35) The acquisition unit 61B of the terminal 6B acquires consent information indicating whether or not the user agrees to using the feedback FB for machine learning of the language model LM, and if the consent information indicates consent, the processing of steps S31 to S35 in which the learning unit 14A uses the feedback FB to machine learn the language model LM is as described above.

[0148] (Effects of Language Model Learning System 1B) As described above, in the language model learning system 1B, the language model learning device 2B generates multiple medical sentences MS and receives feedback FB including information indicating which of the multiple medical sentences MS is appropriate.

[0149] With this configuration, the language model learning system 1B presents medical text MS generated by each of the multiple language models LM to the doctor, thereby increasing the likelihood that appropriate medical text MS will be presented to the doctor. Furthermore, the language model learning system 1B receives feedback FB indicating which of the multiple language models LM generated the medical text MS is appropriate, thereby enabling machine learning of the language model LM that generated inappropriate medical text MS.

[0150] [Example of Software Implementation] Some or all of the functions of the language model learning system 1 and the language model learning devices 2, 2A, and 2B (hereinafter also referred to as "the above systems and devices") may be implemented by hardware such as an integrated circuit (IC chip), or by software.

[0151] In the latter case, the system and each device are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 11. Figure 11 is a block diagram showing the hardware configuration of computer C that functions as the system and each device.

[0152] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to function as the above-mentioned system and each device. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of the above-mentioned system and each device.

[0153] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0154] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0155] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0156] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0157] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0158] (Appendix A1) A language model learning system comprising: a text generation means for generating medical text by inputting medical information obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical text using medical information related to medical care as input; a feedback receiving means for receiving feedback from doctors on the medical text; a consent acquisition means for acquiring consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning means for machine-learning the language model using the feedback if the consent information indicates consent.

[0159] (Supplementary Note A2) The language model training system according to Supplementary Note A1, wherein the feedback receiving means receives feedback including information indicating whether the doctor has determined that the medical text is appropriate.

[0160] (Appendix A3) The language model learning system according to Appendix A1 or A2, further comprising a selection means for selecting into which of a plurality of different language models the medical information acquired from the terminal of the medical institution is to be input, wherein the sentence generation means generates medical sentences by inputting the medical information acquired from the terminal of the one or more medical institutions into the language model selected by the selection means.

[0161] (Appendix A4) The language model learning system described in Appendix A1 or A2, wherein the sentence generation means generates a plurality of medical sentences by inputting medical information obtained from terminals of the one or more medical institutions into each of a plurality of mutually different language models, and the feedback receiving means receives feedback including information indicating which medical sentence among the plurality of medical sentences is appropriate.

[0162] (Appendix A5) The language model learning system according to any one of Appendices A1 to A4, further comprising: a management means for managing the medical information, medical text generated by inputting the medical information into the language model, and feedback from doctors on the medical text in association with each other.

[0163] (Supplementary Note A6) The language model training system according to Supplementary Note A5, wherein the management means encrypts and manages personal information included in the medical information.

[0164] (Appendix A7) A language model learning device comprising: a text generation means for generating medical text by inputting medical information obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical text using medical information related to medical care as input; a feedback receiving means for receiving feedback from doctors on the medical text; a consent acquisition means for acquiring consent information indicating whether or not the doctor agrees to using the feedback in the machine learning of the language model; and a learning means for machine-learning the language model using the feedback if the consent information indicates consent.

[0165] [Appendix B] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0166] (Appendix B1) A language model training method including: a text generation process in which at least one processor generates medical text by inputting medical information related to medical care obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-trained to generate medical text; a feedback acceptance process in which the at least one processor accepts feedback from doctors regarding the medical text; a consent acquisition process in which the at least one processor acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a training process in which the at least one processor, if the consent information indicates consent, trains the language model using the feedback.

[0167] (Supplementary Note B2) The language model training method according to Supplementary Note B1, wherein in the feedback receiving process, the at least one processor receives feedback including information indicating whether the doctor has determined that the medical text is appropriate.

[0168] (Appendix B3) The language model training method described in Appendix B1 or B2, further including a selection process in which the at least one processor selects to which of a plurality of different language models the medical information acquired from the terminal of the medical institution is to be input, and in the sentence generation process, the at least one processor generates medical sentences by inputting the medical information acquired from the terminal of the one or more medical institutions into the language model selected in the selection process.

[0169] (Appendix B4) A language model training method as described in Appendix B1 or B2, wherein in the sentence generation process, the at least one processor generates a plurality of medical sentences by inputting medical information obtained from terminals of the one or more medical institutions into each of a plurality of mutually different language models, and in the feedback reception process, the at least one processor receives feedback including information indicating which of the plurality of medical sentences is appropriate.

[0170] (Appendix B5) The language model training method according to any one of Appendices B1 to B4, further comprising a management process in which the at least one processor manages the medical information, medical text generated by inputting the medical information into the language model, and feedback from doctors on the medical text in association with each other.

[0171] (Supplementary Note B6) The language model training method according to Supplementary Note B5, wherein in the management process, the at least one processor encrypts and manages personal information included in the medical information.

[0172] [Appendix C] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0173] (Appendix C1) A program that causes a computer to function as a language model learning system, the program causing the computer to function as: a sentence generation means that generates medical sentences by inputting medical information related to medical care, obtained from terminals of one or more medical institutions within a closed network, into a language model that has been machine-learned to generate medical sentences; a feedback receiving means that receives feedback from doctors on the medical sentences; a consent acquisition means that acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning means that, if the consent information indicates consent, uses the feedback to train the language model through machine learning.

[0174] (Supplementary Note C2) The program according to Supplementary Note C1, wherein the feedback receiving means receives feedback including information indicating whether the doctor has determined that the medical text is appropriate.

[0175] (Appendix C3) A program described in Appendix C1 or C2, which further causes the computer to function as a selection means for selecting into which of a plurality of different language models the medical information obtained from the terminal of the medical institution will be input, and the sentence generation means generates medical sentences by inputting the medical information obtained from the terminal of the one or more medical institutions into the language model selected by the selection means.

[0176] (Appendix C4) A program described in Appendix C1 or C2, wherein the sentence generation means generates a plurality of medical sentences by inputting medical information obtained from terminals of the one or more medical institutions into each of a plurality of different language models, and the feedback receiving means receives feedback including information indicating which of the plurality of medical sentences is appropriate.

[0177] (Appendix C5) A program described in any of Appendices C1 to C4 that causes the computer to further function as a management means that associates and manages the medical information, medical text generated by inputting the medical information into the language model, and feedback from doctors on the medical text.

[0178] (Appendix C6) The program according to Appendix C5, wherein the management means encrypts and manages personal information included in the medical information.

[0179] (Appendix C7) A program that causes a computer to function as a language model learning device, the program causing the computer to function as: a sentence generation means that generates medical sentences by inputting medical information related to medical care, obtained from terminals of one or more medical institutions within a closed network, into a language model that has been machine-learned to generate medical sentences; a feedback receiving means that receives feedback from doctors on the medical sentences; a consent acquisition means that acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning means that, if the consent information indicates consent, uses the feedback to train the language model in machine learning.

[0180] [Appendix D] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0181] (Appendix D1) A language model learning system comprising at least one processor, the at least one processor executing: a sentence generation process that generates medical sentences by inputting medical information related to medical care obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical sentences; a feedback acceptance process that accepts feedback from doctors on the medical sentences; a consent acquisition process that acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning process that, if the consent information indicates consent, uses the feedback to train the language model on machine learning.

[0182] (Appendix D2) The language model training system according to Appendix D1, wherein in the feedback receiving process, the at least one processor receives feedback including information indicating whether the doctor has determined that the medical text is appropriate.

[0183] (Appendix D3) The language model learning system described in Appendix D1 or D2, wherein the at least one processor further executes a selection process to select to which of a plurality of different language models the medical information acquired from the terminal of the medical institution is to be input, and in the sentence generation process, the at least one processor generates medical sentences by inputting the medical information acquired from the terminal of the one or more medical institutions into the language model selected in the selection process.

[0184] (Appendix D4) The language model learning system described in Appendix D1 or D2, wherein in the sentence generation process, the at least one processor generates a plurality of medical sentences by inputting medical information obtained from terminals of the one or more medical institutions into each of a plurality of mutually different language models, and in the feedback reception process, the at least one processor receives feedback including information indicating which of the plurality of medical sentences is appropriate.

[0185] (Appendix D5) The language model learning system according to any one of Appendices D1 to D4, wherein the at least one processor further executes a management process for managing the medical information, medical text generated by inputting the medical information into the language model, and feedback from doctors on the medical text in association with each other.

[0186] (Supplementary Note D6) The language model training system according to Supplementary Note D5, wherein in the management process, the at least one processor encrypts and manages personal information included in the medical information.

[0187] (Appendix D7) A language model learning device comprising at least one processor, the at least one processor executing: a sentence generation process that generates medical sentences by inputting medical information related to medical care obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate medical sentences; a feedback acceptance process that accepts feedback from doctors on the medical sentences; a consent acquisition process that acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning process that, if the consent information indicates consent, uses the feedback to train the language model by machine learning.

[0188] [Appendix E] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0189] (Appendix E1) A non-transitory recording medium having recorded thereon a program that causes a computer to function as a language model learning system, the program causing the computer to execute: a sentence generation process that generates medical sentences by inputting medical information related to medical care into a language model that has been machine-learned to generate medical sentences, and inputting medical information obtained from terminals of one or more medical institutions within a closed network; a feedback acceptance process that accepts feedback from doctors regarding the medical sentences; a consent acquisition process that acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning process that, if the consent information indicates consent, uses the feedback to train the language model on machine learning.

[0190] (Appendix E2) A non-transitory recording medium having recorded thereon a program that causes a computer to function as a language model learning device, the program causing the computer to execute the following: a sentence generation process that generates medical sentences by inputting medical information related to medical care into a language model that has been machine-learned to generate medical sentences, and inputting medical information obtained from terminals of one or more medical institutions within a closed network; a feedback acceptance process that accepts feedback from doctors regarding the medical sentences; a consent acquisition process that acquires consent information indicating whether or not the feedback is used for machine learning of the language model; and a learning process that, if the consent information indicates consent, uses the feedback to train the language model through machine learning.

[0191] 1, 1A, 1B Language model learning system 2, 2A, 2B Language model learning device 11, 11A, 11B Sentence generation unit 12, 12A, 12B Feedback reception unit 13, 13A Consent acquisition unit 14, 14A Learning unit 22A Selection unit 23A Management unit FB Feedback LM Language model MI Medical information MS Medical text

Claims

1. A text generation means for generating a medical text by inputting medical information acquired from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate a medical text using medical information related to medical treatment as an input; a feedback reception means for receiving feedback from a doctor regarding the medical text; a consent acquisition means for acquiring consent information indicating whether to consent to using the feedback for machine learning of the language model; and a learning means for machine-learning the language model using the feedback when the consent information indicates consent. A language model learning system comprising the above components.

2. The language model learning system according to claim 1, wherein the feedback reception means receives feedback including information indicating whether the doctor has determined that the medical text is appropriate.

3. Further comprising a selection means for selecting which of a plurality of different language models to input the medical information acquired from the terminals of the medical institutions, and the text generation means generates a medical text by inputting the medical information acquired from the terminals of the one or more medical institutions into the language model selected by the selection means. The language model learning system according to claim 1 or 2.

4. The text generation means generates a plurality of medical texts by inputting the medical information acquired from the terminals of the one or more medical institutions into each of a plurality of different language models, and the feedback reception means receives feedback including information indicating which of the plurality of medical texts is appropriate. The language model learning system according to claim 1 or 2.

5. Further comprising a management means for managing the medical information, the medical text generated by inputting the medical information into the language model, and the feedback from the doctor regarding the medical text in association with each other. The language model learning system according to claim 1 or 2.

6. The language model learning system according to claim 5, wherein the management means encrypts and manages personal information included in the medical information.

7. A language model learning device comprising: a text generation means for generating a medical text by inputting medical information obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate a medical text using the medical information related to medical treatment as an input; a feedback reception means for receiving feedback from a doctor regarding the medical text; a consent acquisition means for acquiring consent information indicating whether to consent to using the feedback for machine learning of the language model; and a learning means for machine-learning the language model using the feedback when the consent information indicates consent.

8. A language model learning method including: a text generation process in which at least one processor generates a medical text by inputting medical information obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate a medical text using the medical information related to medical treatment as an input; a feedback reception process in which the at least one processor receives feedback from a doctor regarding the medical text; a consent acquisition process in which the at least one processor acquires consent information indicating whether to consent to using the feedback for machine learning of the language model; and a learning process in which the at least one processor machine-learns the language model using the feedback when the consent information indicates consent.

9. A program for causing a computer to function as a language model learning system, the program causing the computer to function as: a text generation means for generating a medical text by inputting medical information obtained from terminals of one or more medical institutions within a closed network into a language model that has been machine-learned to generate a medical text using the medical information related to medical treatment as an input; a feedback reception means for receiving feedback from a doctor regarding the medical text; a consent acquisition means for acquiring consent information indicating whether to consent to using the feedback for machine learning of the language model; and a learning means for machine-learning the language model using the feedback when the consent information indicates consent.

Images