Medical support system, medical support program, and medical support method

Abstract

To provide a medical support system that supports accurate and quick entry of medical information into a paper form of a predetermined format.SOLUTION: Provided is a medical support system 100 for automatic entry into a medical form related to medical care, the system comprising a medical form generation module 112 trained through machine learning to extract medical information for the items to be entered into the medical form from text information including medical information related to medical care.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a medical support system, a medical support program, and a medical support method. [Background technology]

[0002] When medical personnel such as ambulance personnel, doctors, and nurses check the condition of a patient, they may give a verbal explanation. On the other hand, the patient's condition must be written down in a prescribed form.

[0003] For example, a comprehensive emergency service support system has been disclosed in which information on injured persons received at a firefighting command center is transmitted from an emergency server via a communication line to a terminal used by an emergency worker (Patent Documents 1 and 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-130516 A [Patent Document 2] Patent Publication No. 2021-93228 Summary of the Invention [Problem to be solved by the invention]

[0005] When entering medical information into a form of a specified format, it is necessary to extract the medical information contained in the instructions, embed it in the correct location on the form, and output it accurately and quickly.

[0006] In particular, in the case of illnesses that require urgent treatment, such as stroke and myocardial infarction, a process must be carried out in which a prescribed form is filled out so that medical information, such as the condition that has been verbally explained by medical personnel, can be conveyed quickly and accurately. [Means for solving the problem]

[0007] One aspect of the present invention is a medical support system that automatically fills in medical forms related to medical care, characterized in that the medical support system includes a natural language understanding module that has been machine-learned to extract medical information for items to be filled in the medical form from text information including medical information related to medical care.

[0008] Another aspect of the present invention is a medical support program for automatically filling out medical forms related to medical care, characterized in that the program causes a computer to function as a natural language understanding module that has been machine-learned to extract medical information for items to be filled out in the medical form from text information including medical information related to medical care.

[0009] Another aspect of the present invention is a medical support method for automatically filling in a medical form related to medical care, the medical support method being characterized in that a computer functioning as a natural language understanding module trained by machine learning to extract medical information for items to be filled in the medical form from text information including medical information related to medical care, automatically extracts medical information for items to be filled in the medical form from text information including medical information related to medical care.

[0010] Here, it is preferable that the natural language understanding module is machine-trained to use text information including medical information related to medical care as input data, medical information to be extracted from the text information for the items to be filled in the medical form as training data, and supervised learning data that combines these pieces of information to output the medical information that is the training data when the input data is input.

[0011] In addition, it is preferable that the natural language understanding module is machine-trained to use text information including medical information related to medical care and form information indicating the medical information to be extracted from the text information as input data, and medical information to be extracted from the text for the items to be filled in the medical form as teacher data, and to use supervised learning data that combines these pieces of information to output the medical information that is the teacher data when the input data is input.

[0012] It is also preferable to include a form embedding module that embeds the medical information output from the natural language understanding module into the medical form and outputs the medical information.

[0013] In addition, it is preferable that the form embedding module is machine-trained to use supervised learning data that combines medical information as input data and the medical form with the medical information appropriately filled in as training data, so that when the medical information is input, the medical form, which is the training data, is output.

[0014] In addition, it is preferable that the natural language understanding module and the form embedding module are integrated, and when the text information is input, medical information for the items to be filled in the medical form is extracted from the text information, and the medical information is embedded in the medical form and output.

[0015] It is also preferable that the system further comprises a speech-to-text conversion module that generates the text information from speech information that includes medical information related to medical care.

[0016] It is also preferable that the speech-to-text conversion module outputs the text information by anonymizing specific sensitive information.

[0017] It is also preferred that the speech-to-text conversion module processes the speech information by removing noise using a noise removal algorithm.

[0018] It is also suitable for use in medical emergencies. Effect of the Invention

[0019] According to the present invention, it is possible to provide a medical support system, a medical support program, and a medical support method that support embedding medical information in a predetermined medical form accurately and quickly and outputting the medical information. Other objects of the embodiments of the present invention will become apparent by referring to the entire specification. [Brief description of the drawings]

[0020] [Figure 1] 1 is a diagram showing a configuration of a medical support system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing a configuration of a server according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram illustrating a configuration of a client according to an embodiment of the present invention. [Figure 4] 1 is a functional block diagram showing a configuration of a medical support system according to an embodiment of the present invention. [Diagram 5] FIG. 2 illustrates a speech-to-text conversion module in accordance with an embodiment of the present invention. [Figure 6] FIG. 2 illustrates a speech-to-text conversion module in accordance with an embodiment of the present invention. [Figure 7] FIG. 2 illustrates a speech-to-text conversion module in accordance with an embodiment of the present invention. [Figure 8] FIG. 2 illustrates a speech-to-text conversion module in accordance with an embodiment of the present invention. [Figure 9] FIG. 2 is a diagram showing a medical form generation module in an embodiment of the present invention. [Figure 10] FIG. 4 is a diagram showing an example of text information according to the embodiment of the present invention. [Figure 11] FIG. 4 is a diagram showing an example of form information according to the embodiment of the present invention. [Figure 12] FIG. 4 is a diagram showing an example of output data according to the embodiment of the present invention. [Figure 13] 1 is a flowchart showing a learning method for a natural language understanding module in an embodiment of the present invention. [Figure 14] 2 is a diagram showing an example of a configuration of learning data for a natural language understanding module according to an embodiment of the present invention; FIG. [Figure 15] 2 is a diagram showing an example of a configuration of learning data for a natural language understanding module according to an embodiment of the present invention; FIG. [Figure 16] FIG. 2 is a diagram showing a learning method of a natural language understanding module in an embodiment of the present invention. [Figure 17] FIG. 2 is a diagram showing a learning method of a natural language understanding module in an embodiment of the present invention. [Figure 18] 1 is a flowchart showing a learning method for a form embedding module in an embodiment of the present invention. [Figure 19] 4 is a diagram showing an example of the configuration of learning data for a form embedding module in an embodiment of the present invention; FIG. [Figure 20] FIG. 11 is a diagram showing a learning method of a form embedding module in an embodiment of the present invention. [Figure 21] 1 is a flowchart showing a learning method for a medical form generation module in an embodiment of the present invention. [Figure 22] FIG. 1 is a diagram showing a method for improving a medical support system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] [System Configuration] 1, a medical support system 100 according to an embodiment of the present invention includes a server 102 and a client 104. The client 104 may be a single client or a plurality of clients. The server 102 and the client 104 are connected to each other via an information and communication network 106 such as the Internet so as to be able to exchange information with each other.

[0022] The information and communication network 106 is not limited to the Internet, but may be anything that can communicatively connect the server 102 and the client 104 to each other. For example, it may be a dedicated line, a public line (telephone line, mobile communication line, etc.), a wired LAN (Local Area Network), a wireless LAN, etc., or it may be a combination of the Internet and these.

[0023] As shown in FIG. 2, the server 102 includes a processing unit 10, a storage unit 12, an input unit 14, an output unit 16, and a communication unit 18. The processing unit 10 includes a means for performing arithmetic processing such as a CPU. The processing unit 10 executes a medical support server program stored in the storage unit 12, thereby realizing a process of accurately and quickly entering medical information on a form of a predetermined format in the medical support system 100 in this embodiment. The storage unit 12 includes a storage means such as a semiconductor memory or a memory card. The storage unit 12 is connected to the processing unit 10 so as to be accessible, and stores the medical support server program and information required for the process. The input unit 14 includes a means for inputting information. The input unit 14 includes, for example, a keyboard, a touch panel, a button, etc. for receiving input from an administrator. The output unit 16 includes a means for outputting a processing result in the server 102, such as a user interface screen (UI) for receiving input information from the administrator. The output unit 16 includes, for example, a display for presenting an image to the administrator. The communication unit 18 includes an interface for communicating information with the client 104 via the information and communication network 106. The communication by the communication unit 18 may be wired or wireless.

[0024] The server 102 accepts access from the clients 104 of each user who receives medical support services via the information and communication network 106 and provides the services via the information and communication network 106 to each user.

[0025] As shown in FIG. 3, the client 104 includes a processing unit 20, a storage unit 22, an input unit 24, an output unit 26, and a communication unit 28. The client 104 is also called a communication terminal. The processing unit 20 includes a means for performing arithmetic processing such as a CPU. The processing unit 20 realizes the function as a client terminal in the medical support system 100 in this embodiment by executing a medical support client program stored in the storage unit 22. The storage unit 22 includes a storage means such as a semiconductor memory or a memory card. The storage unit 22 is connected to the processing unit 20 so as to be accessible, and stores the medical support client program and information required for the processing. The input unit 24 includes a means for inputting information. The input unit 24 includes, for example, a keyboard, a touch panel, a button, a motion sensor, a microphone, etc. that receives input from a user. The output unit 26 includes a means for outputting information required for processing in the client 104, such as a screen for receiving input information from a user and a display that displays image information such as a user interface screen (UI). The communication unit 28 includes an interface that communicates information with the server 102 via the information and communication network 106. The communication by the communication unit 28 may be wired or wireless.

[0026] Various information processing devices capable of executing a client program for providing medical support services can be applied as the client 104. For example, the client 104 can be a stationary or portable personal computer (PC), a tablet computer, a smartphone, a mobile phone terminal, a PHS (Personal Handy-phone System) terminal, a portable information terminal (PDA: Personal Digital Assistant), a multi-function television receiver equipped with an information processing function (so-called smart TV), etc.

[0027] In order to ensure a secure access from the client 104 to the server 102, a process such as user authentication may be performed when the client 104 accesses the server 102. The user authentication process may be performed, for example, by having the client 104 input a user ID and password from the input unit 24 of the client 104, and authenticating access to the server 102 from the client 104 if the inputted user ID and password match a combination of the user ID and password stored in advance in the storage unit 12 of the server 102.

[0028] [Medical Support Processing] 4 shows a system configuration diagram of the medical support system 100 according to this embodiment. The medical support system 100 includes a voice-to-text conversion module 110 and a medical form generation module 112, as shown in FIG.

[0029] The voice-to-text conversion module 110 is a module that performs processing to extract medical text information from voice information obtained from medical professionals, patients, etc. Also, the medical form generation module 112 is a module that extracts and outputs medical information corresponding to a predetermined form related to medical information based on the medical text information.

[0030] As shown in Fig. 4, the medical form generation module 112 can be configured to include a natural language understanding module 112a and a form embedding module 112b. The natural language understanding module 112a is a module that performs a process of extracting medical information required to generate a medical form from medical text information. The form embedding module 112b is a module that embeds the medical information output from the natural language understanding module 112a into a predetermined medical form and outputs the medical form. The natural language understanding module 112a and the form embedding module 112b may be independent modules, or may be integrated into one module.

[0031] A medical form is a document such as a medical record used in a medical institution. However, a medical form is not limited to a medical record, but may be any form that includes medical information corresponding to predetermined items related to a patient's condition and medical care. A medical form may be electronic data, or may be printed out on a paper medium from the output unit 16 of the server 102 or the output unit 26 of the client 104.

[0032] The speech-to-text module 110 and the medical form generation module 112 are each described below.

[0033] [Speech to text conversion module] As shown in FIG. 5, the speech-to-text conversion module 110 receives voice information about a patient's medical condition and treatment instructions from medical professionals involved in medical care, such as doctors, nurses, clinic staff, and emergency personnel, and converts the voice information into text information.

[0034] The speech-to-text conversion module 110 can apply existing speech-to-text conversion processing algorithms and AI modules.

[0035] For example, the speech-to-text conversion module 110 can be configured by applying a statistical speech recognition technique. The statistical speech recognition technique is configured from a feature extractor that extracts features from speech, a decoder, and a speech recognition model (a model that integrates an acoustic model, a pronunciation dictionary, and a language model). The feature extractor analyzes a speech signal sequence (for example, time-series speech information) and extracts a feature vector of several tens of dimensions called a logarithmic Mel-frequency spectrum or a Mel-frequency cepstrum. The decoder searches for and outputs a word sequence that is the most likely based on the feature vector. The speech recognition model is configured by integrating an acoustic model, a pronunciation dictionary, and a language model. The acoustic model is a model that obtains a most likely phoneme sequence from a feature vector sequence. The acoustic model is mainly applied with a non-deterministic state transition model having a finite state called HMM. The pronunciation dictionary is a probabilistic model that generates words from a phoneme sequence. The language model is a model that obtains a most likely word sequence from words obtained from the pronunciation dictionary. In other words, the language model is a model that specifies the language-likeness of a given arbitrary symbol sequence (word sequence).

[0036] Also, for example, the speech-to-text conversion module 110 can be configured by applying an AI module based on deep learning. The AI ​​module can be configured in various ways, such as CNN and RNN, but it is often used to learn and use a deep neural network (DNN) that outputs a vector whose elements are the probability of belonging to an HMM state when a feature vector is input.

[0037] Specifically, for example, speech information from a single medical professional and text information transcribed from the speech information are combined as teacher data to generate supervised learning data, the speech data is input into an AI module composed of a neural network, and the AI ​​module is trained to produce output that matches the teacher data, thereby forming the speech-to-text conversion module 110.

[0038] Here, the voice-to-text conversion module 110 can be configured to convert voice information from a single medical worker into text information, as shown in Fig. 5. Fig. 5 shows an example in which voice information related to medical care uttered by doctor 1 is input and the voice information is converted into text information. Also, as shown in Fig. 6, the voice information in the form of a conversation between multiple medical workers or between a medical worker and a patient may be converted into text information. Fig. 6 shows an example in which voice information related to medical care uttered by doctor 1, doctor 2, and nurse 1 is input and the voice information is converted into text information in the form of a chat conversation.

[0039] 7, in order to facilitate recognition of medical terminology, a medical language model and a speech recognition model may be integrated for processing. For example, the speech-to-text conversion module 110 may be constructed using existing techniques such as Shallow Fusion, Cold Fusion, Deep Fusion, and Density Ratio Approach.

[0040] Additionally, by relying on key-value pairs containing medical terms and abbreviations used to train the algorithms built into the medical form generation module 112 described below, free text entries can be converted into structured data that is mapped to a predefined format related to a particular medical condition.

[0041] In a noisy environment such as a hospital or clinic, it is preferable to apply a noise reduction algorithm as shown in Fig. 8. The noise reduction algorithm plays an important role in improving the accuracy of the processing in the speech-to-text conversion module 110. In particular, in the medical speech-to-text conversion process, clarity of the speech information is essential due to the importance and sensitivity of the content.

[0042] Beneficial denoising algorithms for the speech-to-text module 110 include spectral subtraction, Wiener filtering, beamforming and microphone arrays, noise suppression using deep neural networks (DNNs), recurrent neural networks (RNNs) and long short-term memory (LSTM), non-negative matrix factorization (NMF), statistical model-based methods, binaural processing, end-to-end denoising using autoencoders, etc.

[0043] Spectral subtraction is a classical technique that involves subtracting an estimate of the noise spectrum from the noisy speech spectrum. It works by estimating the noise spectrum of non-speech segments and then subtracting this estimated noise from the entire signal.

[0044] Wiener filtering is a statistically based approach. The Wiener filter minimizes the mean squared error between the estimated clean signal and the actual clean signal. It is particularly effective when the characteristics of the noise are known.

[0045] Beamforming and microphone arrays are techniques that reduce noise by using multiple microphones to capture sound. The signals from the multiple microphones are combined in such a way that signals from certain directions are amplified and signals from other directions are suppressed. This helps to focus on the speaker's voice and minimize background noise.

[0046] Deep Neural Network (DNN) noise suppression is a method of reducing noise using deep learning models. By applying a DNN model, it can be trained to distinguish between clean speech and noise. Once trained, the model can suppress noise from novel speech data. With enough training data, DNNs can perform very well even in complex noisy environments.

[0047] Recurrent neural networks (RNNs) and long short-term memory (LSTM) are capable of storing past information and can be used to remove noise. They are particularly effective for time-series information such as audio signals, where context and time-series information are useful for identifying and removing noise.

[0048] Nonnegative matrix factorization (NMF) decomposes the spectrogram of a noise signal into a set of basis spectra and their activities. It is possible to separate speech and noise signals by assuming that they can be approximated by different sets of basis spectra.

[0049] Statistical model-based methods estimate the statistical properties of speech and noise. Based on the statistical model, noise is suppressed and speech is enhanced. Voice activity detection (VAD) is a technique that detects the presence or absence of human speech. During periods when no speech is detected, the system captures samples of background noise to more effectively suppress noise during speech segments. Binaural processing is a technique based on human auditory processing, using two or more microphones (which mimic ears) to identify and suppress noise sources. End-to-end denoising with autoencoders is a technique that uses an autoencoder trained to directly map noisy speech to clean speech. The encoder compresses the noisy speech into a compact representation, and the decoder reconstructs the clean speech from this representation.

[0050] In addition, sensitive information such as personal information may be anonymized from text information indicating medical information. Sensitive information may be defined as appropriate. For example, the type of information that is considered to be sensitive or capable of identifying an individual is defined as sensitive information. Typically, in medical records, sensitive information includes the patient's name, address, telephone number, social security number, and other unique identifiers.

[0051] One process for anonymizing such sensitive information is to tokenize the data. Tokenization involves breaking down text into individual words or tokens and then applying a replacement process to recognize and anonymize specific data points. Anonymization can also be achieved by using named entity recognition (NER). NER is a natural language processing technique that identifies named entities in text and classifies them into predefined categories such as names, organizations, and locations. Traditional NER models may not be able to capture certain medical terminology and nuances, so a pre-trained NER model that is fine-tuned specifically for medical records is preferable.

[0052] Several methods can be applied as a process for anonymizing sensitive information after it is identified. One method of anonymization is to completely delete sensitive information by re-editing text information. Another method of anonymization is to pseudonymize sensitive information contained in text information. When pseudonymizing a person's name contained in text information, for example, if a word contained in text information such as "Taro Tanaka" is recognized as a person's name, a process is performed to replace the word "Taro Tanaka" with information indicating that it is a person's name, "PERSON_NAME". This method is particularly effective when it is necessary to maintain the data structure of the text information. Another method of anonymization is to generalize sensitive information contained in text information. For example, a method is a method of replacing information of a specific item with a more general one, such as replacing a date contained in text information with only the year.

[0053] Specifically, regular expressions can be used to detect typical patterns of specific sensitive information such as birth dates and phone numbers contained in text information. In addition, a dictionary containing known names, place names, or specific terms used within an organization such as a medical institution may be created in advance, and the known terms may be extracted and replaced with other specific terms using the dictionary.

[0054] Machine learning and deep learning may also be used in the anonymization process. By applying a machine learning or deep learning model, text information on medical information and information in which specific sensitive information, such as a date of birth or a phone number, contained in the text information, is combined as training data, and machine learning is performed so that the training data is output by inputting the text information. This makes it possible to build an AI model for anonymization processing that outputs anonymized specific sensitive information, such as a date of birth or a phone number, contained in the text information. By applying machine learning and deep learning, it becomes possible to detect and anonymize inconspicuous text patterns and relationships contained in the text information that may lead to deanonymization.

[0055] In addition, it is preferable that the consistency of the anonymization process is maintained. That is, in a pseudonymization process, for example, if "John Doe" is replaced with "Patient A" in one record in the text information, the same replacement is preferable to be consistent in all records. In addition, it is preferable that the anonymization process is verified, ideally by a human-supervised review process. This allows for the identification of overlooked cases and further improvement of the anonymization process. In addition, by continuously updating the model, it is possible to perform anonymization appropriately, especially when new data types or structures are introduced into the records.

[0056] [Medical form generation module] As shown in Figure 9, the medical form generation module 112 receives as input text information including medical information related to medical care and form information regarding a medical form into which the medical information is embedded, and performs processing to output a medical form into which the medical information contained in the text information is appropriately embedded.

[0057] The text information input to the medical form generation module 112 is text information including medical-related information obtained from a medical professional. The text information may be text data in the form of a chat by a medical professional, as shown in Fig. 10. The text information may also be text data extracted from voice information by the voice-to-text conversion module 110, as described above.

[0058] Moreover, the form information input to the medical form generation module 112 is information indicating the configuration of a medical form in which the medical information extracted from the text information is embedded. The medical form can be, for example, a medical record, a document that needs to be filled in for each disease or each symptom, etc. The medical form can be, for example, a diagnosis form, a treatment form, a patient follow-up form, a research data collection form, etc. Therefore, the form information can be information that combines a question ID, the content of the question, and the type of answer to the question (information type such as numerical value, text, etc.) for each type of medical form, as shown in, for example, FIG. 11. The medical form generation module 112 extracts an appropriate type of information corresponding to the question included in the form information from the input text information, embeds the information in an appropriate medical form, and outputs it.

[0059] The form information may be selected and input by a user such as a medical professional based on the patient's condition, medical procedure, etc. Also, the medical form generation module 112 may automatically select appropriate form information from a plurality of pieces of form information set in advance based on text information input to the medical form generation module 112.

[0060] The output data of the medical form generation module 112 is information including question IDs corresponding to the input form information and answers to each question. The format of the output medical form depends on the type of related question. For example, if the question is multiple choice, single choice, input number, free text, etc., the output answer is formatted accordingly. It is preferable that the output data is output according to a predetermined format that is easy to process later, such as JSON format.

[0061] The natural language understanding module 112a included in the medical form generation module 112 is configured to extract and output medical information to be included in the medical form based on the input text information and form information. Fig. 12 shows an example of output data output from the natural language understanding module 112a. The natural language understanding module 112a receives input of text information and form information, extracts medical information from the text information for the items of medical information required for the medical form specified by the form information, and outputs the medical information. Fig. 12 shows an example of extracted information required to fill in a medical form related to stroke, such as the patient's age "Age", gender "Gender", blood pressure "Blood Pressure", and pulse rate "Heart Rate".

[0062] <Natural Language Understanding Module> The natural language understanding module 112a included in the medical form generation module 112 can be configured by applying an AI module based on deep learning. The natural language understanding module 112a can be an AI module to which a large-scale language model (LLM) such as GPT-4 or LLaMA2 is applied. The large-scale language model (LLM) is a natural language processing model whose accuracy is improved by machine learning using a large-scale data set.

[0063] There are several steps in training a language model for automatically generating a medical form based on medical information. The training process for constructing the natural language understanding module 112a will be described below with reference to the flowchart in FIG.

[0064] In step S10, a process of preparing learning data to be used for training the natural language understanding module 112a is performed. As the learning data, a large amount of text information showing medical records needs to be collected. In addition, it is necessary to prepare an appropriate filled-in medical form as an output for the text information to be input to the medical form generation module 112.

[0065] The natural language understanding module 112a receives as input data text information including medical information and form information specifying the medical information to be extracted from the text information required for embedding in a desired medical form, and prepares supervised learning data by combining the medical information to be output when the text information and form information are input as supervised data.

[0066] For example, as shown in Fig. 14, text information including medical information and form information indicating information (items) to be output from the natural language understanding module 112a are used as input data, and the medical information to be output from the natural language understanding module 112a is used as training data, which are then combined to create training data. In the example of Fig. 14, the form information is instruction information that specifies the patient's medical history as an output item. The training data is "diabetes," which is the patient's medical history to be extracted from the text information included in the training data.

[0067] The instruction information may be, for example, information instructing to output items indicating the patient's condition, such as the patient's age, sex, medical history, blood pressure, pulse, oxygen saturation, the presence or absence of dyspnea, state of consciousness, changes in electrocardiogram, blood test results, etc. The instruction information may be, for example, information instructing to output items indicating medical procedures, diagnostic and therapeutic methods, and examination items to be performed on the patient, such as injections, image diagnosis, and thrombolytic therapy. However, the instruction information is not limited to these, and may be any information indicating the content of information to be extracted from the text information input to the natural language understanding module 112a and output.

[0068] Furthermore, as shown in Fig. 15, the learning data may include form information that specifies multiple items included in a medical form at once. Fig. 15 shows an example of form information that includes instruction information that specifies outputting items required to generate a medical form related to stroke, such as the patient's age, gender, blood pressure, and heart rate. The teacher data includes the items specified in the form information and information to be extracted from text information included in the learning data corresponding to the items.

[0069] The learning data can be divided into a training set, a validation set, and a test set. The training set is used for learning the natural language understanding module 112a. The validation set is used for validating the learned natural language understanding module 112a. The test set is used for finally testing the validated natural language understanding module 112a. This makes it possible to evaluate the performance of the natural language understanding module 112a with respect to unknown data.

[0070] Moreover, it is preferable that sensitive information contained in the text information is anonymized as described above in order to ensure privacy, etc.

[0071] In step S12, data preprocessing is performed. It is preferable that the text information input to the natural language understanding module 112a is formatted in a consistent machine-readable format. Therefore, in this step, the text information is converted into a machine-readable format as necessary. For example, depending on the complexity of the medical form, the text information is structured as a pair of items and their values, or as a more complex schema.

[0072] In addition, it is preferable that medical terms and abbreviations are converted into pairs of items and values. For example, text information may contain various different words such as "acute myocardial infarction," "AMI," "shinkinkosoku," and "Heart Attack" that indicate the same "myocardial infarction." Therefore, using these words as keys, these words are associated with "myocardial infarction" and stored as a medical terminology database. Then, when these words are included in the text information, a process is performed to replace them with "myocardial infarction." The process of replacing words with the same meaning with a common word can be applied to other medical terms and abbreviations as well.

[0073] In step S14, a learning process is performed for the natural language understanding module 112a. The natural language understanding module 112a is trained using the learning data prepared in steps S10 and S12. Specifically, the text information and form information included in the learning data are input to the AI ​​module of the natural language understanding module 112a, and machine learning is performed so that output data, which is teacher data included in the learning data, is output from the natural language understanding module 112a.

[0074] Here, the AI ​​module constituting the natural language understanding module 112a may be, for example, a module including a pre-trained AI model such as GPT-3 or GPT-4. Such an AI model may be fine-tuned to constitute the natural language understanding module 112a.

[0075] FIG. 16 shows a learning method of the natural language understanding module 112a. By inputting supervised learning data to the AI ​​module of the natural language understanding module 112a, output data corresponding to the input text information and form information (including instruction information) is output from the natural language understanding module 112a. The output data is compared with the supervised data in the supervised learning data, and a loss evaluation is performed so that the output data matches the supervised data, and an evaluation function and weighting in the AI ​​module of the natural language understanding module 112a are adjusted. A large number of supervised learning data are prepared, and the learning process is repeated for the large number of supervised learning data, so that the natural language understanding module 112a is constructed so that output data that matches the supervised data is output.

[0076] FIG. 17 shows another example of a learning method of the natural language understanding module 112a. In this example, the natural language understanding module 112a is learned as a module that extracts and outputs multiple items included in a medical form at once. In the example of FIG. 17, the supervised learning data includes form information including instruction information that specifies outputting items required to generate a medical form related to a stroke, such as the patient's age "Age", gender "Gender", blood pressure "Blood Pressure", and pulse "Heart Rate". By inputting the supervised learning data to the AI ​​module of the natural language understanding module 112a, output data corresponding to the input text information and form information (including instruction information) is output from the natural language understanding module 112a. The output data is compared with the supervised data in the supervised learning data, and a loss evaluation is performed so that the output data of each item included in the output data matches the data of each item included in the supervised data, and an evaluation function and weighting in the AI ​​module of the natural language understanding module 112a are adjusted. A large amount of supervised learning data is prepared, and the natural language understanding module 112a is constructed so that output data that matches the supervised learning data is output by repeating the learning process on the large amount of supervised learning data.

[0077] It is more preferable to perform learning so that items considered important in the output data are preferentially extracted. For example, the items included in the form information are weighted in terms of importance, and the natural language understanding module 112a is trained so that the higher the importance of an item, the more correctly the medical information related to that item is extracted.

[0078] Specifically, a weight is assigned to each item for the loss used for evaluation when training the AI ​​module of the natural language understanding module 112a. That is, the higher the importance of an item, the higher the evaluation of the loss when the natural language understanding module 112a outputs something different from the training data. Then, the natural language understanding module 112a is trained so that the loss becomes smaller.

[0079] For example, if the weights of the items of gender, illness, time, and MRI are 1, 5, 3, and 3, respectively, and the output data of the natural language understanding module 112a for the items of gender, illness, time, and MRI are correct, incorrect, incorrect, and correct, respectively, with respect to the teacher data, the weighted loss error is calculated as (5+3) / (1+5+3+3)=0.67. In contrast, if no weighting is applied to each item, the loss error is calculated as (1+1) / (1+1+1+1)=0.50. Therefore, by machine learning the natural language understanding module 112a so as to minimize the weighted loss error, the natural language understanding module 112a can be trained to output more correct output data for items with larger weighting values.

[0080] In addition, if the input text information does not contain necessary medical information, it is preferable to train the system to output special information such as "not included." This is because, when information is unknown in medical care, the fact that the information is unknown itself carries information.

[0081] In step S16, the natural language understanding module 112a is evaluated. The learning data is divided into a training set, a validation set, and a test set, and the natural language understanding module 112a that has been trained using the training set is evaluated using the validation set and the test set. This allows the performance of the AI ​​model of the natural language understanding module 112a for unknown data to be evaluated.

[0082] For example, the accuracy of output data to be filled in a medical form based on the transcription can be evaluated. For example, the natural language understanding module 112a can be evaluated based on metrics such as accuracy rate and mean absolute error, or domain-specific accuracy metrics such as perplexity and BLEU.

[0083] As described above, the natural language understanding module 112a included in the medical form generation module 112 can be constructed by machine learning.

[0084] <Form Embedding Module> In the medical form generation module 112, the medical information extracted from the text information and output by the natural language understanding module 112a is embedded in a desired medical form by the form embedding module 112b and output.

[0085] The form embedding module 112b can be constructed as a machine-learned AI module. There are several steps in the learning of the form embedding module 112b. Hereinafter, the learning process for constructing the form embedding module 112b will be described with reference to the flowchart of FIG.

[0086] In step S20, a process is performed to prepare learning data to be used for learning by the form embedding module 112b. As the learning data, supervised learning data is prepared by combining the medical information output from the natural language understanding module 112a and the medical form in which the medical information is appropriately embedded, as supervised data.

[0087] For example, as shown in FIG. 19, medical information output from the natural language understanding module 112a is used as input data, a medical form in which the medical information is appropriately embedded is used as training data, and these pieces of information are combined to prepare supervised training data.

[0088] The learning data can be divided into a training set, a validation set, and a test set. The training set is used for learning the form embedding module 112b. The validation set is used to validate the learned form embedding module 112b. The test set is used for finally testing the validated form embedding module 112b. This makes it possible to evaluate the performance of the form embedding module 112b for unknown data.

[0089] In step S22, a learning process is performed for form embedding module 112b. Form embedding module 112b is trained using the learning data prepared in step S20. Specifically, input data included in the learning data is input to the AI ​​module of form embedding module 112b, and machine learning is performed so that the medical form, which is teacher data included in the learning data, is output from form embedding module 112b.

[0090] FIG. 20 shows a learning method of the form embedding module 112b. In the example of FIG. 20, in the supervised learning data, medical information related to stroke, such as the patient's age "Age", gender "Gender", blood pressure "Blood Pressure", and pulse "Heart Rate", is used as input data, and a medical form in which this information is embedded is used as teacher data. By inputting the input data to the AI ​​module of the form embedding module 112b, a medical form in which the input data is embedded is output as output data from the form embedding module 112b. The output data is compared with the teacher data in the supervised learning data, and a loss evaluation is performed so that the medical information of each item included in the output data matches the medical information of each item of the medical form included in the teacher data, and an evaluation function and weighting in the AI ​​module of the form embedding module 112b are adjusted. A large number of supervised learning data are prepared, and the learning process is repeated for the large number of supervised learning data, so that the form embedding module 112b is constructed so that output data that matches the teacher data is output.

[0091] In step S24, the form embedding module 112b is evaluated. The learning data is divided into a training set, a validation set, and a test set, and the form embedding module 112b that has been trained using the training set is evaluated using the validation set and the test set. This allows the performance of the AI ​​model of the form embedding module 112b for unknown data to be evaluated.

[0092] As described above, the form embedding module 112b included in the medical form generation module 112 can be constructed by machine learning.

[0093] The medical form actually generated by the form embedding module 112b is stored in the memory unit 12 of the server 102. Furthermore, the generated medical form is presented to a user (such as a medical professional) by the output unit 16 of the server 102 or the output unit 26 of the client 104 as necessary. The generated medical form may be displayed on a display or the like as electronic data, or may be printed out on paper or the like and presented to the user.

[0094] The medical information output from the natural language understanding module 112a is output in association with each item of the medical form specified by the form information. Therefore, the form embedding module 112b may embed the medical information output from the natural language understanding module 112a in association with each item of the medical form, and generate and output the medical form.

[0095] Here, the form embedding module 112b may refer to a medical terminology database stored in advance in the storage unit 12 of the server 102 and determine whether or not the terms included in the medical information extracted by the natural language understanding module 112a match the terms included in the medical terminology database. For example, if the terms included in the medical information extracted by the natural language understanding module 112a do not match the terms included in the medical terminology database, an alert may be output to a user such as a medical professional to warn them. Also, for example, if the terms included in the medical information extracted by the natural language understanding module 112a do not match the terms included in the medical terminology database, the terms may be replaced with terms that are expected to be correct among the terms included in the medical terminology database, and embedded in the medical form for output.

[0096] Also, an alert (warning) may be output when a specific performance evaluation value for a medical form output from the medical support system 100 deteriorates below a predetermined reference value. For example, the importance of each of the above-mentioned items is weighted, and a performance evaluation value is calculated according to the accuracy of the medical information included in the medical form output from the medical form generation module 112, and an alert (warning) is output when the performance evaluation value deteriorates below a reference value.

[0097] Specifically, for example, if the weights of the items Gender, Disease, Time, and MRI are 1, 5, 3, and 3, respectively, and the output data of the natural language understanding module 112a for the items Gender, Disease, Time, and MRI are correct, incorrect, incorrect, and correct, respectively, the weighted loss error is calculated as (5+3) / (1+5+3+3)=0.67. If the weighted loss error is equal to or greater than a predetermined reference value, an alert (warning) is output. Also, for example, an alert (warning) may be output when medical information of a specific required item is missing from the output medical form.

[0098] Furthermore, the form embedding module 112b may detect potential contradictions or inconsistencies in the medical information based on the mutual relationships of the medical information output from the natural language understanding module 112a. For example, when specific medical information should be combined for a plurality of items, the combination of the specific medical information for the plurality of items is stored in advance in the storage unit 12 of the server 102 as a medical information database, and an alert (warning) is output when a combination of medical information other than the specific combination of medical information for the plurality of items is output from the natural language understanding module 112a.

[0099] 21, the medical form generation module 112 may be constructed with the natural language understanding module 112a and the form embedding module 112b as one AI module. In this case, text information and form information are used as input data, and the medical form to be output corresponding to the information is used as training data, and these pieces of information are combined and used as supervised training data. The input data included in the training data is input to an AI module combining the natural language understanding module 112a and the form embedding module 112b, and machine learning is performed so that the medical form, which is training data included in the training data, is output from the AI ​​module.

[0100] Furthermore, when the natural language understanding module 112a and the form embedding module 112b are subjected to machine learning, they can also be trained to predict the type of medical form appropriate for the text information based on the text information.

[0101] In addition, as shown in FIG. 22, it is preferable to analyze errors and mistakes in the output data output from the natural language understanding module 112a. This provides insight into the need for improvement, and makes it possible to improve the AI ​​model and training data of the natural language understanding module 112a based on the insight. In addition, it is preferable to continuously evaluate the natural language understanding module 112a when it is actually used. Feedback from users such as doctors and medical professionals is useful for understanding problems and areas for improvement in the natural language understanding module 112a. In addition, it is preferable to continuously learn (active learning) the AI ​​model of the natural language understanding module 112a based on user feedback.

[0102] In addition, the medical support system 100 may be provided after previously learning the natural language understanding module 112a and the form embedding module 112b, or scratch learning may be performed in which the natural language understanding module 112a and the form embedding module 112b are built by learning them from public data or the company's own service data.

[0103] Additionally, medical forms should be categorized based on the type of illness, type of treatment, medical condition, surgical procedure, and other medical related factors. For medical forms, metadata or tags can be associated with each form type to enable the medical form generation module 112 to quickly identify and select the appropriate form.

[0104] The medical support system 100 can use a patient-specific identifier, biometric authentication, or near-field communication (NFC) technology to identify a patient when the patient arrives or logs in, and output the treatment plan and history of the identified patient in an appropriate medical form. In addition, patient data can be input to the medical support system 100 in real time by synchronizing with an electronic health record (EHR). This allows the medical support system 100 to predict and select an appropriate medical form based on recent diagnoses, medications, and test results, and embed medical information in the medical form and output it.

[0105] Furthermore, sensors such as wearable health devices may be integrated into the medical support system 100. Real-time health data of a patient obtained from the wearable health device may be input as text information to the medical support system 100, and when the health data indicates a specific condition (e.g., an abnormal condition) of the patient, the medical support system 100 may output a medical form in which medical information related to the condition is embedded.

[0106] Also, based on past medical forms, it may be possible to predict which format of medical form will be required next. For example, if there is a history of filling in a monitoring form with the patient's condition every month, form information may be input to the medical support system 100 so that the monitoring form is generated and output at the time of the patient's monthly examination. Also, the patient's medical history may be analyzed, and form information may be input to the medical support system 100 so that a medical form in a format required for the patient is generated and output. For example, for a patient with a history of heart disease, it is preferable to generate and output a medical form in a format specialized for cardiology.

[0107] Also, a geolocation or indoor positioning system may be integrated into the medical support system 100, and location information indicating where a patient or medical equipment is located may be input to the medical support system 100, and a medical form to be generated and output by the medical support system 100 may be selected based on the location information. For example, when a patient is in the cardiology department of a medical institution (hospital, etc.), it is preferable to set the form information so that a medical form related to the cardiology department is generated and output from the medical support system 100.

[0108] Also, medical forms of an appropriate format may be selected in the medical support system 100 depending on the time period, date and time, day of the week, week, month, or season. For example, during influenza epidemic season, it is preferable to set form information and input it to the medical support system 100 so that medical forms for evaluating influenza are preferentially generated and output.

[0109] In addition, the behavior and preferences of medical professionals may be analyzed, and if the medical professional uses a particular medical form frequently, the medical form may be preferentially selected. For example, if a doctor who deals with a particular disease frequently uses a medical form related to the disease, it is preferable to set form information and input it to the medical support system 100 so that the medical form is preferentially generated and output.

[0110] External data sources, such as disease occurrence databases and local health alerts, may also be integrated into the medical support system 100. In this way, if a specific disease suddenly occurs in a region, it is preferable to set form information and input it into the medical support system 100 so that medical forms related to the disease are generated and output preferentially.

[0111] Furthermore, natural language processing (NLP) may be used to implement a chatbot or a voice assistant in the medical support system 100. Using the chatbot or the voice assistant, patients or medical staff may input their current needs (e.g., "I need a post-operative follow-up form") by voice, and form information may be set to generate and output a specified medical form and input to the medical support system 100.

[0112] Furthermore, by integrating the medical support system 100 in this embodiment with an existing clinical decision support system, details of a patient's treatment plan can be input as text information to the medical support system 100. This allows the medical support system 100 to generate and output a medical form in which appropriate information for the treatment plan is embedded, based on the patient's treatment plan input as text information.

[0113] In addition, whether the medical form generated and output by the medical support system 100 is correct or not may be fed back to a user such as a medical professional. Based on this feedback, the medical form generation module 112 may learn to generate and output a more correct medical form.

[0114] In addition, by expressing the learning data in multiple languages ​​and performing machine learning on the medical form generation module 112, the medical support system 100 can be made multilingual and applied to generating medical forms that support multiple languages.

[0115] As described above, according to the medical support system 100 of the present embodiment, text information including medical information related to medical care can be generated from voice information including medical information related to medical care uttered by medical professionals such as doctors, nurses, and emergency personnel. In addition, based on the text information including medical information related to medical care, medical information of items to be filled in a desired medical form can be extracted from the text information. Furthermore, the medical information extracted from the text information including medical information related to medical care can be automatically embedded in the desired medical form, and generated and output as a filled-in medical form.

[0116] The medical support system 100 can be used in various medical settings, and is particularly effective in creating medical forms in emergency medical settings such as acute stroke and acute myocardial infarction (AMI).

[0117] For example, when a patient is brought to the emergency room with symptoms of acute cerebral infarction, a doctor can quickly start a stroke protocol. At this time, by using the medical support system 100, the doctor can verbally instruct the patient's symptoms, onset time, risk factors, and initial physical examination findings, and the medical information can be extracted from the voice information, accurately transcribed, and automatically entered into the desired medical form. This speeds up the process of diagnosis, examination, treatment, etc., and medical personnel can focus on treating the patient, such as imaging diagnosis, examination, and the possibility of thrombolytic therapy or thrombectomy.

[0118] Also, for example, when a patient visits an emergency room with severe chest pain, a doctor suspects acute myocardial infarction. The doctor immediately verbally communicates the patient's medical history, symptoms, changes in electrocardiogram, and the results of the first blood test to the medical support system 100. The medical support system 100 quickly extracts necessary medical information from the voice information, enters the medical information into a desired medical form, and outputs it. This makes it possible to quickly indicate, for example, the possibility of percutaneous coronary intervention (PCI), and save precious time, in which every minute counts, in order to preserve the patient's myocardium and increase the chances of survival.

[0119] In this way, the medical support system 100 enables rapid generation and output of accurate medical forms during medical emergencies, thereby minimizing the time medical personnel spend on paperwork and enabling rapid diagnosis, testing, and treatment in time-sensitive situations such as stroke and AMI, thereby enabling efficient provision of medical services.

[0120] [Configuration of the invention] [Configuration 1] A medical support system for automatically filling out medical forms relating to medical care, comprising: A medical support system comprising a natural language understanding module trained by machine learning to extract medical information for items to be filled in the medical form from text information including medical information related to medical care. [Configuration 2] The medical support system according to configuration 1, The natural language understanding module is a medical support system characterized in that it uses text information including medical information related to medical care as input data, medical information to be extracted from the text information for the items to be filled in the medical form as teacher data, and uses supervised learning data that combines these pieces of information to perform machine learning so that when the input data is entered, the medical information that is the teacher data is output. [Configuration 3] The medical support system according to configuration 2, The natural language understanding module receives as input data text information including medical information related to medical care and form information indicating the medical information to be extracted from the text information, and uses as teacher data the medical information to be extracted from the text for the items to be filled in the medical form.The natural language understanding module is machine-learned to use supervised learning data that combines these pieces of information so that when the input data is entered, the medical information that is the teacher data is output. [Configuration 4] The medical support system according to any one of configurations 1 to 3, A medical support system comprising a form embedding module that embeds medical information output from the natural language understanding module into the medical form and outputs the medical information. [Configuration 5] The medical support system according to configuration 4, The form embedding module is a medical support system characterized in that it is machine-learned to use medical information as input data, the medical form with the medical information properly filled out as training data, and supervised learning data that combines these pieces of information to output the medical form, which is the training data, when the medical information is input. [Configuration 6] The medical support system according to configuration 4 or 5, The natural language understanding module and the form embedding module are integrated, and when the text information is input, medical information for the items to be filled in the medical form is extracted from the text information, and the medical information is embedded in the medical form and output. [Configuration 7] The medical support system according to any one of configurations 1 to 6, A medical support system comprising a voice-to-text conversion module that generates text information from voice information including medical information relating to medical care. [Configuration 8] The medical support system according to configuration 7, The medical support system is characterized in that the speech-to-text conversion module anonymizes specific sensitive information and outputs the text information. [Configuration 9] The medical support system according to configuration 7 or 8, The medical support system is characterized in that the speech-to-text conversion module processes the speech information by removing noise using a noise removal algorithm. [Configuration 10] The medical support system according to any one of configurations 1 to 9, A medical support system characterized by being used in medical emergencies. [Configuration 11] A medical assistance program for automatically filling out medical forms relating to medical care, comprising: A medical support program that causes a computer to function as a natural language understanding module trained by machine learning to extract medical information for items to be filled in the medical form from text information including medical information related to medical care. [Configuration 12] A method for automatically filling out a medical form relating to medical care, comprising: A medical support method, characterized in that a computer functioning as a natural language understanding module that has been machine-learned to extract medical information for items to be filled in the medical form from text information including medical information related to medical care, automatically extracts medical information for items to be filled in the medical form from text information including medical information related to medical care. [Explanation of symbols]

[0121] 10 processing unit, 12 memory unit, 14 input unit, 16 output unit, 18 communication unit, 20 processing unit, 22 memory unit, 24 input unit, 26 output unit, 28 communication unit, 100 medical support system, 102 server, 104 client, 106 information and communication network, 110 speech-to-text conversion module, 112 medical form generation module, 112a natural language understanding module, 112b form embedding module.

Claims

1. A medical support system for automatically filling out medical forms relating to medical care, comprising: A medical support system comprising a natural language understanding module trained by machine learning to extract medical information for items to be filled in the medical form from text information including medical information related to medical care.

2. The medical support system according to claim 1, The natural language understanding module is a medical support system characterized in that it uses text information including medical information related to medical care as input data, medical information to be extracted from the text information for the items to be filled in the medical form as teacher data, and uses supervised learning data that combines these pieces of information to perform machine learning so that when the input data is entered, the medical information that is the teacher data is output.

3. The medical support system according to claim 2, The natural language understanding module receives as input data text information including medical information related to medical care and form information indicating the medical information to be extracted from the text information, and uses as teacher data the medical information to be extracted from the text for the items to be filled in the medical form.The natural language understanding module is machine-learned to use supervised learning data that combines these pieces of information so that when the input data is entered, the medical information that is the teacher data is output.

4. The medical support system according to any one of claims 1 to 3, A medical support system comprising a form embedding module that embeds medical information output from the natural language understanding module into the medical form and outputs the medical information.

5. The medical support system according to claim 4, The form embedding module is a medical support system characterized in that it is machine-learned to use medical information as input data, the medical form with the medical information properly filled out as training data, and supervised learning data that combines these pieces of information to output the medical form, which is the training data, when the medical information is input.

6. The medical support system according to claim 4, The natural language understanding module and the form embedding module are integrated, and when the text information is input, medical information for the items to be filled in the medical form is extracted from the text information, and the medical information is embedded in the medical form and output.

7. The medical support system according to claim 1, A medical support system comprising a voice-to-text conversion module that generates text information from voice information including medical information relating to medical care.

8. The medical support system according to claim 7, The medical support system is characterized in that the speech-to-text conversion module anonymizes specific sensitive information and outputs the text information.

9. The medical support system according to claim 7 or 8, The medical support system is characterized in that the speech-to-text conversion module processes the speech information by removing noise using a noise removal algorithm.

10. The medical support system according to claim 1, A medical support system characterized by being used in medical emergencies.

11. A medical assistance program for automatically filling out medical forms relating to medical care, comprising: A medical support program that causes a computer to function as a natural language understanding module trained by machine learning to extract medical information for items to be filled in the medical form from text information including medical information related to medical care.

12. A method for automatically filling out a medical form relating to medical care, comprising: A medical support method, characterized in that a computer functioning as a natural language understanding module that has been machine-learned to extract medical information for items to be filled in the medical form from text information including medical information related to medical care, automatically extracts medical information for items to be filled in the medical form from text information including medical information related to medical care.

Images