Medical record support apparatus, system, method, and program

Abstract

To improve efficiency in creating a medical record on the basis of a medical interview.SOLUTION: A medical record support apparatus includes: first acquisition means for acquiring text information of an utterance content converted by speech recognition in which speakers are distinguished for conversational speech data between a medical worker and a patient during a medical interview; generation means for generating an instruction sentence for a language model to be output in a medical record format from the text information; and second acquisition means for acquiring medical record information of the patient by inputting the instruction sentence to the language model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a medical record support device, system, method, and program. [Background technology]

[0002] In medical settings, medical professionals such as doctors and nurses are required to interview patients and create medical records. For example, nurses typically share the care of many common patients with other nurses. Therefore, nursing records, which are an example of medical records related to a specific patient, are shared with other nurses and medical professionals.

[0003] Patent Document 1 discloses a technique for creating an electronic medical record from audio data of a conversation between a doctor and a patient. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-026640 Summary of the Invention [Problem to be solved by the invention]

[0005] However, there is a problem in that creating medical records is a heavy burden for medical professionals. The reason is that medical professionals are busy caring for many patients, and yet they have to spend a lot of time creating medical records. This makes it difficult for medical professionals to spend time on their primary duties (examinations, nursing, treatment, etc.).

[0006] In view of the above-mentioned problems, an object of the present disclosure is to provide a medical record support device, system, method, and program for improving the efficiency of creating medical records based on medical interviews. [Means for solving the problem]

[0007] The medical record support device according to the present disclosure comprises: a first acquiring means for acquiring text information of the content of the conversation between the medical staff and the patient during the medical interview, the text information being converted by speech recognition that distinguishes the speaker; a generation means for generating, from the text information, instruction sentences for a language model to be output in the form of a medical record; a second acquiring means for acquiring medical record information of the patient by inputting the instruction sentence into the language model; Equipped with.

[0008] The medical record support system according to the present disclosure comprises: A glasses-type wearable device worn by medical staff during medical interviews, an information processing device communicably connected to the glasses-type wearable terminal, The information processing device includes: a first acquisition means for acquiring text information of the content of the conversation between the medical professional and the patient during the medical interview, the text information being converted by speech recognition that distinguishes between speakers, the first acquisition means being configured to acquire text information of the content of the conversation between the medical professional and the patient during the medical interview, the text information being acquired by the first acquisition means; a generation means for generating, from the text information, instruction sentences for a language model to be output in the form of a medical record; a second acquiring means for acquiring medical record information of the patient by inputting the instruction sentence into the language model; Equipped with The glasses-type wearable terminal displays the medical record information acquired by the second acquisition means on a screen for the wearer.

[0009] The medical record support method according to the present disclosure includes: The computer The text information of the conversation content is obtained by converting the conversation voice data between the medical professional and the patient during the medical interview using speech recognition that distinguishes between the speakers, and generating instructions for a language model to output the text information in the form of a medical record; The patient's medical record information is obtained by inputting the instruction sentence into the language model.

[0010] The medical record support program disclosed herein is a first acquisition process for acquiring text information of the speech content converted from conversational voice data between a medical professional and a patient during a medical interview by speech recognition that distinguishes between speakers; a generation process for generating instruction sentences for a language model to output from the text information in the form of a medical record; a second acquisition process for acquiring medical record information of the patient by inputting the instruction sentence into the language model; to be executed by the computer. [Effects of the Invention]

[0011] The present disclosure makes it possible to efficiently create medical records based on medical interviews. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration of a medical record support device according to the present disclosure. [Figure 2] 1 is a flowchart showing the flow of a medical record support method according to the present disclosure. [Figure 3] 1 is a block diagram showing the overall configuration of a medical record support system according to the present disclosure. [Figure 4] FIG. 1 is a block diagram illustrating a configuration of AR glasses according to the present disclosure. [Figure 5] FIG. 1 is a block diagram showing the hardware configuration of AR glasses according to the present disclosure. [Figure 6] FIG. 2 is a block diagram showing the configuration of a medical record assistance server according to the present disclosure. [Figure 7] FIG. 2 is a block diagram showing the hardware configuration of a medical record assistance server according to the present disclosure. [Figure 8] 1 is a sequence chart showing the flow of a medical record support process according to the present disclosure. [Figure 9] 1 is a sequence chart showing the flow of a medical record support process according to the present disclosure. [Figure 10]1 is a sequence chart showing the flow of a medical record support process according to the present disclosure. [Figure 11] 10A and 10B are diagrams for explaining an example of a display on a screen for a wearer of AR glasses at the start of a medical record support process according to the present disclosure and the concept of a start operation. [Figure 12] FIG. 10 is a diagram showing an example of a display on a screen for a wearer of AR glasses immediately after the start of a medical record support process (update mode) according to the present disclosure. [Figure 13] 10 is a diagram for explaining the concept of an operation for terminating a patient interview and medical record support process by a medical professional according to the present disclosure. FIG. [Figure 14] FIG. 10 is a diagram showing an example of a display on a screen for a wearer of AR glasses during voice recognition in the medical record support process (update mode) according to the present disclosure. [Figure 15] 10A and 10B are diagrams illustrating an example of a text conversion and a generated prompt based on a speech recognition result according to the present disclosure. [Figure 16] 10A-10C illustrate example displays of updated medical records output by an LLM from a generated prompt according to the present disclosure. [Figure 17] FIG. 1 is a block diagram illustrating a configuration of AR glasses according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0014] <Embodiment 1> FIG. 1 is a block diagram showing the configuration of a medical record support device 1. The medical record support device 1 is an information processing device that supports medical professionals in creating medical records by utilizing conversational voice data between a medical professional and a patient during a medical interview. Here, a "medical professional" refers to, for example, a doctor, nurse, physical therapist, or other person qualified to work in the medical field. Furthermore, a "medical record" refers to a medical record (chart) created by a doctor, a nursing record created by a nurse, or any other record related to a patient's medical treatment by a medical professional.

[0015] The medical record support device 1 includes a first acquisition unit 11, a generation unit 12, and a second acquisition unit 13. The first acquisition unit 11 acquires text information of the content of the utterance converted by speaker-distinguishing speech recognition from conversational voice data between a medical professional and a patient during a medical interview.

[0016] The generation unit 12 generates instruction sentences for a language model to output in the form of a medical record from the text information acquired by the first acquisition unit 11. Here, the "language model" is a computer program or information system that receives text data expressing questions or instructions in natural language as input and outputs text data that has been generated, converted, processed, summarized, etc. by performing a predetermined calculation on the input text data. The language model corresponds to a natural language model in an AI (Artificial Intelligence) model. Furthermore, the "instruction sentence" is text data that includes one or more sentences. The "instruction sentence" is input data for the language model and includes at least a portion of the text information acquired by the first acquisition unit 11. Furthermore, the "instruction sentence" is a sentence in a format that causes the language model to output the text information in the form of a medical record.

[0017] The second acquisition unit 13 acquires the patient's medical record information by inputting the instruction sentence generated by the generation unit 12 into a language model. That is, the language model generates and outputs text information of the medical record information by converting, processing, summarizing, etc. the text information included in the instruction sentence according to the input instruction sentence so as to fit it into the format of the medical record specified in the instruction sentence. Therefore, the second acquisition unit 13 acquires the text data output from the language model to which the instruction sentence is input as the medical record information of the patient for whom the medical interview was conducted.

[0018] 2 is a flowchart showing the flow of the medical record support method. First, a first acquisition unit 11 acquires text information of the utterance content converted by speaker-distinguishing speech recognition from conversational voice data between a medical professional and a patient during a medical interview (S1). Next, a generation unit 12 generates instruction sentences for a language model from the text information to output in the form of a medical record (S2). Then, a second acquisition unit 13 inputs the instruction sentences into the language model to acquire the patient's medical record information (S3).

[0019] In this way, the medical record support device 1 acquires medical record information in the form of a medical record using the content of speech between the medical professional and the patient when the medical professional interviews the patient. Therefore, when the medical professional orally interviews the patient, the medical professional can obtain medical record information based on the interview by using the medical record support device 1. Therefore, the medical record can be efficiently created based on the interview.

[0020] The medical record support device 1 includes a processor, a memory, and a storage device (not shown). The storage device stores a computer program that implements the processing of the medical record support method shown in Fig. 2, for example. The processor then loads the computer program from the storage device into the memory and executes the computer program. This allows the processor to implement the functions of an acquisition unit 11, a detection unit 12, a mask unit 13, and an output unit 14.

[0021] Alternatively, each component of the medical record support device 1 may be realized by dedicated hardware. Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and programs. Furthermore, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), quantum processor (quantum computer control chip), etc., may be used as the processor.

[0022] Furthermore, when some or all of the components of the medical record support device 1 are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each is connected via a communication network. Furthermore, the functions of the medical record support device 1 may be provided in a SaaS (Software as a Service) format.

[0023] <Embodiment 2> FIG. 3 is a block diagram showing the overall configuration of a medical record support system 1000. The medical record support system 1000 is an information system that supports a medical professional U1 wearing AR (Augmented Reality) glasses 100 to create a medical record for a patient U2 by having the medical professional U1 interview the patient U2. The medical record support system 1000 includes the AR glasses 100, a medical record support server 200, an authentication system 300, and a medical information system 400. However, the medical record support system 1000 is only required to include at least the AR glasses 100 and the medical record support server 200. The AR glasses 100, the medical record support server 200, the authentication system 300, and the medical information system 400 are communicably connected via a network N. Here, the network N is a wired or wireless communication line.

[0024] As described above, the medical worker U1 is a doctor, a nurse, or the like. The medical worker U1 wears the AR glasses 100 according to the present disclosure and interviews the patient U2. The patient U2 is either an inpatient in a hospital bed or an outpatient who is interviewed by the medical worker U1. If the medical worker U1 is a nurse, the medical record information will be electronic data of a nursing record. If the medical worker U1 is a doctor, the medical record information will be electronic data of a medical record. The medical record information may include a medical record and a nursing record, and may be a so-called "electronic medical record."

[0025] The authentication system 300 is an information system that performs authentication processing for patient identity verification. The authentication system 300 performs facial authentication, a type of biometric authentication, as an example of identity verification. The authentication system 300 may also perform biometric authentication other than facial authentication or identity verification processing other than biometric authentication. The authentication system 300 includes at least an authentication database (DB) 3. The authentication DB 3 is a database that associates and manages a patient ID 31 with facial feature information 32. The authentication DB 3 may also manage patient attribute information (such as name) other than the facial feature information 32 in association with the patient ID 31. The patient ID 31 is identification information for the patient U2. The facial feature information 32 is a feature vector including a set of facial feature points of the patient corresponding to the patient ID 31. For example, the facial feature information 32 is a set of information calculated using, as a feature amount, the distance between each of a plurality of feature points indicating facial features (eyes, nose, mouth, etc.) in the face area detected from the face image of the patient.

[0026] The authentication system 300 determines whether the facial authentication is successful based on the degree of match between facial feature information (query) based on the facial image included in the facial authentication request received via the network N and a plurality of (master) facial feature information 32 in the authentication DB 3. If the facial authentication is successful, the authentication system 300 identifies the patient ID 31 (and attribute information) associated with the facial feature information 32 for which the facial authentication was successful, and replies to the request source with the identified patient ID 31 (and attribute information) included in the facial authentication result.

[0027] The authentication system 300 may further include an iris authentication DB for iris authentication of medical personnel. In this case, the iris authentication DB manages medical personnel IDs and iris feature information in association with each other. The iris authentication DB may also manage attribute information of medical personnel other than iris feature information (e.g., qualification information and name of a doctor, nurse, etc.) in association with the medical personnel ID. The authentication system 300 then determines whether iris authentication is successful based on the degree of match between iris feature information (query) based on an iris image included in an iris authentication request received via the network N and multiple (master) iris feature information in the iris authentication DB3. If iris authentication is successful, the authentication system 300 identifies the medical personnel ID (and attribute information) associated with the iris feature information for which iris authentication was successful, and returns the identified medical personnel ID (and attribute information) to the requestor together with the iris authentication result.

[0028] The medical information system 400 is an information system for managing medical information such as medical records. The medical information system 400 includes at least a medical record DB4. The medical record DB4 is a database that manages the medical record information of patients. The medical record DB4 manages a patient ID 40 in association with medical record information 41 to 4n (n is a natural number equal to or greater than 1). The patient ID 40 is information equivalent to the patient ID 31. Therefore, the patient ID 31 for a specific patient U2 is either the same as the patient ID 40 or is information that can uniquely identify the patient ID 40.

[0029] The medical information system 400 searches the medical record DB 4 using the patient ID included in the request to acquire medical record information received via the network N as a search key, and returns the medical record information 41 etc. found in the search to the request source.

[0030] The AR glasses 100 are eyeglass-type wearable devices that have lenses and a screen through which light from the external surrounding space passes. The AR glasses 100 also display various display information (described later) on a screen, superimposed on a field of view of real space through which light from the outside passes and is visible to the wearer. The AR glasses 100 may be, for example, a head-mounted display (HMD). The AR glasses 100 are an example of an eyeglass-type wearable device worn by a medical worker U1 during a medical interview.

[0031] 4 is a block diagram showing the configuration of the AR glasses 100. The AR glasses 100 include a wearer imaging unit 121, an external imaging unit 122, a sound collection unit 123, a wearer screen 124, an acquisition unit 131, a transmission unit 132, and a display unit 133.

[0032] The wearer photographing unit 121 is a camera that photographs an area including the iris of the eye of the wearer, a medical worker U1, or a circuit or software that controls the camera. The wearer photographing unit 121 outputs an iris image of the iris of the medical worker U1 to the acquisition unit 131.

[0033] The external image capturing unit 122 is a camera that captures an image capturing area including the display area (field of view) of the wearer's screen 124, or a circuit or software that controls the camera. The external image capturing unit 122 captures at least an image in front of the wearer, the medical worker U1, and outputs the captured image to the acquisition unit 131. The external image capturing unit 122 may record the captured image in an internal storage unit (not shown).

[0034] The sound collection unit 123 is a microphone that collects sounds around the AR glasses 100, or a circuit or software that controls the microphone. The sound collection unit 123 collects the voices spoken by the medical staff U1 who is wearing the glasses and the voices spoken by the patient U2. Therefore, the sound collection unit 123 collects the voices of the medical staff U1 and the patient U2 who are present near the AR glasses 100 during a medical interview. The sound collection unit 123 may record the collected sound information, or at least the voice information, in a storage unit.

[0035] The wearer's screen 124 is a display that covers at least a part of the field of view of the medical worker U1 who is wearing the AR glasses 100. The wearer's screen 124 also displays a display image (video) output by the display unit 133 superimposed on the field of view through which light from the external surrounding space passes.

[0036] The acquisition unit 131 acquires an iris image of the medical worker U1 from the wearer photographing unit 121, a face image of the patient U2 from the external photographing unit 122, and conversation voice data of the medical worker U1 and the patient U2 from the sound collection unit 123. Note that the acquisition unit 131 is not limited to conversation voice data, and may acquire voice data of the medical worker U1 speaking alone from the sound collection unit 123.

[0037] The transmission unit 132 may transmit an iris authentication request including the iris image of the medical worker U1 acquired by the acquisition unit 131 to the authentication system 300 via the network N. Then, the acquisition unit 131 acquires the iris authentication result from the authentication system 300 via the network N. If the iris authentication result indicates success, the acquisition unit 131 acquires the medical worker ID (and attribute information) included in the iris authentication result.

[0038] The transmitting unit 132 may also transmit a face authentication request including the face image of the patient U2 acquired by the acquiring unit 131 to the medical record assistance server 200 via the network N. Then, the acquiring unit 131 acquires past medical record information of the patient U2 acquired from the medical information system 400 based on the face authentication result by the authentication system 300 in the medical record assistance server 200.

[0039] Furthermore, the transmission unit 132 transmits the conversational voice data acquired by the acquisition unit 131 to the medical record assistance server 200 via the network N. Then, the acquisition unit 131 acquires text information of the speech content converted from the conversational voice data between the medical worker U1 and the patient U2 during the medical interview using speaker-distinguishing voice recognition. Therefore, the acquisition unit 131 can be said to be an example of the above-mentioned first acquisition unit 11. In particular, the acquisition unit 131 may acquire text information converted in the medical record assistance server 200 based on the conversational voice data. Furthermore, the acquisition unit 131 may acquire text information converted in the medical record assistance server 200 based on speech data uttered by the medical worker U1 before and after the medical interview.

[0040] Furthermore, the acquisition unit 131 acquires, from the medical record assistance server 200, medical record information that has been output by inputting an instruction sentence for outputting text information of utterance content in the form of a medical record into a large-scale language model, which will be described later, in the medical record assistance server 200. Therefore, the acquisition unit 131 can be said to be an example of the above-mentioned second acquisition unit 13. For these reasons, if the AR glasses 100 have the function of a generation unit 224 in the medical record assistance server 200, which will be described later, they can be an example of the above-mentioned medical record assistance device 1.

[0041] The display unit 133 may display the past medical record information of the patient U2 acquired by the acquisition unit 131 on the wearer screen 124. The display unit 133 may also display text information of the utterance content acquired by the acquisition unit 131 on the wearer screen 124. The display unit 133 may also display the medical record information of the patient U2 acquired by the acquisition unit 131 on the wearer screen 124. The display unit 133 may also display the medical record information of the patient U2 acquired by the acquisition unit 131 and generated based on the utterance voice data of the medical interview on the wearer screen 124.

[0042] 5 is a block diagram showing the hardware configuration of the AR glasses 100. The AR glasses 100 include a memory 101, a processor 102, a network interface 103, a display 104, an internal camera 105, an external camera 106, and a microphone 107.

[0043] The memory 101 is configured by a combination of volatile memory and non-volatile memory. The volatile memory is, for example, a volatile storage device such as RAM (Random Access Memory), and is a storage area for temporarily holding information when the processor 102 is operating. The non-volatile memory is, for example, a non-volatile storage device such as flash memory. The memory 101 stores at least a computer program that implements part of the processing of the medical record support method for the AR glasses 100 according to the present disclosure.

[0044] The processor 102 is a control device that controls each component of the AR glasses 100. The processor 102 reads and executes software (computer programs) from the memory 101. As a result, the processor 102 realizes the functions of the acquisition unit 131, the transmission unit 132, and the display unit 133. That is, the processor 102 performs part of the processing of the medical record support method according to the present disclosure. The processor 102 may be, for example, a microprocessor, an MPU (Multi Processing Unit), or a CPU (Central Processing Unit). The processor 102 may also include multiple processors.

[0045] The network interface 103 may be used to communicate with a network node. The network interface 103 may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers. The network interface 103 may also include a wireless local area network (LAN), a wired LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0046] The display 104 is a display device that displays information instructed by the processor 102. The display 104 is, for example, a screen such as a liquid crystal display or an organic electro-luminescence (EL) display. In particular, the display 104 transmits external light and corresponds to the above-mentioned wearer screen 124.

[0047] In response to an instruction from the processor 102, the internal camera 105 captures an image of the area including the iris of the eye of the wearer of the AR glasses 100, and outputs the captured iris image to the processor 102. The internal camera 105 corresponds to the wearer capturing unit 121 described above.

[0048] The external camera 106 captures an image of the external surrounding space including the front direction of the AR glasses 100 in response to an instruction from the processor 102, and outputs the captured image to the processor 102. The external camera 106 is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The external camera 106 is one or more image capturing devices. The external camera 106 corresponds to the external image capturing unit 122 described above.

[0049] The microphone 107 is a sound collector that collects sounds around the AR glasses 100.

[0050] 6 is a block diagram showing the configuration of the medical record assistance server 200. The medical record assistance server 200 is an information processing device that supports the creation of medical record information for a patient U2 interviewed by a medical worker U1, using an authentication system 300, a medical information system 400, etc., in accordance with information received from the AR glasses 100. The medical record assistance server 200 is an example of the above-mentioned medical record assistance device 1. The medical record assistance server 200 includes a storage unit 210, an acquisition unit 221, an authentication control unit 222, a voice recognition unit 223, a generation unit 224, and a display control unit 225.

[0051] The storage unit 210 includes, for example, a non-volatile storage device such as a hard disk or flash memory, and a memory such as RAM, i.e., a volatile storage device. The storage unit 210 stores a program 211 and large language models (LLMs (Large Language Models)) 212. The program 211 is a computer program that implements at least a portion of the medical record support processing according to the present disclosure. The LLM 212 is an example of the above-mentioned language model. The LLM 212 is a trained model that is trained by repeatedly performing deep learning on a natural language model using a large amount of data set. The LLM 212 is a natural language model in which the number of deep learning training sessions, the number of data sets used for training, and the number of trained parameters are larger than those at the beginning of the spread of AI models. The LLM 212 receives as input instruction sentences written in a specific format and including text information to be processed, performs calculations on the text information included in the instruction sentences in accordance with the instruction sentences, and outputs text data as the calculation results. Specifically, the LLM 212 generates and outputs text information of medical record information by converting, processing, summarizing, etc. the text information contained in the instruction sentence to match the format of the medical record specified in the instruction sentence.

[0052] The acquisition unit 221 is an example of the first acquisition unit 11 and the second acquisition unit 13 described above. The acquisition unit 221 acquires text information of the content of speech converted by speech recognition that distinguishes speakers from conversational speech data between the medical worker U1 and the patient U2 during the medical interview, which is collected by the AR glasses 100. Here, the AR glasses 100 are eyeglass-type wearable terminals worn by the medical worker U1 during the medical interview. The acquisition unit 221 also acquires text information converted from the recognition results of speech recognition that distinguishes speakers from the conversational speech data performed by a speech recognition unit 223 (described later). Here, the text information may include, in addition to the speech recognition results from the conversational speech data, text information converted by speech recognition of speech data uttered by the medical worker U2 before and after the medical interview as the content of the speech of the medical worker U2.

[0053] The acquisition unit 221 also acquires the medical record information of the patient U2 by inputting an instruction sentence generated by the generation unit 224 (described later) into the LLM 212. If the face authentication result of the patient U2 received by the authentication control unit 222 (described later) indicates success, the acquisition unit 221 transmits an acquisition request for the medical record information to the medical information system 400, including the patient ID included in the face authentication result. The acquisition unit 221 then receives the medical record information corresponding to the patient ID included in the acquisition request from the medical information system 400, thereby acquiring the medical record information of the patient ID. The acquisition unit 221 then transmits the acquired medical record information to the AR glasses 100.

[0054] The authentication control unit 222 controls the iris authentication of the medical worker U1 and the face authentication of the patient U2 using the authentication system 300. Specifically, the authentication control unit 222 transmits an iris authentication request received from the AR glasses 100 to the authentication system 300, receives the iris authentication result, and returns the iris authentication result to the AR glasses 100. The authentication control unit 222 also transmits a face authentication request received from the AR glasses 100 to the authentication system 300 and receives the face authentication result. The authentication control unit 222 may also return the received face authentication result to the AR glasses 100.

[0055] The voice recognition unit 223 distinguishes speakers and performs voice recognition based on at least one of the volume and sound collection direction of the conversational voice data collected by the AR glasses 100. For example, the voice recognition unit 223 may recognize the speaker of the voice data with a relatively loud volume as the medical worker U1 and the speaker of the voice data with a relatively quiet volume as the patient U2. This is because the medical worker U1 is wearing the AR glasses 100 and is therefore closer to the sound collection unit 123 than the patient U2, and therefore the voice data spoken by the medical worker U1 may be collected at a relatively loud volume. Furthermore, the voice recognition unit 223 is configured to have information about the directionality of the sound collection unit 123 set in advance. In this case, the voice recognition unit 223 may recognize the speaker of the voice data whose sound collection direction is directly below the sound collection unit 123 as the medical worker U1 and the speaker of the voice data whose sound collection direction is in another direction as the patient U2.

[0056] The generation unit 224 is an example of the generation unit 12 described above. The generation unit 224 generates instruction sentences for the LLM 212 to output the text information acquired by the acquisition unit 221 in the form of a medical record. Here, the format of the medical record may be, but is not limited to, the SOAP (Subject Object Assessment Plan) format. The SOAP format is a format in which medical details are recorded by classifying them into Subject (subjective information), Object (objective information), Assessment, and Plan. The Subject (subjective information) is information obtained from what the patient U2 said during the medical interview. The Subject (subjective information) may also include information obtained from the patient U2's family or other staff members of the medical institution. The Object (objective information) is objective information obtained through the medical interview, examination, tests, etc. The Assessment is comprehensive evaluation information derived by the medical professional U1 through analysis, interpretation, and consideration based on the Subject (subjective information) and Object (objective information). The plan is information such as the future treatment policy and content for patient U2 that was decided by medical professional U1 based on the assessment. The treatment policy and content include, for example, follow-up observation, reports from the doctor at the time of examination, prescription content of medication, prescription conditions, prescription timing, etc.

[0057] Specifically, the generation unit 224 may generate a prompt to the LLM 212 as an instruction sentence. The prompt includes the instruction content to the LLM 212, the specification of the format of the medical record as a constraint, the conversation content as text information to be processed, the utterance content of the medical worker U1 alone, etc. The prompt may be a sentence written in, for example, a JSON (JavaScript Object Notation) format.

[0058] The display control unit 225 displays past medical record information (before the current medical interview) of the patient U2 photographed by the AR glasses 100 on the wearer screen 124. The display control unit 225 also displays text information acquired by the acquisition unit 221 on the wearer screen 124. The display control unit 225 also displays the medical record information of the patient U2 acquired by the acquisition unit 221 (output by the LLM 212 based on the instruction text from the current medical interview). Specifically, the display control unit 225 transmits a display request to the AR glasses 100 for each of the past medical record information, the text information, and the medical record information from the current medical interview, together with each piece of information.

[0059] It should be noted that the LLM 212 does not have to be stored in the storage unit 210. In that case, the LLM 212 may be executed in an LLM system connected to the medical record assistance server 200 via the network N. When the LLM system receives an instruction sentence from the medical record assistance server 200 via the network N, it may input the instruction sentence to generate text information of the medical record information and return the generated medical record information to the medical record assistance server 200.

[0060] 7 is a block diagram showing the hardware configuration of the medical record assistance server 200. The medical record assistance server 200 includes a memory 501, a processor 202, and a network interface 203.

[0061] The memory 201 stores at least a computer program (e.g., program 211) that implements at least a portion of the processing of the medical record support method of the medical record support server 200 according to the present disclosure. Furthermore, the memory 201 may store an LLM 212. Other configurations of the memory 201 are the same as those of the memory 101 described above. The processor 202 is a control device that controls each component of the medical record support server 200. The processor 202 reads and executes software (computer programs) from the memory 201. As a result, the processor 202 realizes the functions of an acquisition unit 221, an authentication control unit 222, a voice recognition unit 223, a generation unit 224, and a display control unit 225. In other words, the processor 202 performs at least a portion of the processing of the medical record support method of the medical record support server 200 according to the present disclosure. Other configurations of the processor 202 are the same as those of the processor 102 described above. The network interface 203 has the same configuration as that of the network interface 103 described above.

[0062] Next, a flow of generating medical record information by a medical worker U1 wearing the AR glasses 100 and conducting a medical interview with a patient U2 will be described. As a premise, the medical worker U1 wears the AR glasses 100 before starting the medical interview. At this time, the AR glasses 100 perform iris authentication using the authentication system 300 based on an iris image of the medical worker U1 captured by the wearer's image capturing unit 121. If the iris authentication is successful, the AR glasses 100 allow the wearer to use various functions. Below, a case will be described in which the medical worker U1 uses the medical record support processing function while wearing the AR glasses 100.

[0063] 8, 9, and 10 are sequence charts showing the flow of the medical record support process. First, the AR glasses 100 capture an image of the face of the patient U2 (S101). That is, the medical worker U1 turns his or her face toward the face of the patient U2. Therefore, the AR glasses 100 capture an image of an area including the face of the patient U2 using the external image capturing unit 122, and acquire a facial image of the patient U2. Then, the AR glasses 100 transmit a facial authentication request including the facial image of the patient U2 to the medical record support server 200 via the network N (S102). In response, the medical record support server 200 transmits the received facial authentication request to the authentication system 300 via the network N (S103).

[0064] In response, the authentication system 300 performs face authentication of the patient U2 based on the received face authentication request (S104). Specifically, the authentication system 300 detects a face area from the face image included in the face authentication request, extracts multiple feature points representing the person's facial features (eyes, nose, mouth, etc.) from the detected face area, and calculates the distance between each feature point as a feature amount. The authentication system 300 then extracts a feature vector including a set of the multiple calculated feature amounts as (query) face feature information. Note that the face authentication request may include face feature information extracted by the AR glasses 100 or the medical record assistance server 200 instead of a face image.

[0065] The authentication system 300 then compares the query facial feature information with multiple (master) facial feature information 32 stored in the authentication DB 3 and calculates a matching score indicating the degree of matching as the degree of match. For example, the authentication system 300 may calculate a higher matching score as the distance between the master and query feature vectors is shorter. The authentication system 300 then determines that facial authentication has been successful if the degree of match is equal to or greater than a threshold, and determines that facial authentication has failed if the degree of match is less than the threshold. The authentication system 300 includes the determination result of facial authentication in the facial authentication result. If facial authentication is successful, the authentication system 300 identifies the patient ID 31 associated with the facial feature information 32 whose facial authentication has been successful. The authentication system 300 then includes the identified patient ID 31 in the facial authentication result. The authentication system 300 then returns the facial authentication result to the medical record assistance server 200 via the network N (S105). For ease of explanation, it is assumed below that patient U2 has been successfully authenticated. Therefore, the face recognition result includes a notification that face recognition was successful and the patient ID.

[0066] The medical record assistance server 200 then receives the face authentication result from the authentication system 300 via the network N. The medical record assistance server 200 confirms from the received face authentication result that the face authentication of patient U2 was successful, and acquires the patient ID included in the face authentication result. The medical record assistance server 200 then transmits a request to acquire the most recent medical record information, including the acquired patient ID, to the medical information system 400 via the network N (S106). Note that the "most recent medical record information" includes the most recent of the past medical record information associated with the patient ID. Therefore, the medical record assistance server 200 may transmit an acquisition request for up to a predetermined number of the most recent of the past medical record information as the "most recent medical record information."

[0067] In response to this, the medical information system 400 searches the medical record DB4 using the patient ID included in the received acquisition request as a search key, and returns the most recent medical record information from the medical record information 41 etc. found in the search to the medical record assistance server 200 (S107). Then, the medical record assistance server 200 transmits the received most recent medical record information to the AR glasses 100 via the network N (S108). In response to this, the AR glasses 100 receive the most recent medical record information of the patient U2 from the medical record assistance server 200 via the network N, and display the received most recent medical record information on the wearer's screen 124 (S109).

[0068] FIG. 11 is a diagram illustrating a display example of the wearer's screen 124 of the AR glasses 100 at the start of the medical record assistance process and the concept of the start operation. As described above, first, the medical worker U1 looks at the face of the patient U2 through the AR glasses 100 that he is wearing. Therefore, the external imaging unit 122 of the AR glasses 100 captures an image of the face of the patient U2 (S101). In response to this, the AR glasses 100 display a medical record screen 51 on the wearer's screen 124. The medical record screen 51 includes a display of medical record information 61. The medical record information 61 is the most recent information about the patient U2. In this example, the medical record information 61 is information generated based on the most recent medical interview conducted with the patient U2 by nurse YYY (not shown), who is different from the medical worker U1 (nurse ZZZ). Note that, in order to generate the medical record information 61, nurse YYY may conduct a medical interview with the patient U2 while wearing the AR glasses 100.

[0069] Here, before starting to interview the patient U2, the medical worker U1 visually checks the medical record information 61 through the AR glasses 100 and then performs an operation to start the medical record. In response, the AR glasses 100 detect and accept the operation to start the medical record by the medical worker U1 (S110 in FIG. 8). Examples of the operation to start the medical record include, but are not limited to, the following:

[0070] (Start of medical record) (1: Pressing Operation) First, the medical worker U1 performs an action of pressing the medical record information 61 on the medical record screen 51 with his / her finger (S110-1). In this case, the AR glasses 100 detect the finger of the medical worker U1 from the image captured by the external image capturing unit 122. Then, the display unit 133 of the AR glasses 100 displays a finger beam 62 at a position on the medical record screen 51 that corresponds to the position of the detected finger. Then, when the AR glasses 100 detect that the finger beam 62 has been pressed at a position where it overlaps with the medical record information 61, it detects that the medical worker U1 has pressed the medical record information 61 with his / her finger.

[0071] (2: Voice command) Next, the medical worker U1 utters, for example, "Start medical record" (S110-2). In this case, the AR glasses 100 detects the utterance of "Start medical record" by performing voice recognition on the speech data of the medical worker U1 collected by the sound collection unit 123.

[0072] When the AR glasses 100 detects step S110-2 after step S110-1, the AR glasses 100 detect that the medical record start operation A has been performed, and the state transitions to a medical record update mode.

[0073] The start action of the other medical record may be, for example, only the pressing operation of the AR glasses 100 in step S110-1 or only the detection of the voice command in step S110-2 by the AR glasses 100. Alternatively, the start action of the other medical record may be, for example, only the AR glasses 100 capturing an image of a specific information code posted near the treatment location of the patient U2.

[0074] After step S110 in FIG. 8, the AR glasses 100 display an update mode on the wearer's screen 124 and switch the sound collection unit 123 to ON for collecting sound from medical records (S111).

[0075] FIG. 12 is a diagram showing an example of the display on the wearer's screen 124 of the AR glasses 100 immediately after the start of the medical record assistance process (update mode). When the AR glasses 100 transition to the update mode, they display a medical record screen 52 on the wearer's screen 124. The medical record screen 52 displays an indication that the glasses are in the update mode so that the wearer can visually recognize the update mode. For example, the medical record screen 52 may display the notations "<Update Mode>" and "Recorded by: Nurse ZZZ" at the top, a microphone mark 63 in the center, a voice recognition result display field 64 on the right, and the most recent medical record information 61 on the left. Furthermore, during the update mode, the medical record screen 52 may have a different background color from the medical record screen 51. This makes it easier for the medical worker U1 to visually recognize through the wearer's screen 124 of the AR glasses 100 that the medical record update mode is currently in effect. Furthermore, the medical worker U1 can visually confirm that the conversation content is being added to the voice recognition result display field 64 as needed while looking at the most recent medical record information 61, etc. This allows the medical worker U1 to conduct medical interviews efficiently. Note that the display content and layout of the update mode on the medical record screen 52 are merely examples. The medical record information 61 is the same as the content displayed on the medical record screen 51 in FIG. 11 described above. Meanwhile, the voice recognition result display field 64 currently displays only the start date of voice recognition, and the text information of the voice recognition result is blank.

[0076] Next, the medical worker U1 begins to interview the patient U2. In FIG. 9, first, the AR glasses 100 collect voice (from either the medical worker U1 or the patient U2) using the sound collection unit 123 (S121a) and transmit the collected voice data to the medical record assistance server 200 (S122a). In response, the medical record assistance server 200 performs voice recognition and speaker recognition on the received voice data (S123a). Note that publicly known techniques can be used for the voice recognition process. Furthermore, as described above, the voice recognition unit 223 of the medical record assistance server 200 may perform speaker recognition by recognizing speakers based on at least one of the volume and the direction of sound collection. However, the speaker recognition process does not need to identify individuals; it is sufficient to identify which of multiple speakers the speaker is. Next, the medical record assistance server 200 acquires text information converted by voice recognition (and the distinction between the recognized speakers) (S124a). Then, the medical record assistance server 200 transmits the acquired text information to the AR glasses 100 (S125a). Then, the AR glasses 100 add the received text information as a voice recognition result to the voice recognition result display field 64 and display it (S126a). The processing flow from steps S121a to S126a will be referred to as step S12a.

[0077] Thereafter, speech by the medical worker U1 and speech by the patient U2 are repeated, and similarly to step S12a, the process is performed up to step S12m (m is a natural number equal to or greater than 2). For example, the processes from steps S121m to S126m are equivalent to the processes from steps S121a to S126a, and therefore detailed description thereof will be omitted.

[0078] Thereafter, when the medical worker U1 determines that the medical interview has been sufficiently conducted, the medical worker U1 performs an operation to end the medical record. In response to this, the AR glasses 100 detect and accept the operation to end the medical record by the medical worker U1 (S112). Note that the operation to end the medical record may be an operation equivalent to the start operation described above. Then, the AR glasses 100 end the update mode display on the wearer screen 124 and switch the sound collection unit 123 to medical record sound collection OFF (S111).

[0079] FIG. 13 is a diagram illustrating the concept of the medical interview by medical worker U1 to patient U2 and the operation to end the medical record support process. FIG. 13 illustrates the following medical interview as an example. For example, medical worker U1 asks patient U2, "Mr. XXX, has your stomach pain gone away?" (S121-1). In response, patient U2 replies, "It hurts so much I can't sleep." (S121-2). The AR glasses 100 pick up the conversational voice during the medical interview from step S121-1 onwards, and continue the processing of steps S12a to S12m in FIG. 9 described above. Thereafter, medical worker U1 utters a voice command to end the medical record, saying, "Stop medical record" (S112).

[0080] 14 is a diagram showing an example of the display of the wearer's screen 124 of the AR glasses 100 during voice recognition in the medical record support process (update mode). The medical record screen 53 is an example in which the voice recognition result display field 64 on the medical record screen 52 has been updated to a voice recognition result display field 65. That is, during voice recognition in the update mode, the AR glasses 100 associates the voice-recognized text information, the identified speaker, the sound pickup time, etc., and adds them to the voice recognition result display field 65 for display.

[0081] The voice recognition result at the end of the voice recognition result display field 65 (the text data following "October 26, 20XX, 01:10 Speaker1:") is an example of what the medical professional U1 summarized for the medical record after confirming that there was no response from the patient U2. In other words, the AR glasses 100 not only recognize the conversation (exchange of words) between the medical professional U1 and the patient U2 during the medical interview, but also recognize what the medical professional U1 has spoken alone (not what is spoken to the patient U2). In addition, if the AR glasses 100 are in update mode, they may recognize what the medical professional U1 has spoken alone and add it to the medical record information, even before the medical interview with the patient U2 begins. This allows the medical professional U1 to add his or her voice memo to the medical record, improving the accuracy of the medical record information and reducing the effort required for corrections.

[0082] 9, the AR glasses 100 transmit a request to generate medical record information to the medical record assistance server 200 via the network N (S201). Note that the AR glasses 100 may transmit a notification of the end of the update mode to the medical record assistance server 200 instead of the generation request.

[0083] In response to receiving a generation request or the like, the medical record assistance server 200 generates a prompt for the LLM 212 using the text information group converted during update mode (S202). FIG. 15 is a diagram showing an example of a prompt 7 that has been converted into text and generated based on the speech recognition result. The prompt 7 is an example of an instruction sentence generated according to the input format of the LLM 212. The prompt 7 includes an instruction statement 71, a constraint 72, and speech-recognition text information 73. The instruction statement 71 is text data expressed as a sentence that causes the LLM 212 to convert or summarize the speech-recognition text information 73 into medical record information based on the constraint 72. The constraint 72 is an example in which the SOAP format is specified as the format of the medical record. The speech-recognition text information 73 is text information displayed in the speech recognition result display field 64 of FIG. 14.

[0084] 10, the medical record assistance server 200 inputs the generated prompt to the LLM 212 and acquires the medical record information output from the LLM 212 (S203). That is, the LLM 212 uses a group of parameters that have undergone deep learning to convert and summarize the speech-recognized text information 73 into a SOAP format based on the instruction sentence 71 and constraints 72 of the input prompt 7, thereby generating and outputting text information.

[0085] Then, the medical record assistance server 200 transmits the medical record information acquired from the LLM 212 to the AR glasses 100 via the network N (S204). In response to this, the AR glasses 100 display the medical record information acquired from the medical record assistance server 200 via the network N on the wearer's screen 124 (S205).

[0086] 16 is a diagram showing an example of the display of the latest medical record output by the LLM 212 from the generated prompt 7. The medical record screen 54 includes medical record information 66 and part of medical record information 61. The medical record information 66 is the latest medical record information for the current medical interview conducted by medical worker U1 (nurse ZZZ) for patient U2. The medical record information 61 is the most recent medical record information conducted by another nurse YYY prior to the current interview.

[0087] 10, together with step S204, the medical record assistance server 200 transmits a registration request for medical record information to the medical information system 400 via the network N (S206). At this time, the registration request includes the patient ID of patient U2 received in step S105 and the medical record information acquired in step S203. In response to this, the medical information system 400 associates the received medical record information with the patient ID and registers it in the medical record DB4 (S207).

[0088] In the above description, facial authentication of the patient U2 is performed to display the most recent medical record information on the AR glasses 100, but the method for displaying the most recent medical record information is not limited to this. The method for displaying past medical record information of a specific patient may use the functions of existing medical AR glasses or the functions of a medical institution system.

[0089] The medical record assistance server 200 may also have the functionality of the medical information system 400. In this case, the storage unit 210 of the medical record assistance server 200 further stores a medical record DB4. The medical record assistance server 200 may also cause an external system to perform speech recognition processing using the speech recognition unit 223, and acquire the speech recognition results. Alternatively, the AR glasses 100 may perform the speech recognition processing. For example, the AR glasses 100 may have the functionality of the speech recognition unit 223, and may perform speech recognition by distinguishing speakers based on at least one of the volume and sound collection direction of conversational speech data collected by the sound collection unit 123. In this case, communication between the AR glasses 100 and the medical record assistance server 200 in steps S12a to S12m of FIG. 9 is not necessary, and the communication load can be reduced.

[0090] Here, we will explain again the problems that the medical record support system and the like disclosed herein aim to solve. First, in medical settings, it is common for one medical professional, particularly a nurse, to be responsible for a large number of patients, some of whom are shared with other medical professionals. Because of this, medical professionals are often busy, and the creation of medical records, such as nursing records, is often postponed and completed all at once at the end of their working hours. This places a heavy physical and mental burden on medical professionals, and a large amount of time is spent on the task of creating medical records. This makes it difficult for medical professionals, particularly nurses, to focus on their primary duties, such as treating patients and taking medical interviews. Furthermore, when nurses and others create nursing records and the like while relying on memory after completing all medical interviews and the like, ensuring the accuracy of the medical records places a significant burden on them. In other words, there is a problem that the workload of medical professionals tends to increase because they need to create medical records accurately and quickly.

[0091] Therefore, by using the medical record support system and the like according to the present disclosure, medical personnel such as nurses can quickly record medical record information by utilizing voice input into the eyeglass-type wearable device they are wearing. In particular, since medical personnel U1 wears the AR glasses 100 to interview patient U2, there is no need to handwrite the medical record or input information into an information terminal during the interview. In other words, medical personnel U1 can interview patient U2 with both hands free. Therefore, medical personnel U1 can continue interviewing patient U2 while performing some other task for patient U2, and medical records can be created efficiently.

[0092] This reduces the workload of nurses and other medical professionals, making it easier for them to focus on patient care. This leads to a workplace where medical professionals can work enthusiastically, improving the quality of medical care and patient satisfaction. Furthermore, it can also support managers of medical institutions who want to care for nurses and other medical professionals.

[0093] In addition, medical records are generated and displayed immediately after the medical interview, making it easy for medical professionals to check and make minor corrections. This ensures the accuracy and authenticity of the medical record information, and significantly reduces the time it takes to create medical records.

[0094] Furthermore, after the medical worker U1 puts on the AR glasses 100, the AR glasses 100 transition to the update mode in response to a start operation by the medical worker U1. Therefore, the AR glasses 100 can prevent the speech of the medical worker U1 or other people from being mistakenly picked up, recognized, or converted into text before and after the update mode. This eliminates the need for the medical worker U1 to unintentionally cancel the medical record support process. This further reduces the burden on the medical worker U1 in creating medical records.

[0095] <Embodiment 3> The following describes a case where the glasses-type wearable terminal independently executes the processing of the medical record support method according to the present disclosure without using a medical record support server. FIG. 17 is a block diagram showing the configuration of AR glasses 100a. The AR glasses 100a are glasses-type wearable terminals worn by medical professionals during medical interviews. Compared to the AR glasses 100 shown in FIG. 4 above, the AR glasses 100a have the program 111, LLM 112, authentication control unit 134, voice recognition unit 135, and generation unit 136 added, and the acquisition unit 131a and display unit 133a changed. Therefore, the following description will omit the configuration common to the AR glasses 100 described above. The hardware configuration of the AR glasses 100a is the same as that shown in FIG. 5 above.

[0096] The storage unit 110 corresponds to the memory 101 in FIG. 5 described above. The program 111 is a modified version of the computer program stored in the memory 101 described above. The program 111 is a computer program that implements the processing of the medical record support method according to the present disclosure. The LLM 112 is a large-scale language model equivalent to the LLM 212 in FIG. 6 described above. The LLM 112 does not have to be stored in the storage unit 110. In that case, the LLM 112 may be executed in an LLM system connected to the AR glasses 100a via a network N. When the LLM system receives an instruction sentence from the AR glasses 100a via the network N, it may input the instruction sentence to generate text information of the medical record information and return the generated medical record information to the AR glasses 100a.

[0097] The acquisition unit 131a is an example of the above-described first acquisition unit 11 and second acquisition unit 13. The acquisition unit 131a acquires text information converted based on conversational voice data collected by the sound collection unit 123. In addition to the functions of the acquisition unit 131 described above, the acquisition unit 131a also acquires authentication results and medical record information by communicating with the authentication system 300 and the medical information system 400, like the acquisition unit 221.

[0098] The display unit 133a displays the medical record information acquired by the acquisition unit 131a on the wearer screen 124. The display unit 133a may further display text information acquired by the acquisition unit 131a on the wearer screen 124. The display unit 133a may further display past medical record information of the patient photographed by the external photographing unit 122 on the wearer screen 124.

[0099] The authentication control unit 134 is the same as the above-described authentication control unit 222. The voice recognition unit 135 distinguishes speakers and performs voice recognition based on at least one of the volume and the voice collection direction of conversation voice data collected by the sound collection unit 123. Therefore, the acquisition unit 131a acquires text information converted from the recognition result obtained by the voice recognition unit 135, which distinguishes speakers. The generation unit 136 is the same as the above-described generation unit 224.

[0100] Furthermore, the flow of the medical record assistance process using the AR glasses 100a can be realized by appropriately modifying the above-described second embodiment. Specifically, the communication between the medical record assistance server 200, the authentication system 300, and the medical information system 400 in FIG. 6 is replaced with communication between the AR glasses 100a, the authentication system 300, and the medical information system 400. Furthermore, the communication between the AR glasses 100 and the medical record assistance server 200 in FIG. 9 is omitted, and the AR glasses 100a executes the processes of steps S123a and S124a. Furthermore, the communication between the AR glasses 100 and the medical record assistance server 200 in FIG. 10 is omitted, and the AR glasses 100a executes the processes of steps S202, S203, and S206.

[0101] In this way, the third embodiment can achieve the same effects as the second embodiment described above.

[0102] <Other embodiments> The medical record assistance server 200 may incorporate some or all of the configurations and functions of the authentication system 300 and the medical information system 400 .

[0103] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0104] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0105] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix A1) a first acquiring means for acquiring text information of the content of the conversation between the medical staff and the patient during the medical interview, the text information being converted by speech recognition that distinguishes the speaker; a generation means for generating, from the text information, instruction sentences for a language model to be output in the form of a medical record; a second acquiring means for acquiring medical record information of the patient by inputting the instruction sentence into the language model; A medical record support device comprising: (Appendix A2) the medical record support device is a glasses-type wearable terminal worn by the medical worker at the time of the medical interview, the first acquisition means acquires the converted text information based on the conversational voice data collected by the glasses-type wearable device; The eyeglass-type wearable terminal further includes a display unit that displays the medical record information acquired by the second acquisition unit on a screen for the wearer of the eyeglass-type wearable terminal. A medical record support device as described in Appendix A1. (Appendix A3) The display means The text information acquired by the first acquisition means is further displayed on the wearer's screen. A medical record support device as described in Appendix A2. (Appendix A4) The display means past medical record information of the patient photographed by the eyeglass-type wearable terminal is further displayed on the screen for the wearer. A medical record support device as described in Appendix A2 or A3. (Appendix A5) a voice recognition unit that distinguishes speakers and performs voice recognition based on at least one of the volume and the sound collection direction of the conversational voice data collected by the glasses-type wearable device; The first acquisition means acquires the text information converted from the recognition result of speech recognition performed by the speech recognition means while distinguishing the speaker. A medical record support device according to any one of appendices A2 to A4. (Appendix A6) the first acquisition means acquires the converted text information based on the conversational voice data collected by a glasses-type wearable terminal worn by the medical professional during the medical interview; The device further includes a display control unit for displaying the medical record information on a screen for the wearer of the eyeglass-type wearable terminal. A medical record support device as described in Appendix A1. (Appendix A7) The display control means The text information acquired by the first acquisition means is further displayed on the wearer's screen. A medical record support device as described in Appendix A6. (Appendix A8) The display control means past medical record information of the patient photographed by the eyeglass-type wearable terminal is further displayed on the screen for the wearer. A medical record support device as described in Appendix A6 or A7. (Appendix A9) a voice recognition unit that distinguishes speakers and performs voice recognition based on at least one of the volume and the sound collection direction of the conversational voice data collected by the glasses-type wearable device; The first acquisition means acquires the text information converted from the recognition result of speech recognition performed by the speech recognition means while distinguishing the speaker. A medical record support device according to any one of appendices A6 to A8. (Appendix A10) The text information includes text information converted from speech data of the medical staff before and after the medical interview by speech recognition as the content of the speech of the medical staff. A medical record support device according to any one of appendices A1 to A9. (Appendix A11) the medical professional is a nurse; The medical record information includes nursing record information. A medical record support device according to any one of appendices A1 to A10. (Appendix A12) The medical records are in the SOAP (Subject Object Assessment Plan) format. A medical record support device according to any one of appendices A1 to A11. (Appendix B1) A glasses-type wearable device worn by medical staff during medical interviews, an information processing device communicably connected to the glasses-type wearable terminal, The information processing device includes: a first acquisition means for acquiring text information of the content of the conversation between the medical professional and the patient during the medical interview, the text information being converted by speech recognition that distinguishes between speakers, the first acquisition means being configured to acquire text information of the content of the conversation between the medical professional and the patient during the medical interview, the text information being acquired by the first acquisition means; a generation means for generating, from the text information, instruction sentences for a language model to be output in the form of a medical record; a second acquiring means for acquiring medical record information of the patient by inputting the instruction sentence into the language model; Equipped with The glasses-type wearable terminal displays the medical record information acquired by the second acquisition means on a screen for the wearer. Medical record support system. (Appendix C1) The computer The text information of the conversation content is obtained by converting the conversation voice data between the medical professional and the patient during the medical interview using speech recognition that distinguishes between the speakers, and generating instructions for a language model to output the text information in the form of a medical record; obtaining medical record information of the patient by inputting the instruction sentence into the language model; Medical record support methods. (Appendix D1) a first acquisition process for acquiring text information of the speech content converted from conversational voice data between a medical professional and a patient during a medical interview by speech recognition that distinguishes between speakers; a generation process for generating instruction sentences for a language model to output from the text information in the form of a medical record; a second acquisition process for acquiring medical record information of the patient by inputting the instruction sentence into the language model; A medical record support program that allows a computer to execute the above.

[0106] Some or all of the elements (e.g., configurations and functions) described in Appendix A2 to Appendix A12 that are dependent on Appendix A1 {e.g., device} may also be dependent on Appendix B1 {e.g., system}, Appendix C1 {e.g., method}, and Appendix D1 {e.g., program} in the same dependency relationship as Appendix A2 to Appendix A12. Some or all of the elements described in any appendix may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0107] 1 Medical record support device 11 First Acquisition Section 12 Generation part 13 Second Acquisition Section 1000 Medical Record Support System U1 Healthcare workers U2 patient N Network 100 AR Glasses 100a AR Glasses 200 Medical Record Support Server 300 Authentication System 3 Authentication DB 31 Patient ID 32 Facial feature information 400 Medical Information Systems 4 Medical record database 40 Patient ID 41 Medical Record Information 4n Medical record information 110 Storage section 111 Program 112 LLM 121 Wearer Photography Department 122 External Photography Department 123 Sound pickup section 124 Wearer's Screen 131 Acquisition Department 131a Acquisition Department 132 Transmitter 133 Display section 133a Display section 134 Authentication control section 135 Voice Recognition Unit 136 Generation part 101 Memory 102 processors 103 Network Interface 104 Display 105 Internal Camera 106 External Camera 107 Mike 210 Storage section 211 Program 212 LLM 221 Acquisition Department 222 Authentication control section 223 Voice Recognition Unit 224 Generation part 225 Display control unit 201 Memory 202 processors 203 Network Interface 51 Medical Record Screen 52 Medical Record Screen 53 Medical Record Screen 54 Medical Record Screen 61 Medical Record Information 62 Finger Beam 63 Mike Mark 64 Voice recognition result display field 65 Voice recognition result display field 66 Medical Record Information 7 Prompt 71 Instructions 72 Constraints 73 Speech Recognition Text Information

Claims

1. a first acquiring means for acquiring text information of the content of a conversation between a medical professional and a patient during a medical interview, the text information being converted by speaker-distinguishing speech recognition; a generation means for generating, from the text information, instruction sentences for a language model to be output in the form of a medical record; a second acquiring means for acquiring medical record information of the patient by inputting the instruction sentence into the language model; A medical record support device comprising:

2. the medical record support device is a glasses-type wearable terminal worn by the medical worker at the time of the medical interview, the first acquisition means acquires the converted text information based on the conversational voice data collected by the glasses-type wearable terminal; The eyeglass-type wearable terminal further includes a display unit that displays the medical record information acquired by the second acquisition unit on a screen for the wearer of the eyeglass-type wearable terminal. The medical record support device according to claim 1.

3. a voice recognition unit that distinguishes speakers and performs voice recognition based on at least one of the volume and the sound collection direction of the conversational voice data collected by the glasses-type wearable device; The first acquisition means acquires the text information converted from the recognition result obtained by the speech recognition means while distinguishing the speaker. The medical record support device according to claim 2.

4. the first acquisition means acquires the converted text information based on the conversational voice data collected by a glasses-type wearable terminal worn by the medical professional during the medical interview; The device further includes a display control unit for displaying the medical record information on a screen for the wearer of the eyeglass-type wearable terminal. The medical record support device according to claim 1.

5. a voice recognition unit that distinguishes speakers and performs voice recognition based on at least one of the volume and the sound collection direction of the conversational voice data collected by the glasses-type wearable device; The first acquisition means acquires the text information converted from the recognition result obtained by the speech recognition means while distinguishing the speaker. The medical record support device according to claim 4.

6. The text information includes text information converted from speech data of the medical staff before and after the medical interview by speech recognition as the content of the speech of the medical staff.

5. The medical record support device according to claim 1, 2 or 4.

7. the medical professional is a nurse; The medical record information includes nursing record information.

5. The medical record support device according to claim 1, 2 or 4.

8. A glasses-type wearable device worn by medical staff during medical interviews, an information processing device communicably connected to the glasses-type wearable terminal, The information processing device includes: a first acquisition means for acquiring text information of the content of the conversation between the medical professional and the patient during the medical interview, the text information being converted by speech recognition that distinguishes between speakers, the speech being collected by the eyeglass-type wearable terminal; a generation means for generating, from the text information, instruction sentences for a language model to be output in the form of a medical record; a second acquiring means for acquiring medical record information of the patient by inputting the instruction sentence into the language model; Equipped with The glasses-type wearable terminal displays the medical record information acquired by the second acquisition means on a screen for the wearer. Medical record support system.

9. The computer The text information of the conversation content is obtained by converting the conversation voice data between the medical professional and the patient during the medical interview using speech recognition that distinguishes between the speakers, and generating instructions for a language model to output the text information in the form of a medical record; obtaining medical record information of the patient by inputting the instruction sentence into the language model; Medical record support methods.

10. a first acquisition process for acquiring text information of the content of the conversation between the medical professional and the patient during the medical interview, the text information being converted by speaker-distinguishing speech recognition; a generation process for generating instruction sentences for a language model to output the text information in the form of a medical record; a second acquisition process for acquiring medical record information of the patient by inputting the instruction sentence into the language model; A medical record support program that allows a computer to execute the above.

Images