Information processing method, program, and information processing device

Abstract

To provide an information processing method, and the like, for providing information based on a rule of a medical treatment facility, to a user.SOLUTION: An information processing method causes a computer to execute processes of: acquiring a query in which voice of a person staying in a medical treatment facility is recorded, via a user interface 20; acquiring a response sentence according to the acquired query, using a language model 53 that uses medical treatment facility information including information and rules on the medical treatment facility; and outputting the acquired response sentence via the user interface 20.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing method, a program, and an information processing device. [Background technology]

[0002] BACKGROUND ART An information processing method has been proposed that provides various information to guests by utilizing a user interface installed in a guest room of an accommodation facility (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 244365 Summary of the Invention [Problem to be solved by the invention]

[0004] In medical facilities such as hospitals, various rules are established to ensure smooth treatment of patients, ensure safety, and prevent the leakage of personal information. If the information processing method of Patent Document 1 is introduced into a medical facility, the information provided will not take these rules into consideration, and there is a risk that information that does not comply with the rules of the medical facility will be provided to users.

[0005] In one aspect, an object is to provide an information processing method etc. that can provide information that complies with the rules of medical facilities to users including patients, their families and medical professionals. [Means for solving the problem]

[0006] The information processing method includes receiving, via a user interface, a query recorded with speech from a person staying at a medical facility; inputting the acquired query into a classification model that classifies whether the query is a first request for medical facility information or a second request for medical treatment at a medical facility, and acquiring a classification result; and if the classification result is the first request, Using a language model that uses medical facility information including information about the medical facility and rules, a response sentence corresponding to the acquired query is acquired, and the acquired response sentence is output via the user interface. If the classification result is a second request, output information about a nurse call.The processing is performed by a computer. [Effects of the Invention]

[0007] In one aspect, it is possible to provide an information processing method or the like that can provide a user with information that complies with the rules of a medical facility. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is an explanatory diagram illustrating an outline of the operation of the information processing system. [Figure 2] FIG. 1 is an explanatory diagram illustrating a configuration of an information processing system. [Figure 3] FIG. 10 is an explanatory diagram illustrating an example of medical facility information. [Figure 4] FIG. 10 is an explanatory diagram illustrating the record layout of a classification DB. [Figure 5] FIG. 10 is an explanatory diagram illustrating a procedure for classifying queries. [Figure 6] 10 is a flowchart illustrating the flow of processing of a program. [Figure 7] 10 is a flowchart illustrating the flow of processing of a program according to the second embodiment. [Figure 8] FIG. 10 is an explanatory diagram illustrating the record layout of a log DB. [Figure 9] 11 is a flowchart illustrating the flow of processing of a program according to the third embodiment. [Figure 10] 10 is a flowchart illustrating the flow of processing of a program according to a fourth embodiment. [Figure 11] This is an example of a report that uses the log database. [Figure 12] FIG. 13 is an explanatory diagram illustrating an example of a determination criterion according to the fifth embodiment. [Figure 13] 13 is a flowchart illustrating the flow of processing of a program according to the fifth embodiment. [Figure 14] FIG. 13 is an explanatory diagram illustrating the configuration of an information processing system according to a sixth embodiment. [Figure 15] FIG. 10 is an explanatory diagram illustrating an example of the configuration of a notification determination model. [Figure 16] 13 is a flowchart illustrating the flow of processing of a program according to the sixth embodiment. [Figure 17] FIG. 1 is an explanatory diagram illustrating an example of the configuration of a dementia risk assessment model. [Figure 18] 13 is a flowchart illustrating the flow of processing of a program according to the eighth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] [Embodiment 1] FIG. 1 is an explanatory diagram outlining the operation of an information processing system 40. The information processing method shown in FIG. 1 is used for services to users including patients hospitalized in a medical facility, the patients' families, and medical professionals. In the following explanation, an example will be given in which the patient is hospitalized in a private room in a medical facility. In other words, the patient is an example of a person staying in a medical facility.

[0010] The information processing system 40 includes an information processing device 10 and a user interface 20. The user interface 20 is an information device such as a smart speaker placed in a hospital room 45 (see FIG. 2). The information processing device 10 is a server, a cloud system, or the like connected to the user interface 20 via a network.

[0011] The patient vocalizes a question, a request, etc. The user interface 20 acquires the voice uttered by the user. The user interface 20 transmits a query generated based on the acquired voice to the information processing device 10.

[0012] The information processing device 10 inputs a prompt to instruct a classification model 52 (described later) to classify the query, and classifies the query into a first request and a second request. The first request is a request that does not require a medical professional such as a doctor or nurse to respond. The first request includes a request related to medical facility information 61. The medical facility information 61 includes information and rules related to medical facilities. A specific example of the medical facility information 61 will be described later using FIG. 3.

[0013] The second request is a request that needs to be handled by a medical professional such as a doctor or a nurse. The second request includes a request related to a medical procedure. In the following description, the processing stage of classifying a query into a first request and a second request is referred to as a request classification stage.

[0014] If the classification result is the first request, the medical facility information 61 and a prompt instructing the user to create a response sentence to answer the query are input to the language model 53. The language model 53 is, for example, a large language model (LLM) such as GPT-4 or Gemini. A response sentence to answer the query is output from the language model 53. The response sentence reflects the contents of the medical facility information 61. The response sentence is sent to the user interface 20. The response sentence is output by voice from the user interface 20.

[0015] If the classification result is the second request, a medical professional must respond. Information regarding a so-called nurse call, which calls a medical professional from the patient room 45, is transmitted to the user interface 20. A voice message such as "Please press the nurse call button" is output from the user interface 20.

[0016] When a patient presses a nurse call button 36 (see FIG. 2), a medical professional such as a nurse can respond appropriately to the patient.

[0017] 2 is an explanatory diagram illustrating the configuration of the information processing system 40. As described above, the information processing system 40 includes the information processing device 10 and the user interface 20. The information processing device 10 and the user interface 20 are connected via a network.

[0018] The user interface 20 is placed in the patient's room 45. Furthermore, each patient's room 45 is provided with a nurse call button 36 connected to a nurse call system 35. A nurse call remote control 37 that automatically operates the nurse call button 36 may also be placed in the patient's room 45. The nurse call system 35, nurse call button 36, and nurse call remote control 37 have been used for some time, so detailed explanations will be omitted.

[0019] The information processing device 10 includes a first control unit 11, a main memory device 12, an auxiliary memory device 13, a communication unit 14, a reading unit 19, and a bus. The first control unit 11 is an arithmetic and control device that executes a program 97 of this embodiment. The first control unit 11 uses one or more central processing units (CPUs), graphics processing units (GPUs), multi-core CPUs, etc. The first control unit 11 is connected to each hardware unit that constitutes the information processing device 10 via the bus.

[0020] The main memory device 12 is a memory device such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), a flash memory, etc. The main memory device 12 temporarily stores information required during processing performed by the first control unit 11 and a program 97 currently being executed by the first control unit 11.

[0021] The auxiliary storage device 13 is a storage device such as an SRAM, flash memory, hard disk, or magnetic tape. The auxiliary storage device 13 stores a classification model 52, a language model 53, a classification DB (Database) 54, medical facility information 61, a program 97 to be executed by the first control unit 11, and various data required for executing the program 97. The auxiliary storage device 13 is an example of a storage unit of a medical facility. The language model 53, the classification DB 54, and the medical facility information 61 may be stored in an external mass storage device connected to the information processing device 10.

[0022] The communication unit 14 is an interface for communicating with the network. The reading unit 19 is a device for reading the portable recording medium 96 on which the program 97 is recorded. The first control unit 11 reads the program 97 recorded on the portable recording medium 96, the semiconductor memory 98, or another server computer connected via the network, and stores it in the auxiliary storage device 13.

[0023] The information processing device 10 is a general-purpose personal computer, a tablet, a mainframe computer, or a virtual machine running on a mainframe computer. The information processing device 10 may be configured with hardware such as multiple personal computers or mainframe computers that perform distributed processing. The information processing device 10 may also be configured with a cloud computing system. The information processing device 10 may also be configured with hardware such as multiple personal computers or mainframe computers that operate in cooperation with each other.

[0024] The user interface 20 includes a second control unit 21, a main memory device 22, an auxiliary memory device 23, a communication unit 24, a touch panel 25, a microphone 26, a speaker 27, a camera 28, and a bus. The second control unit 21 is an arithmetic and control device that executes the program 97 of this embodiment. The second control unit 21 uses one or more CPUs, GPUs, multi-core CPUs, etc. The second control unit 21 is connected to each hardware unit that constitutes the user interface 20 via the bus.

[0025] The main memory device 22 is a memory device such as an SRAM, a DRAM, or a flash memory. The main memory device 22 temporarily stores information required during the processing performed by the second control unit 21 and a program 97 being executed by the second control unit 21.

[0026] The auxiliary storage device 23 is a storage device such as an SRAM, a flash memory, a hard disk, a magnetic tape, etc. The auxiliary storage device 23 stores a program 97 to be executed by the second control unit 21 and various data required for executing the program 97.

[0027] The communication unit 24 is an interface for communicating with a network. The user interface 20 is an information device such as a smart speaker, tablet, or smartphone. Although not shown, the hospital room 45 is equipped with a television, lighting, air conditioner, or the like that are controlled by radio waves or infrared rays emitted from the user interface 20.

[0028] The program 97 executed by the second control unit 21 is a part of the program 97 read by the first control unit 11 from the portable recording medium 96, and is stored in the auxiliary storage device 23 via the network. The second control unit 21 may read the program 97 recorded in another server computer connected via the network and store it in the auxiliary storage device 23.

[0029] The touch panel 25, microphone 26, speaker 27, and camera 28 are either built into the user interface 20 or attached externally.

[0030] Fig. 3 is an explanatory diagram illustrating an example of medical facility information 61. As shown in Fig. 3, the medical facility information 61 is composed of multiple documents such as "rules for admission," "facility guide," "in-room equipment instruction manual," and "menu for *month* day." Each document is recorded in any format, such as PDF (Portable Document Format) or XML (Extensible Markup Language).

[0031] FIG. 4 is an explanatory diagram illustrating the record layout of the classification DB 54. The classification DB 54 is a DB that records the content of a query in association with its classification as either a first request or a second request. The classification DB 54 has a query content field and a classification field. The query content is recorded in the query content field. In the following description, the list of items recorded in the query content field will be referred to as item list 541.

[0032] The classification field records whether the content of the query is a first request or a second request. The classification DB 54 has one record for the content of one query.

[0033] 5 is an explanatory diagram illustrating the procedure for classifying a query. The query is converted into a prompt such as "Please classify 'query' into one of the items in the list," and is input to the language model 53 together with the item list 541. Here, "query" means that the query is to be substituted. The language model 53 outputs the item into which the query has been classified.

[0034] Based on the classification DB 54, it is determined whether the output item is a first request or a second request. The determination result is output. In this way, the classification model 52 that classifies a query as a first request or a second request is realized. Note that the classification model 52 may be a machine learning model generated by machine learning using a large-scale language model such as GPT-4 or BERT.

[0035] The classification DB 54 may classify queries into three types: a first request, a second request, and a third request related to the control of devices such as a television, lighting, or air conditioner. For the third request, the first control unit 11 may control the device via a control line or the like (not shown).

[0036] The three-way classification is generated by machine learning using a large amount of data as training data, for example, that associates a query with one of the three labels from the first request to the third request.

[0037] 6 is a flowchart illustrating the processing flow of the program 97. The second control unit 21 acquires a voice uttered by the patient via the microphone 26 (step S501). The second control unit 21 generates a query based on the voice (step S502). The query is a sentence, a word, or the like.

[0038] The second control unit 21 may generate a query in cooperation with a voice recognition server and a language analysis server (not shown). The second control unit 21 may generate a query using an image captured by the camera 28 in addition to the voice.

[0039] The second control unit 21 transmits a query (step S503). The first control unit 11 receives the query (step S601). As described with reference to FIG. 5, the first control unit 11 generates a prompt based on the query. The first control unit 11 inputs the prompt and the item list 541 into the classification model 52 to obtain a classification result (step S602). The first control unit 11 determines whether the obtained classification result is a first request or a second request (step S603).

[0040] If it is determined to be the first request (first request in step S603), the first control unit 11 inputs a prompt instructing the creation of a response sentence to the query received in step S601 and information similar to the query extracted from the medical facility information 61 to the language model 53, as described using FIG. 1. Information similar to the query is extracted, for example, by searching the medical facility information 61 using keywords related to the query. The response sentence to the query is output from the language model 53. The first control unit 11 acquires the output response sentence (step S604). The first control unit 11 transmits the response sentence (step S605).

[0041] A specific example of a prompt will be described. For example, when a patient asks about meal times, the prompt generated by the first control unit 11 is, "You are a guide at ** Hospital. Please tell me what time meals start and what the menu is, based on the results of a meal search from medical facility information." The response sentence output from the language model 53 is, "Meals start at 11:30. The menu is (omitted)."

[0042] If it is determined that the request is the second request (second request in step S603), the first control unit 11 transmits a predetermined response message such as "Please press the nurse call button" (step S606).

[0043] The second control unit 21 receives the response sentence (step S504). The second control unit 21 outputs the response sentence by voice via the speaker 27 (step S505). The second control unit 21 may also output the response sentence by text via the touch panel 25. The touch panel 25 is an example of a display installed in the hospital room 45. Thereafter, the second control unit 21 ends the process.

[0044] According to this embodiment, it is possible to realize an information processing system 40 that automatically provides an appropriate response to a first request that does not require a medical professional's response, and that prompts a medical professional to be contacted for a second request that does require a medical professional's response. The information processing system 40 reduces the burden on medical professionals and contributes to creating an environment where they can concentrate on their primary medical duties.

[0045] This embodiment can provide an information processing system 40 that notifies patients when a nurse call is necessary. Even patients who tend to be reluctant to use the nurse call function will find it easier to press the nurse call button 36 when prompted by the information processing system 40.

[0046] [Embodiment 2] 7 is a flowchart illustrating the processing flow of the program 97 according to the second embodiment. This embodiment is used for patients who have difficulty operating the nurse call button 36 due to injury or paralysis of the upper limbs, for example.

[0047] The program 97 shown in Fig. 7 is the same as the program 97 described using Fig. 6, except for the case where it is determined in step S603 that the request is the second request. Descriptions of the processing common to Fig. 7 will be omitted.

[0048] If it is determined in step S603 that it is a second request (second request in step S603), the first control unit 11 sends an operation signal to the nurse call remote control 37 installed in the patient's hospital room 45, instructing it to operate the nurse call button 36 (step S616).

[0049] Nurse call remote control 37 receives the instruction (step S701). Nurse call remote control 37 operates nurse call button 36 (step S702). Specifically, nurse call remote control 37 is attached to nurse call button 36 and operates to mechanically press nurse call button 36. Nurse call remote control 37 may be a transmitter that transmits a call signal to nurse call system 35 instead of nurse call button 36.

[0050] As a result, it is possible to provide an information processing system 40 that makes a nurse call on behalf of a patient who has difficulty operating the nurse call button 36. In step S616, the first control unit 11 may instruct the user interface 20 to operate the nurse call button 36, and the second control unit 21 may control the nurse call remote control 37.

[0051] [Embodiment 3] This embodiment relates to an information processing system 40 that records the condition of a patient. Explanation of parts common to the first embodiment will be omitted.

[0052] 8 is an explanatory diagram illustrating the record layout of the log DB 57. The log DB 57 is a DB that records logs related to patients. The log DB 57 has an identification information field, a patient name field, a hospital room number field, a date and time field, and an event field.

[0053] The identification information field records identification information such as an ID uniquely assigned to a patient. The patient name field records the patient's name. The room number field records the room number. The date and time field records the date and time. The event field records an event detected by the user interface 20. The log DB 57 has one record for each event. The log DB 57 is recorded in the auxiliary storage device 13.

[0054] 9 is a flowchart illustrating the processing flow of program 97 according to embodiment 3. Second control unit 21 detects via speaker 27 or camera 28 that a person has entered hospital room 45 (step S511). Second control unit 21 starts transmitting in real time the sound detected by speaker 27 and the image captured by camera 28 (step S512).

[0055] The first control unit 11 starts receiving audio and images (step S621). Thereafter, the first control unit 11 continuously performs speech recognition and semantic analysis. The first control unit 11 may analyze the audio using a large-scale multimodal language model such as GPT-4 or BERT. The first control unit 11 determines who is entering or leaving the room, for example, by facial recognition or the like (step S622). The first control unit 11 creates a new record in the log DB 57 and records the person who has entered or left the room (step S623).

[0056] The first control unit 11 determines whether or not a registration instruction for medication information on medication status or treatment information on treatment details has been received from the medical worker (step S624). The registration instruction is made using a sentence including the position and name of the medical worker, the room number and name of the patient to be treated, and a keyword such as "start treatment", such as "This is nurse Sato. I will start treatment for Mr. Tanaka in room 301."

[0057] If it is determined that the registration instruction has been received (YES in step S624), the first control unit 11 creates a new record in the log DB 57. The first control unit 11 analyzes the voices of the medical professional and the patient after the registration instruction to generate medication information or treatment information. The first control unit 11 records the generated medication information, treatment information, etc. in the log DB 57 (step S625). The contents recorded in the log DB 57 may be reflected in an electronic medical record system (not shown).

[0058] Medication information or treatment information will be explained using specific examples. When administering a drug, the nurse will confirm the administration details by saying something like, "Administer ** milliliters of drug P." The first control unit 11 records the medication information, such as "Administer ** ml of drug P," as shown in the event field of the fourth record from the top in FIG. 8.

[0059] The nurse asks the patient about his condition. The patient replies, "I was able to sleep for about three hours after the infusion. I still feel tired, but the chills have gone away." Based on this statement from the patient, the first control unit 11 concisely records the event, such as "Symptoms: Slept for three hours after the infusion. Feels tired. Chills have gone away," as shown in the event field of the second record from the bottom in Figure 8.

[0060] After measuring the patient's vital signs, the nurse explains the current situation, saying, "Your body temperature has dropped to 36.9°C, your pulse rate is 84, and your blood pressure is 103 / 64." The first control unit 11 records the vital data as "body temperature 36.9°C, pulse rate 84, blood pressure 103 / 64," as shown in the event field of the bottom record in Fig. 8.

[0061] The first control unit 11 may input the audio and image data into the language model 53 along with a prompt such as "Please summarize the attached audio and images," and use the obtained summary results as medication information, treatment information, etc.

[0062] If it is determined that the registration instruction has not been received (NO in step S624), or after step S625 is completed, the first control unit 11 determines whether an instruction requesting confirmation of medication information, treatment details, etc. has been received from the medical staff (step S626). The instruction requesting confirmation may be an instruction requesting a history of medication information, treatment details, etc. for a predetermined period of time, such as one day or one week.

[0063] Instructions are given in the form of a sentence that includes the medical professional's position and name, the room number and name of the patient being treated, and keywords such as "please tell me the medication information," such as, "This is nurse Sato. Please tell me today's medication information for Mr. Tanaka in room 301."

[0064] If it is determined that an instruction requesting confirmation has been received (YES in step S626), the first control unit 11 generates a response (step S627). Specifically, the first control unit 11 searches the log DB 57 to extract the instructed information. The first control unit 11 inputs the instructed information and a query such as "Please prepare a report for the nurse" into the language model 53, and obtains a response sentence output from the language model 53.

[0065] For example, in response to the question about the medication information mentioned above, the language model 53 outputs a response such as "Mr. Tanaka in room 301 was administered ** milliliters of medication P at 9:45. Nurse A was in charge."

[0066] The first control unit 11 transmits the response sentence (step S628). The second control unit 21 receives the response sentence (step S513). The second control unit 21 outputs the response sentence by voice via the speaker 27 (step S514). The second control unit 21 may also display the response sentence in text on the screen of the touch panel 25.

[0067] If it is determined that an instruction for confirmation has not been received (NO in step S626), or after step S628 is completed, the first control unit 11 performs emotion analysis of the patient's voice using a language model 53 such as BERT (step S629). The first control unit 11 determines whether a change has occurred in the patient's emotion (step S630). If it is determined that there has been a change (YES in step S630), the first control unit 11 creates a new record in the log DB 57 and records the emotion change (step S631). The first control unit 11 may record the type of emotion, such as anger, surprise, joy, or sadness, in the log DB 57.

[0068] If it is determined that there is no change in the patient's emotions (NO in step S630), or after step S631 is completed, the first control unit 11 determines whether to end the process (step S632). Specifically, if the first control unit 11 no longer transmits audio and images from the user interface 20, the first control unit 11 determines to end the process. If it is determined not to end the process (NO in step S632), the first control unit 11 returns to step S622. If it is determined to end the process (YES in step S632), the first control unit 11 enters a standby state.

[0069] The second control unit 21 detects that a person other than the patient has left based on the data acquired from the camera 28 and the microphone 26 (step S515). The second control unit 21 ends the transmission of audio and images (step S516). The second control unit 21 ends the process.

[0070] According to this embodiment, it is possible to provide an information processing system 40 that allows a medical professional to record a log of medication, treatment, and the like in a patient room 45 without taking his / her hands off the device. According to this embodiment, it is possible to provide an information processing system 40 that automatically records emotional fluctuations of a patient that a medical professional is treating.

[0071] [Embodiment 4] This embodiment relates to a method for utilizing data recorded in the log DB 57 described in embodiment 3. Explanation of parts common to embodiment 3 will be omitted.

[0072] By analyzing the data recorded in the log DB 57, various information such as the compatibility between each medical staff member and patient can be obtained. Reports that visualize and summarize this information can be used to improve the operation of medical facilities.

[0073] 10 is a flowchart illustrating the processing flow of the program 97 according to the fourth embodiment. The following description will be given taking as an example a case where the first control unit 11 creates a report. Any hardware capable of acquiring the log DB 57 via a network may be used to create the report.

[0074] The first control unit 11 acquires data recorded in the log DB 57 (step S641). The first control unit 11 creates a graph in accordance with a predetermined format (step S642). A specific example of the graph will be described later. The first control unit 11 generates a comment related to the graph (step S644). A specific example of the comment will be described later. The first control unit 11 outputs a report including the graph and the comment (step S645). The first control unit 11 ends the process.

[0075] Fig. 11 is an example of a report that uses the log DB 57. The report shown in Fig. 11 is a report on the number of times a patient's emotions changed due to the nurse's response within a predetermined period, such as one week or one month. The report includes a graph column 71 and a comment column 72.

[0076] The horizontal axis of the graph column 71 indicates the combinations of four nurses and four patients. The vertical axis of the graph column 71 indicates the number of times the patient's emotions changed. The graph template is created by a user, such as a head nurse or an office manager, who loads the log DB 57 into general-purpose spreadsheet software and performs various processes. In step S642 of the flowchart described using FIG. 10, the first control unit 11 can create a graph by copying the data recorded in the log DB 57 into the template.

[0077] The first control unit 11 may generate a graph by inputting the log DB 57 and a predetermined prompt such as "Please create a graph counting the number of times the patient's emotional changes occurred for each nurse who treated them based on the log DB. The period is the most recent week" into the language model 53. The first control unit 11 may generate separate graphs for, for example, the number of times the patient's emotions changed to negative states such as anger and anxiety, and the number of times the patient's emotions changed to positive states such as relief and joy.

[0078] The first control unit 11 generates a comment to be displayed in the comment field 72 by inputting the graph created in step S642, the log DB 57, and a prompt such as "Please write a comment explaining this graph in the range of 100 to 150 characters" into the language model 53. The comment field 72 may display a comment entered by a user who has checked the graph.

[0079] According to this embodiment, by generating a report from the data accumulated in the log DB 57, it is possible to provide an information processing system 40 that contributes to improving the management of medical facilities.

[0080] Program 97 is an example of a program product. A computer program can be deployed to be executed on a single computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0081] [Embodiment 5] This embodiment relates to an information processing system 40 that analyzes the emotions of a patient and the emotions of a medical professional who has treated the patient. Explanation of parts common to the third embodiment will be omitted.

[0082] In this embodiment, the second control unit 21 transmits a query including audio data that records the audio of a medical professional such as a nurse from the time that the medical professional enters the hospital room 45 until the time that the medical professional leaves the room, to the information processing device 10. The query may include video data that records both video and audio that are taken from the time that the medical professional such as a nurse enters the hospital room 45 until the time that the medical professional leaves the room. The video captures the whole body or part of the patient or medical professional for at least a part of the time.

[0083] The query may include audio or video data as well as still images taken in the patient room 45. The still or video images preferably include the face of the patient or medical staff, as the face is likely to reveal the emotions of the person in the image.

[0084] The first control unit 11 inputs the received query into a language model 53, such as BERT, to classify the emotions of the patient and medical staff as positive emotions, negative emotions, or neutral emotions that are neither positive nor negative.

[0085] Positive emotions include feelings of security, trust, happiness, gratitude, hope, and confidence. Negative emotions include feelings of disgust, fear, anger, sadness, and helplessness. Neutral emotions include feelings of indifference and calmness.

[0086] The first control unit 11 acquires, for example, from the language model 53, the results of classifying the emotions of the patient and the medical professional at the time when the medical professional finishes talking to the patient and leaves the room. The first control unit 11 may also acquire the results of classifying the emotions of the patient and the medical professional that continued for the longest period of time during which the voice data was recorded.

[0087] FIG. 12 is an explanatory diagram illustrating an example of the determination criteria in the fifth embodiment. In this embodiment, when both the patient and the medical staff are feeling negative emotions, the first control unit 11 sends a "first notification" to a manager such as the hospital director or head nurse. When the patient is feeling negative emotions and the medical staff is not feeling negative emotions, the first control unit 11 sends a "second notification." When the medical staff is feeling negative emotions and the patient is not feeling negative emotions, the first control unit 11 sends a "third notification" to the manager.

[0088] If neither the patient nor the medical staff has negative feelings, the first control unit 11 does not send a notification to the administrator. Table 1 shows examples of the first to third notifications.

[0089] [Table 1]

[0090] Fig. 13 is a flowchart illustrating the processing flow of program 97 according to embodiment 5. When second control unit 21 detects the entry of a medical professional into the room, it starts program 97 in Fig. 13. Second control unit 21 identifies the medical professional who has entered the room by means of facial recognition or the like (step S701).

[0091] The second control unit 21 temporarily records dialogue data 62 relating to communication between the patient and the medical staff in the main storage device 22 or the auxiliary storage device 23 until it detects that the medical staff has left (step S702). The dialogue data 62 includes voice captured via the microphone 26 and still images or video captured via the camera 28. Note that the dialogue data 62 may consist of voice only.

[0092] The second control unit 21 generates a query including information about the medical professional identified in step S701 and the dialogue data 62 recorded in step S702 (step S703). The second control unit 21 transmits the query (step S704). The second control unit 21 ends the process.

[0093] The first control unit 11 receives a query (step S801). The first control unit 11 generates a prompt based on the query. The prompt is, for example, "You are a psychological counselor at ** Hospital. Based on the data recording the conversation between Yamamoto in Room 303 and Nurse A, please perform an emotion analysis for each and tell us whether the emotion is positive, neutral, or negative."

[0094] The first control unit 11 inputs the prompt to the language model 53 and acquires a response sentence indicating the emotion analysis result (step S802). The response sentence is, for example, "Yamamoto-san's emotion is positive. Nurse A's emotion is neutral."

[0095] The first control unit 11 determines whether or not a notification to the administrator is necessary, and if so, which of the first to third notifications to send (step S803), based on the criteria described using Fig. 12. If it is determined that a notification is necessary (YES in step S803), the first control unit 11 sends a notification such as that shown in Table 1 to the administrator's email address or the like (step S804).

[0096] If it is determined that notification is not necessary (NO in step S803), or after step S804 is completed, the first control unit 11 returns to the standby state.

[0097] The first control unit 11 may actually issue a notification when it determines that a notification is necessary (YES in step S803) a predetermined number of times, such as three times or more, within a predetermined period such as one day.

[0098] The first control unit 11 may record the date and time when it was determined that notification was necessary (YES in step S803) for each combination of patient and medical worker, and create a graph or a time series graph such as that described with reference to Fig. 11. The administrator can use the graph to consider specific measures.

[0099] According to this embodiment, the administrator who receives the notification can take appropriate measures such as changing the person in charge, which is expected to reduce stress for both the patient and the medical staff.

[0100] [Embodiment 6] This embodiment relates to an information processing system 40 that detects that a patient is in a critical condition and notifies a medical professional. Explanation of parts common to the first embodiment will be omitted.

[0101] 14 is an explanatory diagram illustrating the configuration of an information processing system 40 according to Embodiment 6. The information processing system 40 includes a second camera 282 and a second information processing device 80 in addition to the information processing device 10 and the user interface 20.

[0102] As in the first embodiment, the information processing device 10 includes a first control unit 11, a main memory device 12, an auxiliary memory device 13, a communication unit 14, and a reading unit 19. The auxiliary memory device 13 stores a notification determination model 58, a language model 53, recorded data 63, a program 97 to be executed by the first control unit 11, and various data required for executing the program 97.

[0103] The configuration of user interface 20 is also the same as in the first embodiment, but in this embodiment, the camera of user interface 20 is referred to as first camera 281. First camera 281 and second camera 282 are collectively referred to as camera 28.

[0104] Second camera 282 is positioned so as to be able to capture the entire interior of hospital room 45, including areas that are blind spots of first camera 281. Second camera 282 is desirably positioned so as to be able to capture the interior of hospital room 45 without any blind spots, even when people such as patients or staff are moving around within hospital room 45. Multiple second cameras 282 may be positioned in hospital room 45 so as to cover each other's blind spots.

[0105] The second camera 282 is a network camera and transmits captured images to the information processing device 10 in real time. The second control unit 21 also transmits images captured by the first camera 281 to the information processing device 10 in real time. The images transmitted to the information processing device 10 are recorded in the recording data 63. The built-in clocks of the user interface 20 and the second camera 282 are synchronized with, for example, an NTP (Network Time Protocol) server or the like.

[0106] The second information processing device 80 includes a third control unit 81, a main memory device 82, an auxiliary memory device 83, a communication unit 84, a touch panel 85, a microphone 86, and a speaker 87. The components constituting the second information processing device 80 are similar to the components constituting the information processing device 10 and the user interface 20, and therefore description thereof will be omitted.

[0107] The second information processing device 80 is an information device such as a smartphone or tablet that a medical professional carries in a medical facility. The second information processing device 80 may be installed in, for example, a nurse's station or an examination room. The second information processing device 80 may be the same model as the user interface 20.

[0108] 15 is an explanatory diagram illustrating an example configuration of the notification determination model 58. The notification determination model 58 includes two models: a posture estimation model 581 and a behavior determination model 582. The posture estimation model 581 is a model that receives input of a still image or video of a person and estimates the posture of the person.

[0109] For example, various pose estimation models 581 are known, such as OpenPose provided by Carnegie Mellon University and Mediapipe Pose provided by Google (registered trademark). These models estimate the pose of a person by detecting the coordinates of landmarks such as the person's joints. A detailed description of the pose estimation model 581 will be omitted.

[0110] The behavior determination model 582 is a model that receives input of patient information and the patient's posture output from the posture estimation model 581, and classifies whether the patient's condition requires notification to a medical professional. The patient information includes information such as the patient's illness and the treatment the patient has received.

[0111] A specific example will be given. For example, in the case of a patient whose patient information is recorded with a doctor's instruction to "stay at bed rest," if the posture estimation model 581 detects a posture in which the patient is attempting to get up from the bed, a notification must be sent to a healthcare professional. In the case of a patient whose patient information is recorded with a patient who has undergone "glaucoma surgery" within one week, if a prone posture is detected, a notification must be sent to a healthcare professional.

[0112] The behavior determination model 582 may be a rule-based model, or a supervised learning model generated using training data that records many sets of patient information, patient posture, and whether or not notification is required.

[0113] The configuration of notification determination model 58 is not limited to that shown in Fig. 15. Notification determination model 58 may be a single machine learning model generated by machine learning to receive an image captured by camera 28 and patient information, and output whether or not notification to a medical professional is necessary.

[0114] Fig. 16 is a flowchart illustrating the flow of processing by the program 97 according to the embodiment 6. The program 97 in Fig. 16 runs constantly in the hospital room 45 where a patient who needs to be monitored for posture and behavior is staying.

[0115] The first control unit 11 acquires patient information from, for example, an electronic medical record system (not shown) (step S711). The first control unit 11 acquires images successively transmitted from the camera 28 (step S712).

[0116] The first control unit 11 inputs the patient information acquired in step S711 and the image acquired in step S712 to the notification determination model 58, and acquires a determination result regarding whether or not notification to a medical professional is necessary (step S713).

[0117] The first control unit 11 determines whether or not a notification is necessary (step S714). For example, when the probability that a notification is necessary exceeds a predetermined threshold value such as 50 percent, the first control unit 11 determines that a notification is necessary.

[0118] If it is determined that a notification is necessary (YES in step S714), the first control unit 11 generates a notification for the medical staff (step S715). The notification includes, for example, a photo showing the patient's condition, the room number of the hospital room 45, and the patient's name.

[0119] The first control unit 11 sends a notification to a medical professional (step S716). For example, the first control unit 11 sends a short message or an email to the user interface 20 carried by the nurse in charge of the patient. The first control unit 11 may display the notification on a monitor installed in the nearest nurse's station.

[0120] If it is determined that no addition is necessary (NO in step S714) or after step S716 is completed, the first control unit 11 determines whether to end the process (step S717). For example, if the patient leaves the hospital room 45 to go to the toilet, or if an event occurs that may cause a change in the patient information, such as seeing a doctor, the first control unit 11 determines to end the process.

[0121] If it is determined not to end the process (NO in step S717), the first control unit 11 returns to step S712. If it is determined to end the process (YES in step S717), the first control unit 11 ends the process.

[0122] According to this embodiment, it is possible to provide an information processing system 40 that promptly notifies a medical professional when a patient assumes an improper posture or is about to assume an improper posture, thereby preventing the patient's condition from worsening due to improper posture and supporting early recovery.

[0123] According to this embodiment, it is possible to provide an information processing system 40 that does not require medical personnel such as nurses to constantly observe the condition of the patient.

[0124] [Embodiment 7] In this embodiment, explanation of parts common to embodiment 6 regarding information processing system 40 that notifies when a patient shows signs of dementia will be omitted. In this embodiment, image data and audio data for a predetermined period, such as one day's worth, are recorded in auxiliary storage device 13, and the processing described below is performed during a time period when the load on first control unit 11 is low, such as at night.

[0125] 17 is an explanatory diagram illustrating an example of the configuration of the dementia risk assessment model 59. The dementia risk assessment model 59 includes an image-to-text conversion model 591, a voice-to-text conversion model 592, a risk event extraction model 593, and a assessment model 594.

[0126] The image-to-text conversion model 591 is an image analysis model that converts multiple still images or videos taken continuously into text. The image-to-text conversion model 591 generates a pair of a timestamp and text that describes the image. An example of the output of the image-to-text conversion model 591 is shown in Table 2.

[0127] [Table 2]

[0128] Note that while the timestamps in Table 2 are listed in minutes, the actual timestamps are recorded in seconds or smaller units.

[0129] The speech-to-text model 592 is a speech analysis model that converts speech data into text, and generates pairs of timestamps and text converted from speech.

[0130] The image-to-text conversion model 591 and the voice-to-text conversion model 592 have conventionally been provided in the form of SaaS (Software as a Service) and package software, and therefore detailed description thereof will be omitted.

[0131] The output of image-to-text conversion model 591 and the output of speech-to-text conversion model 592 are integrated into one time-series data arranged in the order of timestamps. An example of the integrated data is shown in Table 3. In the following explanation, the time-series data shown in Table 3 will be referred to as integrated data.

[0132] [Table 3]

[0133] The risk event extraction model 593 is a model that receives the integrated data and extracts events that are suspected to be dementia. The risk event extraction model 593 is realized by inputting a prompt such as "You are a psychiatrist at ** Hospital. Please read the integrated data and extract any events that are suspected to be dementia or MCI (Mild Neurocognitive Disorder) regarding Mr. Tanaka" to the language model 53 that has been trained in advance with, for example, diagnostic guidelines for dementia.

[0134] The risk event extraction model 593 outputs events that may be related to dementia, such as, "Today, even though Mr. Tanaka had breakfast at 7:30, he asked about today's breakfast menu at 8:00, 8:12, and 8:30."

[0135] The determination model 594 is a model that receives the output of the risk event extraction model 593 and outputs the risk that a patient has developed dementia. The determination model 594 is a model that calculates the dementia risk, i.e., the probability that a patient has developed dementia, in a rule-based manner based on, for example, the number and type of extracted events. The determination model 594 may be a learning model trained by machine learning using training data that records a large number of combinations of the output of the risk event extraction model 593 and diagnosis results by specialists.

[0136] The configuration of the dementia risk assessment model 59 is not limited to that shown in Figure 17. The dementia risk assessment model 59 may be a single model trained by machine learning to accept an image or audio and output a dementia risk. The dementia risk assessment model 59 may be a generative model that accepts an image or audio and executes a prompt to instruct a process to output a dementia risk. The dementia risk assessment model 59 may also accept input of patient information in addition to an image or audio.

[0137] 18 is a flowchart illustrating the processing flow of the program 97 according to the eighth embodiment. The program 97 in FIG. 18 is started, for example, late at night, and processes image data and audio data recorded in the auxiliary storage device 13.

[0138] The first control unit 11 acquires images recorded in the auxiliary storage device 13 (step S721). The first control unit 11 acquires audio data recorded in the auxiliary storage device 13 (step S722).

[0139] The first control unit 11 inputs the image and sound to the dementia risk assessment model 59. The dementia risk assessment model 59 performs the series of processes described using Fig. 17 and outputs the dementia risk. The first control unit 11 acquires the dementia risk output from the dementia risk assessment model 59 (step S723).

[0140] The first control unit 11 determines whether the dementia risk exceeds a predetermined threshold, such as 60 percent (step S724). If it is determined that the dementia risk exceeds the threshold (YES in step S724), the first control unit 11 notifies the doctor or care manager in charge of the patient by means of e-mail or the like that there is a high probability that the patient has developed dementia (step S725).

[0141] If it is determined that the difference does not exceed the threshold (NO in step S724), or after step S725 is completed, the first control unit 11 ends the process.

[0142] According to this embodiment, it is possible to realize an information processing system 40 that can detect signs of dementia at an early stage.

[0143] The threshold value used in the determination in step S724 may be determined, for example, as the amount of increase from one month before, thereby realizing an information processing system 40 that detects patients whose probability of developing dementia is increasing.

[0144] Multiple thresholds may be set for the determination in step S724. For example, a case where two thresholds are set will be described as an example. When a notification is received that the lower threshold has been exceeded, the medical facility will strengthen monitoring, such as by having nurses patrol the patient's room 45 more frequently. For example, measures to prevent the progression of dementia may be taken, such as playing music that is considered effective in preventing dementia, such as "gamma wave sound," in the patient's room 45.

[0145] If the higher threshold is exceeded, it will become difficult to treat the patient in Room 45, which is intended for regular patients, and transfer to a specialized facility for dementia patients will be considered.

[0146] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims.

[0147] Independent and dependent claims may be combined with each other in any combination, regardless of the reference format. Furthermore, while the claims may be written in a format in which a claim references two or more other claims (multiple claim format), this is not a limitation. Multiple claims that reference at least one other claim (multiple multiple claim format) may also be written. [Explanation of symbols]

[0148] 10. Information processing equipment 11 First control section (control section) 12 Main storage 13 Auxiliary storage device 14 Communications Department 19 Reading unit 20 User Interface 21 Second control section 22 Main storage 23 Auxiliary storage device 24 Communications Department 25 Touch panel (display) 26. Mike 27 Speaker 28 Camera 281 Camera 1 282 Second Camera 35 Nurse call system 36 Nurse call button 37 Nurse call remote control 40 Information Processing Systems 45 Hospital Room 52 Classification Models 53 Language Models 54 Classification DB 541 Item List 57 Log DB 58 Notification Judgment Model 581 Pose Estimation Model 582 Behavioral Decision Model 59 Dementia Risk Assessment Model 591 Image-to-Text Conversion Model 592 Speech-to-Text Model 593 Risk Event Extraction Model 594 Decision Model 61 Medical Facility Information 63 Recorded Data 71 Graph column 72 Comments 80 Second information processing device 81 Third Control Section 82 Main storage 83 Auxiliary storage device 84 Communications Department 85 Touch Panel 86 Mike 87 Speaker 96 Portable recording media 97 Programs 98 Semiconductor Memory

Claims

1. obtaining, via a user interface, a voice-recorded query of a person staying at a medical facility; inputting the acquired query into a classification model that classifies whether the query is a first request for medical facility information or a second request for medical treatment at a medical facility, and acquiring a classification result; If the classification result is a first request, obtaining a response sentence in response to the obtained query using a language model that uses medical facility information including information about the medical facility and rules; outputting the acquired response sentence via the user interface; If the classification result is the second request, output information about a nurse call. An information processing method in which processing is performed by a computer.

2. If the classification result is the second request, a message is output to a speaker or a display to request the patient to operate a nurse call button. The information processing method according to claim 1 .

3. If the classification result is the second request, output a nurse call operation signal. The information processing method according to claim 1 .

4. extracting medical facility information similar to the acquired query from a storage unit that stores medical facility information including information about the medical facility and rules; The extracted medical facility information is provided to the language model to obtain the response sentence. The information processing method according to claim 1 .

5. the person is a patient undergoing treatment at the medical facility; generating a report related to the patient using the language model based on information obtained via the user interface; The information processing method according to claim 1 .

6. the person is a patient undergoing treatment at the medical facility; receiving, via the user interface, a voice instruction to register identification information for identifying a patient and medication information relating to a medication status of the patient; The received identification information and the medication information are associated with each other and stored in a storage unit of the medical facility. The information processing method according to claim 1 .

7. receiving, via the user interface, identification information identifying a patient and a query requesting confirmation of medication information; Referring to the storage unit, the medication information regarding the patient is output by voice. The information processing method according to claim 6.

8. receiving, via the user interface, a voice instruction to register identification information for identifying a patient and treatment information relating to treatment details for the patient; The received identification information and the treatment information are stored in the storage unit in association with each other. The information processing method according to claim 7.

9. receiving, via the user interface, identification information for identifying a patient and a query for a medication status and treatment history for the patient; The storage unit is referenced, and a history of medication status and treatment details corresponding to the query is displayed on a screen of the user interface. The information processing method according to claim 8.

10. the query includes records of interactions between patients undergoing treatment at the medical facility and medical personnel; Based on the acquired query, the emotions of the patient and the emotions of the medical staff are acquired, respectively; Outputting a notification when the patient's emotion is negative or when the healthcare worker's emotion is negative. The information processing method according to claim 1 .

11. The patient's emotion and the healthcare worker's emotion are the emotions of the patient and the healthcare worker, respectively, at the end of the interaction. The information processing method according to claim 10.

12. the person is a patient admitted to the medical facility; Acquire image data of the patient from a camera placed in a hospital room where the patient is hospitalized; acquiring a posture of the patient based on the acquired image data; determining whether or not a notification regarding the patient is necessary based on information regarding the patient and the acquired posture; If it is determined that notification is necessary, medical professionals will be notified. The information processing method according to claim 1 .

13. the person is a patient undergoing treatment at the medical facility; acquiring image data of the patient's behavior; Acquire audio data in which the patient's voice is recorded; determining a dementia risk of the patient based on the image data and the voice data; If the dementia risk is determined to be higher than the threshold, a medical professional is notified. The information processing method according to claim 1 .

14. obtaining a voice-recorded query from a person staying at a medical facility via a user interface; inputting the acquired query into a classification model that classifies whether the query is a first request for medical facility information or a second request for medical treatment at a medical facility, and acquiring a classification result; If the classification result is a first request, obtaining a response sentence in response to the obtained query using a language model that uses medical facility information including information about the medical facility and rules; outputting the acquired response sentence via the user interface; If the classification result is the second request, output information about a nurse call. A program that causes a computer to perform a process.

15. An information processing device including a control unit, The control unit obtaining a voice-recorded query from a person staying at a medical facility via a user interface; inputting the acquired query into a classification model that classifies whether the query is a first request for medical facility information or a second request for medical treatment at a medical facility, and acquiring a classification result; If the classification result is a first request, obtaining a response sentence in response to the obtained query using a language model that uses medical facility information including information about the medical facility and rules; outputting the acquired response sentence via the user interface; If the classification result is the second request, output information about a nurse call. Information processing device.

Images