Information processing system

Abstract

Provided is an information processing system capable of managing medication and checking the physical condition of a home-care patient through communication with the patient. When a predetermined condition is satisfied, a robot 10 notifies, from an information output device unit, a home-care patient 2 that it is time for medication. The robot 10 also communicates with the home-care patient 2 through the information output device unit and an information input device unit to acquire communication data using the information input device unit and an imaging device unit. The robot 10 also performs checking based on an external instruction, including checking of medicine D, medication, and the physical condition of the home-care patient 2. The robot 10 also transmits the communication data including the results of the checking to a server 20. The server 20 includes a communication content management unit 111-2 that acquires the communication data transmitted from the robot 10, stores the communication data in a predetermined database, and manages the communication data.

Description

Information Processing Systems

[0001] The present invention relates to an information processing system.

[0002] In recent years, with the progress of the aging society, the importance of medication management has increased, especially for cancer patients. When taking medication at home, forgetting or taking medication incorrectly can have a significant impact on the effectiveness of treatment, so the development of medication confirmation systems is underway. Conventional medication management systems include, for example, devices that notify patients of medication time with an alarm.

[0003] JP 2018-140138 A

[0004] However, conventional technologies, including the technology of Patent Document 1, have not been able to accurately confirm the medication status of patients at home, or to properly grasp the patient's condition or side effects.

[0005] The present invention has been made in view of the above circumstances, and aims to provide an information processing system that enables medication management and physical condition checks through communication with patients at home.

[0006] In order to achieve the above-mentioned object, an information processing system according to one embodiment of the present invention is an information processing system for monitoring a patient at home, comprising: a robot having an imaging device unit, an information input device unit, and an information output device unit for communicating with the patient at home; and a first information processing device that communicates with the robot via a predetermined network, wherein the robot: when a predetermined condition is met, notifies the patient at home from the information output device unit that it is time to take their medication; acquires communication data using the information input device unit and the imaging device unit by communicating with the patient at home using the information output device unit and the information input device unit; confirms the medication, medication, and physical condition of the patient at home based on external instructions; and sends the communication data including the results of the confirmation to the first information processing device; and the first information processing device comprises: a communication content management unit that acquires the communication data sent from the robot, stores it in a predetermined database, and manages it.

[0007] According to the present invention, it is possible to manage medication and check the health condition of patients at home through communication with them. Specifically, it is possible to manage medication and check the health condition of patients at home through communication with them by using a robot.

[0008] 1 is a diagram illustrating an overall configuration of an information processing system according to one embodiment of the present invention and an overview of the service that can be realized by the information processing system. FIG. 2 is a diagram illustrating an example of the configuration of an information processing system according to one embodiment of the present invention. FIG. 3 is a block diagram illustrating an example of the hardware configuration of a server in the information processing system of FIG. 2. FIG. 4 is a functional block diagram illustrating an example of the functional configuration of a robot in the information processing system of FIG. 2. FIG. 5 is a functional block diagram illustrating an example of the functional configuration of a server in the information processing system of FIG. 2. FIG. 6 is a sequence diagram of an information processing system according to one embodiment of the present invention. FIG. 7 is an operational image diagram of a conversation function in Step 1 according to one embodiment of the present invention. FIG. 8 is a diagram of a medication monitoring function by an operator in Step 1 (sounding of the operator console) in one embodiment of the present invention. FIG. 9 is a diagram of a medication monitoring function by an operator in Step 1 (calling through the robot) in one embodiment of the present invention. FIG. 10 is a diagram of a medication monitoring function by an operator in Step 1 (checking medication) in one embodiment of the present invention. FIG. 11 is a diagram of a medication monitoring function by an operator in Step 1 (checking medication) in one embodiment of the present invention. FIG. 1 is a diagram of a medication monitoring function (having AI learn) by an operator in Step 1 in one embodiment of the present invention. FIG. 2 is an operational image diagram of a conversation function by AI automation in Step 2 in one embodiment of the present invention. FIG. 3 is a diagram of a medication monitoring function (automatic system connection) by AI automation in Step 2 in one embodiment of the present invention. FIG. 4 is a diagram of a medication monitoring function (calling through a robot) by AI automation in Step 2 in one embodiment of the present invention. FIG. 5 is a diagram of a medication monitoring function (checking medication) by AI automation in Step 2 in one embodiment of the present invention. FIG. 6 is a diagram of a medication monitoring function (checking medication) by AI automation in Step 2 in one embodiment of the present invention. FIG. 7 is a diagram of a medication monitoring function (follow-up observation) by AI automation in Step 2 in one embodiment of the present invention.

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] First, an overview of the overall configuration of an information processing system according to one embodiment of the present invention and a service that can be realized by the information processing system (hereinafter referred to as "this service") will be described with reference to Fig. 1. Fig. 1 is a diagram showing the overall configuration of an information processing system according to one embodiment of the present invention and an overview of the service that can be realized by the information processing system.

[0011] This service is related to an information processing system 1 that uses a robot 10 to enable medication management and health checks through communication with a patient 2 at home. This service prevents forgetting to take medication, promotes reliable treatment, and reduces the burden on the patient by eliminating the need for separate records. In addition, since data is managed, for example, on a server 20 in a cloud system, it is easy to share information with medical professionals 4. In this service, information from a medicine notebook (electronic information, OCR information obtained from paper media) may be input and managed as part of the information on the server 20.

[0012] This service monitors the medication of a home-based patient 2 (e.g., a cancer patient) at home using a robot 10 as an interface. This service also allows the robot 10 to be close to the home-based patient 2, allowing the patient to enjoy everyday conversations with the robot 10 and to estimate the physical condition of the home-based patient 2 from the conversations.

[0013] This service has two stages: Step 1 and Step 2. In Step 1, the operator 3 monitors the medication of the home-care patient 2 via the robot 10, and also creates a home care record (home care diary 6). In Step 2, the AI ​​learns the information exchanged between the home-care patient 2 and the robot 10, and the home care diary 6 created by the operator 3, and ultimately, the service automatically communicates with the home-care patient 2 and automatically generates the home care diary 6.

[0014] This service also allows for cooperation between the patient 2 at home and, for example, the family 5 .

[0015] This service allows for the home care diary 6 to be fed back to medical professionals 4 such as doctors and pharmacists, thereby confirming the proper use of medication D and supporting the health management of home-care patients 2.

[0016] The information processing system 1 that provides the above-mentioned service is composed of a robot 10 that communicates with a patient 2 at home, a server 20 that is installed, for example, in a cloud system, and an operator console 30 that is operated by an operator 3.

[0017] The robot 10 has an imaging device such as a camera (not shown), an information input device such as a microphone (not shown), and an information output device such as a speaker (not shown), and thus has the function of communicating with the home-care patient 2. Furthermore, the robot 10 can operate in cooperation with various sensors, various medication support devices, various measuring devices, terminals, etc. as necessary.

[0018] The server 20 is communicatively connected to the robot 10 via a predetermined network N. The server 20 has a function of receiving communication data transmitted from the robot 10, storing it in a database, and managing it.

[0019] The operator console 30 is communicatively connected to the robot 10 via the server 20. The operator console 30 has a function of mediating communication with the home patient 2 via the robot 10 and monitoring medication administration.

[0020] In addition, the information processing system 1 allows medical personnel 4 such as doctors and pharmacists to view information such as the home care diary 6 managed on the server 20, and by providing feedback to the medical personnel 4 on the medication status and physical condition of the home-based patient 2, it confirms the appropriate use of drug D and supports the health management of the home-based patient 2.

[0021] The functions of each component of the information processing system 1 will be described in more detail below, including functions not shown in FIG.

[0022] The robot 10 has a function of notifying the home-based patient 2 that it is time to take their medication via an information output device (e.g., a speaker) when a predetermined condition (e.g., a preset time or an instruction from the operator 3) is met. The robot 10 also has a function of communicating with the home-based patient 2 via the information output device (e.g., a speaker) and the information input device (e.g., a microphone) and acquiring communication data using the information input device and the imaging device (e.g., a camera). The robot 10 also has a function of confirming the medication (drug D) of the home-based patient 2, whether the patient has taken the medication, and the patient's physical condition based on instructions from an external source (e.g., the operator 3 or the server 20). The robot 10 also has a function of transmitting communication data including the results of the above confirmations to the server 20.

[0023] The server 20 has a function of acquiring the above-mentioned communication data transmitted from the robot 10, storing it in a predetermined database, and managing it.

[0024] The above functions enable the robot 10 to be used to manage medication and check the patient's health through communication with the patient 2 at home. This prevents forgetting to take medication, promotes reliable treatment, and reduces the burden on the patient by eliminating the need for separate records. Furthermore, since data is managed by the server 20, it is easy to share information with medical personnel 4.

[0025] The operator console 30 has a function of receiving the above-mentioned communication data via the server 20, a function of displaying information for checking the condition of the home-care patient 2 based on the communication data, a function of generating instruction information for the robot 10 based on input from the operator 3, and a function of transmitting the above-mentioned instruction information to the robot 10 via the server 20. The robot 10 has a function of performing the above-mentioned check based on the instruction information transmitted from the operator console 30.

[0026] By introducing the operator console 30 with the above functions, an operator 3 with specialized knowledge can remotely monitor medication administration. This allows the operator 3 to grasp the condition of the patient 2 at home in real time and issue accurate instructions, enabling more accurate medication administration management and health checks. In addition, because it can be operated remotely, it is possible to support many patients with limited human resources.

[0027] The operator console 30 has a function of creating a home care diary 6 including information regarding medication monitoring of the home patient 2. The server 20 has a function of receiving home care diary data, which is data for the home care diary 6, from the operator console 30 and storing it in a predetermined database (such as the storage unit 18 described later).

[0028] The home care diary 6 creation function described above makes it possible to record medication status and changes in physical condition over time. Sharing this data with medical professionals 4 allows for long-term evaluation of the effects and side effects of drug D, which can be useful in formulating and revising appropriate treatment plans. Furthermore, recording the data in a specified database makes it easy to compare and analyze it with past records, and can also be used to recognize patterns in changes in physical condition.

[0029] The operator console 30 has a function of sounding an alarm at a preset time to notify the operator 3, and a function of establishing a connection with the robot 10 based on the response of the operator 3.

[0030] The notification function and connection establishment function for ensuring timely medication confirmation as described above promote compliance with medication times. This allows the appropriate interval between doses to be maintained to maximize the effectiveness of Drug D, leading to improved therapeutic effects. In addition, the quality of monitoring is improved because the operator 3 can respond reliably when necessary.

[0031] The server 20 has a function of training an artificial intelligence model based on the communication data and home care diary data stored in a predetermined database (such as the storage unit 18 described later), a function of generating automatic instruction information for the robot 10 using the trained artificial intelligence model, and a function of transmitting the automatic instruction information to the robot. The robot 10 has a function of performing the above-mentioned confirmation based on the above-mentioned automatic instruction information transmitted from the server 20.

[0032] The AI ​​learning function described above realizes automated medication monitoring without the intervention of an operator 3. This allows for efficient use of human resources and enables continuous monitoring 24 hours a day, 365 days a year. Furthermore, learning from accumulated data enables personalized responses tailored to individual patients, improving the quality of monitoring.

[0033] The server 20 has the function of automatically generating a home care diary 6 based on the above-mentioned communication data transmitted from the robot 10, and the function of storing the data of the automatically generated home care diary 6 in a predetermined database (such as the memory unit 18 described later).

[0034] The automatic generation function of the home care diary 6 as described above significantly reduces the workload of the operator 3. In addition, because the diary is generated objectively from communication data, the accuracy and consistency of the records are improved. This improves the quality of feedback to medical personnel 4 and supports more appropriate medical decisions.

[0035] The robot 10 has a function of automatically estimating the physical condition of the patient 2 at home from the voice or speech content of the patient 2, and a function of transmitting the estimated physical condition information to the server 20 as the above-mentioned communication data.

[0036] The automatic physical condition estimation function described above may be able to detect subtle changes in physical condition that the patient himself or herself is not aware of. In particular, being able to capture symptoms that are difficult to verbalize or changes before subjective symptoms appear can lead to the early detection of side effects or worsening symptoms, preventing them from becoming serious. In addition, since evaluation based on objective data becomes possible, it can support more accurate medical decisions.

[0037] The automatic physical condition estimation function allows for more accurate physical condition evaluation if it estimates the home-based patient 2's physical condition based on the tone of voice, intonation, condition of the vocal muscles, or the content of speech containing specific words. In particular, since it is possible to read changes in physical condition from information that the home-based patient 2 does not consciously communicate, it facilitates objective understanding of the patient's health condition. This enables early detection of side effects of drug D and prompt response to changes in the patient's condition.

[0038] Next, the configuration of an information processing system 1 that realizes the provision of the above-described service will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of an information processing system according to an embodiment of the present invention.

[0039] The information processing system 1 includes a robot 10 , a server 20 , and an operator console 30 .

[0040] The robot 10, server 20, and operator console 30 are connected to each other via a network N such as the Internet. The robot 10 includes an imaging device such as a camera (not shown), an information input device such as a microphone (not shown), and an information output device such as a speaker (not shown). The robot 10 also includes a linking unit (not shown) for linking with various sensors, various medication support devices, various measuring devices, terminals, etc. The robot 10 is an information processing device having a form such as that shown in FIG. 1 (not limited to the form shown in FIG. 1), for example.

[0041] The server 20 is an information processing device managed by a service provider (not shown) of this service. The server 20 is installed, for example, in a cloud system, and executes various processes for realizing this service while appropriately communicating with the robot 10 and the operator console 30.

[0042] The operator console 30 is an information processing device operated by the operator 3, and is configured by a smartphone, a tablet, a personal computer, or the like.

[0043] It should be noted that the information processing system 1 is capable of appropriately communicating with information processing devices (smartphones, tablets, personal computers, etc.) operated by family members 5, for example.

[0044] Next, the hardware configuration of the server 20 in the information processing system 1 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the hardware configuration of the server in the information processing system shown in Fig. 2.

[0045] The server 20 is configured to include a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, a RAM (Random Access Memory) 113, a bus 114, an input / output interface 115, an input unit 116, an output unit 117, a memory unit 118, a communication unit 119, and a drive 120.

[0046] The CPU 111 executes various processes in accordance with a program recorded in the ROM 112 or a program loaded from the storage unit 118 to the RAM 113. The RAM 113 also stores data and the like necessary for the CPU 111 to execute various processes, as appropriate.

[0047] The CPU 111, ROM 112, and RAM 113 are connected to one another via a bus 114. An input / output interface 115 is also connected to this bus 114. An input unit 116, an output unit 117, a storage unit 118, a communication unit 119, and a drive 120 are connected to the input / output interface 115.

[0048] The input unit 116 is configured with, for example, a keyboard and accepts input of various information. The output unit 117 is configured with, for example, a display such as an LCD and a speaker and outputs various information as images and sounds. The storage unit 118 is configured with, for example, a DRAM (Dynamic Random Access Memory) and stores various data. The communication unit 119 communicates with other devices (for example, the robot 10 and operator console 30 in FIG. 2 ) via a network N including the Internet.

[0049] Removable media 130, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately attached to the drive 120. Programs read from the removable media 130 by the drive 120 are installed in the storage unit 118 as needed. The removable media 130 can also store various data stored in the storage unit 118 in the same way as the storage unit 118.

[0050] Although not shown, the robot 10 and the operator console 30 in Fig. 2 can also have basically the same hardware configuration as that shown in Fig. 3. Therefore, a description of the hardware configuration of the robot 10 and the operator console 30 will be omitted.

[0051] The various hardware and software components constituting the information processing system 1 of FIG. 2, including the server 20 of FIG. 3, work together to execute various processes for providing the present service of FIG.

[0052] Next, the functional configuration of the robot 10 in the information processing system 1 will be described with reference to Fig. 4. Fig. 4 is a functional block diagram showing an example of the functional configuration of the robot in the information processing system of Fig. 2.

[0053] 4, the CPU 111R of the robot 10 functions as an imaging unit 111R-1, an information input unit 111R-2, an information output unit 111R-3, a notification execution unit 111R-4, a communication execution unit 111R-5, a data acquisition unit 111R-6, a confirmation execution unit 111R-7, a data transmission unit 111R-8, and a physical condition estimation unit 111R-9. Note that, since the robot 10 has a configuration basically similar to the hardware configuration of the server 20 in FIG. 3, an "R" is added after the CPU reference number 111 to distinguish it from the robot 10.

[0054] The imaging unit 111R-1 controls an imaging device unit (video capturing device) (not shown), such as a camera, which is a hardware component of the robot 10. Specifically, the imaging unit 111R-1 controls an imaging device unit (not shown), such as a camera used to capture images of the face of the home-care patient 2, medication (drug D), etc.

[0055] The information input unit 111R-2 controls an information input device unit (voice input device) (not shown), such as a microphone, which is a hardware component of the robot 10. Specifically, the information input unit 111R-2 controls an information input device unit (not shown), such as a microphone used to collect the voice of the patient 2 at home.

[0056] The information output unit 111R-3 controls an information output device unit (audio output device) (not shown), such as a speaker, which is a hardware component of the robot 10. Specifically, the information output unit 111R-3 controls an information output device unit (not shown), such as a speaker, which is used to output notifications and conversations to the at-home patient 2 as audio.

[0057] When a predetermined condition is satisfied, the notification execution unit 111R-4 executes control to notify the home patient 2 from an information output device (not shown) that it is time to take the medication. Examples of the predetermined condition include a preset time and an instruction from the server 20 or the operator console 30.

[0058] The communication execution unit 111R-5 executes control for communicating with the at-home patient 2 using an information output device unit (voice output device) not shown and an information input device unit (voice input device) not shown.

[0059] The data acquisition unit 111R-6 executes control to acquire data indicating the content of communication with the home patient 2 as communication data using an information input device unit (voice input device) not shown and an imaging device unit (video imaging device) not shown.

[0060] The confirmation execution unit 111R-7 executes control to confirm the medication (drug D) of the home-based patient 2, confirm that the patient has taken the medication, and confirm the patient's physical condition by communicating with the home-based patient 2 using an information output device unit (voice output device) not shown and an information input device unit (voice input device) not shown, based on instructions given from outside (for example, the operator 3 or the server 20) based on the above-mentioned communication data.

[0061] The data transmission unit 111R-8 executes control for transmitting communication data including confirmation of the medication (drug D), confirmation of medication, and confirmation of the physical condition of the at-home patient 2 to the server 20. The data transmission unit 111R-8 executes control for transmitting information on the physical condition (estimated physical condition information) described later to the server 20 as communication data.

[0062] The physical condition estimation unit 111R-9 executes control for automatically estimating the physical condition of the at-home patient 2 from the voice or speech content of the at-home patient 2. By functioning the physical condition estimation unit 111R-9, the physical condition estimation unit 111R-9 can estimate the physical condition of the at-home patient 2 based on the tone of voice, intonation, condition of the vocal muscles, or speech content including specific words. For example, by detecting specific words such as "tired," "can't sleep," and "unmotivated," the physical condition estimation unit 111R-9 can estimate the poor physical condition of the at-home patient 2.

[0063] Next, the functional configuration of the server 20 in the information processing system 1 will be described with reference to Fig. 5. Fig. 5 is a functional block diagram showing an example of the functional configuration of the server in the information processing system of Fig. 2.

[0064] As shown in Figure 5, the CPU 111 of the server 20 functions as a data receiving unit 111-1, a communication content management unit 111-2, a memory control unit 111-3, a learning processing unit 111-4, an automatic instruction generation unit 111-5, a data transmission unit 111-6, and an automatic home care diary generation unit 111-7.

[0065] The data receiving unit 111-1 executes control for receiving communication data transmitted from the robot 10. The communication content management unit 111-2 executes control for storing and managing the received communication data in the storage unit 18. The storage control unit 111-3 executes control for storing the communication data and data such as the home care diary in the storage unit 18.

[0066] The learning processing unit 111-4 executes control for training the artificial intelligence model based on the communication data and home care diary data stored in the storage unit 18. The automatic instruction generation unit 111-5 executes control for generating automatic instruction information for the robot 10 using the trained artificial intelligence model. The data transmission unit 111-6 executes control for transmitting the generated automatic instruction information to the robot 10.

[0067] The automatic home care diary generation unit 111-7 executes control for automatically generating the home care diary 6 based on the communication data transmitted from the robot 10.

[0068] Next, the functional configuration of the operator console 30 in the information processing system 1 will be described with reference to Fig. 6. Fig. 6 is a functional block diagram showing an example of the functional configuration of the operator console in the information processing system of Fig. 2.

[0069] 6, a receiving unit 111C-1, a display unit 111C-2, an instruction generating unit 111C-3, a transmitting unit 111C-4, a home care diary creating unit 111C-5, a sounding unit 111C-6, and a connection establishing unit 111C-7 function in the CPU 111R of the operator console 30. Note that, since the operator console 30 has a configuration basically similar to the hardware configuration of the server 20 in FIG. 3, a "C" is added after the CPU reference number 111 to distinguish it from the operator console 30.

[0070] The receiving unit 111C-1 executes control for receiving communication data via the server 20. The display unit 111C-2 executes control for displaying information for checking the condition of the home patient 2 based on the communication data. The instruction generating unit 111C-3 executes control for generating instruction information for the robot 10 based on input from the operator 3.

[0071] The transmitting unit 111C-4 executes control to transmit the generated instruction information to the robot 10 via the server 20. The home care diary creating unit 111C-5 executes control to create a home care diary 6 including information regarding medication monitoring of the at-home patient 2. The sounding unit 111C-6 executes control to sound a sound at a preset time to notify the operator 3. The connection establishing unit 111C-7 executes control to establish a connection with the robot 10 based on the response of the operator 3.

[0072] The CPU 111C of the operator console 30 has a display unit 111C-2 and an instruction generation unit 111C-3 for display and operation functions. The CPU 111C also has a receiving unit 111C-1 and a transmitting unit 111C-4 for communication functions. The CPU 111C also has a home care diary creation unit 111C-5, a sounding unit 111C-6, and a connection establishment unit 111C-7 for recording and notification functions.

[0073] 7 is a sequence diagram of the information processing system 1 according to an embodiment of the present invention. The information processing system 1 operates in two steps: Step 1 and Step 2.

[0074] In Step 1, medication monitoring is performed via the operator 3. First, a regular notification (S101) is sent from the server 20 to the operator console 30, and the sounding unit 111C-6 of the operator console 30 sounds (S102). When the operator 3 responds (S102), the connection establishment unit 111C-7 of the operator console 30 instructs the robot 10 to establish a connection (S103).

[0075] When the connection is established (S104), the notification execution unit 111R-4 of the robot 10 notifies the home-based patient 2 of the time to take the medication (S105). When the home-based patient 2 responds (S106), the data acquisition unit 111R-6 of the robot 10 acquires the communication data, and the data transmission unit 111R-8 transmits it to the operator console 30 via the server 20 (S107, S108).

[0076] Next, the instruction generation unit 111C-3 of the operator console 30 generates a drug confirmation instruction based on the input from the operator 3, and the transmission unit 111C-4 transmits the instruction to the robot 10 via the server 20 (S109, S110). The confirmation execution unit 111R-7 of the robot 10 makes a drug confirmation request to the home-based patient 2 based on the instruction (S111), and the home-based patient 2 presents the drug (drug D) (S112).

[0077] Following the same process, medication and physical condition are also confirmed (S121). After the series of confirmations is completed, the home care diary creation unit 111C-5 of the operator console 30 creates a home care diary 6 based on the input of the operator 3 (S131) ​​and transmits it to the server 20 (S132). The learning processing unit 111-4 of the server 20 causes the AI ​​to learn the communication data and the data of the home care diary 6 (S133).

[0078] In Step 2, after learning, automated medication monitoring is performed by AI without the intervention of the operator 3. Based on the automatic instruction information generated by the automatic instruction generation unit 111-5 of the server 20, the robot 10 communicates with the patient 2 at home and monitors medication. In addition, the automatic home care diary generation unit 111-7 of the server 20 automatically generates a home care diary 6. The created home care diary 6 is shared with medical professionals 4, such as doctors and pharmacists, who can evaluate the effects and side effects of drug D based on this information and adjust the prescription and dosage of drug D as necessary.

[0079] The flow of operation of the information processing system 1 according to one embodiment of the present invention will be described. First, the content corresponding to Step 1 in FIG. 7 will be described. FIG. 8 is an operational image diagram of the conversation function of Step 1. FIG. 9 is a diagram of the medication monitoring function by the operator of Step 1 (sounding the operator console). FIG. 10 is a diagram of the medication monitoring function by the operator of Step 1 (calling through the robot). FIG. 11 is a diagram of the medication monitoring function by the operator of Step 1 (checking medication). FIG. 12 is a diagram of the medication monitoring function by the operator of Step 1 (checking medication). FIG. 13 is a diagram of the medication monitoring function by the operator of Step 1 (follow-up observation). FIG. 14 is a diagram of the medication monitoring function by the operator of Step 1 (having AI learn).

[0080] In Step 1, as shown in Figures 8 to 14, medication monitoring is performed with the intervention of an operator 3. First, as shown in Figure 8, a daily conversation takes place between the robot 10 and the home-care patient 2 to understand the condition and interests of the home-care patient 2. Next, as shown in Figure 9, at a preset time, the operator console 30 sounds and the operator 3 responds.

[0081] Next, as shown in Fig. 10, the robot 10 notifies the home patient 2 that it is time to take the medication, and the home patient 2 responds. After that, as shown in Fig. 11, the robot 10 confirms the medication (medicine D) with the home patient 2. The operator 3 asks the home patient 2 to present the medication via the robot 10, and the home patient 2 shows the medication to the camera of the robot 10. The operator 3 checks the captured image and determines whether it is the correct medication.

[0082] 12, the robot 10 confirms with the patient 2 whether he or she has taken the medicine. The operator 3 prompts the patient 2 to take the medicine via the robot 10, and the patient 2 takes the medicine. The operator 3 checks how the patient has taken the medicine and determines whether the medicine has been taken correctly.

[0083] 13, the operator 3 checks the physical condition of the home patient 2 through the robot 10. The operator 3 asks the home patient 2 about his / her physical condition through the robot 10, and the home patient 2 replies about his / her physical condition. The physical condition estimation unit 111R-9 of the robot 10 automatically estimates the physical condition of the home patient 2 from the voice and speech content of the home patient 2.

[0084] All communication data obtained through the above series of confirmation processes is recorded in the information processing system 1, as shown in Fig. 14. Information on the interactions between the operator 3 and the home-care patient 2 (information A) and the home care diary 6 created by the operator 3 (information B) are used as data for learning by the AI.

[0085] 14, the operator 3 creates a home care diary 6 using the home care diary creation unit 111C-5 of the operator console 30. Information such as date and time, body temperature, blood pressure, digestive symptoms, symptoms in the hands and feet, bleeding, headache, dizziness, hair loss, shortness of breath, fatigue, and swelling is recorded in this home care diary 6. The created home care diary 6 is shared with medical professionals 4 such as doctors and pharmacists, and serves as material for confirming the appropriate use of drug D.

[0086] The contents corresponding to Step 2 in Fig. 7 will be explained. Fig. 15 is an operational image diagram of the conversation function by AI automation in Step 2. Fig. 16 is a diagram of the medication monitoring function by AI automation in Step 2 (automatic system connection). Fig. 17 is a diagram of the medication monitoring function by AI automation in Step 2 (calling through a robot). Fig. 18 is a diagram of the medication monitoring function by AI automation in Step 2 (checking medication). Fig. 19 is a diagram of the medication monitoring function by AI automation in Step 2 (checking medication). Fig. 20 is a diagram of the medication monitoring function by AI automation in Step 2 (follow-up observation).

[0087] In Step 2, as shown in Figures 15 to 20, automated medication monitoring by AI is performed without the intervention of the operator 3. First, as shown in Figure 15, a daily conversation is automatically conducted between the robot 10 and the home-care patient 2. Next, as shown in Figure 16, the server 20 automatically connects to the robot 10.

[0088] As shown in Fig. 17, the robot 10 notifies the home-based patient 2 that it is time to take the medication, and the home-based patient 2 responds. Next, as shown in Fig. 18, the robot 10 automatically confirms the medication (drug D) with the home-based patient 2 based on the learned artificial intelligence model. The robot 10 asks the home-based patient 2 to present the medication, and the home-based patient 2 shows the medication to the camera of the robot 10. The robot 10 analyzes the captured image and determines whether it is the correct medication.

[0089] Similarly, the robot 10 automatically checks whether the home patient 2 has taken the medication and determines whether the medication has been taken correctly. Next, as shown in Fig. 20, the robot 10 automatically checks the physical condition of the home patient 2. The robot 10 asks the home patient 2 about his or her physical condition, and the home patient 2 replies about his or her physical condition. The physical condition estimation unit 111R-9 of the robot 10 automatically estimates the physical condition of the home patient 2 from the voice and speech content of the home patient 2.

[0090] When the above series of checks are completed, the automatic home care diary generation unit 111-7 of the server 20 automatically generates a home care diary 6 and records it in, for example, the storage unit 18 of the server 20. This home care diary 6 can be checked by medical personnel 4, and serves as an important source of information for doctors, pharmacists, etc. to understand the patient's medication status and the occurrence of side effects, and to confirm the appropriate use of drug D.

[0091] From here on, the process up to the completion of a series of checks such as Step 1 and Step 2 will be described overall without reference to any particular figures or without reference to any reference numerals.

[0092] [Detailed explanation of conversation function] The conversation function in the information processing system is a communication means for monitoring the medication of cancer patients at home, for example, using a robot as an interface. The robot stays close to the patient at home, enjoys everyday conversation, and has the function of estimating the patient's physical condition and other factors from the conversation.

[0093] The conversation function can be activated by a variety of triggers, including the following. First, by pressing a button on the robot. It can also be activated by speaking to the robot. It can also be operated from a tablet device or set to automatically activate at a specific or random time. It can also be activated when there is a change in information from the robot's built-in sensors or various connected measuring devices. It can also be activated from the operator console (remote activation by the operator, family members, or medical professionals).

[0094] The operator console is expected to be dedicated software that runs on a PC, smartphone, or tablet, or a web application that can be run using general browser software, etc. This will enable operations to have real-time conversations through the robot, and triggers to start the robot automatically having conversations.

[0095] The conversation function's operating sequence begins with the robot speaking to the patient at home. For example, "Hello!", "It's 3 o'clock. Have you had a snack?", "Your temperature was high today," etc. Except when activated from the operator console, the robot will use AI or other means to make speech and motions appropriate to the situation. At this time, the content of the speech is determined based on the date and time, weather, the patient's interests and health status, and information from sensors and measuring instruments.

[0096] When started from the operator console, several patterns are assumed for the operation of calling the patient at home using the operator console. One method is to convert text entered in the chat into robot voice using Text-to-Speech (TTS). Another method is to convert voice input into robot voice using TTS after speech recognition, or to have the voice spoken from the robot's speaker (a voice change function can also be used). This makes it appear to the patient at home that a robot is speaking, and the operator does not have to be a fixed person.

[0097] The text patterns that the operator inputs into the chat can be selected from pre-registered ones, or they can input text freely on the spot. The text input by the operator (the robot's speech) is displayed on the operator console, and can also be displayed to the home patient if they have a display device such as a tablet terminal that is linked to the home patient. The robot's motion instructions in conjunction with the operator's voice input and speech can also be recorded.

[0098] Next, the patient responds to the robot's utterances. For example, "Hello!", "Snack time is coming up," or "Yes, just a little." After the patient has spoken, voice input ends and the robot performs voice recognition processing. At this time, the patient's physical condition is automatically confirmed based on their voice and the content of their utterances.

[0099] When checking the patient's physical condition, the patient's physical condition is estimated based on the tone of voice, intonation, vocal muscles, etc. The patient's physical condition is also estimated based on the content of the speech. This is done using specific pre-set words, such as "tired," "can't sleep," and "unmotivated." The results of these physical condition estimations are stored and included in the physical condition and side effect checks. If a significant change is determined, the operator and relevant parties are notified by email or other means.

[0100] The recognized text (what the patient has said at home) is displayed on the operator console and recorded. If the patient has a display device such as a tablet that is linked to the operator's console, the text can also be displayed on the patient's side. The operator can check the patient's voice in real time and record it in audio format.

[0101] When not activated from the operator console, the AI ​​will create the response. For example, "I'm glad you look healthy today!", "Let's exercise after you eat a snack," or "Let's measure yourself again." The results of the health check are reflected in the speech, response, and motion. For example, if it is determined that the user is feeling depressed, the system will use positive expressions and lots of compliments in the conversation.

[0102] When the call is initiated from the operator console, the robot responds to the patient at home by the above-mentioned means, and the robot then responds by speaking the response again to continue the conversation.

[0103] The conversation function is stopped by the following triggers: when the patient's voice is not detected for a certain period of time, or when a button on the robot is pressed. It also stops when the robot is spoken to (e.g., "End of conversation," "That's it," etc.), when it is operated from a tablet device, or when a specific time of day arrives (such as nighttime). It also stops when there is a change in the information from the robot's built-in sensors or the various connected measuring devices (such as when it gets dark), or when the operator console is operated.

[0104] [Detailed explanation of medication monitoring function (with operator)] The medication monitoring function (with operator) in the information processing system is a function that checks the medication, physical condition, and side effects of home-based patients via a robot. A system is provided in which the operator monitors the patient's medication via the robot and creates a record of home care (home care diary).

[0105] The medication monitoring function is activated by the following triggers. One is a preset time, which is set in the system in advance by an operator (not limited to medical personnel), or by the patient at home, their family, medical personnel, etc., using voice instructions to the robot or inputting information into a tablet device, etc. The other is manual operation on the operator console.

[0106] The operational sequence of the medication monitoring function begins with a connection phase. The operator console beeps and the operator responds, or performs manual operation, and then operates the operator console to allow connection to the robot. As a result, the operator console and robot are connected. At the same time, the robot's camera is activated, making it possible to check and record the situation of the patient at home via video. Thereafter, the situation of the patient at home can be continuously checked and recorded via video.

[0107] Next, in the calling phase, the operator calls the patient at home through the robot. There are several possible patterns for the operator to call the patient at home. These include converting text entered by the operator into robot voice using TTS, converting the operator's voice input into voice recognition text and then converting it into robot voice using TTS, or having the operator's voice speak from the robot's speaker (a voice change function can also be used).

[0108] This makes it appear to the patient at home that the robot is speaking, and the operator does not have to be a fixed person. The text patterns that the operator types into the chat can be selected from pre-registered patterns, or they can use text that they type freely on the spot. After this process, the robot will say things like, "It's time for your medicine."

[0109] When making a call, it is expected that the robot will not only make a call but also perform various other actions depending on the settings, etc. For example, when it is time to take medication, the robot can bring the medicine and medicine box to the user (the medicine and medicine box may be built into the robot itself), or it can be linked to various medication support devices and notify the medication support devices so that they can dispense medicine, unlock locks, open doors, play sounds, and perform other actions.

[0110] These actions can be commanded by the operator at any time (when reminding the patient to take the medication, when checking the medication, when taking the medication, etc.), or can be automatically executed in conjunction with startup. After the robot speaks, it waits for voice input (if there is no response, the operator can call again), and the patient at home can say something like "I got it." Then, the voice input ends and the robot's voice recognition process begins.

[0111] During this process, the system also automatically checks the patient's physical condition from their voice and speech. The system estimates the patient's physical condition from the tone of voice, intonation, vocal muscles, etc., and also from the content of speech. Estimation is also made using specific pre-set words, such as "tired," "can't sleep," and "unmotivated." The results of the physical condition estimation are stored and included in the physical condition and side effect checks. If a significant change is determined, the operator and relevant parties are notified by email or other means.

[0112] The recognized text (what the patient has said) is displayed on the operator console and recorded. If the patient has a display device such as a tablet that is linked to the operator's console, it can also be displayed on the patient's side. The operator can check the patient's voice in real time and record it in audio format.

[0113] In the medication confirmation phase, the operator uses the robot to ask the patient at home to confirm the medication, using the method described above. The robot will say something like, "This time it's one tablet A. Prepare the medication and show it to me," and wait for voice input. If there is no response, the operator will try to call again, and the patient at home will then say something like, "This is it." After the voice input is complete, the robot performs voice recognition processing, and, as described above, "automatic health check based on the voice and speech of the patient at home" is performed.

[0114] The operator confirms whether the medication is correct by listening to the audio or viewing the converted text and images. If the medication to be confirmed is not within the field of view, various means are used to adjust it so that it fits within the field of view. For example, the operator can use the robot to encourage the patient to fit within the field of view (by having the robot say, "Please raise the medication a little higher," or by having the robot raise its hand and say, "Place (point) the medication (face) in this direction," etc.), or the operator can operate the robot to adjust the direction and angle of the camera to fit within the field of view. If the patient at home has a display device such as a linked tablet device, the patient can adjust it by looking at the display.

[0115] In the medication confirmation phase, the operator uses the robot to prompt the home-based patient to take their medication using the methods described above. The robot will speak with questions such as "Take your medicine," "Did you take it properly?" and "Show me the medicine shell," and wait for voice input. If there is no response, the operator will try again, and the home-based patient will then say things like "Okay," "I took it," or "Here's the shell." After the voice input is complete, the robot performs voice recognition processing, and as described above, "automatic health checks are performed based on the voice and speech of the home-based patient."

[0116] The operator confirms whether the medication has been taken correctly by listening to the voice or viewing the converted text and images. If the patient or medication to be checked is not within the field of view, the operator adjusts the field of view using the same method as described above. Once the medication has been confirmed, the fact that the medication has been taken can be shared with the patient and relevant parties (family, medical professionals, etc.). The patient at home is notified by the robot's LED display (e.g., changing the eye color), a specific sound, or a tablet device. Reminders can also be sent when using the conversation function (e.g., "You forgot to take your morning medication after meal").

[0117] This function is intended to allow home-based patients, especially those with dementia, to check their own medication records, as they may forget that they have taken their medication and take it multiple times. Those involved can access the system and view the medication records.

[0118] In the follow-up observation phase, the operator uses the robot to check the home patient's physical condition and side effects using the methods described above. The robot will say things like "How are you feeling?" and "Show me your face," and wait for voice input. If there is no response, the operator will try to call out again, and the home patient will say things like "I'm not feeling very well" or "I've had a bit of diarrhea since last night." After the voice input is complete, the robot performs voice recognition processing, and as described above, "automatic health checks based on the home patient's voice and speech" are carried out.

[0119] The operator checks the patient's physical condition and side effects while listening to the audio and viewing the converted text and images. If the part of the patient's body that needs to be checked is not within the viewing angle, the operator adjusts the viewing angle using the same method as described above. To ensure that the operator can check all the check items set for each patient and each medication, the check items are automatically displayed on the operator console, and the operator proceeds with the check accordingly.

[0120] All interactions (voice, text, images, etc.) between the robot (operator) and the home-care patient in all of the above phases are recorded (information A). After the follow-up observation phase is completed, the operator creates a home care diary (information B). The home care diary is expected to be in a standard format that allows medical professionals to record items expected by the home-care patient and medication, and items such as date and time, body temperature, blood pressure, digestive symptoms, symptoms in the hands and feet, bleeding, headache, dizziness, hair loss, shortness of breath, fatigue, and swelling are expected.

[0121] All information recorded in the system, including information A and information B, can be viewed by relevant parties by accessing the system. The scope of disclosure can be set by the viewer. By having AI learn the home care diary (information B) along with the information on interactions with home patients recorded in the system (information A), it will be possible in the future to provide automated services without an operator, as explained below.

[0122] The medication monitoring function is terminated by a trigger such as manual operation on the operator console.

[0123] [Detailed explanation of the medication monitoring function (without operator)] The medication monitoring function (without operator) in the information processing system is a function that uses AI to minimize human intervention and allows a robot to automatically monitor the medication of patients at home. By having the AI ​​learn the conversation data and home care diary accumulated by the above-mentioned "with operator" function, it is possible to automatically interact with patients at home and automatically generate home care diaries.

[0124] The conversation function can be activated by the following triggers: pressing a button on the robot, operating from a tablet device, setting it to a specific or random time, or when information from sensors built into the robot or various linked measuring devices changes. In "No Operator" mode, no operator is involved during normal operation, but it is also possible to activate the conversation function externally by starting the operator console. In this case, the activation trigger and operation sequence will be the same as for "With Operator".

[0125] The conversation function's operational sequence begins with the robot speaking to the patient at home. For example, "Hello!", "It's 3 o'clock. Have you had a snack?", "Your temperature was high today," etc. The robot will use AI to make speech and motions appropriate to the situation. At this time, the content of the speech is determined based on the date and time, weather, the patient's interests and health status, and information from sensors and measuring instruments.

[0126] Next, the patient at home responds to the robot's utterances. For example, "Hello!", "Snack time is coming up," or "Yes, just a little." After the patient at home speaks, voice input ends and the robot performs voice recognition processing. At this time, the patient's physical condition is automatically checked from the patient's voice and speech content. The physical condition is estimated from the tone of voice, intonation, vocal muscles, etc., and from the speech content (estimated using specific pre-set words, e.g., "tired," "can't sleep," "unmotivated," etc.).

[0127] The results of these health condition estimations are stored and included in health condition and side effect checks. If a significant change is determined, the operator and relevant parties are notified by email or other means. The recognized text (what the patient at home says) is recorded. If the patient at home has a display device such as a tablet terminal that is linked to the device, it can also be displayed on the patient's side. The voice of the patient at home can also be recorded in audio format.

[0128] The AI ​​or other device then creates a response. For example, "I'm glad you look healthy today!", "Let's exercise after you eat a snack," or "Let's measure your health again." The results of the health check are reflected in the speech content, response, and motion. For example, if it determines that the person is feeling depressed, it will use positive expressions and lots of compliments in the conversation. The robot then responds again and continues the conversation.

[0129] The conversation function will stop when triggered by the following: when the patient's voice is not detected for a certain period of time, when a button on the robot is pressed, when an operation is performed from a tablet device, when a specific time of day arrives (such as nighttime), or when information from the robot's built-in sensors or various connected measuring devices changes (such as when it becomes dark).

[0130] The medication monitoring function is activated by the following triggers. It can be activated at a preset time (the time can be set in the system in advance by the operator, or by the patient at home or their family member using voice instructions to the robot or input via a tablet device, etc.). In the "without operator" mode, no operator is involved during normal operation, but if something happens (such as a request from the family member or when an important change is detected using the conversation function), the operator can activate it manually. In the case of manual activation, the operating sequence is the same as when "with operator".

[0131] The operation sequence of the medication monitoring function begins with a connection phase. The system automatically connects to the robot, and the two systems are connected. At the same time, the robot's camera is activated, making it possible to record video of the patient's condition at home. From then on, it is possible to continue recording video of the patient's condition at home.

[0132] Next, in the calling phase, the robot calls out to the patient at home. The text generated by the AI ​​is converted into the robot's voice using TTS. The AI ​​also instructs the robot's motions to match the speech, and can record the motions performed. After this process, the robot will say something like, "It's time for your medicine."

[0133] When making a call, it is expected that the robot will not only make a call but also perform various other actions depending on the settings. For example, when it is time to take medication, the robot will bring the medication and medicine box to the user's location (the robot itself may have the medicine and medicine box built in), or it can be linked to various medication support devices and notify the medication support device to take actions such as dispense medication, unlock the lock, open the door, or play sounds or voice messages. These actions can be performed automatically when the robot is started.

[0134] After the robot speaks, it waits for voice input (it will call again if there is no response), and the home patient says something like "I got it." Then the voice input ends and the robot's voice recognition process begins. During this process, the home patient's physical condition is automatically checked from their voice and what they say. Their physical condition is estimated from the tone of voice, intonation, vocal muscles, etc., and also from what they say. Estimation is also made using specific pre-set words, and the results of the physical condition estimation are stored and included in the physical condition and side effect checks. If it is determined that there is a significant change, the operator and relevant parties are notified by email or other means.

[0135] The recognized text (the speech of the patient at home) is recorded. If the patient at home has a display device such as a tablet that is linked to the device, it can also be displayed on the patient's side. The patient's speech can also be recorded in audio format.

[0136] In the medication confirmation phase, the AI ​​uses the above-mentioned means to call out to the home-based patient through the robot to ask them to confirm their medication. The robot will say something like, "This time, it's one tablet A. Prepare the medication and show it to me," and wait for voice input. If there is no response, the robot will call out again, and then the home-based patient will say something like, "This is it." After the voice input is complete, the robot performs voice recognition processing, and as described above, "automatic health check based on the voice and speech content of the home-based patient" is performed.

[0137] The AI ​​determines whether the medication is correct based on the voice, converted text, images, etc. If the medication to be confirmed is not within the field of view, various methods are used to adjust it so that it fits within the field of view. For example, the AI ​​can interact with the patient at home through the robot to bring it within the field of view (such as by having the robot say, "Please raise the medication a little higher," or by having the robot raise its hand and say, "Place (point) the medication (face) in this direction," or the AI ​​can operate the robot to adjust the direction and angle of the camera to fit within the field of view, or if the patient at home has a display device such as a linked tablet device, the patient can adjust it by looking at the display.

[0138] In the medication confirmation phase, the AI ​​prompts the home-based patient to take their medication through the robot using the methods described above. The robot will say things like, "Take your medicine," "Did you take it properly?" and "Show me the medicine shell," and wait for voice input. If there is no response, the robot will call out again, and the home-based patient will say things like, "Okay," "I took it," or "Here's the shell." After the voice input is complete, the robot performs voice recognition processing, and as described above, "automatic health checks based on the home-based patient's voice and speech content" are performed.

[0139] The AI ​​looks at the voice, converted text, images, etc. to determine whether the medication has been taken correctly. If the patient or medication to be confirmed at home is not within the field of view, it adjusts it to fit within the field of view using the same methods as described above. Once the medication has been confirmed, the fact that the medication has been taken can be shared with the patient at home and relevant parties (family, medical professionals, etc.). The patient at home is notified by the robot's LED display (e.g., changing the eye color), a specific sound, a tablet device, etc.

[0140] Some home-care patients may forget that they have taken their medication due to dementia or other reasons and end up taking it multiple times, so the system will allow them to check their own medication records. Those involved will be able to access the system and view the medication records.

[0141] In the follow-up observation phase, the AI ​​uses the robot to check the home patient's physical condition and side effects using the methods described above. The robot will say things like "How are you feeling?" and "Show me your face," and wait for voice input. If there is no response, it will try again, and the home patient will say things like "I'm not feeling very well" or "I've had a bit of diarrhea since last night." After the voice input is complete, the robot performs voice recognition processing, and as described above, "automatic health checks based on the home patient's voice and speech" are carried out.

[0142] The AI ​​checks the patient's physical condition and side effects using voice, converted text, images, etc. If the part of the patient's body that needs to be checked is not within the field of view, it adjusts it so that it fits within the field of view using the same method as described above. The AI ​​proceeds with the check according to the learning data so that it can check all the check items set for each patient at home and each medication being taken.

[0143] All interactions (voice, text, images, etc.) between the robot and the home patient during all of the above phases are recorded (Information A). After the follow-up observation phase is completed, the AI ​​automatically creates a home care diary (Information B). The home care diary is expected to be in a standard format that allows medical professionals to record items expected by the home patient and medication, such as date and time, body temperature, blood pressure, digestive symptoms, symptoms in the hands and feet, bleeding, headache, dizziness, hair loss, shortness of breath, fatigue, and swelling.

[0144] All information recorded in the system, such as information A and information B, can be viewed by relevant parties by accessing the system. The scope of disclosure can be set by the viewer.

[0145] The medication monitoring function ends when triggered by the AI ​​determining that all operational sequences have been completed.

[0146] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope of achieving the object of the present invention are included in the present invention.

[0147] For example, the system configuration shown in FIG. 2 and the hardware configuration of the server 20 shown in FIG. 3 are merely examples for achieving the object of the present invention, and are not particularly limited.

[0148] 4 to 6 are merely examples and are not particularly limited. That is, it is sufficient that the information processing system 1 in FIG. 2 is provided with the functions capable of executing the various processes described above as a whole, and the functional blocks and databases used to realize these functions are not particularly limited to the examples in FIG. 4 to 6.

[0149] 4 to 6, the locations of the function blocks and databases may be arbitrary. For example, at least some of the function blocks and databases arranged on the server 20 side may be provided on the robot 10 side, the operator console 30 side, or another information processing device (not shown).

[0150] The above-described series of processes can be executed by hardware or software, and each functional block can be configured by hardware alone, software alone, or a combination of both.

[0151] When a series of processes is executed by software, the programs constituting the software are installed onto a computer or the like from a network or a recording medium. The computer may be a computer incorporated into dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0152] The recording medium containing such a program may be composed not only of a removable medium (not shown) that is distributed separately from the device main body in order to provide the program to the user, but also of a recording medium that is provided to the user in a state that is pre-installed in the device main body.

[0153] In this specification, the steps describing the program to be recorded on the recording medium include not only processes that are performed in chronological order, but also processes that are not necessarily performed in chronological order but are performed in parallel or individually.

[0154] In summary, an information processing system to which the present invention is applied is sufficient as long as it has the following configuration, and can take various forms. That is, an information processing system to which the present invention is applied (e.g., information processing system 1 in FIG. 1, etc.) is an information processing system that watches over a patient at home (e.g., patient 2 at home in FIG. 1, etc.), and includes: a robot (e.g., robot 10 in FIG. 1) equipped with an imaging device unit (e.g., a camera), an information input device unit (e.g., a microphone), and an information output device unit (e.g., a speaker) for communicating with the patient at home (e.g., the interaction shown in FIG. 8), and a first information processing device (e.g., server 20 in FIG. 1) that communicates with the robot via a predetermined network (e.g., network N in FIG. 2, etc.), wherein the robot notifies the patient at home from the information output device unit that it is time to take medication when a predetermined condition (e.g., a preset time or an instruction from an operator) is met, and acquires communication data using the information input device unit and the imaging device unit by communicating with the patient at home using the information output device unit and the information input device unit, Based on instructions from outside (for example, operator 3 or server 20 in Figure 1), confirmation is made by checking the medication (for example, drug D in Figure 1), whether the patient has taken the medication, and whether the patient is in good physical condition (for example, checking the medication and medication in Figures 11 and 12), and the communication data including the results of the confirmation is sent to the first information processing device, and the first information processing device is only required to be equipped with a communication content management unit (for example, communication content management unit 111-2 in Figure 5) that acquires the communication data sent from the robot and stores and manages it in a specified database (for example, memory unit 118 in Figure 3).

[0155] According to the information processing system to which the present invention is applied, it is possible to use a robot to communicate with patients at home and manage their medication and check their health condition. This prevents patients from forgetting to take their medication, promotes reliable treatment, and reduces the burden on patients by eliminating the need for them to record their medication separately. Furthermore, since data is managed by the first information processing device, it is easy to share information with medical professionals.

[0156] In addition, an information processing system to which the present invention is applied further includes a second information processing device (e.g., operator console 30 in Figure 1, etc.) that is operated by an operator and can communicate with the first information processing device via a predetermined network, and the second information processing device comprises: a receiving unit (e.g., receiving unit 111C-1 in Figure 6, etc.) that receives the communication data via the first information processing device; a display unit (e.g., display unit 111C-2 in Figure 6, etc.) that displays information to check the condition of the home-care patient based on the communication data; an instruction generation unit (e.g., instruction generation unit 111C-3 in Figure 6, etc.) that generates instruction information for the robot based on input from the operator; and a transmitting unit (e.g., transmitting unit 111C-4 in Figure 6, etc.) that transmits the instruction information to the robot via the first information processing device, and the robot can perform the confirmation based on the instruction information transmitted from the second information processing device.

[0157] According to this, by introducing the second information processing device, an operator with specialized knowledge can remotely monitor medication administration. This makes it possible to grasp the condition of home-care patients in real time and issue accurate instructions, enabling more accurate medication administration management and health monitoring. In addition, because it can be operated remotely, it is possible to support many patients with limited human resources.

[0158] In addition, in an information processing system to which the present invention is applied, the second information processing device further includes a home care diary creation unit (e.g., home care diary creation unit 111C-5 in Figure 6) that creates a home care diary (e.g., home care diary 6 in Figure 1) containing information regarding medication monitoring of the patient at home, and the first information processing device receives home care diary data, which is data on the home care diary, from the second information processing device and stores it in the specified database.

[0159] This system allows users to record medication status and changes in physical condition over time using the home care diary creation function. Sharing this data with medical professionals allows for the long-term evaluation of medication effectiveness and side effects, which can be useful in formulating and revising appropriate treatment plans. Recording data in a designated database also facilitates comparative analysis with past records, and can be used to recognize patterns in changes in physical condition.

[0160] In addition, in the information processing system to which the present invention is applied, the second information processing device may further include a sounding unit that sounds a sound at a preset time to notify the operator, and a connection establishment unit that establishes a connection with the robot based on the operator's response.

[0161] This system promotes adherence to medication times through notification and connection establishment functions that ensure medication is taken at the scheduled time. This allows appropriate intervals to be maintained to maximize the effectiveness of medication, leading to improved treatment outcomes. Furthermore, the quality of monitoring is improved because operators can respond reliably when needed.

[0162] In addition, in an information processing system to which the present invention is applied, the first information processing device further comprises: a learning processing unit (e.g., learning processing unit 111-4 in Figure 5) that learns an artificial intelligence model based on the communication data and the home care diary data stored in the specified database; an automatic instruction generation unit (e.g., automatic instruction generation unit 111-5 in Figure 5) that generates automatic instruction information for the robot using the learned artificial intelligence model; and a data transmission unit (e.g., data transmission unit 111-6 in Figure 5) that transmits the automatic instruction information to the robot, and the robot can perform the confirmation based on the automatic instruction information transmitted from the first information processing device.

[0163] This system's AI learning function enables automated medication monitoring without the intervention of an operator. This allows for efficient use of human resources and enables continuous monitoring 24 hours a day, 365 days a year. Furthermore, by learning from accumulated data, personalized responses tailored to individual patients become possible, improving the quality of monitoring.

[0164] In addition, in an information processing system to which the present invention is applied, the first information processing device further includes an automatic home care diary generation unit (e.g., automatic home care diary generation unit 111-7 in Figure 5) that automatically generates a home care diary based on the communication data transmitted from the robot, and the home care diary data generated by the automatic home care diary generation unit is stored in the specified database.

[0165] According to this, the home care diary's automatic generation function significantly reduces the operator's workload. In addition, because the diary is generated objectively from communication data, the accuracy and consistency of records are improved. This improves the quality of feedback to medical personnel and supports more appropriate medical decisions.

[0166] In addition, in an information processing system to which the present invention is applied, the robot may further include a health condition estimation unit (such as health condition estimation unit 111R-9 in Figure 4) that automatically estimates the health condition of the patient at home from the voice or speech content of the patient, and transmits the health condition information estimated by the health condition estimation unit to the first information processing device as the communication data.

[0167] This means that the automatic physical condition estimation function may be able to detect subtle changes in physical condition that the patient himself or herself is not aware of. In particular, by being able to capture symptoms that are difficult to verbalize and changes before subjective symptoms appear, it will lead to the early detection of side effects and worsening symptoms, preventing them from becoming serious. In addition, since evaluation based on objective data will be possible, it will support more accurate medical decisions.

[0168] Furthermore, in the information processing system to which the present invention is applied, the physical condition estimation unit can estimate the physical condition of the patient at home based on the tone of voice, intonation, condition of the vocal muscles, or the content of the speech including specific words (for example, the above-mentioned physical condition estimation).

[0169] According to this, if the automatic physical condition estimation function estimates the physical condition of a home-care patient based on the tone of voice, intonation, the condition of the vocal muscles, or the content of speech containing specific words, a more accurate physical condition assessment becomes possible. In particular, since changes in physical condition can be read from information that the home-care patient does not consciously communicate, an objective understanding of the health condition is promoted. This enables early detection of side effects of medication and prompt response to changes in the patient's condition.

[0170] 1...information processing system, 2...home patient, 3...operator, 4...medical personnel, 5...family, 6...home care diary, 10...robot, 20...server, 30...operator console, 111...CPU, 112...ROM, 113...RAM, 114...bus, 115...input / output interface, 116...input unit, 117...output unit, 118...storage unit, 119...communication unit, 120...drive, 130...removable media, 111R-1...imaging unit, 111R-2...information input unit, 111R-3...information output unit, 111R-4...notification execution unit, 111R-5...communication Communication execution unit, 111R-6...data acquisition unit, 111R-7...confirmation execution unit, 111R-8...data transmission unit, 111R-9...physical condition estimation unit, 111-1...data receiving unit, 111-2...communication content management unit, 111-3...storage control unit, 111-4...learning processing unit, 111-5...automatic instruction generation unit, 111-6...data transmission unit, 111-7...automatic home care diary generation unit, 111C-1...receiving unit, 111C-2...display unit, 111C-3...instruction generation unit, 111C-4...transmitting unit, 111C-5...home care diary creation unit, 111C-6...sounding unit, 111C-7...connection establishment unit

Claims

1. An information processing system for monitoring a patient at home, comprising: a robot having an imaging device unit, an information input device unit, and an information output device unit for communicating with the patient at home; and a first information processing device that communicates with the robot via a predetermined network, wherein the robot: when a predetermined condition is met, notifies the patient at home from the information output device unit that it is time to take their medication; communicates with the patient at home using the information output device unit and the information input device unit to acquire communication data using the information input device unit and the imaging device unit; confirms the medication, medication, and physical condition of the patient at home based on external instructions; and sends the communication data including the results of the confirmation to the first information processing device; and the first information processing device comprises: a communication content management unit that acquires the communication data sent from the robot, stores it in a predetermined database, and manages it.

2. An information processing system as described in claim 1, further including a second information processing device operated by an operator and capable of communicating with the first information processing device via a specified network, wherein the second information processing device comprises: a receiving unit that receives the communication data via the first information processing device; a display unit that displays a display for checking the condition of the home-care patient based on the communication data; an instruction generating unit that generates instruction information for the robot based on input from the operator; and a transmitting unit that transmits the instruction information to the robot via the first information processing device, and wherein the robot performs the checking based on the instruction information transmitted from the second information processing device.

3. The information processing system described in claim 2, wherein the second information processing device further includes a home care diary creation unit that creates a home care diary including information regarding medication monitoring of the home-based patient, and the first information processing device receives home care diary data, which is data of the home care diary, from the second information processing device and stores it in the specified database.

4. The information processing system of claim 2, wherein the second information processing device further comprises: a sounding unit that sounds a sound at a preset time to notify the operator; and a connection establishment unit that establishes a connection with the robot based on the operator's response.

5. The information processing system described in claim 3, wherein the first information processing device further comprises: a learning processing unit that trains an artificial intelligence model based on the communication data and the home care diary data stored in the specified database; an automatic instruction generation unit that generates automatic instruction information for the robot using the trained artificial intelligence model; and a data transmission unit that transmits the automatic instruction information to the robot; and the robot performs the confirmation based on the automatic instruction information transmitted from the first information processing device.

6. The information processing system described in claim 5, wherein the first information processing device further includes an automatic home care diary generation unit that automatically generates a home care diary based on the communication data transmitted from the robot, and stores the home care diary data generated by the automatic home care diary generation unit in the specified database.

7. The information processing system of claim 1, wherein the robot further comprises a health condition estimation unit that automatically estimates the health condition of the patient at home from the voice or speech content of the patient, and transmits the health condition information estimated by the health condition estimation unit to the first information processing device as the communication data.

8. The information processing system according to claim 7, wherein the physical condition estimation unit estimates the physical condition of the patient at home based on the tone of voice, intonation, condition of the vocal muscles, or the content of the speech containing specific words.

Images