Information processing device, information processing method, and program

Abstract

To enable a user U to know that a plurality of devices in a home perform functions affecting each other only by performing a conversation with a watching robot 2 by the user when the function is executed.SOLUTION: A cloud server 1 comprises: an interface control part 31; a device operation setting information acquisition part 33; and an impact information notification part 34. The interface control part 31 performs a control of an interface containing a conversation between a user U and a watching robot 2. The device operation setting information acquisition part 33 acquires each device operation setting information of a plurality of devices which can be connected via a network N. The impact information notification part 34 notifies the user of a determination by the conversation on the basis of a control of the interface control part 31 in the case where it is determined that an execution of a predetermined function is impacted to an operation setting of the other device by referring the device operation setting information in accordance with the acquisition of information that the predetermined function is executed from the predetermined device among the plurality of devices.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Conventionally, there exists a dialogue system that switches dialogue modes in order to avoid erroneous recognition due to ambient noise and smoothly complete current multi-round dialogue tasks (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-554219 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned Patent Document 1 does not propose a dialogue mechanism for informing a plurality of devices when executing functions that affect each other through communication with the devices.

[0005] The present invention has been made in consideration of the above circumstances, and has an object to enable a user to know when multiple devices are executing functions that affect each other, simply by interacting with each other. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device according to an embodiment of the present invention comprises: an interface control means for controlling an interface including a dialogue with a predetermined user; a device operation setting information acquiring means for acquiring device operation setting information of each of a plurality of devices connectable via a communication network; an impact information notifying means for, in response to receiving information indicating that a predetermined function is to be executed from a predetermined device among the plurality of devices, referring to the device operation setting information and determining that the execution of the predetermined function will affect operation settings of other devices, notifying a user of that effect through the dialogue or by interrupting the dialogue under the control of the interface control means; Equipped with. An information processing method and a program corresponding to the information processing device according to one aspect of the present invention are also provided as an information processing method and a program according to one aspect of the present invention. Effect of the Invention

[0007] According to the present invention, a user can know when a plurality of devices are executing functions that affect each other simply by interacting with each other. [Brief description of the drawings]

[0008] [Figure 1] 1 shows a network configuration of an entire information processing system according to an embodiment of the present invention. [Diagram 2] 2 is a block diagram showing a hardware configuration of a cloud server in the information processing system of FIG. 1. [Diagram 3] 2 is a block diagram showing a hardware configuration of a monitoring robot in the information processing system of FIG. 1. [Figure 4] 2 is a functional block diagram showing a functional configuration of a cloud server in the information processing system of FIG. 1. [Diagram 5] 5 is a flowchart showing the operation of the cloud server of FIGS. 3 and 4; [Figure 6] FIG. 2 is a diagram showing a first embodiment of a living robot care service in the information processing system of FIG. 1. [Figure 7] FIG. 2 is a diagram showing a second embodiment of a living robot care service in the information processing system of FIG. 1. [Figure 8] FIG. 13 is a diagram showing a third embodiment of a living robot care service in the information processing system of FIG. [Figure 9] FIG. 13 is a diagram showing a fourth embodiment of a living robot care service in the information processing system of FIG. [Figure 10] FIG. 13 is a diagram showing a fifth embodiment of a living robot care service in the information processing system of FIG. [Figure 11] FIG. 13 is a diagram showing a first example of a sixth embodiment of a living robot care service in the information processing system of FIG. [Figure 12] 5 is a functional block diagram showing a second functional configuration of the cloud server of FIG. 4. [Figure 13] FIG. 13 is a diagram showing a seventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. [Figure 14] FIG. 13 is a diagram showing an eighth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. [Figure 15] FIG. 13 is a diagram showing a ninth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. [Figure 16] FIG. 13 is a diagram showing a tenth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. [Figure 17] FIG. 13 is a diagram showing an eleventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. [Figure 18] FIG. 12 is a diagram showing a twelfth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] FIG. 1 shows the overall configuration of an information processing system according to one embodiment of the present invention. The information processing system shown in Figure 1 is configured so that a cloud server 1, a monitoring robot 2 (interface), a control target device 3, a user terminal 4, an ambulance terminal 5, and an external terminal 6 are interconnected via a predetermined network N such as the Internet, thereby enabling communication.

[0011] The cloud server 1 is an information processing device that executes various processes to manage the operations of interfaces such as the monitoring robot 2, the control target device 3, the user terminal 4, the ambulance team terminal 5, and the external terminal 6.

[0012] Specifically, in this embodiment, the cloud server 1 manages each operation of the watching robot 2 and the user terminal 4 as follows. That is, the cloud server 1 is managed by a service provider, such as a service operating company, that provides this service, and is operated by the person in charge. The cloud server 1 manages various information (lifestyle information, conversation information, equipment operation information, specific report information, confidential information, abnormality determination method information, detection information from each sensor, etc.) obtained when the watching-over robot 2 operates the control target device 3 during a conversation between the user U and the watching-over robot 2, and executes processing based on each piece of information. The process executed by the cloud server 1 will be described later in detail with reference to FIG. The controlled devices 3 are some of the devices placed in the user U's home, such as a television, video, air conditioner, refrigerator, washing machine, bathroom dryer, electric kettle, etc., and are devices that are controlled by control signals from the monitoring robot 2 or the cloud server 1. In addition, the controlled devices 3 also include devices that detect things by themselves and transmit detection signals to the monitoring robot 2 or the cloud server 1, such as various sensors such as bathroom fall sensors, odor sensors, fire alarms, environmental monitoring sensors, excretion sensors, sleep vital signs / wake-up sensors, and surveillance cameras. The user terminal 4 is an information processing device managed and operated by an elderly user U, such as an elderly person, and is managed for each user. The user terminal 4 is a mobile terminal that the user U can carry or wear to browse and input information, such as a smartphone (hereinafter referred to as "smartphone"), a tablet terminal, or a laptop computer. In addition, the user terminal 4 also includes a remote controller (hereinafter referred to as "remote control") capable of operating home appliances such as a television (hereinafter referred to as "television"), a video recorder, and lighting equipment. The ambulance team terminal 5 is, for example, a personal computer (PC), a tablet terminal, a smartphone, or the like, which is operated by an information processing device for processing emergency calls installed in, for example, a fire station, or an ambulance technician rushing from the fire station. The external terminal 6 is an information processing device managed, for example, by the children or grandchildren of an elderly user U, a staff member of a service provider S (service operating company) that provides monitoring services, or a staff member in charge of small-scale delivery, and is, for example, a personal computer (PC), a tablet terminal, a smartphone, etc.

[0013] FIG. 2 is a block diagram showing a hardware configuration of a cloud server in the information processing system of FIG.

[0014] As shown in FIG. 2, the cloud server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, an input unit 17, a memory unit 18, a communication unit 19, and a drive 20.

[0015] The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 18 into the RAM 13 . The RAM 13 also stores data and the like necessary for the CPU 11 to execute various processes.

[0016] The CPU 11, ROM 12, and RAM 13 are connected to one another via a bus 14. An input / output interface 15 is also connected to this bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.

[0017] The output unit 16 is composed of a display such as a liquid crystal display, a speaker, etc., and outputs various information as images and sounds. The input unit 17 is configured with, for example, a keyboard, and outputs various information. The storage unit 18 is configured with a dynamic random access memory (DRAM) or the like, and stores various data. The communication unit 19 communicates with other devices (eg, the watching robot 2 and the external terminal 6 in FIG. 3) via a network N including the Internet.

[0018] Removable media 21, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately loaded into the drive 20. A program read from the removable media 21 by the drive 20 is installed in the storage unit 18 as necessary. Further, the removable medium 21 can store various data stored in the storage unit 18 in the same manner as the storage unit 18 .

[0019] Although not shown, the user terminal 4, the ambulance crew terminal 5, the external terminal 6, etc. also have basically the same hardware configuration as the cloud server 1 shown in Fig. 4. Therefore, explanations of the hardware configurations of the user terminal 4, the ambulance crew terminal 5, the external terminal 6 (for example, a terminal of a family member of the user U or a terminal of the service provider S) etc. will be omitted.

[0020] Such cooperation between the various hardware and software in FIG. 2 makes it possible to execute a series of processes relating to the dialogue between the user U and the watching-over robot 2.

[0021] FIG. 3 is a block diagram showing a hardware configuration of a monitoring robot in the information processing system of FIG. As shown in FIG. 3, the monitoring robot 2 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a bus 104, an input / output interface 105, an output unit 106, an input unit 107, a memory unit 108, a communication unit 109, and a drive 110. The hardware configuration of the watching robot 2 is almost the same as that of the cloud server 1, but the configurations of the output unit 106 and the input unit 107 are different. In the case of the monitoring robot 2, the body has an appearance resembling a human shape so as to facilitate natural conversation with the user U. Therefore, the microphone of the output unit 106 is located in the place corresponding to the ear, the camera in the place corresponding to the eye, the sensors in the limbs, the speaker and buzzer in the place corresponding to the mouth, and the display is located, for example, on the front part of the body.

[0022] FIG. 4 is a functional block diagram showing a functional configuration of the cloud server in the information processing system of FIG.

[0023] A DB 140 including lifestyle information 141, dialogue information 142, equipment operation setting information 143, specific report information 144, abnormality determination method information 145, and secret information 146 is stored in one area of ​​the memory unit 18 (Figure 2) of the cloud server 1. Lifestyle information 141 includes, for example, planned wake-up / wake-up time, planned meal / meal time (meal start time / meal end time), meal content (menu, calories, etc.), planned activity / activity record, planned bedtime / bedtime, planned bath time (entry / exit) / bath time (entry / exit), etc. The dialogue information 142 is voice data or text data of the contents of the conversation (both parties) in which the watching robot 2 responds to a question from the user U. If there is no limit to the storage capacity, the dialogue information 142 may include videos or images including the actions of the user U captured by the watching robot 2 during the dialogue with the user U. The device operation setting information 143 includes, for example, detection settings and sensitivity settings of a bathroom fall sensor, sensitivity settings of an odor sensor, detection settings and sensitivity settings of a fire alarm, operation settings of a television, recording settings of a video, operation settings and temperature settings of an air conditioner, operation settings and temperature settings of a refrigerator, operation settings of a washing machine, operation settings of a bathroom dryer, operation settings of an electric kettle, etc. Specific report information 144 is report information from a specific device, such as contact information and a message to notify the fire department when a bathroom fall sensor detects a person's fall in the bathroom, for example. The abnormality determination method information 145 is information on a determination process or procedure for a device such as a bathroom fall sensor to determine an abnormality and issue a specific notification. The secret information 146 is information stored based on a specific management attribute designated by the user U, and when the specific management attribute is, for example, a secret conversation, the information is not disclosed to others when the user U has a secret conversation with the watching robot 2. Specifically, for example, the user U's monologue, diary, feelings at that time, etc. are not output to anyone other than the user U, cannot be accessed, and are stored in the DB 140 as information that cannot be erased even by the user U.

[0024] In addition, in the CPU 11 (Figure 2) of the cloud server 1, as shown in Figure 4, when executing a series of processes related to the living robot care service for user U, an interface control unit 31, a dialogue unit 32, an equipment operation setting information acquisition unit 33, an impact information notification unit 34, a specific notification unit 35, and an abnormality determination method update unit 36 ​​function.

[0025] The interface control unit 31 controls the interface including the dialogue with the user U. For example, the interface control unit 31 treats elderly people (old people) living at home, people who need care, people with disabilities, etc. as a user U (a specified user), and controls the interface including the dialogue between this user U and the monitoring robot 2.

[0026] Specifically, when a specific device among multiple devices present in the home, such as a bathroom fall sensor, attempts to notify the fire department, the interface control unit 31 controls the output to confirm this to the user U. Specifically, for example, when the bathroom fall sensor is about to report a fall in the bathroom to the fire department, the interface control unit 31 can have the monitoring robot 2 speak to ask the user U whether it is okay to report the fall to the fire department, even though the bathroom fall sensor has detected a fall in the bathroom, which may be due to a malfunction caused by noise from the bathroom dryer.

[0027] The interface control unit 31 acquires lifestyle information of the user U based on a dialogue with the user U, and the specific reporting unit 35 performs an abnormality determination based on the lifestyle information. This allows the user U to have his / her own lifestyle information managed, for example, the time the user U normally takes a bath, simply by interacting with the device, thereby improving the accuracy of specific reports. Managing lifestyle information includes, for example, managing the implementation record of "schedule management (calling out)," "music therapy," and "brain games." By managing the lifestyle information of user U in this way, for example, if the bathroom fall sensor detects a fall when the user is not taking a bath and the bathroom dryer is operating, it can be determined that the false detection is due to the noise of the bathroom dryer, preventing false reports in such cases and improving reporting accuracy.

[0028] Furthermore, when the user U converses with another user who has a specific relationship with the user U, such as the elderly user U's child or grandchild, a staff member of the service provider S (service operating company) that provides the monitoring service, or a staff member of small-lot delivery, via the monitoring robot 2, the interface control unit 31 performs predetermined input / output processing (for example, automatically starting the eye camera of the monitoring robot 2 to capture and capture the elderly user U who is the subject of the conversation, and distributes the captured image to the external terminal 6 of the child or grandchild. In addition, the interface control unit 31 may also perform operations such as having the monitoring robot 2 vocally communicate to the user U what has happened since the previous monitoring visit. This can improve the convenience of conversation between the elderly user U and his / her children or grandchildren, and provide topics of conversation, thereby making the user U feel more positive.

[0029] The dialogue unit 32 dialogues with the user U via the watching robot 2. Words and the like used for the dialogue may be realized by a thought process such as AI, or phrases such as answers to questions may be registered in advance in the DB 140 as dialogue information 142, and appropriate words and the like may be read out from the dialogue information 142 during the dialogue with the user U.

[0030] The device operation setting information acquisition unit 33 acquires device operation setting information 143 for each of a plurality of devices that can be connected via a communication network such as network N, such as a bathroom fall sensor, odor sensor, fire alarm, television, video, air conditioner, refrigerator, washing machine, bathroom dryer, electric kettle, etc. The device operation setting information 143 includes, for example, the detection settings and sensitivity settings of a bathroom fall sensor, the sensitivity settings of an odor sensor, the detection settings and sensitivity settings of a fire alarm, the operation settings of a television, the recording settings of a video, the operation settings and temperature settings of an air conditioner, the operation settings and temperature settings of a refrigerator, the operation settings of a washing machine, the operation settings of a bathroom dryer, the operation settings of an electric kettle, etc.

[0031] When the impact information notification unit 34 receives information that an emergency call will be made from a specific device among multiple devices, such as a bathroom fall sensor, and determines by referring to the device operation setting information 143 that the execution of a specific function will affect the operation settings of other devices, it notifies, for example, the user U, other family members K, service provider S, etc., through dialogue or by interrupting the dialogue, based on the control of the interface control unit 31.

[0032] As a specific example, if the impact information notification unit 34 determines that the operation of a bathroom dryer will cause a malfunction of the bathroom fall sensor, it will send a message to the monitoring robot 2 indicating that the operation of the bathroom dryer may be causing the bathroom fall sensor to malfunction, and then have the monitoring robot 2 speak a message to notify the user U.

[0033] As a method of notifying the message, other than vocalizing the message from the watching robot 2, the message may be displayed as a pop-up on the user terminal 4, for example. As a result, the user U can learn, simply by interacting with the monitoring robot 2 or by operating and viewing the user terminal 4, that there is a possibility of malfunction when executing functions that affect each other, such as the bathroom dryer, bathroom fall sensor, and emergency reporting function.

[0034] The device operation setting information acquisition unit 33 acquires the device operation setting information 143 based on a dialogue with the user U under the control of the interface control unit 31 . This allows the user U to configure the operation settings of the devices in his / her home simply by interacting with the watching robot 2. For example, during a conversation between user U and the monitoring robot 2, if user U says, "It's a little hot, isn't it?", the equipment operation setting information acquisition unit 33 acquires the operation settings of the air conditioner and transmits them to the interface control unit 31, and the interface control unit 31 adjusts the settings, such as raising the set temperature included in the operation settings of the air conditioner by one degree.

[0035] The impact information notification unit 34 notifies the user of an impact (such as a possibility of malfunction) between functions occurring between devices based on a dialogue with the user U under the control of the interface control unit 31. Specifically, if, during a conversation between user U and monitoring robot 2, user U says something like "the bathroom dryer is running," and the bathroom fall sensor detects a fall even though user U is not in the bathroom, it will determine that if the bathroom dryer is running, there is a possibility that the noise from the bathroom dryer will cause the bathroom fall sensor to malfunction, and will notify user U that the noise from the bathroom dryer will affect the bathroom fall sensor's fall detection function (possibility of malfunction, etc.). As a result, by simply interacting with the monitoring robot 2, the user U can learn, for example, that the execution of the drying function of the bathroom dryer will affect the operation of the bathroom fall sensor to detect falls, report emergency calls, and the like.

[0036] When the specific notification unit 35 determines that an abnormality has occurred based on the device operation setting information 143 of a plurality of devices, the specific notification unit 35 issues a specific notification, such as an emergency notification to a fire department, under the control of the interface control unit 31. Here, abnormality determination means that when the bathroom fall sensor (specific device) detects a fall, it determines whether the detection information of the bathroom fall sensor is abnormal by referring to the operation information of devices other than the bathroom fall sensor, such as the bathroom dryer, and comments by user U ("I'm fine" or "Help me!"). This makes it possible to improve the accuracy of a specific call (for example, an emergency call triggered by a fall detection by a bathroom fall sensor) based on device operation setting information 143 of a plurality of devices.

[0037] Based on the control of the interface control unit 31, the specific reporting unit 35 further outputs information regarding the abnormality determination (for example, the location of the bathroom where the user U has fallen) to a responder, such as a paramedic C from the fire department, who has responded to the emergency call (specific call). Specifically, when paramedic C of the ambulance team arrives at the scene and says something like "This is the ambulance team," the interface control unit 31 identifies paramedic C and causes the monitoring robot 2 to say something like "He's collapsed in the bathroom at the end of the hallway on the right side!" This makes it possible, for example, to inform a paramedic C of the ambulance that has arrived at the scene where the user U has collapsed.

[0038] The abnormality determination method update unit 36 ​​updates the determination logic and determination method (abnormality determination method) of the abnormality determination by the specific notification unit 35 based on the abnormality determination result, under the control of the interface control unit 31, and based on the acquired information on the need for a specific notification. This allows, for example, if the device determines that there is an abnormality and attempts to make a specific report, and dialogue with the user U reveals that the specific report is unnecessary, the method of determining the abnormality thereafter can be modified.

[0039] Based on the control of the interface control unit 31, the secret information management unit 37 manages the secret conversation based on the specific management attribute when the user U specifies, for example, a secret conversation between the user U and the watching robot 2 and the cloud server 1 acquires the information of the secret conversation via the watching robot 2. Information on secret conversations, such as user U's monologue, diary, and feelings at the time, are stored in a storage means such as a non-volatile memory that is not output to anyone other than user U, cannot be accessed, and cannot be erased even by user U. This allows the user U to develop an attachment to the watching robot 2 through the exchange of secret information that he or she has confessed only to the watching robot 2.

[0040] Next, the operation of the cloud server in this information processing system will be described with reference to FIG. FIG. 5 is a flowchart showing the operation of the cloud server of FIGS. In the cloud server 1, the interface control unit 31 controls the interface including the dialogue between the user U and the watching robot 2 in step S11. In step S12, the device operation setting information acquisition unit 33 acquires the device operation setting information 143 of each of a plurality of control target devices connectable via the network N. In step S13, in response to acquiring information that a specified function will be executed from a specified device among the multiple controlled devices, the impact information notification unit 34 refers to the device operation setting information 143 of each device related to the specified device and determines that the execution of the specified function will affect the operation settings of other devices, and notifies the user U of this through dialogue or by interrupting the dialogue based on the control of the interface control unit 31 in step S11.

[0041] In this manner, according to the operation of the cloud server 1, while the monitoring robot 2 is interacting with the user U, the cloud server acquires that information, and the interface control unit 31 takes into consideration the operating status of the devices operating in the user U's home and notifies the user U that the execution of a specified function of a specified device will affect the operating settings of other devices. Therefore, by simply interacting with the monitoring robot 2, the user can learn when multiple devices in the home are executing functions that affect each other (such as how the execution of a specified function of a specified device will affect and spread to the other devices).

[0042] Various embodiments of the living robot care service in the information processing system of FIG. 1 will be described below. First, a first embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 6 is a diagram showing a first embodiment of the living robot care service in the information processing system of FIG.

[0043] The first embodiment of the living robot care service shown in FIG. 6 shows an overall image of an information processing system that realizes the living robot care service.

[0044] In the home of a user U, various devices including sensors and services (a monitoring robot 2, control target devices 3) are arranged. The control target devices 3 include home electrical appliances (hereinafter referred to as "home appliances") such as toilets, refrigerators, electric kettles, lighting fixtures, air conditioners, and automatic opening and closing curtains. The sensors include, for example, sensors for monitoring the bathroom (falls, submersion in water), excretion sensors, cameras for monitoring presence in the room, sleep vital signs / wake-up sensors, environmental monitoring sensors (temperature, humidity, abnormal sounds, etc.), etc. Each sensor transmits the detected information to cloud server 1 via network N. The sleep vitals / wake-up sensor is one of the devices in a sleep monitoring system that provides a sleep monitoring service. The bathroom monitoring sensor has the function of alerting (contacting) emergency responders if it detects a fall or drowning.

[0045] Services include sleep monitoring services, lunch box delivery services, flat-rate taxi shuttle services, brain game services, and music therapy services. In addition, in the user U's home, a wireless router serves as a base station, and the monitoring robot 2 and the controlled device 3 are connected to a cloud server 1, a service provider S, and each of the external terminals 6 of the family K via a network N such as the Internet.

[0046] When the watching robot 2 talks to the user U or when the user U operates the television remote control while interacting with the user U, the watching robot 2 receives a notification of the operation and transmits the operation information to the cloud server 1. The cloud server 1 can control each device via the network N and obtain information from each device. The cloud server 1 acquires lifestyle information of the user U in advance and manages it using a dashboard D1 and a scheduler.

[0047] When the cloud server 1 acquires abnormality information from the monitoring robot 2 or the controlled device 3, it comprehensively assesses the information, taking into account the influence of other devices, etc., and if it determines that an abnormality has occurred, it sends an abnormality notification to the external terminal 6. When the external terminal 6 receives the abnormality notification, it notifies the service provider S and the family K by issuing a notification (display, voice, etc.) of the received abnormality notification. In addition, not only when an abnormality is detected, but also at a preset time or at regular intervals, the user U is periodically checked for monitoring, and the user U's own condition is notified. According to the first embodiment, a platform can be constructed in which various sensors, home appliances, and services are connected and the monitoring robot 2 serves as a UI (user interface). As a result, a comprehensive living robot care service that provides total care for the user U can be provided.

[0048] Next, a second embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 7 is a diagram showing a second embodiment of the living robot care service in the information processing system of FIG. The second embodiment of the living robot care service shown in FIG. 7 is an example of optimizing the states of the sensors and home appliances through the actions of a watching robot 2 in an information processing system that realizes the living robot care service.

[0049] In this case, the cloud server 1 causes the watching robot 2 to make a sound based on the life information, and thus a dialogue with the user U is started. Specifically, when it is time to go to the toilet based on the daily life information of the user U, the watching robot 2 will say, for example, "It's about time to go to the toilet." Also, if the user U wakes up a little later than his usual wake-up time, the watching robot 2 will say, "Didn't you sleep well last night? Let's do some exercises together." Furthermore, when it is detected that the user U has gotten up from the bed, the watching robot 2 will say, "Good morning, I'll open the curtains." Also, when the temperature in the room exceeds a certain temperature, the watching robot 2 will say, "It's hot, so I'm going to turn on the air conditioner, make sure you drink plenty of water," etc.

[0050] The detection information of various sensors acquired by the cloud server 1 is presented on the dashboard D1, so that users of the external terminal 6, such as the service provider S and family K, can constantly monitor the condition of the user U and the user U's home.

[0051] According to the second embodiment, the watching robot 2 can interact with the user U to deepen the relationship with the user U, and can optimize the states of the sensors and home appliances based on the user U's lifestyle information to suit the user U's current state. Also, feedback can be given to the user U himself. As a result, by communicating with the user U, the monitoring robot 2 can create an optimal environment for the user U's life.

[0052] Next, a third embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 8 is a diagram showing a third embodiment of the living robot care service in the information processing system of FIG. 8 is an example of a processing operation executed by the watching robot 2 when an abnormality in daily life is detected by a sensor in the house in an information processing system that realizes the living robot care service. More specifically, it is an example in which the watching robot 2 detects an abnormality that occurs in the daily life of the user U and reports the abnormality to others on behalf of the user U.

[0053] In the third embodiment, when a bathroom monitoring sensor in the house detects a person's fall in the bathroom, for example, an abnormality detection signal is transmitted to the cloud server 1 from the bathroom monitoring sensor. When the cloud server 1 receives the abnormality detection signal, it controls the watching robot 2 to call out to the person and give a notification (alert). As for messages to people, for example, the monitoring robot 2 may say to the user U who has collapsed, "Are you OK? Shall I call for help?". In addition, the monitoring robot 2 may issue a message to family member K, such as "Go and check on the bathtub," and may control home appliances to blink lights. Furthermore, if family member K is not nearby, the cloud server 1 may send a message or an abnormality notification to an external terminal 6 (for example, a family member's smartphone), and the external terminal 6 may issue an alert (for example, displaying the report content on a screen or outputting it as a voice), thereby informing family member K and the service provider S of the abnormality of user U.

[0054] When the user U responds to the voice of the watching robot 2 by saying "Everything seems fine," the cloud server 1 does not issue a specific report from the specific reporting unit 35, and the interface control unit 31 controls the watching robot 2 to notify the external terminal 6 of family member K of the voice saying "Everything seems fine." Furthermore, when the user U utters a command such as "Call for help!" or "Help me!", the cloud server 1 immediately makes an emergency call to the fire brigade under the control of the interface control unit 31.

[0055] In addition, if there is no response from the user U despite being called out to, the cloud server 1 determines whether or not to treat the detection information of the bathroom fall sensor as abnormal based on the equipment operation setting information of multiple devices in the house, for example by referring to the operation information of the bathroom dryer and lifestyle information 141, and if it determines that there is an abnormality, it makes an emergency call to the fire department based on the control of the interface control unit 31.

[0056] When paramedic C is dispatched from the fire department in response to the emergency call and rushes to the user U's home, the cloud server 1 identifies paramedic C using the words of the paramedic C and the camera in the eye of the monitoring robot 2, and controls the monitoring robot 2 to say something like, "He's collapsed in the bathroom at the end of the hallway on the right side!", thereby communicating the location of user U to paramedic C.

[0057] Thus, according to the third embodiment, an alert is issued after the sensor detects an abnormality, and at the same time, the on-site monitoring robot 2 interacts with the user U, family member K, and emergency medical technician C who has rushed to the scene, thereby enabling smooth and rapid response according to the situation at the scene.

[0058] Next, a fourth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 9 is a diagram showing a fourth embodiment of the living robot care service in the information processing system of FIG. A fourth embodiment of the living robot care service shown in FIG. 9 is an example in which the life of a user U is improved by schedule management in an information processing system that realizes the living robot care service.

[0059] In this fourth embodiment, family member K of user U operates the application screen of the external terminal 6 or the browser of the external terminal 6 to access the cloud server 1, displaying user U's My Page (for family member K), and operates the "Schedule setting, confirmation" button in the My Page, causing the interface control unit 31 of the cloud server 1 to launch the schedule management function (scheduler) of the cloud server 1 and display the scheduler screen D2 (schedule management screen).

[0060] This scheduler screen D2 allows the user U to set and confirm items related to plans among the lifestyle information 141, and by operating the family member K, plans such as time to take medicine, time to exercise, and other plans such as time to watch television (rakugo) can be registered or changed in the schedule. When the schedule of the user U is updated on the scheduler screen D2, the interface control unit 31 of the cloud server 1 controls the watching over robot 2 to notify the user U of the schedule update by speaking.

[0061] Also, the user U or the family K can register an event in the schedule by talking to the watching robot 2, such as "I'll go out at 10 o'clock the day after tomorrow." Furthermore, the user U or the family K can operate a remote control in hand to display the schedule screen on the screen of the controlled device 3, such as a TV, and operate the contents of the schedule, thereby registering or changing an event in the schedule in the same manner as described above. Then, when the schedule is successfully registered or changed, the watching robot 2 will say, "It's been registered in your schedule! Would you like to check it on TV?" or "You've registered an appointment to go out the day after tomorrow at 10 o'clock!", etc., so that the user U can confirm that the appointment has been successfully registered or changed in the schedule. When the user U says "Yes" to the watching robot 2's question, "Would you like to check it on TV?", the interface control unit 31 of the cloud server 1 will display the schedule screen D2 on the TV screen, so that the user U can check the appointment registered in the schedule.

[0062] Then, when the actual scheduled time arrives, the interface control unit 31 of the cloud server 1 controls the watching robot 2 to have the watching robot 2 say something like, "It's time for your medicine / exercise / rakugo...!" to notify the user U.

[0063] Thus, according to this fourth embodiment, the schedule management function (scheduler) of the cloud server 1 can register or change (including delete) the schedule of the user U through dialogue between the family K or the user U himself and the monitoring robot 2. In addition, the user U's daily schedule can be managed on the cloud server 1, and appropriate information can be provided to the user U in a timely manner through dialogue from the watching robot 2.

[0064] Next, a fifth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 10 is a diagram showing a fifth embodiment of the living robot care service in the information processing system of FIG. The fifth embodiment of the living robot care service shown in Fig. 10 is an example in which a schedule management function is linked with various services in an information processing system that realizes the living robot care service. More specifically, it is an example in which the watching robot 2 cooperates with the service to notify the user of the schedule of services such as home delivery and transportation based on a pre-set schedule.

[0065] In this fifth embodiment, by registering plans such as home delivery, flat-rate taxi service, music therapy, and brain games in the scheduler, when the user U and the monitoring robot 2 interact with each other before the scheduled time, for example, when waking up, the monitoring robot 2 will say something like, "Today we'll be going out by taxi starting at 11 o'clock!" In addition, based on past information left in the scheduler, Monitoring Robot 2 can provide topics of conversation, such as, "It's been a week since the delivery guy last came! This and that happened."

[0066] According to this fifth embodiment, by linking the schedule management function with various services, during the dialogue between the user U and the monitoring robot 2, the monitoring robot 2 will vocalize the schedule of services the user U will receive in the future and the contents of services that the user U has received in the past, providing topics of conversation, so that the user U can always experience new activities or reminisce about past topics.

[0067] Next, a sixth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 11 is a diagram showing a sixth embodiment of the living robot care service in the information processing system of FIG. The sixth embodiment of the living robot care service shown in FIG. 11 is a basic example in which the watching robot 2 itself serves as a UI in an information processing system that realizes the living robot care service. More specifically, this is an example in which the monitoring robot 2 utilizes controllable devices such as a TV connected to the network N to realize two-way communication.

[0068] In this sixth embodiment, when the user U and the watching robot 2 are conversing, for example, if the user U says, for example, "I want to call my grandchild," the watching robot 2 will say, "I'll make a call," "I'll turn on the TV," etc. The interface control unit 31 of the cloud server 1 starts up the TV and attempts to make a call to the external terminal 6 of a distant family member K using the videophone function. When connected to the external terminal 6, an image of family member K captured by the camera of the external terminal 6 is displayed on the TV screen and a videophone call is initiated, so that the user U can make a call to family member K (grandchild) simply by interacting with the monitoring robot 2.

[0069] According to the sixth embodiment, predetermined keywords that the user U speaks to the monitoring robot 2 are associated with actions corresponding to those keywords and registered in the cloud server 1. As a result, when the user U speaks a single word during a conversation between the user U and the monitoring robot 2, the monitoring robot 2 becomes a UI and can cause devices in the home to execute a series of operations that the user wishes to perform on behalf of the user U, allowing the user U to spend his or her time in comfort.

[0070] Next, a second functional configuration of the cloud server 1 of an embodiment of the information processing device according to the present invention will be described with reference to Fig. 12. In describing the second functional configuration, the same components as those in Fig. 4 are denoted by the same reference numerals, and description thereof will be omitted.

[0071] In the second functional configuration, in the CPU 11 of the cloud server 1 (see Figures 2 and 4), as shown in Figure 12, an interface control unit 31 and a push notification unit 38 function when executing a series of processes related to the living robot care service for the user U.

[0072] The interface control unit 31 controls the watching robot 2 so as to link the smartphone T operated by the user U with the dialogue with the user U. Specifically, the interface control unit 31 links the monitoring robot 2, which is the human interface, and the smartphone T in a symbiotic relationship (two UIs), so that, for example, the monitoring robot 2 is mainly responsible for calling out to people and giving simple voice instructions from people, while complex operations such as entering schedules and using services are mainly performed by the smartphone T or a remote control, etc. By linking the smartphone T with the monitoring robot 2 in this manner, the user U can use the interface in a way that suits his or her convenience and situation, allowing the user U to receive care services more comfortably.

[0073] The interface control unit 31 starts control of the watching robot 2, including dialogue with the user U, when a predetermined condition is satisfied, such as activating the conversation mode of the watching robot 2 by using a preset specific word or physically touching the watching robot 2. This allows the user U to wake up the watching robot 2 and talk to it at a time of his or her choice.

[0074] The push notification unit 38 acquires the contents to be push-notified to a predetermined user U when at least some of the devices among the plurality of devices execute a predetermined function, and pushes the contents to the user U or an administrator through dialogue based on the control of the interface control unit 31. Administrators include people designated by the user U (e.g., family members, etc.). This allows the monitoring robot 2 to call out to the user U via a push notification.

[0075] Hereinafter, various embodiments of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG. 12 will be described. First, a seventh embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 13 is a diagram showing a seventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG.

[0076] The seventh embodiment of the living robot care service shown in FIG. 13 is a modification of the first embodiment of the living robot care service shown in FIG.

[0077] In the seventh embodiment, as shown in FIG. 13, a smartphone T is placed in the user U's home as a mobile terminal operated by the user U, for example. An application program for smartphones (hereinafter referred to as the "smartphone app") provided by the service provider is installed on smartphone T. The smartphone app works in conjunction with the watching robot 2 under the control of the cloud server 1 to provide information to the user U. The smartphone app also transmits to the cloud server 1 input information and instructions input by the user U through operations on the smartphone app screen.

[0078] By having the monitoring robot 2 coexist with the smartphone T and become the interface for the user U, the user U can be cared for through two user interfaces: the smartphone T and the monitoring robot 2.

[0079] In this information processing system, all operations can be performed by the user U using the smartphone T, but the monitoring robot 2 is mainly responsible for calling out to the user U and receiving simple voice instructions from the user U, and complex operations (such as entering a schedule or using a service) are mainly performed by operating the smartphone T or TV screen (for example, a graphic user interface: GUI or TV remote control).

[0080] The interaction between user U and monitoring robot 2 is reflected in real time on the status displayed on the smartphone screen of smartphone T. The voice exchange between the user U and the monitoring robot 2 is also reflected in chat format on the smartphone screen of the smartphone T, allowing the user U to monitor the process of the exchange in text form.

[0081] Even if the monitoring robot 2 misrecognizes the voice, the user U can check the text that the monitoring robot 2 has recognized on the smartphone screen, and can thus know the reason for the success / failure of the voice recognition.

[0082] In addition, when the user U corrects the incorrect parts of the interaction between the user U and the monitoring robot 2 via the smartphone T, correct / incorrect data is accumulated in the cloud server 1, and the correct / incorrect data is used to machine-learn the conversational AI of the monitoring robot 2 under the control of the cloud server 1, thereby improving the voice recognition accuracy of the monitoring robot 2.

[0083] Next, an eighth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 14 is a diagram showing an eighth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG.

[0084] The eighth embodiment of the living robot care service shown in FIG. 14 is a modification of the second embodiment of the living robot care service shown in FIG.

[0085] In the eighth embodiment, the watching robot 2 has a "push call mode" and a "conversation mode." In the "push call mode" of the monitoring robot 2, the monitoring robot 2 calls out to the user U by push notification and controls home appliances and sensors accordingly. The control of home appliances and sensors is performed according to the time, environment, and various sensor information (sleep vital sensor, etc.).

[0086] In the case of the eighth embodiment, the control content is not just a reading out of data, such as the results of a comparison with past data, but is also digested. Specifically, as shown in the speech bubble in Figure 14, a message is spoken to the user U to provide guidance or advice on his or her next actions. At that time, there is no interaction between the user U and the monitoring robot 2, and no voice is reflected, but the push call content of the monitoring robot 2 is sent from the cloud server 1 to the smartphone T and reflected on the smartphone screen, so the user U can check the push call content by the text displayed on the smartphone screen.

[0087] The “conversation mode” is a function that enables the user U to wake up the watching robot 2 by performing a predetermined action and enjoy a conversation with the watching robot 2. Specifically, when user U takes an action such as speaking a specific word (e.g., Mr. Robot) or physically touching the monitoring robot 2, the cloud server 1 detects the specified action and controls the monitoring robot 2 to activate the "conversation mode" of the monitoring robot 2.

[0088] After activating the "conversation mode," the user U's vocal instructions (e.g., saying "open the curtains") are recognized by the monitoring robot 2, and the curtains are opened under control of the cloud server 1. At the same time, the monitoring robot 2 responds by saying, "Good morning, I'll open the curtains," etc., and if the "conversation mode" is activated later than usual, the monitoring robot 2 makes push calls to the user U, saying, "Didn't you sleep better than usual last night?", "Let's do some exercises together," etc., and continues the conversation with the user U.

[0089] In addition, if the temperature detected by a temperature sensor installed in the room is higher than usual, the monitoring robot 2 will say to the user U, "It's hot, so I'll turn on the air conditioner. Make sure you drink plenty of water," etc. The air conditioner is operated under the control of the cloud server 1.

[0090] Furthermore, when a predetermined time has passed since the user U last went to the toilet, or when a certain time arrives, the watching robot 2 calls out to the user U saying, "It's about time to go to the toilet." The above series of exchanges will be reflected on the chat screen displayed on Smartphone T when "conversation mode" is activated. As a result, the user U can say "Open the curtains" in the text displayed on the chat screen of the smartphone T, causing the monitoring robot 2 to activate "conversation mode", after which the user U can check the content of the conversation with the monitoring robot 2 and the operating status of the home appliances in the room. Also, on the dashboard D1, an administrator or the like can monitor the status of each sensor as well as the content of the conversation between the user U and the operating status of the home appliances in the room.

[0091] According to the eighth embodiment, the monitoring robot 2's "push call mode" allows it to call out to the user U and check the user U's condition based on the user U's response, and the monitoring robot 2 can also optimize the state of each sensor and home appliance in the room to a state preferred by the user U. Furthermore, according to the eighth embodiment, a "conversation mode" is provided, so that the user U can wake up the watching robot 2 by uttering specific words or the like and have a conversation with it. When the user U activates the "conversation mode", the watching robot 2 determines that a conversation request has arisen, and will speak to the user U, for example, saying, "Didn't you sleep better than usual yesterday?" or "Let's do some exercises together", and provide feedback to the user U himself / herself, so that the user U can enjoy a conversation with the watching robot 2 whenever he / she wants to chat.

[0092] Next, a ninth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 15 is a diagram showing a ninth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG.

[0093] The ninth embodiment of the living robot care service shown in FIG. 15 is a modification of the third embodiment of the living robot care service shown in FIG.

[0094] In this ninth embodiment, in addition to the monitoring robot 2 as an interface for the user U, a smartphone T is provided, and when the monitoring robot 2 detects an abnormality in daily life, the detected content is notified to the smartphone T and displayed on the application screen, and depending on the content of the abnormality, the smartphone T's own alarm function such as a warning sound, a warning light, and vibration is used to notify the user U of the abnormality.

[0095] The monitoring robot 2 also notifies of the abnormality with warning sounds and movements. The monitoring robot 2 rushes to the area where the abnormality was detected and connects to the outside world by calling out to the person, taking pictures with a camera, and communicating with the outside world by communication means such as a TV phone. Specifically, the monitoring robot 2 heads to the location of the sensor that detected the abnormality or the source of the abnormal sound (scream, sound of something falling, sound of glass breaking) by autonomous driving using a camera and LiDAR, and uploads on-site information such as audio and images acquired at the site to the cloud server 1. If it is confirmed that there is no malfunction of the sensor or that there is no problem with the user U himself, the abnormality detection action ends there. On the other hand, if an abnormal state occurs, the watching-over robot 2 notifies at least one of the family, the service operating company, and the security company, and monitors the state until someone arrives. The monitoring robot 2 transmits the robot's own location (location information) and behavioral history, etc. to the smartphone T via the cloud server 1, so that the user U can always check the location and behavior, etc. of the monitoring robot 2 on the app screen of the smartphone T.

[0096] Next, a tenth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 16 is a diagram showing a tenth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG.

[0097] The tenth embodiment of the living robot care service shown in FIG. 16 is a modification of the fourth embodiment of the living robot care service shown in FIG.

[0098] In this tenth embodiment, in addition to the monitoring robot 2 as an interface for the user U, a smartphone T is provided, and the schedule management function of the cloud server 1 manages the user U's schedule on the app screen of the smartphone T or on the screen of a family member's smartphone R.

[0099] This schedule management function manages the user U's daily schedule on the application screen of the smartphone T, and the monitoring robot 2 and the smartphone T notify the user U at the start and end times of events set in the schedule, allowing the user U to go about their daily life according to the set schedule.

[0100] Schedules and other information can be entered manually into the smartphone T itself, or through a television, a family member's smartphone R, or conversation with the monitoring robot 2. Input methods include voice input and manual input using the touch or switch parts of the robot. For users U who have difficulty inputting information into their smartphones, the monitoring robot 2 will start up the television and guide them through the television input.

[0101] In addition, the schedule is uploaded to the cloud server 1, and by transmitting the schedule from the cloud server 1 to the family's smartphone R or the like, the user U can share (input / check) his / her own schedule with the family. Thus, according to the tenth embodiment, the user U's schedule can be managed by the monitoring robot 2 and the smartphone T, thereby improving the user U's life.

[0102] Next, an eleventh embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 17 is a diagram showing an eleventh embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG.

[0103] The eleventh embodiment of the living robot care service shown in FIG. 17 is a modification of the fifth embodiment of the living robot care service shown in FIG.

[0104] In addition to the schedule management function, the eleventh embodiment also has a linking function for linking the schedule with various services. With this collaboration function, calendar appointments, push reminder notifications from the monitoring robot 2, etc. are notified to the smartphone T via the cloud server 1, and the status of each is displayed on the application screen of the smartphone T, so that the user U can check his or her own schedule from the status displayed on the application screen of the smartphone T. In addition to smartphone T, by notifying user U's schedule to, for example, a television or a family member's smartphone R, user U can check his or her schedule on the television as well. Furthermore, family members can check User U's schedule for the day on their own smartphone screens.

[0105] As described above, according to the eleventh embodiment, by providing a linking function for linking a schedule with various services, the schedule of the user U can be shared between the user and his / her family.

[0106] Next, a twelfth embodiment of the living robot care service in the information processing system of FIG. 1 will be described with reference to FIG. FIG. 18 is a diagram showing a twelfth embodiment of a living robot care service in an information processing system including a cloud server having the second functional configuration of FIG.

[0107] The twelfth embodiment of the living robot care service shown in FIG. 18 is a modification of the sixth embodiment of the living robot care service shown in FIG.

[0108] This twelfth embodiment is a basic configuration example in which a monitoring robot 2 and a smartphone T each serve as a UI for a user U. In this case, when the user U speaks to the monitoring robot 2, for example as shown in Figure 18, saying, "I want to call my grandchild," the monitoring robot 2 responds to the user U by saying, "I'm going to call you, so please have your smartphone ready," and transmits an instruction to make a call to the cloud server 1. The cloud server 1 then controls the outgoing call to a family member's smartphone R that has been registered in advance on the smartphone T, and a phone call or video call is made from the user U's smartphone T to the family member's smartphone R. This allows the user U's current appearance, state, surrounding situation, etc. to be displayed on the family member's smartphone R. In addition to telephone calls between the user U's smartphone T and the family member's smartphone R, it is also possible to hold internet telephone calls / web meetings using SNS on tablets, PCs, etc.

[0109] Thus, according to the 12th embodiment, the monitoring robot 2 and the smartphone T each become the UI of the user U, so that when the user U speaks to the monitoring robot 2 saying, for example, "I want to call my family," the user U can make a call from the smartphone T to the family's smartphone R without having to operate the smartphone T directly, so that even a user U who is unsure about operating the smartphone T can easily call their family members.

[0110] Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the scope that can achieve the object of the present invention are included in the present invention.

[0111] For example, the hardware configuration shown in FIG. 2 is merely an example for achieving the object of the present invention, and is not particularly limited.

[0112] In addition, the functional block diagram shown in Fig. 4 is merely an example and is not particularly limited. In other words, it is sufficient that the information processing device is provided with a function capable of executing the above-mentioned series of processes as a whole, and the functional blocks and databases used to realize this function are not particularly limited to the example in Fig. 4.

[0113] Furthermore, the locations of the functional blocks and databases are not limited to those shown in FIG. 4 and may be arbitrary. Furthermore, one functional block and database may be configured as a single piece of hardware, a single piece of software, or a combination thereof.

[0114] When the processing of each functional block and database is executed by software, the program constituting the software is installed into a computer or the like from a network or a recording medium. The computer may be a computer built into dedicated hardware, or may be a computer capable of executing various functions by installing various programs, such as a cloud server, a general-purpose smartphone, a personal computer, or the like.

[0115] A recording medium containing such a program is not only composed of removable media that is distributed separately from the device body in order to provide each user with the program, but also composed of a recording medium that is provided to each user in a state in which it is pre-installed in the device body.

[0116] In this specification, the steps of describing a program to be recorded on a recording medium include not only processes that are performed chronologically according to the order, but also processes that are not necessarily performed chronologically but are executed in parallel or individually.

[0117] In addition, in this specification, the term "system" refers to an overall device that is composed of a plurality of devices, a plurality of means, etc. In the above-described embodiment, examples of other users (related parties) to user U (a specific user) include family members (children, grandchildren, etc.) and service providers S (a service operating company and its personnel, etc.), but the examples may include other users.

[0118] In summary, the information processing apparatus to which the present invention is applied is sufficient if it has the following configuration, and can take various different embodiments. That is, an information processing device to which the present invention is applied (for example, the cloud server 1 in FIG. 4 ) An interface control unit (e.g., the interface control unit 31 in FIG. 4 ) that controls an interface including a dialogue with a predetermined user (e.g., the user U in FIG. 4 ); a device operation setting information acquisition means (such as device operation setting information acquisition unit 33 in FIG. 4 ) for acquiring device operation setting information (such as detection settings and sensitivity settings of a bathroom fall sensor, sensitivity settings of an odor sensor, detection settings and sensitivity settings of a fire alarm, operation settings of a television, recording settings of a video, operation settings and temperature settings of an air conditioner, operation settings and temperature settings of a refrigerator, operation settings of a washing machine, operation settings of a bathroom dryer, operation settings of an electric kettle, etc.) of a plurality of devices (such as a bathroom fall sensor, an odor sensor, a fire alarm, a television, a video, an air conditioner, a refrigerator, a washing machine, a bathroom dryer, an electric kettle, etc.) that can be connected via a communication network (such as network N in FIG. 1 ); an impact information notification means (e.g., impact information notification unit 34 of FIG. 4 ) for notifying a user (e.g., user U, other family member K, service operator S of FIG. 4 ) of the fact (e.g., that the operation of the bathroom dryer may be causing a malfunction of the bathroom fall sensor) through the dialogue or by interrupting the dialogue under control of the interface control means (e.g., interface control unit 31 of FIG. 4 ) in response to obtaining information that a predetermined function (e.g., an emergency call) will be executed from a predetermined device (e.g., a bathroom fall sensor) among the plurality of devices, and then referring to the device operation setting information and determining that the execution of the predetermined function will affect the operation settings of other devices (e.g., it is determined that the operation of the bathroom dryer will cause a malfunction of the bathroom fall sensor); Equipped with. This allows the user to know when multiple devices (such as a bathroom dryer, bathroom fall sensor, emergency call function, etc.) are executing functions that affect each other simply by interacting with them.

[0119] In the information processing device (for example, cloud server 1 in FIG. 4 ), The device operation setting information acquisition means (e.g., the device operation setting information acquisition unit 33 in FIG. 4) acquires operation setting information of the device based on a dialogue with the specified user (e.g., user U in FIG. 4) under the control of the interface control means (e.g., the interface control unit 31 in FIG. 4). This allows the user to configure the operation of the device simply by interacting with the device.

[0120] In the information processing device (for example, cloud server 1 in FIG. 4 ), The impact information notification means (e.g., the impact information notification unit 34 in Figure 4) makes the notification based on a dialogue with the specified user (e.g., user U in Figure 4) under the control of the interface control means (e.g., the interface control unit 31 in Figure 4). As a result, a user (such as user U in Figure 4) can learn that the execution of a specified function (such as a drying function) of a specified device (such as a bathroom dryer) affects the operation (fall detection, emergency reporting, etc.) of other devices (such as a bathroom fall sensor) simply by interacting with the monitoring robot 2.

[0121] In the information processing device (for example, cloud server 1 in FIG. 4 ), The interface control means (e.g., interface control unit 31 in FIG. 4, etc.) controls the output to confirm to a specified user (e.g., user U in FIG. 4, etc.) when a specific device among the multiple devices (e.g., a bathroom fall sensor, etc.) is about to make a specific report (e.g., a report to the fire department, etc.). Specifically, for example, when reporting a fall in the bathroom to the fire department, the user U can confirm this.

[0122] In the information processing device (for example, cloud server 1 in FIG. 4 ), a specific notification means (e.g., specific notification unit 35 in FIG. 4 ) that makes the specific notification (e.g., an emergency notification to a fire department) under the control of the interface control means (e.g., interface control unit 31 in FIG. 4 ) when an abnormality is determined based on the device operation setting information of the plurality of devices (e.g., determining whether or not to treat the detection information of a bathroom fall sensor as abnormal while also referring to the operation information of a bathroom dryer), Equipped with. This makes it possible to improve the accuracy of specific reports based on the device operation setting information of multiple devices.

[0123] In the information processing device (for example, cloud server 1 in FIG. 4 ), The specific notification means (e.g., specific notification unit 35 in FIG. 4 ) further outputs information regarding the abnormality determination to a responder (e.g., a paramedic from a fire station) who has responded to the specific call (e.g., an emergency call) based on the control of an interface control means (e.g., interface control unit 31 in FIG. 4 ). This makes it possible to provide an arriving ambulance with information regarding where a specific user (eg, user U in FIG. 4) has collapsed.

[0124] In the information processing device (for example, cloud server 1 in FIG. 4 ), an abnormality determination method update unit (e.g., an abnormality determination method update unit 36 ​​in FIG. 4 ) that updates the method of abnormality determination (e.g., a determination logic or a determination method) by the specific notification unit (e.g., the specific notification unit 35 in FIG. 4 ) based on the information on necessity of the specific notification acquired under the control of the interface control unit (e.g., the interface control unit 31 in FIG. 4 ) for the determination result of the abnormality determination; It further comprises: This allows, for example, if the device determines that there is an abnormality and attempts to make a specific report, and then, through dialogue with the user, it becomes clear that the specific report is unnecessary, the method of determining the abnormality thereafter can be modified.

[0125] In the information processing device (for example, cloud server 1 in FIG. 4 ), The interface control means (e.g., the interface control unit 31 in FIG. 4, etc.) acquires lifestyle information of the predetermined user based on a dialogue with the predetermined user, The specific notification means (for example, the specific notification unit 35 in FIG. 4) performs the abnormality determination based on the life information. This allows the user to manage their own lifestyle information (for example, the time when user U usually takes a bath) simply by interacting with the device, improving the accuracy of specific reports. For example, if the bathroom fall sensor detects a fall when the user is not taking a bath, if the bathroom dryer is operating, it can be determined that this is a false detection due to the noise of the bathroom dryer, and such a report can be prevented.

[0126] In the information processing device (for example, cloud server 1 in FIG. 4 ), The interface control means (e.g., interface control unit 31 in FIG. 4, etc.) performs predetermined input / output processing (e.g., automatically starting the eye camera of the monitoring robot 2 to capture and photograph the elderly user U, and distributing the photograph to the other user's terminal (e.g., external terminal 6 in FIG. 4, etc.), speaking to user U about events that have occurred since the last monitoring visit, etc.) when the specified user (e.g., user U in FIG. 4, etc.) interacts with another user who has a specific relationship with the specified user (e.g., a child or grandchild of elderly user U, a staff member of a service management company that provides a monitoring service, a staff member of a small-lot delivery company, etc.). This can improve the convenience of conversations with other users and provide topics of conversation, making the user feel more positive.

[0127] In the information processing device (for example, cloud server 1 in FIG. 4 ), The system further includes a specific information management means (e.g., secret information management unit 37 in FIG. 4 ) for managing the specific information (e.g., secret conversation, etc.) based on a specific management attribute (e.g., a secret conversation between user U and the monitoring robot 2 only) specified from the specific user (e.g., user U in FIG. 4 ) based on the control of the interface control means (e.g., interface control unit 31 in FIG. 4 ), in response to acquiring the specific information (e.g., information about a secret conversation, etc.) based on the specific management attribute. Information about secret conversations, such as user U's monologue, diary, and feelings at the time, are not output (cannot be accessed) to anyone other than user U, and are stored in a storage means such as a non-volatile memory that cannot be erased even by user U. This allows the user U to develop an attachment to the watching robot 2 through the exchange of secret information that he or she has confessed only to the watching robot 2.

[0128] The interface control means (e.g., the interface control unit 31 in FIG. 12) controls the interface (e.g., the robot 2 in FIG. 12) so as to link a mobile terminal (e.g., the smartphone T in FIG. 12) operated by the predetermined user to a dialogue with the predetermined user (e.g., the user U in FIG. 12). It is possible.

[0129] a push notification means (e.g., the push notification unit 38 in FIG. 12 ) that acquires content to be push-notified to the predetermined user when at least some of the plurality of devices execute a predetermined function, and push-notifies the predetermined user or an administrator of the content through the dialogue under the control of the interface control means; Further comprising: It is possible.

[0130] The interface control means (e.g., the interface control unit 31 in FIG. 12) starts the control of the interface (e.g., the robot 2 in FIG. 12), including the dialogue with the predetermined user (e.g., the user U in FIG. 12), when a preset condition (a specific word or a physical touch on the robot 2 activates the conversation mode of the robot 2) is satisfied as a trigger. It is possible.

[0131] In the information processing method executed by the information processing device (for example, the cloud server 1 in FIG. 4 ), An interface control step (e.g., step S11 in FIG. 5) for controlling an interface including a dialogue with a predetermined user (e.g., user U in FIG. 4), A device operation setting information acquisition step (e.g., step S12 in FIG. 5) of acquiring device operation setting information of each of a plurality of devices connectable via a communication network; an impact information notifying step (e.g., step S13 in FIG. 5 ) of notifying a user of the effect by the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined by referring to the device operation setting information that the execution of the predetermined function will affect the operation settings of other devices in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices; Includes. This allows the user to know when multiple devices (such as a bathroom dryer, bathroom fall sensor, emergency call function, etc.) are executing functions that affect each other simply by interacting with them.

[0132] In a computer (such as the CPU 11 of the cloud server 1 in FIG. 4), An interface control step (e.g., step S11 in FIG. 5) for controlling an interface including a dialogue with a predetermined user (e.g., user U in FIG. 4), A device operation setting information acquisition step (e.g., step S12 in FIG. 5) of acquiring device operation setting information of each of a plurality of devices connectable via a communication network; an impact information notifying step (e.g., step S13 in FIG. 5 ) of notifying a user of the effect by the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined by referring to the device operation setting information that the execution of the predetermined function will affect the operation settings of other devices in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices; The control process includes: This allows the user to know when multiple devices (such as a bathroom dryer, bathroom fall sensor, emergency call function, etc.) are executing functions that affect each other simply by interacting with them. [Explanation of symbols]

[0133] 1···Cloud server, 2···Monitoring robot, 3···Controlled device, 4···User terminal (e.g., smartphone or remote control), 5···Emergency team terminal, 6···External terminal (e.g., smartphone), 31···Interface control unit, 32···, 33···Device operation setting information acquisition unit, 34···Impact information notification unit, 35···Specific notification unit, 36···Abnormality determination method update unit, 37···Confidential information management unit, 38···Push notification unit, K···Family (children, grandchildren, etc.), N···Network, S···Service provider (service operating company, etc.), U···User

Claims

1. an interface control means for controlling an interface including a dialogue with a predetermined user; a device operation setting information acquiring means for acquiring device operation setting information of each of a plurality of devices connectable via a communication network; an impact information notifying means for, in response to receiving information indicating that a predetermined function is to be executed from a predetermined device among the plurality of devices, referring to the device operation setting information and determining that the execution of the predetermined function will affect operation settings of other devices, notifying a user of that effect through the dialogue or by interrupting the dialogue under the control of the interface control means; An information processing device comprising:

2. the device operation setting information acquisition means acquires operation setting information of the device based on a dialogue with the predetermined user under the control of the interface control means; The information processing device according to claim 1 .

3. the impact information notifying means notifies based on a dialogue with the predetermined user under the control of the interface control means; The information processing device according to claim 1 .

4. the interface control means, when a specific device among the plurality of devices is about to make a specific report, executes control regarding an output for confirming that fact to a predetermined user. The information processing device according to claim 1 .

5. a specific notification means for issuing the specific notification under the control of the interface control means when an abnormality is determined based on the device operation setting information of the plurality of devices; The information processing device according to claim 4 , further comprising:

6. The specific notification means further outputs information regarding the abnormality determination to a responder who responds to the specific notification under the control of the interface control means. The information processing device according to claim 5 .

7. an abnormality determination method update means for updating a method of the abnormality determination by the specific notification means based on information on necessity of the specific notification acquired under control of the interface control means with respect to a result of the abnormality determination; The information processing device according to claim 5 , further comprising:

8. The interface control means acquires lifestyle information of the predetermined user based on a dialogue with the predetermined user, The specific notification means performs the abnormality determination based on the life information. The information processing device according to claim 1 .

9. the interface control means performs a predetermined input / output process when the predetermined user interacts with another user who has a specific relationship with the predetermined user. The information processing device according to claim 1 .

10. a specific information management means for managing the specific information based on a specific management attribute in response to acquisition of the specific information by the specific user under control of the interface control means, The information processing device according to claim 1 .

11. the interface control means controls the interface so as to link a mobile terminal operated by the predetermined user to a dialogue with the predetermined user; The information processing device according to claim 1 .

12. a push notification means for acquiring content to be push-notified to the predetermined user when at least a part of the plurality of devices executes a predetermined function, and for pushing-notifying the predetermined user or an administrator of the predetermined user through the interaction based on the control of the interface control means; The information processing device according to claim 1 , further comprising:

13. the interface control means starts the control of the interface, including the dialogue with the predetermined user, when a preset condition is satisfied as a trigger. The information processing device according to claim 1 .

14. An information processing method executed by an information processing device, an interface control step for controlling an interface including a dialogue with a predetermined user; a device operation setting information acquisition step of acquiring device operation setting information of each of a plurality of devices connectable via a communication network; an impact information notifying step of notifying a user of the effect by the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined by referring to the device operation setting information that the execution of the predetermined function will affect operation settings of other devices in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices; An information processing method comprising:

15. On the computer, an interface control step for controlling an interface including a dialogue with a predetermined user; a device operation setting information acquisition step of acquiring device operation setting information of each of a plurality of devices connectable via a communication network; an impact information notifying step of notifying a user of the effect by the dialogue or by interrupting the dialogue based on control in the interface control step when it is determined by referring to the device operation setting information that the execution of the predetermined function will affect operation settings of other devices in response to obtaining information indicating that a predetermined function will be executed from a predetermined device among the plurality of devices; A program that executes control processing including:

Images