Information processing system, information processing method, and program

Abstract

To realize, for example, smooth communication between a user and an information processing apparatus.SOLUTION: An information processing apparatus comprises: a receiving unit for receiving a command from a user; and a determination unit for determining whether or not a command is able to be executed in a plurality of other information processing apparatuses to be commanded. If it is determined that the command is able to be executed, the plurality of information processing apparatuses substitute the command from the user based on the priority set in advance by the user.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to Information Processing System The present invention relates to an information processing method and a program. [Background technology]

[0002] Currently, various information processing devices are increasingly used in daily life and business. For example, there are cases where multiple information processing devices are installed in a home. As information processing devices, smart speakers that communicate with users, and robots that can communicate and also travel (move) autonomously have been developed and commercialized. In this way, as multiple information processing devices are used in a home or office, Patent Documents 1 to 3 also consider the execution of processing by coordinating between information processing devices. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2003-345435 A [Patent Document 2] JP 2017-52075 A [Patent Document 3] JP 2002-178233 A Summary of the Invention [Problem to be solved by the invention]

[0004] In such fields, it is desirable to facilitate smooth communication between users and various information processing devices including robots.

[0005] The present disclosure provides a method for realizing smooth communication between an information processing device and a user. Information Processing System One of the objects of the present invention is to provide an information processing method and a program. [Means for solving the problem]

[0006] The present disclosure relates to, for example, The information processing apparatus includes at least an instruction target information processing device that is a target of an instruction from a user, a first information processing device, and a second information processing device, The first information processing device and the second information processing device each include User commands When the command is received, the command is executed by the command-receiving information processing device. A decision section that decides whether it is possible or not Equipped with Whether the determination by the determination unit is made by the first information processing device or the second information processing device is determined. Based on pre-defined priorities It is decided based on It is an information processing system. .

[0007] The present disclosure relates to, for example, An information processing method performed in an information processing system including at least an instruction target information processing device that is a target of an instruction from a user, a first information processing device, and a second information processing device, a determination unit included in each of the first information processing device and the second information processing device, when receiving a command from a user, determines whether the command can be executed by the command target information processing device; Whether the determination by the determination unit is to be made by the first information processing device or the second information processing device is determined based on a preset priority. It is an information processing method.

[0008] The present disclosure relates to, for example, A program for causing a computer to execute an information processing method performed in an information processing system including at least an instruction target information processing device that is a target of an instruction from a user, a first information processing device, and a second information processing device, a determination unit included in each of the first information processing device and the second information processing device, when receiving a command from a user, determines whether the command can be executed by the command target information processing device; Whether the determination by the determination unit is to be made by the first information processing device or the second information processing device is determined based on a preset priority. It is a program that causes a computer to execute the information processing method. Effect of the Invention

[0009] According to at least one embodiment of the present disclosure, smooth communication can be performed between an information processing device that is the target of an instruction and a user. The effects described herein are not necessarily limited, and may be any of the effects described in the present disclosure. In addition, the contents of the present disclosure should not be interpreted as being limited by the effects exemplified. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining a usage state of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a configuration of the mobile terminal according to the embodiment. [Diagram 3]FIG. 3 is a block diagram showing the configuration of the robot according to the embodiment. [Figure 4] FIG. 4 is a diagram for explaining a usage state of the information processing system according to the embodiment. [Diagram 5] FIG. 5 is a flow diagram showing a command proxy process executed by the mobile terminal according to the embodiment. [Figure 6] FIG. 6 is a flow diagram showing the first status confirmation process executed by the mobile terminal according to the embodiment. [Figure 7] FIG. 7 is a flow diagram showing the second status confirmation process executed by the mobile terminal according to the embodiment. [Figure 8] FIG. 8 is a diagram for explaining a usage state of the information processing system according to the embodiment. [Figure 9] FIG. 9 is a flow diagram showing an instruction destination change process executed in the mobile terminal according to the embodiment. [Figure 10] FIG. 10 is a diagram for explaining the usage status of the information processing system according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The description will be made in the following order. <1. First embodiment> <2. Second embodiment> <3. Modifications> The embodiments and the like described below are preferred specific examples of the present disclosure, and the contents of the present disclosure are not limited to these embodiments.

[0012] <1. First embodiment> FIG. 1 is a diagram for explaining a usage situation of an information processing system according to an embodiment. In this embodiment, a situation is assumed in which a user X has a mobile terminal 1 in his / her home and is using a robot 2 capable of walking (moving). A communication facility capable of connecting to a communication network C such as the Internet is provided in the home. The communication facility includes a router 3 connected to the communication network C, and an access point 4 connected to the router. The mobile terminal 1 and the robot 2 can communicate with an information processing server 5 and the like connected to the communication network C by wirelessly communicating with the access point 4.

[0013] FIG. 2 is a block diagram showing the configuration of a mobile terminal 1 according to an embodiment. The mobile terminal 1 of this embodiment is a so-called information processing device such as a smartphone or tablet. The mobile terminal 1 executes command proxy processing, which will be described later. However, the device performing command proxy processing is not limited to the mobile terminal 1, and various information processing devices such as a robot, a smart speaker (AI speaker), and a television can be used. The mobile terminal 1 of this embodiment is configured to include a control unit 11, an operation unit 19, a camera 12, a communication unit 14, a GPS (Global Positioning System) receiving unit 15, a touch panel display unit 16, a speaker 17, and a microphone 18.

[0014] The control unit 11 is configured with a CPU (Central Processing Unit) capable of executing various programs, a ROM and RAM for storing various programs and data, and is a part that controls the entire mobile terminal 1. The operation unit 19 is a part that receives operations from the user, such as a power button and a volume button. The camera 12 is provided on the front or rear of the mobile terminal 1, or both, and is a part that captures still images or videos in response to user operations, etc.

[0015] The communication unit 14 is a part that communicates with various external devices, and in this embodiment, is configured to use the Wifi standard in order to communicate with the access point 4. In addition to the above, the communication unit 14 may use short-range communication means such as Bluetooth (registered trademark) and infrared communication, or a portable communication means that can connect to the communication network C via a portable communication network instead of the access point 4.

[0016] The GPS receiver 15 is a position detector that can receive signals from satellites to determine the position (latitude, longitude) of the mobile terminal 1. As a position detector in the mobile terminal 1, in addition to the GPS receiver 15, indoor positioning that detects the position indoors, or storing the surrounding environment as mapping data and capturing an image by the camera 12 to determine the position of the mobile terminal 1 may be used.

[0017] The touch panel display unit 16 is a user interface in which a display unit using liquid crystal, organic electroluminescence, or the like is superimposed on a touch panel using a pressure-sensitive method, a capacitance method, or the like, and is capable of displaying various types of information on the display unit. In addition, various types of input from the user can be received by touching various objects displayed on the display unit. The speaker 17 can notify the user by sound. The microphone 18 can capture sounds around the mobile terminal 1.

[0018] 3 is a block diagram showing the configuration of a robot 2 according to an embodiment. The robot 2 according to this embodiment is a pet-type (dog-type) robot 2 capable of various types of communication with a user. In this embodiment, a walking (moving) robot 2 is taken as an example of a command target of a user X, but the robot 2 is not limited to the robot 2, and various information processing devices capable of receiving commands, such as a non-moving robot, a smart speaker (AI speaker), a television, etc., can be used.

[0019] In this embodiment, the robot 2 is configured to include a control unit 21, an operation unit 22, a camera 23, a contact sensor 24, a communication unit 25, a GPS receiving unit 26, a display unit 27, a speaker 28, a microphone 29, and a plurality of actuators 30a, 30b, etc. Note that the form of the robot 2 is merely an example, and various forms can be adopted.

[0020] The control unit 21 is configured with a CPU capable of executing various programs, a ROM (Read-only Memory) that stores various programs and data, a RAM (Random Access Memory), etc., and controls the robot 2 in an integrated manner. The operation unit 22 is a portion that receives operations from a user, such as a power button. The camera 23 is a sensor provided on the head or the like of the robot 2, which captures and captures images of the surrounding environment. The contact sensor 24 is a sensor provided on the head, abdomen, back, etc. of the robot 2, which is capable of detecting contact by the user.

[0021] The communication unit 25 is a section for communicating with various external devices, and in this embodiment, is configured to use the Wifi standard in order to communicate with the access point 4. As with the mobile terminal 1, the communication unit 25 can use short-range communication means such as Bluetooth (registered trademark) and infrared communication, as well as mobile communication means that can connect to the communication network C via a mobile communication network instead of the access point 4.

[0022] The display unit 27 is a display means such as a liquid crystal or an organic electroluminescence (EL) provided at the eye position on the head of the robot 2, and can express the emotions of the robot 2 by changing the display contents. Other display devices can be provided as the display unit 27, and for example, an LED (Light Emitting Diode) or the like can be provided to notify the outside of various states of the robot 2 (for example, battery state, communication state, etc.). The speaker 28 can notify the user by sound, and can express the emotions of the robot 2 by emitting, for example, a cry. The microphone 18 can capture ambient sounds of the mobile terminal 1, and can, for example, capture commands uttered by the user and execute various processes.

[0023] The actuators 30a, 30b are provided on the joints of the limbs, the neck, the tail, etc. of the robot 2, and are driven by commands from the control unit 21. In particular, by driving the actuators provided on the joints of the limbs, the robot 2 is able to walk (move).

[0024] Fig. 1 shows an environment in which the mobile terminal 1 and the robot 2 described in Fig. 2 and Fig. 3 are used in a home. Conventionally, when using the robot 2 in a home or the like, the user X can communicate with the robot 2 by giving voice commands to the robot 2. However, when the robot 2 is turned off, is in a hibernation state due to a dead battery, or is located in room B and cannot fully receive the voice of the user X, the robot 2 may not be able to respond to a command given by the user X.

[0025] In such a situation, the user X may not understand why the robot 2 did not respond to his / her command, and may give the command again, resulting in a loss of interaction between the user X and the robot 2. One of the objectives of the information processing system of this embodiment is to suppress loss of interaction between the user and various information processing devices with which the user can communicate, such as the robot 2, under such a situation, and to ensure smooth communication.

[0026] Therefore, in the information processing system of this embodiment, a command proxy process is performed on the mobile terminal 1, which is a separate individual other than the robot 2 that is the command target individual, for commands given to the robot 2. This command proxy process will be described below using the situation in FIG. 1 as an example.

[0027] In FIG. 1, User X, a mother, is preparing a meal in Room A (the kitchen), but is unable to check on the situation in Room B (the living room) because her hands are busy. When User X is about to finish preparing the meal, she issues a command to Robot 2 in Room A to call her child. Here, we assume that Robot 2 is named "Robo." User X's voice command is "Robo, call my child." The command is issued to Robot 2 by combining "Robo," which indicates the target of the command, with the command "Call my child." When User X issues the command, Robot 2 is in sleep mode and is in a state where it cannot receive commands.

[0028] Under such circumstances, a command proxy process is being executed in the portable terminal 1 carried by the user X, and it becomes possible to proxy commands issued to the robot 2.

[0029] FIG. 5 is a flow diagram showing a command proxy process executed by the mobile terminal 1 according to the embodiment. In this description, the information processing device including the mobile terminal 1 and the robot 2 may be simply called a "terminal". The command proxy process of this embodiment is a process executed by the mobile terminal 1, and during the execution, the voice input to the microphone 18 is analyzed to determine whether or not a command has been received (S101). In this embodiment, as described above, since the command is composed of a combination of a command target and command contents, it is possible to clearly determine the command target. In addition to this form, a command without a command target may also be used. In that case, it is considered that the command target is determined based on various situations, such as an analysis of the voice content itself, an analysis of the past conversation content, or the surrounding situation captured by the camera 23.

[0030] In the case where the received command target is not limited (S102: No), in this embodiment, a notification is given by voice from the speaker 17 to prompt the user X to re-utter the command (S103). For example, in a space where a plurality of robots 2 exist, when a command without a command target or a command with an ambiguous command target is uttered, the mobile terminal 1 notifies the user X by voice or display such as "The command target cannot be specified. Please specify the command target." Here, a command command with an ambiguous command target refers to, for example, a command uttered without a command target in an environment where the number of terminals that can be command targets is other than one (0 or 2 or more) within a radius of N meters of the mobile terminal 1. Specifically, this corresponds to a command such as "do something" without explicitly uttering the command target in a space where two or more command targets exist.

[0031] On the other hand, when the command target is limited among the received commands (S102: Yes), it is determined whether the command is for the own terminal (in this case, the mobile terminal 1) or for another terminal (S104). If the command is for the own terminal (S104: No), the command is executed in the mobile terminal 1 (S105). On the other hand, if the command is for another terminal (S104: Yes), the communication unit 14 is used to acquire the status of the other terminal (S106). In the case of FIG. 1, when acquiring the status of the robot 2 from the mobile terminal 1, the status of the robot 2 may be acquired directly by both of them communicating wirelessly via the access point 4, or the status of the robot 2 may be uploaded to the information processing server 5, and the mobile terminal 1 may indirectly acquire the status of the robot 2 by accessing the information processing server 5. Alternatively, when the status cannot be acquired directly, the both may be used in combination, such as indirect acquisition.

[0032] After acquiring the status of the other terminal, it is determined whether the other terminal can execute the command (S107). If the command can be executed (S107: Yes), the command will be processed by the other terminal, and the process returns to the beginning of the command proxy process. On the other hand, if the other terminal cannot execute the command (S107: No), it is determined whether the other terminal is in the vicinity of the mobile terminal 1 (S108). The determination of whether the other terminal is in the vicinity can be made by referring to the position information included in the acquired status of the robot 2 and the position information determined by the GPS receiver 15 of the mobile terminal 1. If both pieces of position information are within a predetermined distance, it is determined that the other terminal is in the vicinity (S108: Yes), and the first status confirmation process (S200) is executed. On the other hand, if both pieces of position information are outside the predetermined distance, it is determined that the other terminal is not in the vicinity (S108: No), and the second status confirmation process (S300) is executed.

[0033] The situation in Fig. 1 is a case where a command to the robot 2 is uttered by the user X, but since the robot 2 is in a resting state, it is determined that the command by the user X cannot be executed (S107: No). It is also a case where it is determined that the mobile terminal 1 and the robot 2 are in the vicinity. Therefore, in the situation in Fig. 1, the first status confirmation process (S200) is executed.

[0034] 6 is a flow diagram showing a first state confirmation process (S200) executed by a mobile terminal according to an embodiment. In this embodiment, the state of the other terminal acquired in S106 is classified into three states, namely, (1) a state in which another command is being executed, (2) a state in which the intervention of user X is required to transition the other terminal to a command-receiving state, for example, a sleep state, and (3) a state in which the intervention of user X is not required to transition the other terminal to a command-receiving state, for example, a sleep state. In the first state confirmation process, it is determined which of these three states the other terminal is in (S201).

[0035] (1) When the other terminal is executing another command (S201: executing another command), first, a notification is given to the user X by voice from the speaker 17 of the mobile terminal 1 or by using the touch panel display unit 16 of the mobile terminal 1, asking whether or not to cancel the command (the other command) currently being executed on the other terminal (S202).

[0036] When the user X responds by voice or inputs to the touch panel display unit 16 to interrupt the execution of the other command (S203: Yes), the other command is interrupted in the other terminal and the other terminal is brought into a command-acceptable state (S204). The command-acceptable state means that the other terminal can accept the command of the user X and can execute the command. Then, the command held in the mobile terminal 1, that is, the command received in S101, is transmitted to the other terminal (S205). On the other hand, when the execution of the other command is not interrupted (S203: No), the command from the user X is rejected (S206). Thus, in this embodiment, when the command cannot be executed in the terminal to which the command is to be issued, that is, the other command is interrupted and the command is executed, or when the terminal to which the command is to be issued is inactive or powered off as described later, the received command is rejected. When the command is rejected, the mobile terminal 1 may notify the user X that the command has been rejected and the reason for the rejection.

[0037] 1 shows an example in which the robot 2 is at rest, but if the robot 2 is executing another command when the user X issues a command, the user X is asked whether to interrupt the execution of the other command (S202). If the user X inputs an instruction to interrupt (S203: Yes), the command that has already been issued is transmitted to the robot 2 (S205). Therefore, the user X does not need to repeatedly issue the same command, and can communicate smoothly with the robot 2.

[0038] (2) A state in which the intervention of the user X is required to transition the other terminal to a command-receiving state, in which if the other terminal is in a sleep or powered-off state (S201: sleep, powered-off), the command of the user X is rejected (S206). In this case, even though the user issues a command to the other terminal (robot 2 in the case of FIG. 1), no reaction or response is obtained from the robot 2. Therefore, the mobile terminal 1 may perform a notification process to notify the user X of the status of the robot 2 by voice from the speaker 17 or by display on the touch panel display unit 16.

[0039] (3) In a state in which the intervention of the user X is not required to transition the other terminal to a command-receiving state, here, when the other terminal is in a sleep state (S201: sleep), the mobile terminal 1 communicates with the other terminal that is the target of the command, thereby transitioning the other terminal to a command-receiving state (S207). After transitioning the other terminal to a command-receiving state, the command held in the mobile terminal 1, i.e., the command received in S101, is transmitted to the other terminal (S208). In this case, as in (1) the state in which the other terminal is executing a different command, the user X does not need to repeatedly utter the same command, and it is possible to achieve smooth communication with the robot 2.

[0040] Next, the second state confirmation process (S300), i.e., the case where the other terminal is not in the vicinity of the mobile terminal 1 (S108: No), will be described. FIG. 4 is a diagram for explaining the usage status of the information processing system according to the embodiment. The situation in FIG. 4 shows a case where the robot 2 is in a room B different from the user X, and the other terminal (robot 2) is not in the vicinity of the mobile terminal 1. In such a case, it is possible that the robot 2 cannot receive a command uttered by the user X, and the command of the user X will be wasted, that is, a loss of interaction between the user X and the robot 2 will occur.

[0041] The second status confirmation process (S300), like the first status confirmation process (S200), classifies the status of the other terminal into three states, namely, (1) a state in which another command is being executed, (2) a state in which the intervention of user X is required to transition the other terminal to a command-receiving state, such as a hibernation or power-off state, and (3) a state in which the intervention of user X is not required to transition the other terminal to a command-receiving state, such as a sleep state. In the first status confirmation process, it is determined which of these three states the other terminal is in (S301).

[0042] (1) When the other terminal is executing a different command (S301: executing a different command), first, a notification is given to the user X by voice from the speaker 17 of the mobile terminal 1 or by using the touch panel display unit 16 of the mobile terminal 1 to ask whether or not to cancel the command (different command) currently being executed on the other terminal (S302).

[0043] If there is an input from user X to interrupt the execution of the other command by voice response or input to the touch panel display unit 16 (S303: Yes), the other command is interrupted at the other terminal and the other terminal is made to transition to a command acceptable state (S304). On the other hand, if the execution of the other command is not interrupted (S303: No), the command from the user is rejected (S308).

[0044] After the instruction to interrupt the execution of another command is input, the mobile terminal 1 judges whether the other terminal is a movable body, i.e., a mobile terminal like the robot 2 (S305). This judgment can be made by storing information on the other terminal in advance in the mobile terminal 1, or by acquiring the information when acquiring the state of the other terminal in S106. If it is judged to be a movable body (S305: Yes), the other terminal is moved to the vicinity of the user X, i.e., to the vicinity of the mobile terminal 1 (S306). In the case of FIG. 4, the robot 2 is moved from the room B to the room A where the mobile terminal 1 is located. After the movement is completed, the command held in the mobile terminal 1, i.e., the command received in S101, is transmitted to the other terminal (S307). On the other hand, if the other terminal is not a movable body (S305: No), the command received in S101 is transmitted to the other terminal without moving the other terminal (S307).

[0045] In this way, when the other terminal is a movable object, by moving the other terminal to the vicinity of the mobile terminal 1, i.e., to the vicinity of the user X, and then having the other terminal execute commands from the user X, it becomes possible to appropriately carry out communication between the user X and the other terminal (the robot 2 in the case of FIG. 4) at a close distance.

[0046] (2) If the other terminal is in a sleep state or powered off state (S301: sleep, powered off), the user's command is rejected (S308). In this case, similar to the first status confirmation process, the mobile terminal 1 may perform a notification process to notify the user X of the status of the robot 2.

[0047] (3) When the other terminal is in a sleep state (S301: sleep), the mobile terminal 1 communicates with the other terminal that is the target of the command, thereby transitioning the other terminal to a command-receiving state (S309). Thereafter, by executing the processes of S310 to 312, which are the same as the processes of S305 to 307, if the other terminal is a movable object (S310: Yes), the mobile terminal 1 moves the other terminal to the vicinity of the user X (S311) and then transmits the command received in S101 to the other terminal (S312). On the other hand, if the other terminal is not a movable object (S310: No), the mobile terminal 1 transmits the command received in S101 to the other terminal without moving the other terminal (S312).

[0048] The above describes the information processing system according to the first embodiment. In the information processing system of this embodiment, command proxy processing is performed in an information processing device (mobile terminal 1 in the case of FIG. 1) other than the information processing device that is the target of the command (robot 2 in the case of FIG. 1), thereby making it possible to reduce interaction loss and achieve smooth communication between the user and the information processing device that is the target of the command.

[0049] <2. Second embodiment> Fig. 8 is a diagram for explaining a usage situation of the information processing system according to the embodiment. In the first embodiment, a case where there is one robot 2 as a command target has been explained. In the future, as the robot 2 becomes more widespread, it is conceivable that a plurality of robots 2 will be used in one home. The second embodiment takes such a situation into consideration, and shows a situation where a plurality of robots 2a, 2b are used in a home.

[0050] In the example of FIG. 8, the robot 2a in room A is ready to receive commands, and the robot 2b in room B is inactive. Here, the name of the robot 2a is "Robo a" and the name of the robot 2b is "Robo b". For example, as in the first embodiment, the user X utters to the robot 2b, "Robo b, call my child." The robot 2b is located in room B away from the user X and is inactive, so it cannot process the command of the user X. In the second embodiment in which multiple robots 2a and 2b exist, the robot 2a can execute commands for the robot 2b instead of the robot 2b.

[0051] Fig. 9 is a flow diagram showing an instruction destination change process executed by a mobile terminal according to the embodiment. In the instruction destination change process, in the first status check process (S200) described in Fig. 6 or the second status check process (S300) described in Fig. 7, it is monitored (S351) whether the instruction execution has become impossible in the other terminal that is the instruction destination. Here, the case where the instruction has become impossible to execute means that the instruction cannot be processed in any way in the terminal that is the instruction destination, which corresponds to the case where the user's instruction is rejected in the first status check process (S200) or the second status check process (S300).

[0052] For example, this corresponds to the case where, in the first status confirmation process (S200), it is instructed not to interrupt the execution of another command (S203: No) and the other terminal is in a paused state or the power is off (S201: pause, power off), and where, in the second status confirmation process (S300), it is instructed not to interrupt the execution of another command (S303: No) and the other terminal is in a paused state or the power is off (S301: pause, power off).

[0053] When it is determined that the command cannot be executed at the terminal (other terminal) that is the command destination (S351: Yes), the presence of another terminal that can execute the command is determined (S356). When there are multiple other terminals that can execute the command, the other terminals in the vicinity of the user X, i.e., close to the mobile terminal 1, are selected as candidates. When the command is executed at another terminal, it will be executed at a terminal different from the command of the user X. Therefore, in this embodiment, the user is notified of the existence of the other terminal and whether the command can be executed at the other terminal on behalf of the user X, and the user is confirmed (S358). When an instruction to execute the command at the other terminal is input from the user X (S359: Yes), the user's command is transmitted to the other terminal, and the command is executed at the other terminal (S360). On the other hand, when an instruction not to execute the command at the other terminal is input (S359: No), the user's command is rejected (S357).

[0054] In the example of FIG. 8, the user X utters to the robot 2b, "Robot b, call my child." However, since the robot 2b is inactive, the mobile terminal 1 notifies the user X of the content, "Robot b is inactive. Do you want robot a to take over the process?" (S358). If the user X instructs the robot to take over (S359: Yes), the mobile terminal 1 transmits the command to the robot 2b to the robot 2a (S360). On the other hand, if the user X instructs the robot not to take over (S359: No), the command to the robot 2b is rejected (S357).

[0055] In this way, in the second embodiment in which a plurality of robots 2a and 2b exist, the robot 2a can execute commands for the robot 2b instead of the robot 2b.

[0056] <3. Modifications> Although a number of embodiments of the present disclosure have been specifically described above, the contents of the present disclosure are not limited to the above-described embodiments, and various modifications based on the technical ideas of the present disclosure are possible. Modifications will be described below.

[0057] (When multiple information processing devices exist) FIG. 10 is a diagram for explaining the usage of the information processing system according to the embodiment. In the first and second embodiments, a mobile terminal 1 owned by a user X is used as an information processing device that executes command proxy processing and command destination change processing. Currently, in a home, information processing devices capable of interactive correspondence with a user include not only a mobile terminal 1a but also a television 1b and a speaker 1c (sometimes called a smart speaker, an AI speaker, etc.). In such information processing devices, various controls can be performed by the user X's voice. In an environment in which a plurality of information processing devices are used, it is preferable to execute command proxy processing and command destination change processing in any one of the information processing devices.

[0058] For example, by setting as shown below, it is possible to select one information processing device that executes the command substitution process and the command destination change process.

[0059] (1) The user sets it in advance. By setting the information processing device that executes the command proxy processing and the command destination change processing for the robot 2 by the user himself, it is possible to determine one information processing device that executes the command proxy processing and the command destination change processing.

[0060] (2) Give priority to wearable devices. The reason for giving priority to wearable devices is that they are expected to be close to the user's mouth, making it easier to recognize the user's speech, and are expected to be in a state where the user's conversation is constantly being received. When there are multiple wearable devices, it is possible to select the one that is expected to be closer to the user's mouth depending on the type of wearable device.

[0061] (3) Prioritize an information processing device located closer to the user. It is possible to measure the distance between the information processing device and the user using various sensors such as a GPS receiver, a camera, or an infrared sensor provided in the information processing device, and select the information processing device located closest to the user. Note that, in an information processing device having a microphone, it is also possible to determine that the information processing device that can receive the user's voice more loudly is the information processing device located closest to the user.

[0062] (4) A single information processing device is determined by performing communication between a plurality of information processing devices. At that time, the determination may be made taking into consideration the conditions described in (1) to (3) above.

[0063] (5) One information processing device is determined by the information processing server 5. The information processing server 5 may acquire information on a plurality of information processing devices and determine an information processing device suitable for executing the command proxy processing and the command destination change processing.

[0064] (If there are multiple users) For example, in the first state confirmation process (S200) described in FIG. 6, when the robot 2 is executing another command, a notification is made to ask the user X whether or not to stop the command (another command) currently being executed on another terminal (S202). When there are multiple users, it is considered that the user who requested the other command is different from the user who requested the current command. Therefore, in the notification of S202, it is preferable to notify the user who requested the other command as well. In this case, when receiving a command, it is preferable to recognize the user who requested the command by using speaker recognition using the command utterance of the requesting user, or face recognition using the face image of the requesting user, or both, and to store the user in association with the command. In addition, it is also considered that it is not preferable for the command to be interrupted by another user. Therefore, when interrupting the execution of the other command, the requesting user may be inquired of whether or not to interrupt the execution of the command.

[0065] (Non-voice commands) In the first and second embodiments, voice commands are used as commands to the robot 2. However, commands to the robot 2 may be issued not only by voice commands, but also by user operations using, for example, a remote control device, gestures, etc.

[0066] (Modification of instruction destination change process) In the command destination change process described in Fig. 9, if the command destination terminal cannot execute the command (S351), another terminal capable of executing the command is searched for, and if another terminal is present, the user's command is sent to the other terminal. Instead of this form, in the command destination change process, even if the command destination terminal can execute the command, if another terminal capable of executing the command more quickly exists, the user's command may be sent to the other terminal.

[0067] In the situation of FIG. 8, both the robot 2a and the robot 2b are in a state in which they can execute commands. In this situation, if the user X utters to the robot 2b, "Robo b, what's the weather today?", the robot 2b will execute the command after moving from room B to room A. Meanwhile, the robot 2a in the state in which it can execute commands exists in room A. It is considered that the robot 2a, which is closer to the user X, can execute the command faster than the robot 2b moving from room B to room A to execute the command. Therefore, it is possible to process the command of the user X quickly by sending the command of the user X to another terminal that can execute the command faster. Even in this case, it is preferable to notify the user X of the existence of the other terminal and whether the command can be executed on behalf of the user X at the other terminal, and to confirm with the user X. In addition, when executing the command on behalf of the user, in addition to the form in which the most suitable robot is automatically selected by the mobile terminal 1, the user may select the robot to act as the substitute. For example, the mobile terminal 1 may ask the user, "You have specified robot b, but there is robot a nearby. What do you want to do?" and allow the user to select which robot 2a or 2b to execute the command.

[0068] (Variations of ambiguous commands) In the command proxy process described in FIG. 5, if the command target is not limited, the user is prompted to re-utter the command. In addition, as a command having ambiguity, there is a case where the command target is specified but the command content itself remains ambiguous. For example, a case where the user X utters "Robot, um, bring that, that." corresponds to this case. In such a case, since the command target is clear, there is no problem even if the mobile terminal 1 executes the process (S106) of acquiring the state of the terminal that is the command target in the command proxy process described in FIG. 5. However, it is preferable that the mobile terminal 1 notifies the user X when the user X finishes speaking, and if the command to the terminal itself remains ambiguous even if the user X finishes speaking, it is preferable that the mobile terminal 1 does not transmit the command to the terminal. In addition, in order to determine whether the command is ambiguous, for example, a method of determination such as a case where the reliability score is significantly low and a command outside the acceptable command set is recognized can be considered.

[0069] The present disclosure can also be realized by an apparatus, a method, a program, a system, etc. For example, a program that performs the functions described in the above-mentioned embodiments can be made downloadable, and an apparatus that does not have the functions described in the embodiments can download the program and perform the control described in the embodiments in the apparatus. The present disclosure can also be realized by a server that distributes such a program. Furthermore, the matters described in each embodiment and modified example can be combined as appropriate.

[0070] The present disclosure may also have the following configurations. (1) A receiving unit for receiving an instruction from a user; a determination unit that determines whether or not the command can be executed in a plurality of other information processing devices that are the target of the command; When it is determined that the command can be executed, the information processing apparatuses execute the command on behalf of the user based on a priority level preset by the user. Information processing device. (2) The priority is determined based on whether or not the user has set the priority for each of the information processing devices. An information processing device as described in (1). (3) The priority is higher for wearable devices. An information processing device as described in (1). (4) The closer the wearable device is to the user's mouth, the higher the priority is. (3) An information processing device. (5) The priority is based on the distance to the user. An information processing device as described in (1). (6) The closer to the user, the higher the priority. (5) An information processing device according to the present invention. (7) The priority is based on a result of communication between the plurality of information processing devices. An information processing device according to any one of (1) to (4). (8) The priority is determined by an information processing device that communicates with each of the plurality of information processing devices. An information processing device as described in (1). (9) A receiving unit receives a command from a user, a determination unit that determines whether or not the instruction can be executed in a plurality of other information processing devices that are the target of the instruction; When it is determined that the command can be executed, the information processing apparatuses execute the command on behalf of the user based on a priority level preset by the user. Information processing methods. (10) A receiving unit receives a command from a user, a determination unit that determines whether or not the instruction can be executed in a plurality of other information processing devices that are the target of the instruction; When it is determined that the command can be executed, the information processing apparatuses execute the command on behalf of the user based on a priority level preset by the user. A program that causes a computer to execute an information processing method. [Explanation of symbols]

[0071] 1: Mobile terminal (information processing device) 17: Speaker 1a: Mobile terminal 18: Microphone 1b: TV 19: Operation unit 1c: Speaker 21: Control unit 2 (2a, 2b): Robot 22: Operation unit 3: Router 23: Camera 4: Access point 24: Contact sensor 5: Information processing server 25: Communications department 11: Control unit 26: GPS receiver unit 12: Camera 27: Display 14: Communication unit 28: Speaker 15: GPS receiver 29: Microphone 16: Touch panel display unit 30a, 30b...: Actuators

Claims

1. A system including at least an instruction target information processing device that is the target of an instruction from a user, a first information processing device, and a second information processing device, The first information processing device and the second information processing device each include a determination unit that, when receiving a command from a user, determines whether the command is executable by the command-recipient information processing device; Whether the determination by the determination unit is to be made by the first information processing device or the second information processing device is determined based on a preset priority. Information processing system.

2. The priority is determined based on whether the user has set the first information processing apparatus and the second information processing apparatus. The information processing system according to claim 1 .

3. The priority is higher for wearable devices. The information processing system according to claim 1 .

4. The closer the wearable device is to the user's mouth, the higher the priority is. The information processing system according to claim 3 .

5. The priority is based on the distance to the user. The information processing system according to claim 1 .

6. The closer to the user, the higher the priority.

6. The information processing system according to claim 5.

7. The priority is determined based on a communication result of communication between the first information processing device and the second information processing device. The information processing system according to claim 1 .

8. An information processing server that communicates with the first information processing device and the second information processing device, The priority is determined by the information processing server. The information processing system according to claim 1 .

9. An information processing method carried out in an information processing system including at least an instruction target information processing device that is a target of an instruction from a user, a first information processing device, and a second information processing device, a determination unit included in each of the first information processing device and the second information processing device, when receiving a command from a user, determines whether the command can be executed by the command target information processing device; determining whether the determination by the determination unit is to be made by the first information processing device or the second information processing device based on a preset priority; Information processing methods.

10. A program for causing a computer to execute an information processing method performed in an information processing system including at least an instruction target information processing device that is the target of an instruction from a user, a first information processing device, and a second information processing device, a determination unit included in each of the first information processing device and the second information processing device, when receiving a command from a user, determines whether the command can be executed by the command target information processing device; determining whether the determination by the determination unit is to be made by the first information processing device or the second information processing device based on a preset priority; A program that causes a computer to execute an information processing method.

Images