Remote operation system, remote operation method and remote operation program

Abstract

Provided are a remote operation system, a remote operation method, and a remote operation program that suppress the increase in load caused by a remote operator remotely operating multiple robots in parallel. A remote operation system in which a remote operator remotely operates multiple robots and provides services that can be provided by performing multiple tasks to a service recipient. The remote operation system includes: a task assignment unit that assigns multiple tasks to each of the multiple robots; and a remote operation condition assignment unit that assigns, to each of the multiple robots, remote operation conditions corresponding to the contents of the multiple tasks assigned to each of the multiple robots.

Description

Technical Field

[0001] The present invention relates to a remote operation system, a remote operation method and a remote operation program, and in particular to a remote operation system, a remote operation method and a remote operation program for remotely operating a plurality of robots. Background Art

[0002] An example of such a remote operation system is disclosed in Japanese Patent Gazette No. 5070441. In this example of the remote operation system, a predetermined operation terminal is selected based on a condition list of conditions that an operator who performs remote operation of a robot should satisfy and terminal information including attribute information of the operator. Thus, such an operator is selected and determined from a plurality of remote operator candidates. Summary of the invention

[0003] The inventors of the present application have found the following problems.

[0004] There are tasks that require communication and are difficult to handle with just one robot. For example, there is a service that takes a group photo of the user and the robot.

[0005] In such a service, in the above-mentioned technology, multiple remote operators perform tasks by remotely operating multiple robots assigned to each remote operator, thereby providing the service to users. However, in order to determine the tasks to be performed by each remote operator, communication between the remote operators is required.

[0006] On the other hand, if one remote operator performs remote operations on a plurality of robots to execute a task, the remote operator must perform a plurality of remote operations in parallel, so the load on the remote operator may increase.

[0007] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a remote operation system, a remote operation method, and a remote operation program that suppress an increase in load caused by one remote operator remotely operating a plurality of robots in parallel.

[0008] The present invention provides a remote operation system, in which a remote operator remotely operates multiple robots and provides services that can be provided by performing multiple tasks to service recipients, wherein the remote operation system has:

[0009] a task assigning unit that assigns the plurality of tasks to each of the plurality of robots; and

[0010] The remote operation condition assigning unit assigns, to each of the plurality of robots, a remote operation condition corresponding to the contents of the plurality of tasks assigned to each of the plurality of robots.

[0011] According to such a structure, the remote operation conditions can be determined according to the contents of the multiple tasks assigned, and multiple robots can be remotely operated. Therefore, the overall amount of remote operation is reduced by using the remote operation conditions. Therefore, the increase in load caused by one remote operator remotely operating multiple robots in parallel can be suppressed.

[0012] In addition, the present invention may also be characterized in that the plurality of robots include a first robot and a second robot.

[0013] The plurality of tasks include a first task assigned to the first robot and a second task assigned to the second robot,

[0014] The first robot accepts the remote operation performed by the remote operator according to the remote operation condition, communicates with the service recipient, and performs the first task.

[0015] The second robot performs the second task through autonomous control according to the remote operation condition.

[0016] According to such a structure, at least a part of the second task can be performed by the autonomous control of the second robot. Therefore, at least a part of the remote operation of the second robot by the remote operator can be omitted. Therefore, the remote operator can centrally perform the remote operation of the first robot, which can reduce the load of the remote operation.

[0017] In addition, the first robot may communicate with the service recipient at a position where a combined photograph of the service recipient and the first robot can be taken through remote operation by the remote operator.

[0018] The second robot takes the collective photo using a camera device through autonomous control.

[0019] According to such a structure, it is possible to take a group photo by autonomous control. Therefore, at least a part of the remote operation of the second robot can be omitted. Therefore, the remote operator can centrally perform the remote operation of the first robot, which can reduce the load of the remote operation. Thus, it is possible to provide a service of taking a group photo of the service recipient and the first robot while communicating with the service recipient.

[0020] In addition, it may also be characterized in that the second robot adjusts the field angle through autonomous control before using the camera device to take the collective photograph.

[0021] According to such a configuration, the viewing angle can be adjusted by autonomous control, and thus at least a part of the remote operation of the second robot can be omitted.

[0022] In addition, the invention may be characterized in that after the second robot takes the collective photo according to the given remote operation condition, it prints the collective photo through autonomous control to obtain a photo print.

[0023] After the first robot grasps the photograph print by autonomous control, it delivers it to the service recipient through remote operation performed by the remote operator.

[0024] According to such a structure, it is possible to obtain a photo print by autonomous control and deliver the photo print to the service recipient by remote operation. Therefore, at least a part of the remote operation of the second robot can be omitted, and the photo print can be delivered to match the body of the service recipient, such as the height of the height of the service recipient's body, such as the height of the service recipient's body, the height of the service recipient's hand, etc., by remote operation of the first robot.

[0025] In addition, the first robot may be characterized in that the first robot receives remote control from the remote operator and communicates with the service recipient.

[0026] The second robot transports the item to the service recipient through autonomous control according to the remote operation conditions.

[0027] According to such a structure, the article can be transported to the service recipient by autonomous control. Therefore, at least a part of the remote operation of the second robot can be omitted. Therefore, the remote operator can centrally perform the remote operation of the first robot, which can reduce the load of the remote operation. Thus, it is possible to provide a service of distributing articles to the service recipient while communicating with the service recipient.

[0028] In addition, the first robot may be characterized in that the first robot receives remote control from the remote operator and communicates with the service recipient.

[0029] The second robot moves to the vicinity of the service recipient while holding the container under autonomous control according to the remote operation conditions, and collects the items into the container.

[0030] According to such a structure, it is possible to collect items from the service recipient through autonomous control. Therefore, at least a part of the remote operation of the second robot can be omitted. Therefore, the remote operator can centrally perform the remote operation of the first robot, which can reduce the load of the remote operation. Thus, it is possible to provide a service of collecting items from the service recipient while communicating with the service recipient.

[0031] The present invention provides a remote operation method, which is executed in a remote operation system. In the remote operation system, a remote operator remotely operates multiple robots to provide services that can be provided by performing multiple tasks to a service recipient. The remote operation method comprises:

[0032] The step of assigning the plurality of tasks to each robot of the plurality of robots; and

[0033] A step of assigning, to each of the plurality of robots, a remote operation condition corresponding to the contents of the plurality of tasks assigned to each of the plurality of robots.

[0034] According to such a structure, the remote operation conditions can be determined according to the contents of the multiple tasks assigned, and multiple robots can be remotely operated. Therefore, the overall amount of remote operation is reduced by using the remote operation conditions. Therefore, the increase in load caused by one remote operator remotely operating multiple robots in parallel can be suppressed.

[0035] The present invention provides a remote operation program, which is executed by a computer that acts as a server in a remote operation system. In the remote operation system, a remote operator remotely operates a plurality of robots via the server to provide services that can be provided by performing a plurality of tasks to a service recipient, wherein the remote operation program causes the execution of:

[0036] The step of assigning the plurality of tasks to each robot of the plurality of robots; and

[0037] A step of assigning, to each of the plurality of robots, a remote operation condition corresponding to the contents of the plurality of tasks assigned to each of the plurality of robots.

[0038] According to such a structure, the remote operation conditions can be determined according to the contents of the multiple tasks assigned, and multiple robots can be remotely operated. Therefore, the overall amount of remote operation is reduced by using the remote operation conditions. Therefore, the increase in load caused by one remote operator remotely operating multiple robots in parallel can be suppressed.

[0039] The present invention can suppress an increase in load caused by one remote operator remotely operating a plurality of robots in parallel.

[0040] The above and other objects, features and advantages of the present disclosure will be more fully understood from the detailed description given below and the accompanying drawings which are given by way of illustration only and thus should not be considered to limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1This is a schematic diagram showing the structure of the remote operation system according to the first embodiment.

[0042] Figure 2 This is a side view showing a state in which the robot operator is communicating with a communication partner via the robot.

[0043] Figure 3 is a block diagram showing the system configuration of the robot.

[0044] Figure 4 This is a block diagram showing the system configuration of another robot.

[0045] Figure 5 is a block diagram showing the system configuration of the operation terminal.

[0046] Figure 6 is a block diagram showing the system configuration of the management server.

[0047] Figure 7 This is a flowchart of the operation of the first example of the remote operation system according to the first embodiment.

[0048] Figure 8 This is a schematic diagram showing the operation of the first example of the remote operation system according to the first embodiment.

[0049] Fig. 9 This is a flowchart of the operation of the second example of the remote operation system according to the first embodiment.

[0050] Fig.10 This is a schematic diagram showing the operation of the second example of the remote operation system according to the first embodiment.

[0051] Fig.11 This is a flowchart of the operation of the third example of the remote operation system according to the first embodiment.

[0052] Fig.12 This is a schematic diagram showing the operation of the third example of the remote operation system according to the first embodiment. DETAILED DESCRIPTION

[0053] Hereinafter, with reference to the accompanying drawings, a specific embodiment to which the present invention is applied will be described in detail. However, the present invention is not limited to the following embodiments. In addition, in order to make the description clear, the following description and the accompanying drawings are appropriately simplified.

[0054] (Implementation Method 1)

[0055] Hereinafter, Embodiment 1 will be described with reference to the drawings.

[0056] First, use Figure 1 to Figure 6 , the structure and operation of the remote operation system of this embodiment are described. Figure 7 to Figure 12, illustrating multiple examples of actions in the remote operation system of this embodiment. In addition, of course, Figure 1 The right-handed XYZ coordinates shown in other figures are convenient for explaining the positional relationship of the components. Usually, the positive direction of the Z axis is the vertical direction, and the XY plane is a horizontal plane, which is common among the figures.

[0057] <Remote Operating System>

[0058] Figure 1 1 is a diagram for explaining a remote operation system of an embodiment. Figure 1 As shown, the remote operation system 1 of this embodiment is composed of a robot 10a, a robot 10b, an operation terminal 30, and a management server 40. The remote operation system 1 of this embodiment is a remote operation system in which an operator who uses the operation terminal 30 to remotely operate the robot 10a and the robot 10b is connected to the service recipient C50 (see Figure 2 ) communicates with robot 10a and robot 10b.

[0059] Specifically, the robot 10a and the robot 10b are arranged at a position away from the operation terminal 30, and the remote operator M60 can use the operation terminal 30 to remotely operate the robot 10a and the robot 10b. The robot 10a, the robot 10b and the operation terminal 30 are configured to be able to communicate with each other via the management server 40. The robot 10a is provided with a camera A (14) and a display unit 15. The robot 10b is provided with a camera A (14). In addition, the operation terminal 30 is also provided with a camera C (31) and a display unit 32. The service recipient C50 (refer to Figure 2 ) is taken by the camera A (14) of the robot 10a. Then, the image of the service recipient C50 is displayed on the display unit 32 of the operation terminal 30 of the remote operator M60. Similarly, the remote operator M60 who operates the operation terminal 30 is photographed by the camera C (31) of the operation terminal 30. Then, the photographed remote operator M60 is displayed on the display unit 15 of the robot 10a. In addition, the robot 10a and the operation terminal 30 are each equipped with a microphone and a speaker. Therefore, the service recipient C50 and the remote operator M60 can communicate while looking at each other's faces.

[0060] Furthermore, the remote operating system 1 of this embodiment is as follows Figure 2 The configuration shown is such that a camera B (17) for photographing can be used to photograph a service recipient C50 in front of the robot 10a. The configuration and operation of the remote operation system 1 of this embodiment will be described in detail below.

[0061] like Figure 1 As shown, the remote operation system 1 of the present embodiment includes a robot 10a, a robot 10b, an operation terminal 30, and a management server 40. The robot 10a and the robot 10b are arranged at a position away from the operation terminal 30, and are configured to be able to communicate with each other via the management server 40. In addition, the management server 40 may be arranged at a position away from the robot 10a, the robot 10b, and the operation terminal 30, or may be arranged near the operation terminal 30, or may be arranged near the robot 10a and the robot 10b.

[0062] <Robot Structure>

[0063] First, the structure of the robot is described. Figure 1 , Figure 2 As shown, the robot 10a and the robot 10b have a head 11, a body 12, and a moving unit 13. A camera A (14) and a display unit 15 are provided on the head 11 of the robot 10a. A camera A (14) is provided on the head 11 of the robot 10b. The camera A (14) is a camera used for the main actions of the robot 10a and the robot 10b. For example, it can be used to detect the presence of obstacles when the robot 10a and the robot 10b move, or to search for service recipients. In addition, the camera A (14) is used between the remote operator M60 of the operation terminal 30 and the service recipient C50 (refer to Figure 2 The display unit 15 displays a remote operator M60 who performs remote operation using the operation terminal 30.

[0064] The body 12 is provided with an arm 16. A gripper 26 is provided at the front end of the arm 16 of the robot 10a. A camera B (17) is provided at the front end of the arm 16 of the robot 10b. The camera B (17) is used to provide a service to the customer C50 (see Figure 2 ) is a camera that takes photos of the image. The position of the gripping portion 26 can be adjusted using the arm 16 of the robot 10a. The position of the camera B (17) can be adjusted using the arm 16 of the robot 10b.

[0065] In addition, the body 12 of the robot 10b is provided with a take-out port 18. For example, the robot 10b has a built-in printer (in Figure 1 Not shown in the figure), can be used by C50 (refer to Figure 2 ) after taking the photo, use the printer to print the photo and obtain the photo print. The photo print can be delivered to the service recipient C50 through the take-out port 18.

[0066] The moving unit 13 is a unit for moving the robot 10a and the robot 10b. For example, wheels (not shown) are provided on the lower surface of the moving unit 13, and the robot 10a and the robot 10b can be moved by driving the wheels.

[0067] Next, the system configuration of the robot 10a will be described. Figure 3 It is a block diagram used to illustrate the system structure of a robot. Figure 3 As shown, the robot 10a includes a control unit 20 for controlling the operation of the robot 10a. The mobile unit 13, the camera A (14), the display unit 15, the speaker 21, the microphone 22, the communication unit 23, the arm driving unit 24, and the gripping unit 26 are connected to the control unit 20.

[0068] The mobile unit 13 drives the wheels (not shown) of the mobile unit 13 according to the control signal supplied from the control unit 20. Thus, the robot 10a and the robot 10b can be moved. For example, the control unit 20 can generate a control signal for controlling the mobile unit 13 according to the operation signal sent from the operation terminal 30 (management server 40). In addition, the control unit 20 can also detect obstacles around the robot 10a using information obtained from the camera A (14) or the laser scanner (not shown), and generate a control signal using information related to the detected obstacles. Thus, the control unit 20 can control the mobile unit 13 in a manner that allows the robot 10a to move while avoiding obstacles.

[0069] The control unit 20 transmits the image data acquired from the camera A ( 14 ) to the operation terminal 30 via the communication unit 23 .

[0070] The control unit 20 displays the image data transmitted from the operation terminal 30, that is, the image of the remote operator M60 operating the operation terminal 30, on the display unit 15. In addition, the control unit 20 outputs the sound data transmitted from the operation terminal 30, that is, the voice of the remote operator M60 operating the operation terminal 30, from the speaker 21. In addition, the control unit 20 transmits the sound data acquired by the microphone 22, that is, the voice of the service recipient C50, to the operation terminal 30 via the communication unit 23.

[0071] The arm driving unit 24 is a unit for driving the arm 16, and can be configured using a motor, etc., for example. The arm driving unit 24 drives the arm 16 according to a control signal supplied from the control unit 20. Thus, the arm 16 can be moved. For example, by moving the arm 16, the position of the gripping unit 26 can be adjusted.

[0072] The control unit 20 supplies a control signal to the gripping unit 26. The gripping unit 26 grips the gripped object or stops gripping according to the supplied control signal.

[0073] The communication unit 23 is configured to be communicable with the operation terminal 30 via the management server 40. For example, the communication unit 23 is connected to the management server 40 by wire or wirelessly.

[0074] Next, the system configuration of the robot 10b will be described. Figure 4 is a block diagram used to illustrate the system structure of another robot. Figure 4 As shown, the robot 10b includes a control unit 20 for controlling the operation of the robot 10b. The robot 10b has the same configuration as the robot 10a except that it does not have the display unit 15 and the gripping unit 26 and includes a camera B (17) and a printer 25.

[0075] The control unit 20 transmits the image data acquired from the camera B (17) to the operation terminal 30 via the communication unit 23. At this time, the control unit 20 may selectively transmit the image data acquired from the camera A (14) and the image data acquired from the camera B (17) to the operation terminal 30. In addition, by moving the arm 16, the position of the camera B (17) can be adjusted.

[0076] The control unit 20 is configured to be able to transmit the image data (image data of the service recipient C50) acquired from the camera B (17) to the printer 25. The printer 25 prints a photograph based on the image data acquired from the control unit 20.

[0077] The above-described configurations of the robot 10 a and the robot 10 b are merely examples, and in the remote control system 1 of the present embodiment, the configurations of the robot 10 a and the robot 10 b are not limited to the above-described configurations.

[0078] <Structure of the operation terminal>

[0079] Next, the operation terminal 30 will be described. Figure 1 As shown, the operation terminal 30 is a terminal for the remote operator M60 to remotely operate the robot 10 a and the robot 10 b , and can be configured using, for example, a personal computer, a tablet terminal, or the like. Figure 5 This is a block diagram for explaining the system structure of the operation terminal. Figure 5 As shown, the operation terminal 30 includes a camera C (31), a display unit 32, a speaker 33, a microphone 34, an input unit 35, a communication unit 36, and a control unit 37. The control unit 37 has a function of controlling the operation terminal 30, and the camera C (31), the display unit 32, the speaker 33, the microphone 34, the input unit 35, and the communication unit 36 ​​are connected to the control unit 37.

[0080] The control unit 37 transmits the image data acquired from the camera C (31) to the robot 10a via the communication unit 36. The image data transmitted to the robot 10a is displayed on the display unit 15 of the robot 10a.

[0081] In addition, the control unit 37 displays the image data transmitted from the robot 10a and the robot 10b, such as the image of the service recipient C50, on the display unit 32. In addition, the control unit 37 outputs the voice data transmitted from the robot 10a and the robot 10b, such as the voice of the service recipient C50 standing in front of the robot 10a, from the speaker 33. In addition, the control unit 20 transmits the voice data acquired by the microphone 34, that is, the voice of the remote operator M60 operating the operation terminal 30, to the robot 10a and the robot 10b via the communication unit 36.

[0082] The input unit 35 is a unit for inputting various information for the remote operator M60 to remotely operate the robot 10a and the robot 10b. For example, the input unit 35 can be configured using a touch panel or a keyboard. When the input unit 35 is configured using a touch panel, for example, the remote operator M60 presses an icon displayed on the display unit 32, thereby remotely operating the robot 10a and the robot 10b. In addition, when the input unit 35 is configured using a keyboard, for example, predetermined information is input from the keyboard, thereby remotely operating the robot 10a and the robot 10b.

[0083] The communication unit 36 ​​is configured to be able to communicate with the robot 10a and the robot 10b via the management server 40. For example, the communication unit 36 ​​is connected to the management server 40 by wire or wirelessly.

[0084] The above-described configuration of the operation terminal 30 is an example, and in the remote operation system 1 of the present embodiment, the configuration of the operation terminal 30 is not limited to the above-described configuration.

[0085] <Structure of Management Server>

[0086] Next, the management server 40 will be described. Figure 1 As shown, the management server 40 is a server for managing the robot 10 a and the robot 10 b . The robots 10 a and 10 b and the operation terminal 30 are configured to be able to communicate with each other via the management server 40 . Figure 6 This is a block diagram for explaining the system structure of the management server. Figure 6 As shown, the management server 40 includes a communication unit 41 , a robot motion control unit 42 , and a robot control information storage unit 43 .

[0087] The communication unit 41 communicates with the communication units 23 of the robot 10a and the robot 10b (see Figure 3 , 4 ) and the communication unit 36 ​​of the operation terminal 30 (refer to Figure 5 For example, the communication unit 41 is connected to the communication units 23 of the robot 10a and the robot 10b and the communication unit 36 ​​of the operation terminal 30 by wire or wireless.

[0088] The robot motion control unit 42 is a computing device such as a CPU (Central Processing Unit) and an MPU (Micro Processing Unit) that controls the circuit and the device. The robot control information storage unit 43 is a storage device that stores various data such as programs and robot control information. The robot motion control unit 42 controls the motion of the robot according to the robot control information stored in the robot control information storage unit 43. That is, in the remote operation system 1 of the present embodiment, the types and sequences of the motions of the robots 10a and 10b (that is, the scripts related to the motions of the robots) are predetermined, and such information is pre-stored in the robot control information storage unit 43 as robot control information.

[0089] The robot motion control unit 42 obtains the current state information of the robot 10a and the robot 10b from the robot 10a and the robot 10b. Then, the robot motion control unit 42 refers to the obtained current state information of the robot 10a and the robot 10b and the robot control information stored in the robot control information storage unit 43, and creates information related to the next control content (next script) of the robot 10a and the robot 10b.

[0090] The information on the next control contents of the robot 10a and the robot 10b is determined according to the service provided to the service recipient C50, such as a group photo shooting service, a distribution service, a collection service, and the like.

[0091] The service includes multiple tasks. The collection photo shooting service includes, for example, the task of being the subject and the shooting task. The distribution service includes, for example, the communication task and the delivery task. The recycling service includes, for example, the communication task and the recycling task.

[0092] The multiple tasks can be assigned to the robot 10a and the robot 10b, respectively. The robot motion control unit 42 includes a task assignment unit 42a and a remote operation condition assignment unit 42b. The task assignment unit 42a refers to the current state information of the robot 10a and the robot 10b and the robot control information obtained above, and assigns the multiple tasks to the robot 10a and the robot 10b, respectively. In addition, the remote operator M60 can also input the required information into the operation terminal 30 to assign the multiple tasks to the robot 10a and the robot 10b, respectively.

[0093] The remote operation condition assigning unit 42b assigns remote operation conditions to the robot 10a and the robot 10b respectively according to the plurality of tasks. The remote operation conditions are, for example, remote operation or autonomous control performed by the remote operator M60. The robot 10a and the robot 10b are remotely operated according to the remote operation conditions assigned to them. For example, the robot 10a and the robot 10b can perform tasks through remote operation or autonomous control performed by the remote operator M60.

[0094] The robot motion control unit 42 transmits information on the next control contents (next script) of the created robots 10 a and 10 b to the operation terminal 30 .

[0095] After that, information related to the next control content (next script) of the robot 10a and the robot 10b is displayed on the display unit 32 of the operation terminal 30. For example, the content of the next action recommended for the robot 10a and the robot 10b is displayed on the display unit 32 of the operation terminal 30, and the remote operator M60 can perform remote operations in accordance with the content. In addition, the display unit 32 of the operation terminal 30 displays an icon for the remote operator M60 to select the next action of the robot 10a and the robot 10b, and the remote operator M60 can determine the next action of the robot 10a and the robot 10b by pressing the icon.

[0096] For example, the remote operator M60 presses an icon representing a predetermined service, thereby determining the service to be provided to the service recipient C50. The remote operator M60 presses an icon representing a predetermined task and the robot 10a or the robot 10b, thereby determining the allocation of the predetermined task to the robot 10a or the robot 10b. Furthermore, the remote operator M60 presses an icon representing an action included in the task, thereby enabling the robot 10a or the robot 10b to perform the action. The remote operator M60 presses an icon representing a predetermined service, thereby enabling the robot 10a and the robot 10b to perform the task included in the predetermined service.

[0097] The information on the next action of the robot 10a and the robot 10b (information on the selected scenario) determined by the remote operator M60 is transmitted to the robot 10a and the robot 10b via the management server 40. The robot 10a and the robot 10b perform actions according to the received information on the next action (information on the selected scenario).

[0098] In the remote operation system 1 of the present embodiment, the above-mentioned actions, that is, the acquisition of the current state information of the robot 10a and the robot 10b, the creation of the next control content (next script) of the robot 10a and the robot 10b, and the determination of the next action of the robot 10a and the robot 10b are repeated, so that the robot 10a and the robot 10b can be operated. The management server 40 manages the progress of the control content (script) of the robot 10a and the robot 10b. Furthermore, the remote operation conditions can be determined according to the contents of the multiple tasks assigned to the robot 10a and the robot 10b, respectively, and the robot 10a and the robot 10b can be remotely operated. Therefore, the overall amount of remote operation is reduced by using the remote operation conditions. Therefore, the increase in load caused by a remote operator remotely operating the robot 10a and the robot 10b in parallel can be suppressed.

[0099] In addition, the control contents (scripts) of the robots 10a and 10b can be appropriately changed according to the usage scenarios of the robots 10a and 10b. The control contents (scripts) of the robots 10a and 10b can be changed by rewriting the robot control information stored in the robot control information storage unit 43. In addition, the management server 40 can also be configured to manage a plurality of robots. In addition, the management server 40 can also be configured as the same system as the operation terminal 30.

[0100] <Description of the First Example of Remote Operation System Operation>

[0101] Next, use Figure 7 The flowchart shown and Figure 8 The schematic diagram shown in FIG. 1 illustrates a first example of the operation of the remote control system of the present embodiment. As an example of the operation of the robot 10a and the robot 10b, the robot 10a searches for a service user (search mode), then communicates with the service user (communication mode), and the robot 10b takes a group photo of the robot 10a and the service user (group photo shooting mode). In this example of the operation, a service of taking a group photo of the robot 10a and the service user is provided to the service user.

[0102] First, the robot 10a performs an action for searching for a service recipient (step S1). For example, the remote operator M60 operates the operation terminal 30 to select a search mode script as the action mode of the robot 10a. As a result, the action mode of the robot 10a is set to the search mode. For example, the robot 10a can use the camera A (14) to search for a service recipient. In addition, the robot 10a can expand the range of searching for service recipients by driving around.

[0103] After that, the robot 10a determines whether the service recipient can communicate (step S2). For example, when a person is captured by the camera A (14) of the robot 10a, the robot 10a talks to the person to confirm whether the person can become a communication partner. If the person does not agree to become a communication partner (step S2: No), the robot 10a continues to search for the service recipient (step S1).

[0104] On the other hand, when the person agrees to become a communication partner (step S2: Yes), the remote operator M60 of the operation terminal 30 starts communicating with the person via the robot 10a (step S3). Specifically, the remote operator M60 operates the operation terminal 30 and selects a script of a communication mode as the action mode of the robot 10a. The image of the service recipient C50 in front of the robot 10a is displayed on the display unit 32 of the operation terminal 30 of the remote operator M60. In addition, the image of the remote operator M60 operating the operation terminal 30 is displayed on the display units 15 of the robots 10a and 10b. Therefore, the service recipient C50 and the remote operator M60 can communicate while looking at each other's faces and talking.

[0105] Afterwards, the robot 10a (remote operator M60) confirms with the service recipient C50 whether a group photo including the robot 10a and the service recipient C50 can be taken (step S4). Then, when the service recipient C50 has obtained permission for taking the group photo (step S4: yes), the remote operator M60 starts enjoying the group photo shooting service for the person via the robots 10a and 10b (step S5). Specifically, the remote operator M60 operates the operation terminal 30 and selects a script for the group photo shooting service as a service provided by the robot 10a and the robot 10b. In the script for the group photo shooting service, the task of the subject to be photographed is pre-assigned to the robot 10a, and the task of shooting is pre-assigned to the robot 10b. In addition, the remote operator M60 may also operate the operation terminal 30 to assign the task of the subject to be photographed to the robot 10a, and the task of shooting is pre-assigned to the robot 10b.

[0106] Next, the position, orientation, and posture of the robot 10a are adjusted so that the robot 10a becomes the subject of the group photo including the robot 10a and the service recipient C50 (step S6). Specifically, the robot 10a approaches the service recipient C50 through autonomous control and moves to a position where a group photo including the robot 10a and the service recipient C50 can be taken. Next, the robot 10a accepts remote operation performed by the remote operator M60, and adjusts the distance between the robot 10a and the service recipient C50, the position, orientation, and posture of the robot 10a and the arm 16 of the robot 10a in consideration of the condition of the service recipient C50, the shooting of the group photo, etc. The remote operator M60 performs remote operation in consideration of the characteristics of the service recipient C50, such as the body size of the service recipient C50, so the robot 10a can make appropriate adjustments to the service recipient C50, which is satisfactory. Appropriately, the remote operator M60 can also communicate with the service recipient C50 via the remotely operated robot 10a.

[0107] Next, the robot 10b adjusts the camera B (17) for taking a group photo of the robot 10a and the service recipient C50 (step S7). Specifically, the robot 10b moves to a position where the group photo can be taken by autonomous control, and adjusts the position and field of view of the camera B (17). Alternatively, steps S6 and S7 may be performed in parallel.

[0108] Next, the robot 10b takes a group photo of the robot 10a and the service recipient C50 by autonomous control (step S8). For example, a photo button is displayed on the operation terminal 30, and the remote operator M60 can take a group photo by pressing the photo button.

[0109] When the remote operator M60 takes a plurality of photos in step S7, the remote operator M60 selects a photo to be delivered to the service recipient C50 from the images representing the plurality of photos taken (step S9). At this time, the actions of steps S6 and S7 may be repeated to take a plurality of photos. In addition, when only one photo is taken in step S7, the action of step S9 may be omitted.

[0110] Afterwards, the remote operator M60 operates the operation terminal 30 to instruct the robot 10b to print the photos selected in step S9. The robot 10b uses the printer built into the robot 10b through autonomous control to print the collective photos of the service recipient C50 and obtains the photo print. Then, the robot 10a takes out the photo print from the outlet 18 of the robot 10b and delivers it to the service recipient C50 (step S9). The photo print can also be delivered to the service recipient C50 via the outlet 18. The collective photo shooting service ends (step S10). Specifically, the robot 10b moves to the vicinity of the robot 10a through autonomous control. The robot 10a uses the gripping part 26 of the arm 16 to grip the photo print through autonomous control. The robot 10a delivers the gripped photo print to the service recipient C50 through remote operation performed by the remote operator M60. The remote operator M60 performs the remote operation in consideration of the characteristics of the service recipient C50, such as the body size of the service recipient C50, so the robot 10a can appropriately deliver the service to the service recipient C50, which is satisfactory. In addition, step S9 may be appropriately omitted.

[0111] In addition, if the service recipient C50 does not agree to take a photo of the service recipient C50 in step S4 (step S4: No), the operator of the operation terminal 30 continues the conversation (communication) with the service recipient via the robot 10a (step S11). In this case, the collective photo shooting mode of steps S5 to S10 is omitted.

[0112] By completing the above operations, the communication between the remote operator M60 of the operation terminal 30 and the service recipient C50 via the robot 10a and the robot 10b is completed (step S12). Thereafter, by repeating the operations of steps S1 to S12, the robot 10a (remote operator M60) can further communicate with other service recipients C50.

[0113] In the present embodiment, the actions of the robot 10a and the robot 10b are predetermined based on the robot control information (script) stored in the robot control information storage unit 43 of the management server 40. In addition, the display unit 32 of the operation terminal 30 displays information related to the next control content (next script) of the robot 10a and the robot 10b. For example, the display unit 32 of the operation terminal 30 displays an icon for selecting the next action of the robot 10a and the robot 10b, and the operator can determine the next action of the robot 10a and the robot 10b by pressing the icon. In this way, in the present embodiment, the operator can determine the next action of the robot 10a and the robot 10b by pressing the icon displayed on the display unit 32 of the operation terminal 30. Therefore, it is possible to suppress the operation of the robot by the remote operator M60 from becoming complicated. In addition, the next action of the robot 10a and the robot 10b is performed by the robot 10a or the robot 10b according to the content through remote operation or autonomous control performed by the remote operator M60. Therefore, at least a part of the task can be performed by the autonomous control of the robot 10a or the robot 10b. Therefore, at least a part of the remote operation of one of the robots 10a and 10b by the remote operator can be omitted. Therefore, the remote operator can centrally perform the remote operation of the other of the robots 10a and 10b, and the load of the remote operation can be reduced.

[0114] <Description of the Second Example of Remote Operation System Operation>

[0115] Next, use Fig. 9 The flowchart shown and Fig.10 The schematic diagram shown in the figure illustrates the second example of the action of the remote control system of the present embodiment. Below, as an example of the action of the robot 10a and the robot 10b, the robot 10a searches for a service recipient (search mode), then the robot 10a communicates with the service recipient (communication mode), and the robot 10b distributes items to the service recipient (distribution mode) is described. That is, in one example of this action, the service of distributing items to the service recipient is provided to the service recipient. In addition, the robot 10b has a retainer 61, and the retainer 61 retains an item 62. The item 62 can be any item as long as it can be distributed, such as a pet bottle of drinking water.

[0116] First, with Figure 7 as well as Figure 8 In the first example of the operation of the remote operation system shown, steps S1 to S3 are performed in the same manner.

[0117] After that, the robot 10a (remote operator M60) confirms with the service recipient C50 whether the item 62 can be distributed (see Fig.10 )(Step S24). Then, when the permission of the service recipient C50 is obtained regarding the distribution of the items (Step S24: Yes), the remote operator M60 starts to distribute the service to the person through the robots 10a and 10b (Step S25). Specifically, the remote operator M60 operates the operation terminal 30 to select a script for the distribution service as a service provided by the robots 10a and 10b. In the script for the distribution service, the communication task is pre-assigned to the robot 10a, and the distribution task is pre-assigned to the robot 10b. In addition, the remote operator M60 may also operate the operation terminal 30 to assign the communication task to the robot 10a, and the distribution task to the robot 10b.

[0118] Next, the robot 10a (remote operator M60) communicates with the service recipient C50 (step S26). The content of the communication is, for example, the description of the item 62 and the intention of distributing the item 62. The remote operator M60 considers the characteristics of the service recipient C50, such as the degree to which the service recipient C50 understands the item 62, to perform the remote operation, so the robot 10a can communicate appropriately with the service recipient C50, which is satisfactory.

[0119] Next, the robot 10b approaches the service recipient C50 from a predetermined position by autonomous control and carries the article 62 (step S27). The predetermined position is, for example, a place where articles of the same type as the article 62 are gathered. Specifically, the robot 10b may also receive remote control from the remote operator M60 to drive the arm 16 to adjust the position of the retainer 61.

[0120] Then, the robot 10b makes the retainer 61 face the service recipient C50 through autonomous control, and distributes the article 62 to the service recipient C50 (step S28). In addition, in this step S28, the robot 10a (remote operator M60) can also urge the service recipient C50 to accept the distribution of the article 62.

[0121] Furthermore, the robot 10a may also take out the article 62 from the retainer 61 of the robot 10b and deliver the article 62 to the service recipient C50. Specifically, the robot 10a grasps the article 62 using the gripping portion 26 of the arm 16 through autonomous control. The robot 10a delivers the grasped article 62 to the service recipient C50 through remote operation performed by the remote operator M60. Information about the service recipient C50 is hardly acquired in advance, and there is a tendency that the service recipient C50 is changed to another person every time step S3 starts. Therefore, the remote operator M60 considers the characteristics of the service recipient C50, such as the body size of the service recipient C50, to perform remote operation, so that the article 62 can be distributed to the service recipient C50 appropriately, which is satisfactory.

[0122] The robot 10b returns to the original predetermined position by autonomous control (step S29). Then, the delivery service ends (step S30).

[0123] In addition, when the distribution of the article 62 to the service recipient C50 is not agreed to in step S24 (step S24: No), the operator of the operation terminal 30 continues the dialogue (communication) with the service recipient via the robot 10a (step S11). In this case, the distribution mode of steps S25 to S30 is omitted.

[0124] By completing the above operations, the communication between the remote operator M60 of the operation terminal 30 and the service recipient C50 via the robot 10a and the robot 10b is completed (step S12). Afterwards, by repeating the operations of steps S1 to S3, S24 to S30, S11, and S12, the robot 10a (remote operator M60) can further communicate with other service recipients C50.

[0125] and Figure 7 as well as Figure 8 Similarly, in this embodiment, the operator can determine the next action of the robot 10a and the robot 10b by pressing an icon displayed on the display unit 32 of the operation terminal 30. Therefore, it is possible to prevent the operation of the robot by the remote operator M60 from becoming complicated. Figure 7 as well as Figure 8Similarly, the next action of the robot 10a and the robot 10b is performed by the robot 10a or the robot 10b through remote operation or autonomous control by the remote operator M60 according to the content. Therefore, at least a part of the task can be performed by the autonomous control of the robot 10a or the robot 10b. Therefore, at least a part of the remote operation of one of the robots 10a and 10b by the remote operator can be omitted. Therefore, the remote operator can centrally perform the remote operation of the other robot 10a and the robot 10b, and the load of the remote operation can be reduced.

[0126] <Description of the Third Example of Remote Operation System Operation>

[0127] Next, use Fig.11 The flowchart shown and Fig.12 The schematic diagram shown in the figure illustrates the third example of the action of the remote operation system of the present embodiment. Below, as an example of the action of the robot 10a and the robot 10b, the robot 10a searches for a service recipient (search mode), then the robot 10a communicates with the service recipient (communication mode), and the robot 10b collects items from the service recipient (recovery mode) is described. That is, in one example of this action, a service of collecting items from the service recipient is provided to the service recipient. In addition, the robot 10b has a container 71, and the container 71 can accommodate an item 72. Specifically, the item 72 is an item that is not needed by the service recipient C50 and an item that should be collected by the remote operator. More specifically, the item 72 is garbage, a loaned item to the service recipient C50, etc.

[0128] First, with Figure 7 as well as Figure 8 In the first example of the operation of the remote operation system shown, steps S1 to S3 are performed in the same manner.

[0129] Afterwards, the robot 10a (remote operator M60) confirms with the service recipient C50 whether the item 72 can be collected (see Fig.12)(Step S34). Then, when the service recipient C50 has obtained permission for the recycling of the item (Step S34: Yes), the remote operator M60 starts the recycling service for the person through the robots 10a and 10b (Step S35). Specifically, the remote operator M60 operates the operation terminal 30 to select the script of the recycling service as the service provided by the robots 10a and 10b. In the script of the recycling service, the communication task is assigned to the robot 10a and the recycling task is assigned to the robot 10b in advance. In addition, the remote operator M60 may also operate the operation terminal 30 to assign the communication task to the robot 10a and the recycling task to the robot 10b.

[0130] Next, the robot 10a (remote operator M60) communicates with the service recipient C50 (step S36). The content of the communication is, for example, the promotion of the recycling of the item 72 and the explanation of its intention. The remote operator M60 performs remote operation considering the characteristics of the service recipient C50, such as the extent to which the service recipient C50 understands the recycling of the item 72, so the robot 10a can communicate appropriately with the service recipient C50, which is satisfactory.

[0131] Next, the robot 10b approaches the service recipient C50 from a predetermined position by autonomous control (step S37). The predetermined position is, for example, a place where articles of the same type as the article 72 are gathered.

[0132] Next, the robot 10b extends the container 71 to the service recipient C50 through autonomous control (step S38). When the service recipient C50 places the item 72 in the container 71, the robot 10b can recover the item 72 from the service recipient C50. The robot 10b can also accept remote operation performed by the remote operator M60 to drive the arm 16 to adjust the position of the container 71. The remote operator M60 considers the characteristics of the service recipient C50, such as the body size of the service recipient C50, to perform remote operation, so the robot 10b can properly recover the item 72 from the service recipient C50, which is satisfactory. In addition, in this step S38, the robot 10a (remote operator M60) can also communicate with the service recipient C50 to urge the recovery of the item 72.

[0133] In addition, the robot 10a can also receive the article 72 from the service recipient C50 and store it in the container 71 of the robot 10b. Specifically, the robot 10a receives the article 72 from the service recipient C50 using the gripping portion 26 of the arm 16 through remote operation performed by the remote operator M60. The robot 10a places the article 72 in the container 71 through remote operation performed by the remote operator M60. The remote operator M60 performs remote operation taking into account the characteristics of the service recipient C50, such as the body size of the service recipient C50, so the robot 10a can appropriately receive the article 72 from the service recipient C50, which is satisfactory.

[0134] The robot 10b returns to the original predetermined position by autonomous control and carries the article 72 (step S39). Then, the collection service ends (step S40).

[0135] In addition, when the collection of the article 72 from the service recipient C50 is not agreed to in step S34 (step S34: No), the operator of the operation terminal 30 continues the dialogue (communication) with the service recipient via the robot 10a (step S11). In this case, the collection mode of steps S35 to S40 is omitted.

[0136] By completing the above actions, the communication between the remote operator M60 of the operation terminal 30 and the service recipient C50 via the robot 10a and the robot 10b is completed (step S12). Afterwards, by repeating the actions of steps S1 to S3, S34 to S40, S11, and S12, the robot 10a (remote operator M60) can further communicate with other service recipients C50.

[0137] and Figure 7 to Figure 10 In the present embodiment, the operator can determine the next action of the robot 10a and the robot 10b by pressing an icon displayed on the display unit 32 of the operation terminal 30. Therefore, the operation of the robot by the remote operator M60 can be prevented from becoming complicated. Figure 7 to Figure 10Similarly, the next actions of the robot 10a and the robot 10b are performed by the robot 10a or the robot 10b through remote operation or autonomous control by the remote operator M60 according to the content. Therefore, at least a part of the task can be performed by the autonomous control of the robot 10a or the robot 10b. Therefore, at least a part of the remote operation of one of the robots 10a and 10b by the remote operator can be omitted. Therefore, the remote operator can centrally perform the remote operation of the other robot 10a and the robot 10b, and the load of the remote operation can be reduced.

[0138] In addition, in the above remote operation system 1, a configuration example is described in which the camera A (14) for the main action of the robot 10b and the camera B (17) for taking a photo of the communication partner are provided. However, in the remote operation system 1 of this embodiment, the camera mounted on the robot 10b may be one. For example, the camera A (14) may be used to take a photo of the communication partner.

[0139] In addition, in the above examples, the program is stored using various types of non-transitory computer readable media and can be provided to the computer. Non-transitory computer readable media include various types of tangible recording media (tangible storage medium). Examples of non-transitory computer readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), optical magnetic recording media (e.g., optical magnetic disks), CD-ROMs, CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs, programmable read-only memories), EPROMs (Erasable PROMs, erasable programmable read-only memories), flash ROMs, RAMs). In addition, the program can also be provided to the computer via various types of transitory computer readable media (transitory computer readable media). Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to the computer via wired communication lines such as electric wires and optical fibers or wireless communication lines.

[0140] Furthermore, in the above-mentioned various embodiments, as an example of explaining the processing in the remote operation system, the present disclosure can also be adopted as a mode of a remote operation method executed in the remote operation system. Figure 7 , Fig. 9 as well as Fig.11An example of the information management method is shown. In addition, regarding another example, it is the same as the example described in the above embodiment. In addition, the above program can be said to be a program for causing a remote operating system such as a management server to execute such a remote operation method.

[0141] The present invention has been described above according to the above-mentioned embodiments, but the present invention is not limited to the structures of the above-mentioned embodiments, but includes various deformations, modifications, and combinations that can be performed by those skilled in the art within the scope of the invention of the claims of the present patent application.

[0142] From the disclosure thus described, it will be apparent that the embodiments of the disclosure can be modified in various ways. These changes should not be considered as departing from the spirit and scope of the disclosure, and for those skilled in the art, all such modifications are certainly intended to be included within the scope of the appended claims.

Claims

1. A remote operation system, in which a remote operator remotely operates multiple robots and provides services that can be provided by performing multiple tasks to service recipients, wherein: The remote operating system has: a task assigning unit that assigns the plurality of tasks to each of the plurality of robots; and a remote operation condition assigning unit for assigning, to each of the plurality of robots, a remote operation condition corresponding to the contents of the plurality of tasks assigned to each of the plurality of robots; The plurality of robots include a first robot and a second robot, The plurality of tasks include a first task assigned to the first robot and a second task assigned to the second robot, The first robot accepts the remote operation performed by the remote operator according to the remote operation condition, communicates with the service recipient, and performs the first task. The second robot performs the second task by autonomous control according to the remote operation condition. The first robot communicates with the service recipient at a position where a photo showing a group of the service recipient and the first robot can be taken through remote operation by the remote operator, The second robot takes the collective photo using a camera device through autonomous control.

2. The remote operation system according to claim 1, characterized in that: The second robot adjusts the field of view angle by autonomous control before taking the collective photograph using the camera device.

3. The remote operation system according to claim 1, characterized in that: The second robot takes the group photo according to the remote operation condition given, and then prints the group photo through autonomous control to obtain a photo print. After the first robot grasps the photograph print by autonomous control, it delivers it to the service recipient through remote operation performed by the remote operator.

4. The remote operation system according to claim 1, characterized in that: The first robot receives remote control from the remote operator and communicates with the service recipient. The second robot transports the item to the service recipient through autonomous control according to the remote operation conditions.

5. The remote operation system according to claim 1, characterized in that: The first robot receives remote control from the remote operator and communicates with the service recipient. The second robot moves to the vicinity of the service recipient while holding the container through autonomous control according to the remote operation conditions, and collects the items into the container.

6. A remote operation method, executed in a remote operation system, in which a remote operator remotely operates multiple robots to provide services that can be provided by performing multiple tasks to a service recipient, wherein: The remote operation method comprises: the step of assigning the plurality of tasks to each robot of the plurality of robots; as well as a step of assigning, to each of the plurality of robots, a remote operation condition corresponding to the contents of the plurality of tasks assigned to each of the plurality of robots, The plurality of robots include a first robot and a second robot. The plurality of tasks include a first task assigned to the first robot and a second task assigned to the second robot, The first robot accepts the remote operation performed by the remote operator according to the remote operation condition, communicates with the service recipient, and performs the first task. The second robot performs the second task by autonomous control according to the remote operation condition. The first robot communicates with the service recipient at a position where a photo showing a group of the service recipient and the first robot can be taken through remote operation by the remote operator, The second robot takes the collective photo using a camera device through autonomous control.

7. A storage medium storing a remote operation program, the remote operation program being executed by a computer operating as a server in a remote operation system, wherein a remote operator remotely operates a plurality of robots via the server to provide services that can be provided by performing a plurality of tasks to a service recipient, wherein: The remote operation program causes the computer to execute: the step of assigning the plurality of tasks to each robot of the plurality of robots; as well as a step of assigning, to each of the plurality of robots, a remote operation condition corresponding to the contents of the plurality of tasks assigned to each of the plurality of robots, The plurality of robots include a first robot and a second robot. The plurality of tasks include a first task assigned to the first robot and a second task assigned to the second robot, The first robot accepts the remote operation performed by the remote operator according to the remote operation condition, communicates with the service recipient, and performs the first task. The second robot performs the second task by autonomous control according to the remote operation condition. The first robot communicates with the service recipient at a position where a photo showing a group of the service recipient and the first robot can be taken through remote operation by the remote operator, The second robot takes the collective photo using a camera device through autonomous control.

Images