Operator selection method, terminal device, and program

Abstract

An operator selection method according to the present disclosure is an operator selection method for selecting, from among a plurality of operators, an operation execution operator who remotely operates a robot that can autonomously move. A terminal device used for each of the plurality of operators displays prediction information for predicting the generation of a remote operation request for requesting a remote operation for the robot, and receives a preliminary preparation operation for remotely operating the robot. When the remote operation request is generated, an information processing device selects any one of the plurality of operators as the operation execution operator on the basis of an execution state of the preliminary preparation operation in each of the plurality of operators. The information processing device causes the operation execution operator to perform the remote operation on the robot by using the terminal device.

Description

Operator selection method, terminal device and program

[0001] The present disclosure relates to an operator selection method, a terminal device, and a program.

[0002] With recent advances in information technology, robots are now able to assess their surroundings and move autonomously. For example, it is predicted that robots will be able to autonomously avoid obstacles and continue moving even if they encounter them on their route. However, it is not always possible to avoid obstacles 100% of the time and continue moving. In reality, exceptional events may occur, such as robots failing to avoid obstacles and getting stuck, or robots that can move autonomously but stop or slow down excessively, resulting in inefficient movement. A robot that can be remotely controlled by an operator can efficiently deal with such exceptional events without the need for a worker to rush to the robot's location.

[0003] Patent Literature 1 describes a technology in which an operation terminal used by an operator in a remote location is connected to a robot on-site, and the operator remotely operates the robot. In particular, Patent Literature 1 describes a method for matching a robot with an operator who will remotely operate it.

[0004] However, in conventional technology, although operators can set their desired conditions, such as specifying the time periods when they are available to work, they end up wasting time waiting to be matched. For example, in matching services such as food delivery, delivery personnel can take actions to increase the probability of being matched, such as moving to a store that receives many orders in advance. Such actions lead to faster delivery of items to the orderer, which is beneficial for both the orderer and the store. Furthermore, Patent Document 2 describes technology that supports operators in improving their operating skills of work machines. However, Patent Document 2 does not describe matching operators.

[0005] International Publication No. 2023 / 053520 Patent No. 7544198

[0006] The present disclosure aims to provide an operator selection method, a terminal device, and a program that can appropriately select an operator to remotely operate a robot from among a plurality of operators.

[0007] The operator selection method according to the present disclosure is an operator selection method for selecting, from a plurality of operators, an operation performing operator to remotely operate an autonomously mobile robot, wherein a terminal device used by each of the plurality of operators displays prediction information predicting the occurrence of a remote operation request requesting remote operation of the robot and accepts advance preparation operations for remotely operating the robot, and when the remote operation request occurs, an information processing device selects one of the plurality of operators as the operation performing operator based on the implementation status of the advance preparation operations by each of the plurality of operators, and the information processing device causes the operation performing operator to remotely operate the robot using the terminal device.

[0008] FIG. 1 is a diagram illustrating a configuration of a robot control system according to an embodiment. FIG. 2 is a diagram illustrating an outline of processing in the robot control system. FIG. 3 is a diagram illustrating a functional configuration of an information processing device. FIG. 4 is a diagram illustrating a functional configuration of a terminal device. FIG. 5 is a sequence diagram illustrating a processing flow in the robot control system. FIG. 6 is a flowchart illustrating a processing flow of an information processing device. FIG. 7 is a flowchart illustrating a processing flow of selecting prediction information. FIG. 8 is a flowchart illustrating a processing flow of accepting a preparation operation. FIG. 9 is a flowchart illustrating a processing flow of changing the layout of an operation screen. FIG. 10 is a flowchart illustrating a processing flow by a terminal device when a remote operation request is received. FIG. 11 is a diagram illustrating a first specific example of a method for generating prediction information. FIG. 12 is a diagram illustrating a second specific example of a method for generating prediction information. FIG. 13 is a diagram illustrating a third specific example of a method for generating prediction information. FIG. 14 is a diagram illustrating an example of preparation information. FIG. 15 is a diagram illustrating an example of a prediction information confirmation screen. FIG. 16 is a diagram illustrating a first example of a preparation implementation screen. FIG. 17 is a diagram illustrating a second example of a preparation implementation screen. FIG. 18 is a diagram illustrating a third example of a preparation implementation screen. FIG. 19 is a diagram showing an operation screen using a first-person perspective view. FIG. 20 is a diagram showing an operation screen using an omnidirectional view. FIG. 21 is a diagram showing an operation screen combining a first-person perspective view and an omnidirectional view. FIG. 22 is a diagram showing an example of an operation screen when remote operation is being performed. FIG. 23 is a diagram showing an example of an evaluation screen. FIG. 24 is a flowchart showing the flow of a selection process for an operation performing operator. FIG. 25 is a diagram showing the results of tallying up judgments of whether prediction information is "right" or "wrong". FIG. 26 is a diagram showing a schedule and advance preparation information. FIG. 27 is a diagram for explaining a process for providing a reward. FIG. 28 is a diagram for explaining the contents of a specific example of a process for providing a reward. FIG. 29 is a diagram showing the hardware configuration of an information processing device.

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings as appropriate. However, more detailed description than necessary may be omitted. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0010] FIG. 1 is a diagram illustrating an example of the configuration of a robot control system 10 according to an embodiment.

[0011] The robot control system 10 includes one or more robots 20 (20-1 to 20-M) (M is an integer of 1 or more), multiple terminal devices 30 (30-1 to 30-N) (N is an integer of 2 or more), an information processing device 40, and a service system 50. The one or more robots 20, the information processing device 40, and the multiple terminal devices 30 are communicatively connected via a network NW or the like.

[0012] The robot 20 operates autonomously. For example, the robot 20 moves autonomously according to a preset schedule. For example, the robot 20 delivers an object according to a preset schedule. Furthermore, the robot 20 can temporarily suspend its autonomous movement, receive remote control from any of the terminal devices 30 used by an operator via the network NW, and move according to the operator's remote control. The robot 20 issues a remote control request when autonomous movement becomes difficult due to, for example, the presence of an obstacle or the like in the movement route.

[0013] Each of the multiple terminal devices 30 has a hardware configuration similar to that of a normal computer, including a processor and a storage device, and executes information processing according to a program. Each of the multiple terminal devices 30 is used by one of the multiple operators. Each of the multiple terminal devices 30 displays information to the operator and accepts information and operation inputs from the operator. Each of the multiple operators is a user pre-registered in the robot control system 10, and uses the terminal device 30 they are using to remotely control one or more robots 20 that has issued a remote control request.

[0014] The information processing device 40 has a hardware configuration similar to that of a normal computer equipped with a processor and a storage device, and executes information processing according to a program. The information processing device 40 may be a server connected to a network NW, or may be a cloud computer in which one or more servers operate in conjunction with each other. When a remote operation request is issued from one or more robots 20, the information processing device 40 selects one of multiple operators as an operation execution operator and has the selected operation execution operator remotely operate the robot 20 that issued the remote operation request.

[0015] The service system 50 has a hardware configuration similar to that of a normal computer, including a processor and a storage device, and executes information processing according to a program. The information processing device 40 may be a server connected to the network NW, or may be a cloud computer in which one or more servers operate in conjunction with each other. The service system 50 manages and controls the entire robot control system 10. For example, the service system 50 generates schedules for one or more robots 20 and causes each of the one or more robots 20 to operate autonomously according to the schedules.

[0016] FIG. 2 is a diagram for explaining an outline of the processing of the robot control system 10. As shown in FIG.

[0017] The robot 20 among the one or more robots 20 operates according to a preset schedule. For example, the robot 20 moves autonomously along a movement route according to the schedule so as to arrive at a destination at a time according to the schedule.

[0018] There are cases where the robot 20 has difficulty moving autonomously. For example, there are cases where the robot 20 has to stop moving autonomously due to an obstacle on the route, or where the robot 20 is able to move autonomously but the movement efficiency is significantly reduced, resulting in a movement time that is significantly longer than scheduled.

[0019] When autonomous movement of the robot 20 becomes difficult, the robot 20 generates a remote control request requesting remote control by an operator and transmits the request to the information processing device 40. Note that the service system 50 may generate the remote control request and transmit it to the information processing device 40 instead of the robot 20.

[0020] The information processing device 40 predicts the occurrence of a remote control request for remote control of one or each of the robots 20. Then, the information processing device 40 generates prediction information that predicts the occurrence of a remote control request.

[0021] For example, the information processing device 40 generates prediction information including information for identifying the robot 20 that will issue a remote control request, a request generation area where the remote control request will be issued, and a predicted time when the remote control request will be issued, based on the schedule of the robot 20 and the past movement history of one or more robots 20. Then, before the remote control request is issued, the information processing device 40 transmits the generated prediction information to the terminal device 30 used by each of the multiple operators.

[0022] Each of the multiple terminal devices 30 receives the prediction information from the information processing device 40 and displays the received prediction information. Subsequently, each of the multiple terminal devices 30 accepts from the operator a preparatory operation for remotely operating the robot 20 when a remote operation request based on the prediction information is issued. The preparatory operation may include multiple items or multiple stages. Each of the multiple terminal devices 30 transmits to the information processing device 40 the implementation status of the preparatory operation accepted from the operator. Each time the preparatory operation by the operator progresses, each of the multiple terminal devices 30 may transmit the implementation status of the preparatory operation to the information processing device 40.

[0023] When a remote operation request predicted by the prediction information is actually issued, the information processing device 40 selects one of the multiple operators as an operation performing operator who will remotely operate the robot 20, based on the implementation status of the advance preparation operations for the prediction information acquired from each of the multiple terminal devices 30. The information processing device 40 selects, for example, any operator who has properly completed all of the advance preparation operations as the operation performing operator. If there is no operator who has properly completed all of the advance preparation operations, the information processing device 40 may select, as the operation performing operator, any operator who has properly completed all of the advance preparation operations or any operator who is most advanced in the advance preparation operations. Then, the information processing device 40 causes the selected operation performing operator to remotely operate the robot 20 using the terminal device 30.

[0024] The information processing device 40 may select one of the multiple operators as the operation performing operator based on the implementation status of the preparatory operation for the prediction information acquired from each of the multiple terminal devices 30 and other parameters. The other parameters may be, for example, values ​​evaluating the skills of each of the multiple operators. The skill evaluation value may be, for example, the average value of the time required for one remote operation. The other parameters may also be, for example, values ​​evaluating the experience of each of the multiple operators. The experience evaluation value may be, for example, the number of times or the total time for which remote operations have been performed.

[0025] In addition, the information processing device 40 may select an operation performing operator from among multiple operators based on the quality of the pre-preparation operation, instead of or in addition to the degree of progress of the pre-preparation operation.

[0026] The operation operator remotely controls the robot 20 using the terminal device 30 that he or she uses. This allows the robot 20 to move even in a situation where autonomous movement is difficult. For example, the robot 20 can bypass obstacles and pass through the request area in a short time.

[0027] When the remote operation by the operation implementing operator is completed, the information processing device 40 transmits an operation restart instruction to the robot 20. When the robot 20 receives the operation restart instruction, it restarts autonomous movement according to the schedule.

[0028] Then, the information processing device 40 performs a process of giving a reward to the operation performing operator who performed the remote operation of the robot 20. Furthermore, the information processing device 40 may perform a process of giving a reward to an operator other than the operation performing operator who appropriately performed the advance preparation operation.

[0029] The robot control system 10 as described above generates prediction information that predicts the occurrence of a request for remote operation of the robot 20, and can have each of the multiple operators perform advance preparation operations based on the prediction information. The robot control system 10 can then appropriately select an operation performing operator who will remotely operate the robot 20 from the multiple operators based on the implementation status of the advance preparation operations performed by each of the multiple operators.

[0030] Furthermore, each of the multiple operators can increase the probability of receiving a reward by performing a preparatory operation more appropriately and earlier. Therefore, each of the multiple operators can increase the possibility of receiving a larger reward for their own actions. Furthermore, by having each of the multiple operators take actions that increase the possibility of receiving a larger reward, the robot control system 10 can select an operation execution operator who is likely to perform a more appropriate operation in a short time when a remote operation request is issued.

[0031] FIG. 3 is a diagram showing the functional configuration of the information processing device 40. As shown in FIG.

[0032] The information processing device 40 includes a first communication unit 60, a robot information acquisition unit 62, a memory unit 64, a prediction information generation unit 66, an operator information acquisition unit 68, an operator selection unit 70, an operation request generation unit 72, and a reward processing unit 74.

[0033] The first communication unit 60 communicates with one or more robots 20, a plurality of terminal devices 30, and the service system 50 via the network NW.

[0034] The robot information acquisition unit 62 acquires robot information from each of the one or more robots 20 via the first communication unit 60. The robot information acquisition unit 62 may acquire the robot information for each of the one or more robots 20 from the service system 50 instead of from each of the one or more robots 20.

[0035] The robot information includes, for example, a schedule, movement information indicating the current position and speed of the robot 20, and video image data captured by a camera mounted on the robot 20. The robot information acquisition unit 62 also acquires a remote control request from each of one or more robots 20. Note that instead of acquiring a remote control request from each of one or more robots 20, the robot information acquisition unit 62 may acquire a remote control request for each of one or more robots 20 from the service system 50.

[0036] The storage unit 64 stores the robot information for each of one or more robots 20 acquired by the robot information acquisition unit 62 .

[0037] The prediction information generation unit 66 generates prediction information for each of one or more robots 20, predicting a remote control request that the robot 20 will issue, based on the robot information stored in the storage unit 64. The prediction information generated by the prediction information generation unit 66 is transmitted by the first communication unit 60 to each of the multiple terminal devices 30 via the network NW.

[0038] For example, the prediction information generator 66 may detect the position and type of an object for each of one or more robots 20 by video analysis using deep learning or the like on video data, determine whether the object whose position and type have been detected is an obstacle on the path, and generate prediction information based on the determination result. In this case, the prediction information generator 66 may generate prediction information for each of one or more robots 20 based on video data of other robots 20. For example, when detecting obstacles on the path based on video data captured by the robot 20 itself, the prediction information generator 66 can basically only detect obstacles that will be present on the path in the near future, for example, at a position where the robot will be traveling in several tens of seconds. However, by generating prediction information for each of one or more robots 20 based on video data of other robots 20, the prediction information generator 66 can generate prediction information for obstacles that will be present on the path in the more distant future. For example, the predictive information generation unit 66 can generate predictive information based on video data of another robot 20 traveling in the oncoming lane, thereby generating predictive information for obstacles that may be located on the route the robot will travel in the more distant future.

[0039] The operator information acquisition unit 68 acquires preparation information indicating the implementation status of preparation operations from each of the terminal devices 30 via the first communication unit 60 .

[0040] When the robot information acquisition unit 62 acquires a remote operation request, the operator selection unit 70 selects, from among the multiple operators, an operation execution operator who will remotely operate the robot 20 that issued the acquired remote operation request. In this case, the operator selection unit 70 selects the operation execution operator based on the implementation status of advance preparation operations of each of the multiple operators for prediction information that predicts the issued remote operation request.

[0041] The operation request generation unit 72 transmits a request for remote operation to the terminal device 30 used by the operation performing operator selected by the operator selection unit 70 via the first communication unit 60. As a result, the operation request generation unit 72 allows the operation performing operator to use the terminal device 30 to remotely operate the robot 20 for which the remote operation request has been issued.

[0042] The reward processing unit 74 performs processing to give a reward to the operation performing operator who performed the remote operation. The reward processing unit 74 may also perform processing to give a reward to operators other than the operation performing operator when the operator appropriately performs the advance preparation operation.

[0043] FIG. 4 is a diagram showing the functional configuration of the terminal device 30.

[0044] The terminal device 30 includes a second communication unit 76, a prediction information confirmation screen processing unit 78, a pre-preparation implementation screen processing unit 80, a pre-preparation information generation unit 82, an operation screen processing unit 84, an operation information generation unit 86, and a terminal control unit 88.

[0045] The second communication unit 76 communicates with one or more robots 20, information processing devices 40, and service systems 50 via the network NW.

[0046] The prediction information confirmation screen processing unit 78 acquires prediction information from the information processing device 40 via the second communication unit 76. The prediction information confirmation screen processing unit 78 generates a prediction information confirmation screen for allowing the operator to confirm the prediction information and displays it on the monitor. This allows the prediction information confirmation screen processing unit 78 to display the prediction information to the operator. In addition, the prediction information confirmation screen processing unit 78 accepts, from the operator, a selection operation of prediction information to be subjected to advance preparation operation for remote operation via the prediction information confirmation screen.

[0047] When prediction information is selected via the prediction information confirmation screen, the preparation screen processing unit 80 generates a preparation screen for accepting preparation operations to be performed in the event of a remote operation request indicated in the selected prediction information. The preparation screen processing unit 80 displays the generated preparation screen on a monitor. The preparation screen processing unit 80 then accepts, from the operator via the preparation screen, preparation operations to be performed in the event of a remote operation request indicated in the selected prediction information.

[0048] The preparation information generation unit 82 manages the implementation status of preparation operations performed via the preparation implementation screen. The preparation information generation unit 82 generates preparation information indicating the implementation status of the preparation operations and transmits the generated preparation information to the information processing device 40 via the second communication unit 76. The preparation information generation unit 82 may periodically transmit the preparation information to the information processing device 40, or may generate updated preparation information each time the preparation operations progress and transmit the updated preparation information to the information processing device 40.

[0049] The operation screen processing unit 84 acquires information representing a request for remote operation from the information processing device 40 via the second communication unit 76. When the operation screen processing unit 84 acquires the information representing the request for remote operation, it generates an operation screen for remotely operating the robot 20 indicated in the information representing the request for remote operation and displays it on the monitor. Then, the operation screen processing unit 84 accepts remote operation of the robot 20 by the operator via the operation screen.

[0050] The operation information generating unit 86 generates operation information indicating the content of the remote operation received from the operation screen. The operation information generating unit 86 transmits the generated operation information to the robot 20 to be remotely operated via the second communication unit 76. This allows the robot 20 to operate in accordance with the remote operation performed by the operator on the operation screen.

[0051] The terminal control unit 88 performs overall management of the terminal device 30. The terminal control unit 88 also controls input units such as a keyboard, mouse, and controller, as well as a monitor, etc., to display screens generated by the prediction information confirmation screen processing unit 78, advance preparation execution screen processing unit 80, and operation screen processing unit 84 on the monitor, etc., and provides information received by the input units to the prediction information confirmation screen processing unit 78, advance preparation execution screen processing unit 80, and operation screen processing unit 84.

[0052] 5 is a sequence diagram showing the flow of processing in the robot control system 10. The robot control system 10 executes processing according to the flow shown in FIG.

[0053] The robot 20 transmits robot information to the information processing device 40 periodically or when some event occurs (S11). In response to receiving the robot information, the information processing device 40 generates prediction information predicting a future remote control request from the robot 20 that transmitted the robot information, and transmits the generated prediction information to each of the multiple terminal devices 30 (S12). When each of the multiple terminal devices 30 receives the prediction information, it displays the received prediction information on a monitor. Each of the multiple terminal devices 30 accepts a preparatory operation from an operator based on the received prediction information, and transmits the preparatory information to the information processing device 40 periodically or as the preparatory operation progresses (S13). Then, the robot 20, the information processing device 40, and each of the multiple terminal devices 30 repeatedly executes the processes of S11, S12, and S13 during the autonomous movement of the robot 20.

[0054] Furthermore, when it becomes difficult for the robot 20 to move autonomously, the robot 20 transmits a remote control request to the information processing device 40 (S14). The service system 50 that manages the movement of the robot 20 may transmit the remote control request to the information processing device 40 instead of the robot 20.

[0055] When the information processing device 40 receives a remote operation request from the robot 20, it selects an operation performing operator from among the multiple operators to remotely operate the robot 20 that sent the remote operation request, based on the implementation status of each of the multiple operators' advance preparation operations. The information processing device 40 then transmits information representing a request for remote operation to the terminal device 30 used by the selected operation performing operator (S15). When the terminal device 30 that received the information representing the request for remote operation performs an operation to accept the request, the terminal device 30 transmits information representing acceptance to the information processing device 40 (S16). When the information processing device 40 receives the information representing acceptance, the information processing device 40 transmits information indicating that the operator has made a decision to the robot 20 that sent the remote operation request (S17). In this case, the information processing device 40 also transmits information identifying the terminal device 30 used by the operation performing operator to the robot 20. If the operator does not accept the request for remote operation, the information processing device 40 selects a new operation performing operator from among the multiple operators and transmits information representing a request for remote operation to the terminal device 30 used by the new operation performing operator (S15).

[0056] When the robot 20 receives the information indicating that the operator has made the decision, the robot 20 switches the control so that it can operate in accordance with the operation information from the operation performing operator (S18). This allows the operation performing operator to remotely control the robot 20.

[0057] For example, the robot 20 may autonomously move to a safe location and then stop, and switch control after stopping. For example, the robot 20 may move to the shoulder of a road and then stop, or stop after crossing a crosswalk, and then switch control after stopping. Furthermore, if the robot 20 can seamlessly move in accordance with a control operation from a state of autonomous movement, it may interrupt its autonomous movement and operate in accordance with the operation information when it receives information indicating that the operator has made a decision. Furthermore, in this case, if there is a delay in deciding on an operator to perform the operation, the robot 20 may autonomously move and then stop, and switch control after stopping.

[0058] When the terminal device 30 used by the operation performing operator receives an operation to complete the remote operation from the operation performing operator, it transmits information indicating the completion of the operation to the information processing device 40 (S19). When the information processing device 40 receives the information indicating the completion of the operation, it transmits an operation resume instruction to the remotely operated robot 20 (S20). Then, when the remotely operated robot 20 receives the operation resume instruction, it ends the operation by remote control and resumes autonomous movement according to the schedule.

[0059] 6 is a flowchart showing the flow of processing by the information processing device 40. The information processing device 40 executes processing for each of one or more robots 20 according to the flow shown in FIG.

[0060] First, in S31, the information processing device 40 receives robot information from the robot 20.

[0061] Next, in S32, the information processing device 40 predicts a remote control request that will occur in the future based on the robot information, and generates prediction information.

[0062] Next, in S33, the information processing device 40 transmits the prediction information generated in S32 to each of the multiple terminal devices 30.

[0063] Subsequently, in S34, the information processing device 40 receives, from each of the plurality of terminal devices 30, advance preparation information for the prediction information generated in S32.

[0064] Next, in S35, the information processing device 40 determines whether the remote control request predicted by the prediction information generated in S32 has actually been issued. If the remote control request has not been issued (No in S35), the information processing device 40 ends this flow. If the remote control request has actually been issued (Yes in S35), the information processing device 40 proceeds to S36.

[0065] In S36, the information processing device 40 selects an operation performing operator from among the multiple operators based on the received remote operation request and the implementation status of the pre-preparation operation indicated in the pre-preparation information acquired from each of the multiple terminal devices 30.

[0066] Next, in S37, the information processing device 40 transmits information representing a request for remote operation to the terminal device 30 used by the operation performing operator. This allows the information processing device 40 to allow the operation performing operator to remotely operate the robot 20 for which remote operation has been requested. After completing the processing of S37, the information processing device 40 ends this flow.

[0067] 7 is a flowchart showing the flow of the process of selecting prediction information by the terminal device 30. The prediction information confirmation screen processing unit 78 of the terminal device 30 executes the process according to the flow shown in FIG.

[0068] First, in S41, the prediction information confirmation screen processing unit 78 acquires prediction information from the information processing device 40. Subsequently, in S42, the prediction information confirmation screen processing unit 78 updates the prediction information confirmation screen based on the acquired prediction information. Subsequently, in S43, the prediction information confirmation screen processing unit 78 determines whether the operator has operated the prediction information confirmation screen to select new prediction information for the advance preparation operation. If no selection has been made (No in S43), the prediction information confirmation screen processing unit 78 ends this flow. If a selection has been made (Yes in S43), the prediction information confirmation screen processing unit 78 proceeds to S44. In S44, the prediction information confirmation screen processing unit 78 provides the selected prediction information to the advance preparation execution screen processing unit 80, and ends this flow.

[0069] 8 is a flowchart showing the flow of a process for accepting a preparation operation by the terminal device 30. The preparation execution screen processing unit 80 of the terminal device 30 executes the process according to the flow shown in FIG.

[0070] First, in S51, the preparation screen processing unit 80 determines whether any prediction information has been selected by operating the prediction information confirmation screen. If no prediction information has been selected (No in S51), the preparation screen processing unit 80 waits for processing in S51. If any prediction information has been selected, the preparation screen processing unit 80 proceeds to S52.

[0071] In S52, the preparation execution screen processing unit 80 generates a preparation operation screen based on the selected prediction information. Note that the preparation execution screen processing unit 80 may transmit a request to generate a preparation operation screen to the information processing device 40 and obtain the preparation execution screen generated by the information processing device 40.

[0072] Next, in S53, the preparation screen processing unit 80 determines whether all preparation operations performed by the operator have been completed. If all preparation operations have been completed (Yes in S53), the preparation screen processing unit 80 proceeds to S55. If all preparation operations have not yet been completed (No in S53), the preparation screen processing unit 80 proceeds to S54. In S54, the preparation screen processing unit 80 determines whether the preparation operations for the selected prediction information have been interrupted. If the preparation operations have been interrupted (Yes in S54), the preparation screen processing unit 80 proceeds to S55. If the preparation operations have not been interrupted (No in S54), the preparation screen processing unit 80 returns to S53 and repeats the process from S53.

[0073] In S55, the preparation execution screen processing unit 80 provides the prediction information confirmation screen processing unit 78 with information indicating that the selection of the selected prediction information has been cancelled.

[0074] Subsequently, in S56, the preparation execution screen processing unit 80 transmits the preparation information to the information processing device 40, and the flow ends.

[0075] 9 is a flowchart showing the flow of processing for changing the layout of the operation screen by the terminal device 30. When an operation screen for operating any of the robots 20 is displayed, the operation screen processing unit 84 of the terminal device 30 executes processing according to the flow shown in FIG.

[0076] First, in S61, the operation screen processing unit 84 determines whether or not the operator has selected another robot 20. If another robot 20 has not been selected (No in S61), the operation screen processing unit 84 ends this flow. If another robot 20 has been selected (Yes in S61), the operation screen processing unit 84 proceeds to S62. In S62, the operation screen processing unit 84 acquires advance preparation information for the selected robot 20. Subsequently, in S63, the operation screen processing unit 84 generates and displays an operation screen for operating the selected robot 20 based on the advance preparation information for the selected robot 20. After completing the processing of S63, the operation screen processing unit 84 ends this flow.

[0077] 10 is a flowchart showing the flow of processing by the terminal device 30 when a remote control request is received. The operation information generating unit 86 of the terminal device 30 executes the processing according to the flow shown in FIG.

[0078] First, in S71, the operation information generation unit 86 determines whether or not information indicating a request for remote operation has been received from the information processing device 40. If information indicating a request for remote operation has not been received (No in S71), the operation information generation unit 86 ends this flow. If information indicating a request for remote operation has been received (Yes in S71), the operation information generation unit 86 proceeds to S72.

[0079] In S72, the operation information generation unit 86 determines whether the robot 20 for which remote operation has been requested is the same as the robot 20 operated on the currently displayed operation screen. If they are the same (Yes in S72), the operation information generation unit 86 proceeds to S75. If they are not the same (No in S72), the operation information generation unit 86 proceeds to S73.

[0080] In S73, the operation information generation unit 86 acquires advance preparation information for the robot 20 for which a request for remote operation has been made. Subsequently, in S74, the operation information generation unit 86 provides the acquired advance preparation information to the operation screen processing unit 84, causing the operation screen processing unit 84 to generate and display an operation screen. After completing the processing of S74, the operation information generation unit 86 proceeds to the processing of S75.

[0081] In S75, the operation information generation unit 86 updates the operation screen in response to the operator's operation, and transmits information representing consent to the information processing device 40. Thereafter, the operation information generation unit 86 transmits operation information to the robot 20 to be operated in response to the operator's operation on the operation screen, thereby controlling the robot 20. Then, when the operation information generation unit 86 has completed remote operation of the robot 20 to be operated, it transmits operation completion information to the information processing device 40, and ends this flow.

[0082] FIG. 11 is a diagram for explaining a first specific example of a prediction method for generating prediction information.

[0083] The information processing device 40 generates prediction information based on the schedule of the robot 20 and the past movement history of one or more robots 20. For example, the information processing device 40 may generate prediction information using a prediction method based on whether the robot 20 stops or moves slowly, as shown in FIG.

[0084] For example, as shown in A of FIG. 11 , the information processing device 40 refers to a map showing a group of routes that one or more robots 20 can travel, and divides the group of routes into a plurality of regions. For example, the information processing device 40 divides the group of routes into a plurality of regions using intersections as dividing points. In A of FIG. 11 , the information processing device 40 divides the group of routes into eight regions, R01, R02, R03, R04, R05, R06, R07, and R08. Alternatively, the information processing device 40 may divide the group of routes into one region at an intersection. Alternatively, the information processing device 40 may divide one route from an intersection to the next intersection into two regions, each region being separated by a traveling direction.

[0085] In the map of Figure 11, each route is represented by a straight line and an intersection is represented by a circle. This also applies to Figures 11 and subsequent figures.

[0086] 11B, the information processing device 40 identifies, among multiple areas, areas where the stagnation time, which is the cumulative value of the time that the passing robot 20 either stopped or moved slowly, exceeds a certain value based on the past movement history of one or more robots 20. Note that the stagnation time may be the ratio of the total cumulative value of the time that the robot 20 either stopped or moved slowly along the route to the average running time of one or more robots 20 that passed through the route. Alternatively, the stagnation time may be the difference time obtained by subtracting a set running time calculated from a set speed and the length of the route from the average running time.

[0087] In the example of FIG. 11, the information processing device 40 identifies areas R03, R04, and R06 as stagnant areas.

[0088] Next, as shown in C of FIG. 11 , when the information processing device 40 acquires the robot information, it refers to the schedule included in the robot information and determines whether or not to pass through the identified stasis area. If the robot will pass through the stasis area, the information processing device 40 identifies the passing stasis area as a request generation area. Furthermore, the information processing device 40 refers to the schedule and identifies the time at which the robot will pass through the stasis area as a predicted time. Then, the information processing device 40 generates prediction information including information identifying the robot 20, the identified request generation area, and the identified predicted time.

[0089] When one robot 20 passes through two or more stagnant areas at different times, the information processing device 40 generates prediction information for each of the two or more stagnant areas.

[0090] This allows the information processing device 40 to generate prediction information that predicts the occurrence of a remote control request using a prediction method based on whether the robot 20 will stop or move slowly.

[0091] The information processing device 40 may set and change a reference time of past movement history that is referenced to identify a stagnant area. For example, the information processing device 40 may receive a reference time range and generate prediction information by referencing past movement history within the reference time range from the current time. The information processing device 40 may also generate prediction information for each of a plurality of different reference time ranges. For example, the information processing device 40 may generate prediction information by referencing movement history within the past hour from the current time, prediction information by referencing movement history within the past three hours from the current time, and prediction information by referencing movement history within the past day from the current time.

[0092] Furthermore, the information processing device 40 may generate prediction information for each performance or type of the robot 20. For example, the information processing device 40 may include information that identifies the performance or type of the robot 20 in the prediction information.

[0093] FIG. 12 is a diagram for explaining a second specific example of a prediction method for generating prediction information.

[0094] The information processing device 40 may generate prediction information using a prediction method based on the request occurrence rate as shown in FIG.

[0095] For example, the information processing device 40 divides the group of routes into a plurality of regions as shown in A of Fig. 12. The division method shown in A of Fig. 12 is the same as the example shown in A of Fig. 11.

[0096] 12B, the information processing device 40 identifies, among the multiple areas, areas where the request occurrence rate, which represents the ratio of the number of robots 20 that have issued remote control requests to the number of passing robots 20, is equal to or greater than a certain value as high occurrence rate areas based on the past movement history of one or more robots 20. Note that the number of robots 20 that have issued remote control requests may be the number of robots 20 that have issued remote control requests for a predetermined specific reason.

[0097] In the example of FIG. 12, the information processing device 40 identifies areas R07 and R08 as request occurrence areas.

[0098] Next, as shown in C of FIG. 12 , when the information processing device 40 acquires the robot information, it refers to the schedule included in the robot information and determines whether or not the robot will pass through the identified high occurrence rate area. If the robot will pass through the high occurrence rate area, the information processing device 40 identifies the high occurrence rate area that the robot will pass through as a request occurrence area. Furthermore, the information processing device 40 refers to the schedule and identifies the time that the robot will pass through the high occurrence rate area as a predicted time. Then, the information processing device 40 generates prediction information that includes information that identifies the robot 20, the identified request occurrence area, and the identified predicted time.

[0099] As a result, the information processing device 40 can generate prediction information that predicts the occurrence of a remote control request that requests remote control of the robot 20 using a prediction method based on the request occurrence rate.

[0100] 11 , when one robot 20 passes through two or more request occurrence areas at different times, the information processing device 40 generates prediction information for each of the two or more request occurrence areas. Also, as in the case of FIG. 11 , the information processing device 40 may set and change a reference time range of the past movement history of one or more robots 20 that is referenced to identify a high occurrence rate area. Also, as in the case of FIG. 11 , the information processing device 40 may generate prediction information for each performance or type of robot 20.

[0101] FIG. 13 is a diagram for explaining a third specific example of a prediction method for generating prediction information.

[0102] The information processing device 40 may generate prediction information using a prediction method based on the number of passes of the robot 20 as shown in FIG.

[0103] For example, the information processing device 40 divides the group of routes into a plurality of regions as shown in A of Fig. 13. The division method shown in A of Fig. 13 is the same as the example shown in A of Fig. 11.

[0104] Next, as shown in B of Figure 13, the information processing device 40 identifies, among the multiple areas, areas through which the number of times the robots 20 have passed is less than a predetermined number as areas of unknown status based on the past movement history of one or more robots 20 in the time range from time T, which is a predetermined time before the current time, to the current time.

[0105] Furthermore, the information processing device 40 may specify, as an unknown state area, an area through which the number of robots 20 passing is less than a predetermined specified number, on the condition that it is determined that the stagnation time is equal to or greater than a certain value based on the past movement history of one or more robots 20 from time T' before time T to time T. This allows the information processing device 40 to exclude, from selection as a request generation area, an area in which the robot 20 has not stopped or slowed down in the past and is unlikely to generate a remote control request.

[0106] In the example of FIG. 13, the information processing device 40 identifies the region R01 as an unknown state region.

[0107] Next, as shown in C of FIG. 13 , when the information processing device 40 acquires the robot information, it refers to the schedule included in the robot information and determines whether or not to pass through the identified unknown state area. If the robot will pass through the unknown state area, the information processing device 40 identifies the unknown state area that will be passed through as a request occurrence area. Furthermore, the information processing device 40 refers to the schedule and identifies the time at which the robot will pass through the unknown state area as a predicted time. Then, the information processing device 40 generates prediction information that includes information identifying the robot 20, the identified request occurrence area, and the identified predicted time.

[0108] When multiple robots 20 pass through one unknown state region at different times or the same time, the information processing device 40 generates prediction information for each of the multiple robots 20. When one robot 20 passes through two or more unknown state regions at different times, the information processing device 40 generates prediction information for each of the two or more unknown state regions.

[0109] This allows the information processing device 40 to generate prediction information that predicts the occurrence of a remote control request requesting remote control of the robot 20 using a prediction method based on the number of times the robot 20 passes by.

[0110] 11 , when one robot 20 passes through two or more request occurrence areas at different times, the information processing device 40 generates prediction information for each of the two or more request occurrence areas. Also, as in the case of FIG. 11 , the information processing device 40 may set and change a reference time range of the past movement history of one or more robots 20 that is referenced to identify an unknown status area. Also, as in the case of FIG. 11 , the information processing device 40 may generate prediction information for each performance or type of robot 20.

[0111] 14 is a diagram showing an example of advance preparation information. The terminal device 30 generates advance preparation information representing the content shown in FIG.

[0112] In principle, pre-preparation operations are performed for each piece of prediction information. The operator selects one piece of prediction information from the prediction information confirmation screen and then performs pre-preparation operations for the selected piece of prediction information. The terminal device 30 displays the prediction ID, predicted time, and implementation status of the pre-preparation operations for each piece of prediction information. By referring to this information, the operator can confirm the content of the prediction information and the progress of the implementation of the pre-preparation operations. In the example of FIG. 14 , the predicted time is expressed as the time difference between the current time and the predicted time. In addition, in the example of FIG. 14 , the implementation status of the pre-preparation operations is expressed as a percentage, which becomes 100% when all of the pre-preparation operations are completed.

[0113] In addition, in this embodiment, the advance preparation operation includes a preparation operation for grasping the purpose of remote operation and a preparation operation for grasping the advance setting of an appropriate layout of the operation screen.

[0114] Furthermore, the preparatory operation for grasping the purpose of the remote operation includes an input operation of the reason for the remote operation request, an input operation of the start position for starting the remote operation, and an input operation of the end position for ending the remote operation. Furthermore, the preparatory operation for grasping the pre-setting of an appropriate operation screen layout includes an operation of switching the layout of the operation screen.

[0115] The advance preparation information includes, for each piece of prediction information, information indicating whether the input operation for the reason for the remote control request has been completed, whether the input operation for the start position at which the remote control will begin has been completed, and whether the input operation for the end position at which the remote control will end has been completed.

[0116] Furthermore, the terminal device 30 is capable of executing an operation to switch the layout of the operation screen on the condition that the input operation of the reason for the remote operation request, the input operation of the start position, and the input operation of the end position for ending the remote operation are completed. The advance preparation information includes information indicating whether the operation to switch the layout of the operation screen has been completed. Note that the advance preparation information may also include information indicating the elapsed time since the operation to switch the layout of the operation screen was completed. This allows the information processing device 40 to detect the operator who switched the layout of the operation screen earliest, i.e., the operator who completed all of the advance preparation operations earliest.

[0117] In this example, the implementation status of the preparatory operations is expressed as a percentage indicating the completion rate of the input operation for the reason for the remote operation request, the input operation for the start position for starting the remote operation, the input operation for the end position for ending the remote operation, and the operation for switching the layout of the operation screen. For example, in this example, the implementation status of the preparatory operations is 100% when all four operations are completed, 75% when three operations are completed, 50% when two operations are completed, 25% when one operation is completed, and 0% when none of the operations are completed.

[0118] The terminal device 30 transmits such advance preparation information to the information processing device 40, for example, periodically or every time the prediction information is updated.

[0119] FIG. 15 is a diagram showing an example of the prediction information confirmation screen.

[0120] The terminal device 30 may display to the operator, for example, a forecast information confirmation screen showing the contents shown in FIG.

[0121] The terminal device 30 displays currently valid prediction information, for example, prediction information for which the predicted time has not passed and the robot 20 has not issued a remote control request, on the prediction information confirmation screen. If there are multiple currently valid prediction information, the terminal device 30 lists the multiple prediction information and displays them on the prediction information confirmation screen. For example, the terminal device 30 displays a prediction ID, a predicted time, and the implementation status of the advance preparation operation as information identifying the prediction information.

[0122] Furthermore, the terminal device 30 may switch the prediction information to be displayed depending on the prediction method used to generate the prediction information. For example, the terminal device 30 may be able to select one or more of a prediction method based on whether the robot 20 will stop or slow down, a prediction method based on a request occurrence rate, and a prediction method based on the number of passes of the robot 20, and may display the prediction information predicted using the selected prediction method.

[0123] Furthermore, the terminal device 30 may be able to set a reference time range of past movement history referenced in generating the prediction information, as shown in the small screen for detailed settings in FIG. 15 . For example, the terminal device 30 can select prediction information generated by referencing movement history within the past hour from the current time, or can select prediction information generated by referencing movement history within an arbitrary set time period from the current time. In this case, the terminal device 30 displays prediction information generated by referencing past movement history within the selected reference time range.

[0124] 15, the terminal device 30 may be able to select the performance or type of the robot 20. In this case, the terminal device 30 extracts and displays the predicted information about the selected performance or type.

[0125] The terminal device 30 may also display, on the prediction information confirmation screen, a map showing a group of routes along which one or more robots 20 can travel. Furthermore, the terminal device 30 may display information identifying the position of a request occurrence area included in the prediction information on the map. For example, the terminal device 30 displays the information identifying the position of the request occurrence area by changing the color of or highlighting the request occurrence area included in the prediction information on the map. Furthermore, for example, the terminal device 30 may distinguish the prediction methods by changing the color or highlighting method for each type of prediction method in the prediction information. Furthermore, for example, the terminal device 30 may change the color or highlighting method depending on the progress of the implementation status of the advance preparation operation in the prediction information.

[0126] FIG. 16 is a diagram showing a first example of the advance preparation execution screen.

[0127] When any piece of forecast information is selected on the forecast information confirmation screen, the terminal device 30 displays a preparation execution screen as shown in FIG. 16 for performing preparation operations for the selected forecast information.

[0128] The terminal device 30 acquires the robot information and displays, on the advance preparation execution screen, information indicating the status in the request occurrence area included in the selected prediction information.

[0129] For example, the terminal device 30 displays on the advance preparation screen a video captured from any of the robots 20 when that robot 20 passes through the request generation area by remote control. For example, the terminal device 30 displays on the advance preparation screen a video captured from any of the robots 20 when that robot 20 passed through the request generation area by remote control in the most recent past from the current time. This allows the operator to understand whether or not an obstacle or the like is present in the request generation area included in the selected prediction information.

[0130] For example, the terminal device 30 displays a front image captured in front of the robot 20 in the direction of travel, a rear image captured in rear of the robot 20 in the direction of travel, a right side image captured in the right side of the robot 20 in the direction of travel, and a left side image captured in the left side of the robot 20 in the direction of travel. The terminal device 30 may also display a play button for playing the moving image, a stop button for stopping the playback of the moving image, a seek bar for playing from any time position, etc.

[0131] The terminal device 30 may also display a planar map of the request generation area, for example, a planar view of the entire range of the request generation area as seen from above. Furthermore, the terminal device 30 may display, on the planar map, obstacles on the route, the movement trajectory of the robot 20, the position of the robot 20 at the start of the video, the position of the robot 20 at the end of the video, and the position of the robot 20 at the playback time of the video.

[0132] Furthermore, the terminal device 30 may display, on the advance preparation screen, a reason input completion button to be pressed by the operator when the input operation of the reason for the occurrence of the remote operation request is completed. Furthermore, the terminal device 30 may display, on the advance preparation screen, a position input completion button to be pressed by the operator when the input operation of the start position for starting the remote operation and the input operation of the end position for ending the remote operation are completed.

[0133] FIG. 17 is a diagram showing a second example of the advance preparation execution screen.

[0134] The terminal device 30 may receive, as an input operation for indicating the reason for the remote control request, an input from the operator specifying the position of an obstacle that will hinder the movement of the robot 20 in the moving image displayed on the advance preparation execution screen. For example, the terminal device 30 receives a handwritten input by the operator of a line indicating the position of the obstacle in the moving image. The terminal device 30 also receives an input of a line indicating the position of the obstacle in a planar map of the request generation area.

[0135] The terminal device 30 also receives from the operator an input of a start position mark written in the planar map of the request occurrence area as an input operation of a start position where the remote operation starts. The terminal device 30 also receives from the operator an input of an end position mark written in the planar map of the request occurrence area as an input operation of an end position where the remote operation ends.

[0136] Furthermore, the terminal device 30 may receive, from the operator, input of a movement route when the robot 20 travels through the request generation area, within the planar map of the request generation area. The terminal device 30 may also receive input of text describing precautions to be referred to when the operator remotely operates the robot 20.

[0137] Furthermore, the terminal device 30 may detect the position and type of an object through video analysis using deep learning or the like on video image data, determine whether the object whose position and type have been detected is an obstacle on the route, and present the determination result to the operator. The terminal device 30 may then receive a confirmation result from the operator as to whether the determination result is correct. Furthermore, if the operator confirms that the determination result is incorrect, the terminal device 30 may receive an input from the operator to correct the determination result.

[0138] FIG. 18 is a diagram showing a third example of the advance preparation execution screen.

[0139] The terminal device 30 may accept, as a preparation operation, an input of information that there is no reason for a remote control request, instead of an input operation of the reason for a remote control request. For example, when the operator refers to a video of one of the robots 20 passing through a request generation area and finds that there is no obstacle, for example, in the request generation area and there is no reason for a remote control request, the operator inputs, as a preparation operation, information that there is no reason for a remote control request. In this way, when the operator inputs that there is no reason for a remote control request, but the remote control request is not actually issued, the information processing device 40 can determine that the operator has properly grasped the situation in the remote control area and is making preparations.

[0140] In addition, the terminal device 30 may not have any robots 20 that have passed through the request generation area by remote control in the past. In such cases, the terminal device 30 may display video captured from the robots 20 that have autonomously passed through the request generation area in the immediate past from the current time. This allows the operator to, for example, check whether there is a factor in the request generation area that may cause a remote control request, thereby identifying obstacles, etc., in the request generation area in advance and manually inputting the location of the obstacles or inputting a better movement route. If there is no robot 20 that has passed through the request generation area by remote control or autonomous movement in the immediate past from the current time (for example, if an area with an unknown status has been identified as the request generation area), the terminal device 30 may display the latest video captured from the robot 20 that is first to pass through the area. This allows the operator to quickly notice obstacles in the request generation area, manually inputting the location of the obstacles, or inputting a better movement route.

[0141] 19, 20, and 21 are diagrams showing examples of images displayed on the operation screen. Fig. 19 is a diagram showing an operation screen with a first-person perspective view. Fig. 20 is a diagram showing an operation screen with an omnidirectional view. Fig. 21 is a diagram showing an operation screen that combines a first-person perspective view and an omnidirectional view.

[0142] During remote control, the terminal device 30 displays an operation screen including real-time images captured by a camera mounted on the robot 20.

[0143] The layout of the operation screen varies depending on the arrangement and type of cameras mounted on the robot 20. For example, the layout of the operation screen may be a first-person perspective view including a front image, a rear image, a right side image, and a left side image, as shown in Fig. 19. Alternatively, for example, the layout of the operation screen may be an omnidirectional view including a 360° image in all directions, as shown in Fig. 20. Alternatively, for example, the layout of the operation screen may be a view that combines a first-person perspective view including a front image and a rear image with an omnidirectional view including a 360° image in all directions, as shown in Fig. 21.

[0144] Then, the terminal device 30 accepts an operation to switch the layout of the operation screen as a preparatory operation. For example, by selecting the type of robot 20, the terminal device 30 switches to an operation screen with an appropriate layout for remotely operating the selected robot 20.

[0145] When a remote control request is made, the information processing device 40 determines whether remote control can be performed appropriately using an operation screen with a first-person perspective view, an operation screen with an omnidirectional view, or an operation screen with a combination of a first-person perspective view and an omnidirectional view, based on the type of the robot 20 that made the remote control request. The information processing device 40 may preferentially select, as the operation performing operator, an operator who has switched to an appropriate operation screen layout from among the multiple operators. Furthermore, the information processing device 40 may preferentially select, as the operation performing operator, an operator who has a large track record of performing operations using an appropriate operation screen from among the multiple operators.

[0146] FIG. 22 is a diagram showing an example of an operation screen during remote operation.

[0147] During remote operation, the terminal device 30 displays the information input on the advance preparation screen together with the operation screen. For example, the information processing device 40 stores information input in past moving images in association with the position information of the robot 20 at the time of input. The terminal device 30 displays the input information on the real-time operation screen when the distance between the position information of the robot 20 to be operated and the position information of the robot 20 at the time of input becomes less than a certain value.

[0148] For example, the terminal device 30 displays the position of an obstacle input as the reason for the remote control request, superimposed on a real-time image captured by the remotely controlled robot 20. Also, for example, the terminal device 30 displays the optimal travel route, start position mark, and end position mark for the robot 20 when traveling through the request generation area, superimposed on the real-time image displayed on the operation screen. Furthermore, the terminal device 30 displays text describing precautions that the operator should refer to when remotely operating the robot 20, on the operation screen.

[0149] For example, the information processing device 40 can perform camera calibration for the camera's lens distortion, mounting position, etc., and calculate distance information for each pixel of the captured image based on the camera calibration results. Alternatively, the information processing device 40 may calculate distance information for each pixel of a two-dimensional image using deep learning image processing technology that converts distance information in three-dimensional space into an estimated distance image. The information processing device 40 combines the distance information obtained from the image using such distance estimation technology with the position information of the robot 20 at the time of input, thereby linking information input to past video images, such as the positions of obstacles, the travel route, the start position mark, and the end position mark, to map information and storing the position information.

[0150] The terminal device 30 then displays the input information on the real-time operation screen when the distance between the position information of the robot 20 being operated and the position information of the robot 20 at the time of input becomes less than a certain value. In this case, the terminal device 30 calculates relative position information based on the current position of the robot 20 and the position information linked to the input information for the past video. The terminal device 30 then calculates the pixel position at which to superimpose the input information on the real-time image based on the calculated relative position information and the camera calibration result.

[0151] Furthermore, even if the layout of the past video used on the advance preparation implementation screen is different from the layout of the operation screen, more specifically, even if the type and installation position of the camera that captured the past video is different, by obtaining the results of camera calibration linked to map information and stored by the information processing device 40, the terminal device 30 can superimpose and display input information at the appropriate pixel position on the real-time image.

[0152] Furthermore, if the layout of the past video used on the advance preparation screen differs from the layout of the operation screen, the terminal device 30 displays the advance preparation screen in a window separate from the operation screen. In this case, the terminal device 30 automatically selects a time of past video in which the robot 20 was positioned close by, and presents it to the operator during remote operation along with the information entered on the advance preparation screen.

[0153] FIG. 23 is a diagram showing an example of an evaluation screen displayed on the terminal device 30, which displays a preparation implementation screen created by another operator.

[0154] The terminal device 30 may acquire from the information processing device 40 a pre-preparation execution screen created by one or more other operators and display it on the evaluation screen. When the number of operators is large relative to the number of remote operation requests, each of the multiple operators will have fewer opportunities to perform remote operation. Even in such a situation, highly motivated operators can check the pre-preparation operations already completed by other operators and improve their own skills. This allows operators to refer to the pre-preparation operations performed by operators with high evaluations and use them as reference for their own pre-preparation operations, for example. Furthermore, the information processing device 40 may reward the operators who input evaluations after checking the pre-preparation operations already completed by other operators when they input evaluations in favor of the pre-preparation operations and actually perform the operations, or when they input evaluations in opposition to the pre-preparation operations and actually perform the operations, for example, when there are no remote operation requests.

[0155] Furthermore, the terminal device 30 may receive, via the evaluation screen, information indicating whether the operator approves of a preparation operation performed by one or more other operators. For example, the operator refers to the preparation implementation screen created by another operator, and if the preparation operation is good, for example, if the input of the reason for the remote operation request is appropriate, the created route is appropriate, or the notes written in text are appropriate, the operator inputs information indicating approval of the preparation operation via the evaluation screen.

[0156] Furthermore, the information processing device 40 may use an operator route, which is a route created in advance by an operator and a route traveled by operation of the operator, to improve the performance of autonomous movement by the robot 20. For example, the information processing device 40 may replace a route generated during autonomous movement with the operator route. Furthermore, the information processing device 40 may correct the route generated during autonomous movement by a predetermined amount each time the same route is passed so as to bring the route closer to the operator route.

[0157] The information processing device 40 collects information indicating whether or not each of the operators agrees, which information is input by each of the operators, and stores the information in association with the advance preparation information generated by each of the operators.

[0158] When selecting an operation performing operator from among a plurality of operators, the information processing device 40 may select an operator who has performed a preparatory operation with a high degree of approval as the operation performing operator, over an operator who has performed a preparatory operation with a low degree of approval. In this way, the information processing device 40 can select an operator who has performed an appropriate preparatory operation from among the plurality of operators as the operation performing operator, and have the operator appropriately perform remote operation of the robot 20.

[0159] Furthermore, the terminal device 30 may display tutorial information to an operator with little experience, such as an operator logging in for the first time, or to an operator who has not logged in for a certain period of time. The terminal device 30 may present, as the tutorial information, for example, content recommending that the operator check the preparation implementation screen used by an operator who performed a preparation operation that received a high level of approval. The terminal device 30 may also present, as the tutorial information, for example, by playing back a video captured of a series of operations performed when creating a preparation operation that received a high level of approval.

[0160] Fig. 24 is a flowchart showing the flow of a selection process of an operation performing operator by the information processing device 40. The information processing device 40 executes the process, for example, according to the flow shown in Fig. 24, as a process of selecting an operator who has performed a preparatory operation with a high degree of approval as an operation performing operator, in preference to an operator who has performed a preparatory operation with a low degree of approval.

[0161] First, in S81, the information processing device 40 determines whether or not a remote control request issued by any of the robots 20 has been confirmed. If the request has not been confirmed (No in S81), the information processing device 40 puts the process on hold in S81. If the request has been confirmed (Yes in S81), the information processing device 40 proceeds to S82.

[0162] In S82, the information processing device 40 determines whether or not there is an operator who has input the reason for the remote control request and who has completed the advance preparation operation (a completion operator). If there is a completion operator who has input the reason for the remote control request (Yes in S82), the information processing device 40 proceeds to S83. If there is not a completion operator who has input the reason for the remote control request (No in S82), the information processing device 40 proceeds to S86.

[0163] In S83, the information processing device 40 compares the number of operators who agreed with the completion operator who input the reason for the remote operation request, and determines whether there is a completion operator whose number of operators who agreed (number of agreements) is significantly different from other completion operators. Having a significantly different number of agreements from other completion operators means, for example, that the number of agreements is the largest and the difference from the second-highest number of agreements or the average number of agreements is greater than a predetermined value or a predetermined percentage.

[0164] If there is a completion operator who has entered the reason for the remote control request and has a significantly higher number of approvals than other completion operators (Yes in S83), in S84, the information processing device 40 selects the completion operator who has entered the reason for the remote control request and has the most approvals as the operation execution operator, and terminates this flow.

[0165] If there is no completion operator who has entered the reason for the remote control request and has a significantly higher number of supporters than other completion operators (No in S83), in S85, the information processing device 40 selects the completion operator who has completed the preliminary preparation operation earliest among the completion operators who have entered the reason for the remote control request as the operation execution operator, and terminates this flow.

[0166] In S86, it is determined whether there is an operator (completion operator) who has input that there is no reason for a remote operation request to be generated and who has completed the advance preparation operation. If there is a completion operator who has input that there is no reason for a remote operation request to be generated (Yes in S86), the information processing device 40 proceeds to S87. If there is no completion operator who has input that there is no reason for a remote operation request to be generated (No in S86), the information processing device 40 proceeds to S90.

[0167] In S87, the information processing device 40 compares the number of operators who agreed with the completion operators who input that there is no reason for a remote control request to be generated, and determines whether there is a completion operator who has a significantly higher number of agreements than the other completion operators and has input that there is no reason for a remote control request to be generated. If there is a completion operator who has a significantly higher number of agreements than the other completion operators and has input that there is no reason for a remote control request to be generated (Yes in S87), in S88, the information processing device 40 selects the completion operator who has the most agreements and has input that there is no reason for a remote control request to be generated as the operation execution operator, and ends this flow.

[0168] If there is no completion operator who has a significant difference in the number of approvals compared to other completion operators and has input that there is no reason for a remote control request to occur (No in S87), in S89, the information processing device 40 selects the completion operator who completed the pre-preparation operation earliest among the completion operators who have input that there is no reason for a remote control request to occur as the operation execution operator, and terminates this flow.

[0169] In S90, the information processing device 40 determines whether there is an operator who is performing the advance preparation operation but has not yet completed it. If there is an operator who is performing the advance preparation operation (Yes in S90), the information processing device 40 selects the operator who is most advanced in the advance preparation operation or the operator who started the advance preparation operation earliest as the operator who will perform the operation, and ends this flow.

[0170] If no operator is currently performing the preparatory operation (No in S90), in S92 the information processing device 40 selects, from the multiple operators, the operator with the most experience operating the robot 20 that issued the remote operation request as the operation executing operator. Alternatively, the information processing device 40 may select, from the multiple operators, the operator with the most experience operating the robot 20 that issued the remote operation request as the operation executing operator, for example. Alternatively, the information processing device 40 may select, from the multiple operators, the operator with the most experience operating the robot 20 in the request issuing area as the operation executing operator using another method. When the information processing device 40 completes the processing of S92, it ends the flow.

[0171] In this way, when selecting an operator to perform an operation from among multiple operators, the information processing device 40 can select an operator who performed a pre-preparation operation with a lot of approval as the operator to perform the operation, giving priority to an operator who performed a pre-preparation operation with less approval.

[0172] The number of approvals is an example of a value for evaluating the quality of the preparatory operation. Instead of or in addition to selecting an operator who performed a preparatory operation with a large number of approvals as an operation performing operator over an operator who performed a preparatory operation with a small number of approvals, the information processing device 40 may select an operation performing operator based on another value for evaluating the quality of the preparatory operation.

[0173] For example, the information processing device 40 may use the number of times that cautions to be observed in remote control are input as a value for evaluating the quality of the advance preparation operation. In this case, the information processing device 40 determines that an operator who has inputted a large number of cautions to be observed in remote control is an operator with high risk awareness, and preferentially selects the operator to perform the operation.

[0174] Furthermore, for example, the information processing device 40 may use the playback speed of the moving image as a value for evaluating the quality of the advance preparation operation. In this case, the information processing device 40 preferentially selects an operator who plays back the moving image at a playback speed slower than the standard speed in the advance preparation operation as an operator who checks risks in detail, as an operator who will perform the operation.

[0175] Furthermore, for example, the information processing device 40 may use the time required for the advance preparation operation as a value for evaluating the quality of the advance preparation operation. In this case, the information processing device 40 preferentially selects an operator who has spent a long time on the advance preparation operation as the operator to perform the operation.

[0176] Furthermore, for example, the information processing device 40 may use the number of times that the video has been repeatedly played as a value for evaluating the quality of the preparatory operation. In this case, the information processing device 40 may preferentially select an operator who has repeatedly played back the video many times in the preparatory operation as the operator performing the operation.

[0177] Furthermore, for example, the information processing device 40 may calculate an evaluation value representing a composite value of the above information as a value for evaluating the quality of the pre-preparation operation, and may select an operator with a high value for evaluating the quality of the pre-preparation operation as the operator to perform the operation, in preference to an operator with a low value for evaluating the quality of the pre-preparation operation.

[0178] 25 is a diagram showing the results of tallying the judgments of whether the prediction information will be "right" or "wrong." The information processing device 40 may tally and manage the judgments of a plurality of operators as to whether the prediction information will be "right" or "wrong."

[0179] If the operator determines that there is an obstacle or the like in the request generation area and that the robot 20 will issue a remote control request, he or she will input the reason for the remote control request, assuming that the predicted information will be "correct" as predicted by the information processing device 40. On the other hand, if there is no obstacle or the like in the request generation area and the operator determines that the robot 20 will not issue a remote control request, he or she will input that there is no reason for the remote control request, assuming that the predicted information will be "wrong."

[0180] The information processing device 40 counts the number of operators who input the reason for the occurrence of a remote control request for each piece of prediction information as the number of people who judged that the prediction based on the prediction information would be "correct." In addition, the information processing device 40 counts the number of operators who input that there is no reason for the occurrence of a remote control request as the number of people who judged that the prediction based on the prediction information would be "wrong."

[0181] The terminal device 30 may then obtain from the information processing device 40 the results of tallying the number of people who judged that the prediction based on the prediction information would be "correct" and the number of people who judged that the prediction based on the prediction information would be "wrong," and display this to the operator.

[0182] Furthermore, the information processing device 40 calculates, for each piece of prediction information, the ratio of the number of people who judged that the prediction based on the prediction information would be "correct" to the total number of people who judged that the prediction based on the prediction information would be "correct" and the number of people who judged that the prediction based on the prediction information would be "wrong" as the occurrence probability. The terminal device 30 may also acquire the occurrence probability for each piece of prediction information from the information processing device 40 and display it to the operator together with the aggregated results of the judgment of whether the prediction information would be "correct" or "wrong".

[0183] The information processing device 40 may store log data representing a pair of information on which the operator has determined whether the prediction will be "correct" or "incorrect" and the determination result of whether the prediction will be "correct" or "incorrect" for each determination made by the operator. Furthermore, the information processing device 40 may perform machine learning using the information on which the operator has determined whether the prediction will be "correct" or "incorrect" as input and the determination result of whether the prediction will be "correct" or "incorrect" as output, thereby generating multiple AIs (artificial intelligences) that make determinations on behalf of the operator. The information processing device 40 may then estimate the occurrence probability based on the determination results of the multiple AIs. This allows the information processing device 40 to calculate the estimated occurrence probability even if the number of operators actually making the determination is small.

[0184] For example, as shown in FIG. 25, the terminal device 30 may display, in a table format, for each piece of prediction information, the prediction ID, predicted time, occurrence probability, robot ID, reservation completion information, the number of people who judged it to be a "win," the number of people who judged it to be a "lose," and the number of operators currently performing advance preparation operations. The robot ID is information that identifies the robot 20. The reservation completion information indicates "full" if advance preparation has been completed and an operator is present who is displaying the operation screen for operating the robot 20 of the type indicated by the robot ID. Furthermore, the reservation completion information indicates "empty" if no such operator is present.

[0185] By displaying information such as that shown in FIG. 25, the terminal device 30 can prompt the user to perform advance preparation operations for reservation information for which there is a high possibility of remote operation being performed, and for which no operator has completed the advance preparation operations.

[0186] Furthermore, by displaying information such as that shown in Fig. 25, the terminal device 30 can provide the operator with information that serves as a guide for determining, from among a plurality of pieces of prediction information, which prediction information the operator will use to perform the advance preparation operation and which prediction information the operator will use to evaluate the advance preparation operation of another operator. For example, by being provided with information such as that shown in Fig. 25, the operator can determine the prediction information to perform the advance preparation operation when the reservation completion is "empty," and the prediction information to evaluate the advance preparation operation of another operator when the reservation completion is "full." Furthermore, the terminal device 30 may display, for each piece of prediction information, the estimated time required for the advance preparation operation and the estimated time required for evaluation, with reference to past history.

[0187] FIG. 26 is a diagram showing the schedule of the robot 20 and advance preparation information.

[0188] There are cases where multiple robots 20 pass through one area at different times. For example, in the example schedule shown in Fig. 26, robots 20 A10, A11, A12, and B11 pass through area R07 at different times.

[0189] In such a case, the terminal device 30 may reflect the advance preparation operation performed for one piece of reservation information as an advance preparation operation for the reservation information of another robot 20 passing through the same request generation area, thereby enabling the operator to perform advance preparation operations efficiently without repeating the same work.

[0190] For example, the terminal device 30 may reflect a preparatory operation for understanding the purpose of remote control, which has been performed for one piece of reservation information, as a preparatory operation for reservation information for another robot 20 passing through the same request occurrence area. More specifically, when the terminal device 30 performs an input operation for the reason for the remote control request, an input operation for the start position for starting the remote control, and an input operation for the end position for ending the remote control, which have been performed for one piece of reservation information, the terminal device 30 also reflects the information in the reservation information for the other robot 20 passing through the same request occurrence area.

[0191] Note that the operation of switching the layout of the operation screen, which is a preparatory operation for grasping the appropriate preset layout of the operation screen, differs for each type of robot 20. In such a case, even if the preparatory operation for grasping the appropriate preset layout of the operation screen is completed, the terminal device 30 considers that the preparatory operation for grasping the appropriate preset layout of the operation screen has not been completed for reservation information about a robot 20 with a different operation screen.

[0192] Furthermore, the terminal device 30 may display a predicted frequency of remote operation requests within a predetermined time period, for example, within one hour, for each area on the map displayed on the reservation information confirmation screen. This allows the terminal device 30 to inform the operator of areas where advance preparation operations can be performed efficiently for multiple pieces of reservation information.

[0193] The terminal device 30 may set an upper limit on the number of pieces of prediction information for which the operator can perform advance preparation operations. This prevents the terminal device 30 from mistaking the prepared content for another remote control request due to the operator performing advance preparation operations for a large number of pieces of prediction information, thereby preventing the operator from being unable to perform remote control efficiently.

[0194] For example, when an operator performs a preparatory operation for any of the prediction information, the terminal device 30 increments the count value by 1. When it is determined that the prediction information for which the operator performed a preparatory operation is to be assigned to another operator, or when remote control for the prediction information assigned to the terminal device 30 is completed, the terminal device 30 decrements the count value by 1. When the count value is less than the upper limit, the terminal device 30 allows the operator to perform a preparatory operation for a new remote control request.

[0195] FIG. 27 is a diagram illustrating the process of giving a reward.

[0196] The information processing device 40 performs a process of giving a reward to the operation performing operator who remotely operates the robot 20.

[0197] Furthermore, the information processing device 40 may perform a process of rewarding an operator other than the operation-performing operator who has appropriately performed the preparatory operation. For example, when a remote operation is performed by another operator, the information processing device 40 may perform a process of rewarding an operator who has input the reason for the remote operation request, completed the preparatory operation, and has a number of approvals equal to or greater than a predetermined value. This allows the information processing device 40 to motivate each of the multiple operators to appropriately and promptly perform the preparatory operation.

[0198] Furthermore, the information processing device 40 may provide a reward to an operator who performed a preparatory operation but was not selected as an operation execution operator, according to the degree of progress or quality of the preparatory operation. Furthermore, the information processing device 40 may determine a reward for the operator at the stage when the preparatory operation is completed. By proactively paying a reward to operators who performed a preparatory operation in this way, the information processing device 40 can increase the overall number of operators who perform preparatory operations and reduce service downtime, i.e., the time the robot 20 waits until it is remotely operated.

[0199] In addition, if remote operation is not performed by any operator, that is, if a remote operation request is not made from the robot 20, the information processing device 40 may perform a process of giving a reward to an operator who inputs that there is no reason for a remote operation request to be made.

[0200] From the perspective of the administrator who manages the operation of the robot 20, it is ideal to allow the robot 20 to move by itself in areas where autonomous movement is not a problem, and to have an operator support the robot 20 in moving only in areas where autonomous movement is difficult. This allows the administrator to reduce the amount of work done by the operator and lower costs. Therefore, the administrator expects the operator to remotely control the robot 20 in areas where autonomous movement is difficult, and also expects the operator to clarify whether the area is difficult or not for the robot 20 to move autonomously. Therefore, the information processing device 40 can reward the operator who realizes the expectation of clarifying whether the area is difficult or not for the robot 20 to move autonomously by rewarding the operator who inputs that there is no reason for the remote operation request.

[0201] Furthermore, if a reward is given only when the robot 20 is remotely controlled, the operator may always input the reason for the remote control request so that the operator can receive the reward. However, if no remote control request is made by the robot 20, the administrator can make the operator more appropriately determine whether or not a remote control request is made by giving a reward to the operator who inputs that there is no reason for the remote control request.

[0202] Furthermore, the information processing device 40 may provide a reward to other operators who agree with the operator who performed the remote operation. Because the remote operation is performed by a single operator, if another operator has already completed the preparatory operations and the preparatory operations are appropriate, the other operator who completed the preparatory operations earlier is likely to perform the remote operation. Therefore, the operator may only perform the preparatory operations for prediction information for which another operator has already completed the preparatory operations and for which the preparatory operations are appropriate, and may not perform the preparatory operations, but may proceed with the preparatory operations for other prediction information for which the operator is more likely to receive a reward. In this manner, the information processing device 40 can proceed with the preparatory operations for a large number of prediction information, thereby enabling smooth operation control of multiple robots 20.

[0203] Furthermore, the information processing device 40 may impose a penalty on an operator who has input a reason for a remote control request a predetermined number of times or more even though no remote control request has been issued from the robot 20, and an operator who has input a reason for no remote control request a predetermined number of times or more even though a remote control request has been issued from the robot 20. For example, the information processing device 40 may, as a penalty, have the operator review advance preparation information by an operator who made a correct decision, or review a video or the like showing the operation performed by an operator who performed remote control. In this way, the information processing device 40 can provide a learning opportunity to an operator who did not make a correct decision.

[0204] FIG. 28 is a diagram for explaining the contents of a specific example of the process of giving a reward.

[0205] The information processing device 40 may provide the operator with a reward in accordance with the correct input of the reason for the occurrence of the remote control request. In this case, the information processing device 40 may provide a larger reward the more consecutively the operator correctly inputs the reason for the occurrence of the remote control request.

[0206] For example, assume that a first operator has responded to six pieces of forecast information as shown in FIG. 28 within one hour.

[0207] Specifically, for the first piece of prediction information, the first operator inputs the reason for the remote operation request and performs the remote operation.

[0208] For the second piece of forecast information, the first operator inputs the reason for the request for remote control, but another operator will perform the remote control.

[0209] For the third and fourth pieces of prediction information, the first operator inputs that there is no reason for a remote control request to be issued, and the robot 20 does not actually issue a remote control request.

[0210] For the fifth piece of prediction information, the first operator inputs the reason for the request for remote control, but another operator will perform the remote control.

[0211] For the sixth piece of prediction information, the first operator inputs the reason for the remote operation request and performs the remote operation.

[0212] In the above case, the information processing device 40 gives 500 yen as a reward for one remote operation. Therefore, in the above case, the information processing device 40 gives the first operator 500 yen x 2 as a reward for performing the remote operation.

[0213] In the above case, the information processing device 40 rewards the first operator with 100 yen per input for correctly inputting the reason for the remote control request less than five times in a row. In addition, the information processing device 40 rewards the first operator with 125 yen per input from the fifth input onwards for correctly inputting the reason for the remote control request five or more times in a row. Therefore, in the above case, the information processing device 40 rewards the first operator with 100 yen x 4 + 125 yen x 2 yen for correctly inputting the reason for the remote control request.

[0214] As a result, in the above case, the information processing device 40 gives a total of 1,650 yen to the first operator.

[0215] In this way, by giving a larger reward the more consecutively correct inputs of the reason for the remote control request, the information processing device 40 can encourage the operator to input the reason for the remote control request more accurately.

[0216] The information processing device 40 may change the basic reward depending on the remote operation time, the type of robot 20, etc. In this case, the terminal device 30 may display multiple pieces of predicted information sorted in order of the magnitude of the basic reward. The terminal device 30 may also display multiple pieces of predicted information sorted by the remote operation time, the type of robot 20, etc. The terminal device 30 may also accept a minimum reward setting from the operator in advance. In this case, the terminal device 30 may present to the user predicted information that will earn a reward higher than the set minimum reward. By displaying the information processing device 40 and the terminal device 30 in this manner, the user can prioritize advance preparation for operations with high rewards, or can select predicted information for advance preparation depending on the remote operation time, the type of robot 20, etc. desired by the user.

[0217] FIG. 29 is a diagram showing the hardware configuration of the terminal device 30.

[0218] The terminal device 30 has a hardware configuration that utilizes a normal computer, with a CPU (Central Processing Unit) 102, a ROM (Read Only Memory) 104, a RAM (Random Access Memory) 106, an I / F 108, an input device 110, and a display device 112, etc., interconnected by a bus.

[0219] The CPU 102 is a computing device that controls the terminal device 30. The ROM 104 stores programs and the like that enable the CPU 102 to process information. The RAM 106 stores data necessary for various processes by the CPU 102. The I / F 108 is an interface that is connected to a storage unit, a sensor, a communication unit, and the like, and is used to send and receive data. The input device 110 accepts information input from an operator. The display device 112 displays information to the operator.

[0220] In the terminal device 30, the CPU 102 reads out a program from the ROM 104 onto the RAM 106 and executes it, thereby realizing each function of the terminal device 30 on the computer.

[0221] The programs for executing the processes executed by the terminal device 30 may be stored in a hard disk drive (HDD). Also, the programs for executing the processes executed by the terminal device 30 may be provided in advance by being stored in the ROM 104.

[0222] Furthermore, the program for executing the processing executed by the terminal device 30 may be provided as a computer program product by being stored in an installable or executable file on a computer-readable storage medium such as a CD-ROM, CD-R, memory card, DVD (Digital Versatile Disk), or flexible disk (FD). Furthermore, the program for executing the information processing executed by the terminal device 30 may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Furthermore, the program for executing the information processing executed by the terminal device 30 may be provided or distributed via a network such as the Internet.

[0223] Similarly to the terminal device 30, the information processing device 40 and the service system 50 also have a hardware configuration using a computer. However, the information processing device 40 and the service system 50 may have a configuration in which a plurality of computers operate in cooperation with each other.

[0224] Although the embodiments have been described above, they are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. This embodiment and its modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as defined in the claims.

[0225] REFERENCE SIGNS LIST 10 Robot control system 20 Robot 30 Terminal device 40 Information processing device 50 Service system 60 First communication unit 62 Robot information acquisition unit 64 Storage unit 66 Prediction information generation unit 68 Operator information acquisition unit 70 Operator selection unit 72 Operation request generation unit 74 Reward processing unit 76 Second communication unit 78 Prediction information confirmation screen processing unit 80 Advance preparation implementation screen processing unit 82 Advance preparation information generation unit 84 Operation screen processing unit 86 Operation information generation unit 88 Terminal control unit

Claims

1. An operator selection method for selecting, from a plurality of operators, an operation performing operator to remotely operate an autonomously mobile robot, wherein a terminal device used by each of the plurality of operators displays prediction information predicting the occurrence of a remote operation request requesting remote operation of the robot and accepts advance preparation operations for remotely operating the robot, and an information processing device, when the remote operation request occurs, selects one of the plurality of operators as the operation performing operator based on the implementation status of the advance preparation operations by each of the plurality of operators, and the information processing device causes the operation performing operator to remotely operate the robot using the terminal device.

2. The operator selection method according to claim 1, wherein the information processing device predicts a request occurrence area in which the remote control request will occur in the robot, and the terminal device displays the status of the request occurrence area and accepts input of the reason for the remote control request as the advance preparation operation.

3. The operator selection method according to claim 2, wherein the terminal device displays past video images of when the robot passed through the request generation area by remote control and a map of the request generation area, and accepts input for the video images or the map specifying the location of an obstacle that will hinder the robot's movement as the reason for the remote control request.

4. The operator selection method according to claim 3, wherein the terminal device accepts an input of "no reason for the remote control request" instead of the reason for the remote control request.

5. The operator selection method according to claim 1, wherein the information processing device predicts a request occurrence area where the remote control request for the robot will occur, and the terminal device accepts input of a start position where the remote control will begin and an end position where the remote control will end in the request occurrence area as the advance preparation operation.

6. The operator selection method according to claim 5, wherein the terminal device displays past video images of when the robot passed through the request occurrence area by remote control and a map of the request occurrence area, and accepts input of the robot's route in the request occurrence area onto the map as the advance preparation operation.

7. The operator selection method according to claim 1, wherein the terminal device accepts, as the advance preparation operation, an operation to display an operation screen for remotely operating the robot.

8. The operator selection method according to claim 1, wherein when the remote operation request occurs, the information processing device selects, from among the plurality of operators, one of the operators who has completed all operations predetermined in the advance preparation operations as the operation performing operator.

9. The operator selection method according to claim 1, wherein when the remote operation request occurs, the information processing device selects, from among the plurality of operators, an operator with a high quality of the advance preparation operation as the operator to perform the operation, in preference to an operator with a low quality of the advance preparation operation.

10. The operator selection method according to claim 1, wherein the information processing device executes a process of giving a reward to the operator who performed the remote operation.

11. The operator selection method according to claim 4, wherein, if the remote control request does not occur, the terminal device executes a process of rewarding an operator who inputs information indicating that there is no reason for the remote control request to occur as the preparatory operation.

12. The operator selection method according to claim 1, wherein the information processing device generates the prediction information including the request generation area where the remote control request will be generated and the predicted time when the remote control request will be generated based on the schedule and past movement history of the robot, and transmits the prediction information to the terminal device used by each of the multiple operators.

13. The operator selection method according to claim 12, wherein a plurality of the robots are in operation, the information processing device generates the prediction information for each of the robots and each of the request generation areas, and the terminal device accepts the advance preparation operation for each of the prediction information.

14. A terminal device used by one of a plurality of operators who remotely operate an autonomously mobile robot, the terminal device displays prediction information that predicts the occurrence of a remote operation request that requests remote operation of the robot, accepts advance preparation operations for remotely operating the robot, and transmits advance preparation information indicating the implementation status of the advance preparation operations to an information processing device, and when the operator is selected by the information processing device from among the plurality of operators as an operation performing operator who will remotely operate the robot, transmits operation information corresponding to the operation by the operation performing operator to the robot.

15. A program for causing an information processing device to select an operation performing operator from among a plurality of operators to remotely operate an autonomously mobile robot, wherein a terminal device used by each of the plurality of operators displays predictive information predicting the occurrence of a remote operation request requesting remote operation of the robot and accepts advance preparation operations for remotely operating the robot, and when the remote operation request occurs, the information processing device selects one of the plurality of operators as the operation performing operator based on the implementation status of the advance preparation operations by each of the plurality of operators, and causes the operation performing operator to remotely operate the robot using the terminal device.

Images