Information processing method, information processing device, and program

Abstract

This information processing method includes: acquiring a remote response request for each of a plurality of moving bodies (S11); acquiring a response required time and a response completion extension time for each of two or more remote response requests that have not been responded to (S13); determining whether there is a remote response order in which the remote response of each of the two or more remote response requests can be completed by each response completion extension time (S14, S16); and, when a determination is made that the there is a remote response order, executing a predetermined processing for the two or more remote response requests on the basis of the remote response order (S19).

Description

Information processing method, information processing device, and program

[0001] The present disclosure relates to an information processing method, an information processing device, and a program.

[0002] Patent Document 1 discloses a remote control device and an automatic driving system that prevents a decrease in vehicle safety when an operator is unable to intervene in the control of an autonomously driven vehicle.

[0003] Japanese Patent Application Laid-Open No. 2020-154631

[0004] When one operator remotely controls multiple mobile objects, the operator may be notified of multiple remote response requests. If there is a completion time (response completion grace time) for completing each of the multiple remote response requests, it is desirable for the single operator to complete each of the multiple remote response requests by the completion time.

[0005] However, Patent Document 1 does not disclose a technique for allowing one operator to complete the remote response of each of a plurality of remote response requests by the completion time.

[0006] Therefore, the present disclosure provides an information processing method, an information processing device, and a program that support one operator in completing the remote response of each of two or more remote response requests by the completion time.

[0007] An information processing method according to one aspect of the present disclosure acquires remote response requests for each of a plurality of mobile bodies, acquires the required response time and the response completion grace time for each of two or more unaddressed remote response requests, determines whether there is a remote response order that allows the remote response of each of the two or more remote response requests to be completed by the respective response completion grace times, and if it is determined that such a remote response order exists, executes a predetermined process for the two or more remote response requests based on the remote response order.

[0008] An information processing device according to one aspect of the present disclosure includes a first acquisition unit that acquires remote response requests for each of a plurality of mobile bodies, a second acquisition unit that acquires the required response time and the response completion grace time for each of two or more unaddressed remote response requests, a judgment unit that determines whether there is a remote response order that can complete the remote response for each of the two or more remote response requests by the respective response completion grace times, and a control unit that, when it is determined that the remote response order exists, executes a predetermined process for the two or more remote response requests based on the remote response order.

[0009] A program according to one aspect of the present disclosure is a program for causing a computer to execute the above-described information processing method.

[0010] According to one aspect of the present disclosure, it is possible to realize an information processing method, etc., that can assist one operator in completing remote responses to two or more remote response requests by the completion time.

[0011] FIG. 1 is a diagram illustrating a schematic configuration of an information processing system according to an embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the information processing system according to an embodiment. FIG. 3 is a flowchart illustrating the operation of the information processing system according to an embodiment. FIG. 4 is a diagram illustrating a method for determining a remote response order when multiple remote responses can be completed by remote responses by operators alone. FIG. 5 is a diagram illustrating a method for determining a remote response order when multiple remote responses are simultaneously notified to an operator. FIG. 6 is a diagram illustrating an example of a method for determining a remote response order when vehicle control is performed. FIG. 7 is a diagram illustrating another example of a method for determining a remote response order when vehicle control is performed. FIG. 8 is a diagram illustrating a method for determining a remote response order when a request for emergency response is made. FIG. 9 is a flowchart illustrating the operation of an information processing system according to a modified example of an embodiment. FIG. 10 is a diagram illustrating a method for determining a readjusted remote response order. FIG. 11 is a first diagram illustrating an example of a display on an operator terminal according to an embodiment. FIG. 12 is a second diagram illustrating an example of a display on an operator terminal according to an embodiment. FIG. 13 is a third diagram illustrating an example of a display on an operator terminal according to an embodiment.

[0012] (Background to the Invention of the Present Disclosure) Before describing the embodiments of the present disclosure, the background to the invention of the present disclosure will be described.

[0013] As explained in the "Problem to be Solved by the Invention" section, a remote control system in which one operator remotely controls multiple mobile objects is being considered. For example, when a mobile object is in a situation where it cannot move autonomously, a remote control server notifies the operator of a remote response request. The remote response request is a request to an operator located remotely from the mobile object to remotely respond to the movement of the mobile object (for example, remotely control the mobile object).

[0014] When the operator receives the notification, the operator determines whether to respond, the timing of the response, the content of the response, etc., and then performs the remote response. For example, if the operator is notified of multiple remote response requests, the multiple remote response request notifications may be displayed simultaneously on the screen. In this case, the operation flow is that the operator checks the screen on which the multiple remote response request notifications are displayed simultaneously, determines the order in which to respond, and responds to each notification in order.

[0015] Here, the remote response request has different response time requirements (requested response time) and response completion time requirements (response completion grace period) depending on its content. Some remote response requests have long or short response time requirements, and some have long (future) or short (close) response completion grace periods. For example, a remote response request with a short response time and short response completion grace period is a request when a fallen object is detected. An example of the remote response in this case is a vehicle stop instruction (e.g., a temporary stop instruction). Another example of a remote response request with a short response time and a long response completion grace period is a request to confirm the start of travel during forwarding. An example of the remote response in this case is a travel start instruction. Another example of a remote response request with a long response time and a short response completion grace period is a request when the vehicle gets stuck on an obstacle. An example of the remote response in this case is avoidance by remote control (remote operation). Furthermore, for example, an example of a remote response request with a long response request time and response completion grace period is a request for remote control to a standby position after the service is completed. In this case, the remote response is, for example, remote control to the standby position.

[0016] In this way, when multiple remote response requests with different response request times and response completion grace periods are displayed simultaneously, the operator must decide the order of the remote responses themselves, which places a heavy burden on the operator. Furthermore, when the operator decides the order of the remote responses themselves, there is a risk of making an error in the order of the remote responses. For example, it may happen that the remote response cannot be completed by the response completion grace period. If the remote response cannot be completed by the response completion grace period, the service provision time may be delayed or safe operation may be affected. Therefore, it is desirable to complete the remote response for each of the multiple remote response requests by its respective response completion grace period.

[0017] Therefore, the inventors of the present application conducted intensive research into information processing methods etc. that can assist one operator in completing remote responses to two or more remote response requests by the respective response completion grace times, and have devised the information processing methods etc. shown below.

[0018] An information processing method according to a first aspect of the present disclosure acquires remote response requests for each of a plurality of mobile bodies, acquires the required response time and the response completion grace time for each of two or more unanswered remote response requests, determines whether there is a remote response order that allows the remote response of each of the two or more remote response requests to be completed by the respective response completion grace times, and if it is determined that such a remote response order exists, executes a predetermined process for the two or more remote response requests based on the remote response order.

[0019] As a result, predetermined processing for each of the two or more unprocessed remote response requests is executed in the order of remote response that can be completed by the response completion grace time, thereby realizing an information processing method that supports one operator in completing each of the two or more remote response requests by the completion time (response completion grace time).

[0020] Also, for example, an information processing method according to a second aspect may be the information processing method according to the first aspect, wherein the two or more remote response requests include a first remote response request and a second remote response request, and further includes acquiring a first required response time and a first response completion grace period for the first remote response request, and acquiring a second required response time and a second response completion grace period for the second remote response request, determining whether the remote response of the first remote response request can be completed by the first response completion grace period based on a first time at which the first remote response request can be notified and the first required response time, and determining whether the remote response of the second remote response request can be completed by the second response completion grace period based on a second time at which the second remote response request can be notified and the second required response time.

[0021] This can help ensure that the remote responses to the first remote response request and the second remote response request can be completed by the response completion grace time.

[0022] Also, for example, the information processing method according to the third aspect may be the information processing method according to the second aspect, in which the second response completion grace period is a time later than the first response completion grace period, and the second time is a time determined based on the first time and the first response required time.

[0023] This makes it possible to assist in completing each remote response by the grace period when a remote operation for a first remote response request is performed and then a remote operation for a second remote response request is performed.

[0024] Also, for example, an information processing method according to a fourth aspect may be an information processing method according to a second aspect, wherein the two or more remote response requests include a first remote response request and a second remote response request, and when it is determined that the remote response order exists, it is determined whether there is a remote response request that can be moved up or down in the remote response order, and when it is determined that there is a remote response request that can be moved up or down, the specified processing for the remote response request that can be moved up or down may be executed simultaneously.

[0025] This allows the operator to decide which remote response to execute first when there is a remote response request that can be changed in the order of responses.

[0026] Furthermore, for example, an information processing method according to a fifth aspect may be an information processing method according to the fourth aspect, wherein if the time at which the first remote response request and the second remote response request can be notified and the time based on the sum of the first response required time and the second response required time are earlier than the first response completion grace time and the second response completion grace time, respectively, it may be determined that the response order of the first remote response request and the second remote response request can be reversed.

[0027] This makes it possible to determine whether the response order of the first remote response request and the second remote response request can be changed using the first response required time, the second response required time, the first response completion deferred time, and the second response completion deferred time.

[0028] Furthermore, for example, the information processing method according to the sixth aspect is an information processing method according to any one of the first to fifth aspects, and when it is determined that the remote response order does not exist, the predetermined processing may involve sending a notification to the terminal device of the on-site staff member requesting them to respond on-site.

[0029] As a result, even if it is difficult to complete the remote response of each of two or more remote response requests by the response completion grace time using only the remote response order, by requesting on-site response, it is possible to improve the certainty that the remote response can be completed by the response completion grace time.

[0030] Furthermore, for example, an information processing method according to a seventh aspect may be an information processing method according to any one of the first to fifth aspects, and when it is determined that the remote response order does not exist, it may determine whether or not the remote response order exists when assuming that a predetermined vehicle control is to be performed on one or more of the multiple moving bodies, and when it is determined that the remote response order exists, it may execute the predetermined processing for the two or more remote response requests based on the remote response order, and cause the one or more moving bodies to perform the predetermined vehicle control.

[0031] This means that even if it is difficult to complete the remote response of each of two or more remote response requests by the response completion grace time using only the remote response order, by executing a specified vehicle control, it is possible to improve the certainty that the remote response can be completed by the response completion grace time.

[0032] Furthermore, for example, the information processing method according to the eighth aspect may be the information processing method according to the seventh aspect, wherein the remote response requests acquired from the one or more mobile bodies are excluded from the remote response requests that are the target of executing the specified processing, and based on the required response time and the response completion grace time for the remote response requests of each of the one or more other mobile bodies among the plurality of mobile bodies excluding the one or more mobile bodies, it may be determined whether or not there is a remote response order that can complete the remote response of the remote response request of each of the other one or more mobile bodies by the response completion grace time for each of the other one or more mobile bodies.

[0033] This allows a determination to be made as to whether or not there is a remote response order, excluding remote response requests from the moving body that controls the vehicle, thereby improving the certainty that the remote response can be completed by the response completion grace time.

[0034] Furthermore, for example, the information processing method according to the ninth aspect may be the information processing method according to the seventh or eighth aspect, further acquiring the response completion grace period for each of the one or more moving bodies when the specified vehicle control is performed on the one or more moving bodies, the response completion grace period being later than when the specified vehicle control is not performed, and determining whether or not the remote response order exists based on the acquired response completion grace period for each of the one or more moving bodies.

[0035] This means that when the response completion grace period for a remote response request from a mobile body can be delayed by vehicle control, the reliability of completing the remote response by the response completion grace period can be improved without excluding each of two or more remote response requests.

[0036] Also, for example, an information processing method according to a tenth aspect is an information processing method according to any one of the seventh to ninth aspects, and the specified vehicle control may include at least one of a route change instruction, a speed change instruction, and a stop instruction to the one or more moving bodies.

[0037] As a result, by issuing at least one of a route change instruction, a speed change instruction, and a stop instruction, it is possible to exclude the remote response request from the targets for which a predetermined process is to be executed, or to delay the response completion grace time.

[0038] Furthermore, for example, the information processing method according to the eleventh aspect is an information processing method according to any one of the seventh to tenth aspects, and when it is assumed that the specified vehicle control is to be performed on the one or more moving bodies, and it is determined that the remote response order does not exist, the specified processing may involve sending a notification to a terminal device of an on-call response personnel requesting on-call response.

[0039] This means that even if it is difficult to complete the remote response for each of two or more remote response requests by the response completion grace time using only the remote response order and vehicle control, by requesting on-site response, the certainty of completing the remote response by the response completion grace time can be further improved.

[0040] Furthermore, for example, an information processing method according to a twelfth aspect is an information processing method according to any one of the second to fifth aspects, and when the order of the remote response request is determined to be the first remote response request, the second remote response request, the third remote response request is acquired after the specified processing is performed for the first remote response request, a third required response time and a third response completion deferred time are acquired for the third remote response request, and the remote response order may be updated based on the second required response time, the second response completion deferred time, the third required response time, and the third response completion deferred time.

[0041] This allows the remote response order to be updated when an additional third remote response request is received, thereby helping to complete the remote operation for the third remote response request by the third response completion grace time.

[0042] Also, for example, an information processing method according to a thirteenth aspect may be an information processing method according to any one of the first to twelfth aspects, wherein the plurality of mobile bodies include a mobile body capable of providing a specified service, and the two or more remote response requests may include remote response requests obtained from a server device that manages the specified service.

[0043] This allows the remote response of the remote response request acquired from the server device that manages the predetermined service to be completed by the response completion grace time.

[0044] Also, for example, an information processing method according to a fourteenth aspect may be an information processing method according to any one of the first to thirteenth aspects, and the two or more remote response requests may include a remote response request generated based on operation information of the plurality of moving bodies.

[0045] This allows the remote response to the remote response request created based on the operation information to be completed by the response completion grace time.

[0046] Also, for example, an information processing method according to a fifteenth aspect is an information processing method according to any one of the first to fourteenth aspects, and the predetermined processing may include processing for presenting a notification of each of the two or more remote response requests on an operator's terminal device based on the remote response order in accordance with the remote response order.

[0047] This allows the operator to know which remote response will be performed next. Since the operator does not need to determine the order of the remote responses, it is possible to prevent the remote responses from being performed in an order that cannot be completed by the response completion grace time due to an operator's incorrect judgment.

[0048] In addition, an information processing device according to a sixteenth aspect includes a first acquisition unit that acquires remote response requests for each of a plurality of mobile objects, a second acquisition unit that acquires a required response time and a response completion grace time for each of two or more unaddressed remote response requests, a determination unit that determines whether a remote response order exists that allows the remote response of each of the two or more remote response requests to be completed by the respective response completion grace times, and a control unit that, when it is determined that the remote response order exists, executes a predetermined process for the two or more remote response requests based on the remote response order. In addition, a program according to a seventeenth aspect is a program for causing a computer to execute the information processing method according to any one of the first to fifteenth aspects.

[0049] This provides the same effect as the above-described information processing method.

[0050] These general or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or as any combination of the system, method, integrated circuit, computer program, or recording medium. The program may be pre-stored in the recording medium, or may be supplied to the recording medium via a wide area communication network including the Internet.

[0051] Hereinafter, embodiments and the like will be specifically described with reference to the drawings.

[0052] The embodiments and the like described below are all comprehensive or specific examples. The numerical values, shapes, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments and the like, components that are not recited in independent claims are described as optional components.

[0053] Furthermore, each figure is a schematic diagram and is not necessarily an exact illustration. Therefore, for example, the scales of the figures do not necessarily match. Furthermore, in each figure, substantially the same components are given the same reference numerals, and redundant explanations are omitted or simplified.

[0054] Furthermore, in this specification, terms indicating relationships between elements such as "simultaneity," as well as numerical values ​​and numerical ranges, are not expressions that only express the strict meaning, but are expressions that also include a substantially equivalent range, for example, a difference of about several percent (or about 10%).

[0055] Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components unless otherwise specified, but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0056] (Embodiment) Hereinafter, an information processing method according to the present embodiment will be described with reference to FIGS.

[0057] 1. Configuration of Information Processing System First, the configuration of an information processing system according to this embodiment will be described with reference to Figures 1 and 2. Figure 1 is a diagram showing a schematic configuration of an information processing system 1 according to this embodiment.

[0058] As shown in Fig. 1, the information processing system 1 is a system for remotely controlling a plurality of robots 10, which may be operated robots, by a single operator U1. The remote control includes remote monitoring and remote operation. Remote operation means that the operator U1 issues instructions to the robots 10 to stop and start traveling, controls acceleration / deceleration, and controls the steering angle via a network N as needed.

[0059] The information processing system 1 includes a remote control server 20 and an operator terminal 30. The information processing system 1 may also include a plurality of robots 10, a staff terminal 40, and a service server 50. When the information processing system 1 includes the robots 10, the number of robots 10 included in the information processing system 1 is not particularly limited as long as it is two or more. The information processing system 1 may also include operator terminals 30 for a plurality of operators U1.

[0060] The remote control server 20 is a processing device that executes various processes for one operator U1 to remotely control multiple robots 10. As will be described in detail later, the remote control server 20 executes processes to support the completion of each of multiple remote response requests from the multiple robots 10 by their respective response completion grace times.

[0061] The remote control server 20 is installed in a location where it can be communicatively connected to the network N, and its physical location may be anywhere. As an example, if the remote control server 20 is installed inside or near the space where the operator terminal 30 is installed, there is an advantage that communication with the operator terminal 30 can be established in a short time. Furthermore, the remote control server 20 is communicatively connected to the multiple robots 10, the staff terminals 40, and the service server 50 via the network N.

[0062] Fig. 2 is a block diagram showing the functional configuration of the information processing system 1 according to this embodiment. Fig. 2 shows the functional configuration of only one robot 10 out of the multiple robots 10. The functional configuration of the other robots 10 may be the same as that of the one robot 10. Also, Fig. 2 does not show the network N.

[0063] As shown in FIG. 2 , the robot 10 is a robot that can be remotely controlled by an operator U1. The robot 10 is a mobile body that can switch between autonomous driving and remotely controlled driving. In this embodiment, the robot 10 includes an autonomous mobile robot that can provide a predetermined service such as delivery. The robot 10, for example, drives on a sidewalk, but may also drive on a roadway. The mobile body may be, for example, a vehicle with or without a human on board, or an air vehicle such as a drone.

[0064] The robot 10 includes a robot information acquisition unit 11 , a remote response request unit 12 , and a vehicle control unit 13 .

[0065] The robot information acquisition unit 11 acquires robot information indicating the status of the autonomously traveling robot 10 (host robot) from various sensors mounted on the host robot, which is used to determine whether remote response is necessary for the host robot. The robot 10 acquires image data of the surroundings of the host robot from one or more cameras (an example of a sensor) mounted on the host robot. The robot information acquisition unit 11 also acquires the host robot's speed, acceleration, jerk (jerk), steering angle, remaining battery level, operation status of the turn signal, operation status of the ABS (Anti-lock Braking System) or AEB (Automatic Emergency Braking), presence, number and type of surrounding objects, distance to surrounding objects, speed difference, acceleration difference, etc. from one or more sensors that detect these. The robot information acquisition unit 11 may be configured to include, for example, a communication circuit (communication module).

[0066] The robot information acquisition unit 11 outputs the acquired robot information to the remote response request unit 12. The robot information acquisition unit 11 also transmits the acquired robot information to the remote control server 20 via the network N. Note that the robot information may include, for example, information indicating the traveling position and traveling route of the robot 10.

[0067] The remote response request unit 12 determines whether to request remote response from the operator U1 based on the robot information acquired from the robot information acquisition unit 11. The remote response request unit 12 may determine whether to request remote response from the operator U1 based on, for example, a first table in which the state of the robot 10 indicated by the robot information is associated with information indicating whether remote response is requested. Note that the method by which the remote response request unit 12 determines whether to request remote response is not limited to the above, and any known method may be used. When the remote response request unit 12 determines that remote response is requested, it transmits a remote response request to the remote response planning unit 23.

[0068] The vehicle control unit 13 performs vehicle control (e.g., driving control) of the robot 10 based on instructions from the remote control server 20. For example, the vehicle control unit 13 controls the speed, steering angle (travel direction), etc. of the robot itself based on instructions from the remote control server 20.

[0069] The remote control server 20 is an example of an information processing device, and includes a storage unit 21 , a remote response request unit 22 , a remote response planning unit 23 , a control unit 24 , and a remote response communication unit 25 .

[0070] The memory unit 21 is a storage device that stores various types of information. The memory unit 21 stores robot information from the robot 10 and service information from the service server 50, which are received via a communication circuit (not shown). The service information is information about the service provided by the robot 10, and includes information such as the deadline for product delivery and whether the product is currently being delivered or is being sent for delivery. The service information may also include information such as the delivery destination and the product to be delivered. The robot information and service information are, for example, periodically acquired and updated. The memory unit 21 may be realized, for example, by a semiconductor memory or the like, but is not limited to this.

[0071] The storage unit 21 also stores remote control information required for remote control by the operator U1. The remote control information may include operator information, such as the number of operators U1, their proficiency level, their fatigue level, and correspondence information between the operator terminal 30 and the operator U1. The remote control information may also include a second table, which will be described later. The remote control information may also include the content of a remote response request, information indicating whether vehicle control is possible, whether vehicle control can eliminate the need for remote response, and the like. The remote control information may also include vehicle control information, including vehicle control candidates corresponding to the remote response request and the implementation priority of vehicle control for each vehicle control candidate. The vehicle control information may further include information indicating the amount of time by which the response completion grace time for each vehicle control candidate can be delayed. The remote control information may also include information indicating the response priority for rushing to respond to each remote response request.

[0072] Robot information, service information, and remote control information are collectively referred to as operation information.

[0073] The vehicle control is a control for changing the driving conditions of the robot 10 from the current driving conditions in response to an instruction from the remote control server 20, and includes at least one of an instruction to temporarily stop the robot 10 (stop instruction), an instruction to change the speed of the robot 10 (speed change instruction), and an instruction to change the moving path of the robot 10 (path change instruction). The vehicle control information may also include information indicating that the vehicle control is not possible when the robot gets stuck on an obstacle, for example.

[0074] The remote response request unit 22 determines whether to request the operator U1 to perform a remote response on the robot 10 based on operation information (e.g., at least one of robot information and service information). The remote response request unit 22 may determine whether to request the operator U1 to perform a remote response, for example, based on a first table in which the state of the robot 10 indicated by at least one of the robot information and service information is associated with information indicating whether a remote response is requested. Note that the method by which the remote response request unit 22 determines whether to request a remote response is not limited to the above, and any known method may be used. When the remote response request unit 22 determines that a remote response is required, it outputs the remote response request to the remote response planning unit 23.

[0075] The remote response planning unit 23 acquires a remote response request from any of the remote response request units 12, 22, and 52, and when the number of acquired remote response requests becomes multiple, executes processing to complete the remote response for each of the multiple remote response requests by the respective response completion grace times. The multiple remote response requests may include remote response requests acquired from the remote response request unit 12 of the robot 10, remote response requests generated by the remote response request unit 22 based on operation information of the multiple robots 10, or remote response requests acquired from the remote response request unit 52 of the service server 50.

[0076] When the remote response planning unit 23 acquires a new remote response request, it adds the remote response request to a queue (storage area). In other words, the remote response planning unit 23 does not uniformly transfer each new remote response request to the operator terminal 30 every time it acquires it. The remote response planning unit 23 then plans the notification order of the remote response requests so that all remote response requests included in the queue can be completed by their respective response completion deferred times. As will be described in detail later, the remote response planning unit 23 acquires the required response time and response completion deferred time for each remote response request, and determines a plan including the notification order and notification timing of the remote response requests to be notified to the operator terminal 30 of operator U1 based on the acquired required response time and response completion deferred time.

[0077] Based on the plan determined by the remote response planning unit 23, the control unit 24 performs at least one of notifying the operator terminal 30 or the staff terminal 40 of a remote response request and issuing a vehicle control instruction to the vehicle control unit 13. Furthermore, when the control unit 24 notifies the operator terminal 30 of the operator U1 of a remote response request or when the remote response of the notified remote response request is completed, the control unit 24 deletes the remote response request from the queue.

[0078] The remote communication unit 25 causes the vehicle control unit 13 to control the traveling of the robot 10 based on a control signal for remote communication received from the operator terminal 30. The remote communication unit 25 may be configured to include, for example, a communication circuit (communication module).

[0079] The operator terminal 30 is a terminal device used by the operator U1 to realize remote control by the operator U1. The operator terminal 30 includes a display unit 31, an input unit 32 (see FIG. 1 ), and an interface control unit 33.

[0080] The display unit 31 displays information related to remote control from the remote control server 20. The display unit 31 displays, for example, a notification of a remote response request from the remote control server 20. The display unit 31 is realized, for example, by a liquid crystal display device or the like, but is not limited to this. Furthermore, the operator terminal 30 may have a configuration that presents notifications by sound, light, or the like instead of or in addition to the display unit 31.

[0081] Furthermore, when the operator U1 remotely controls the robot 10, the display unit 31 displays information (for example, an image) showing the situation around the robot 10.

[0082] The input unit 32 is connected to the interface control unit 33 and is a device into which various operations are input from the operator U1. The input unit 32 is, for example, a steering wheel, foot pedals (e.g., an accelerator pedal and a brake pedal), etc., and is a device for remotely operating the robot 10.

[0083] The input unit 32 may also include a stop button for stopping the robot 10, a permission button for allowing the robot 10 to start running, and the like.

[0084] The input unit 32 may be configured to accept input from the operator U1 by voice, gestures, or the like.

[0085] The interface control unit 33 is connected to the input unit 32, and transmits information for remotely responding to the robot 10, which is input to the input unit 32, to the remote response communication unit 25. The interface control unit 33 may be configured to include, for example, a communication circuit (communication module).

[0086] The staff terminal 40 is a terminal device carried by an on-site staff member U2 who responds to the situation of the robot 10 in accordance with instructions from the on-site response notification unit 41. When a situation occurs that cannot be handled by at least one of remote operation and vehicle control by the operator U1, the on-site staff member U2 responds to the robot 10. The on-site staff member U2 waits in a location different from the operator U1. The on-site staff member U2 waits in, for example, multiple waiting areas, and when an on-site response request is received, the on-site staff member U2 in the nearest waiting area rushes to the location of the robot 10 that requires on-site response and responds to the situation. It is assumed that the on-site staff member U2 is able to complete the response within the response completion grace time corresponding to the remote response request from the robot 10.

[0087] The on-call staff may be dedicated staff or general users who have signed a contract to provide on-call support.

[0088] The staff terminal 40 is implemented by a mobile terminal such as a smartphone or a tablet terminal, but may also be implemented by a stationary terminal such as a PC (Personal Computer). The staff terminal 40 includes an emergency response notification unit 41.

[0089] The emergency response notification unit 41 notifies the emergency response staff U2 of the emergency response request from the control unit 24. The emergency response notification unit 41 notifies the emergency response staff U2 by, for example, display, sound, light, etc., but the notification method is not particularly limited.

[0090] The service server 50 is a server device that executes processes related to a predetermined service executed by the robot 10. The service server 50 may be configured to include, for example, a server device managed by a business that provides the predetermined service. The service server 50 includes a service information acquisition unit 51 and a remote support request unit 52.

[0091] The service information acquisition unit 51 acquires service information related to a predetermined service to be executed by the robot 10. The service information acquisition unit 51 is communicably connected to a terminal device that inputs service information, and acquires information input to the terminal device by an employee or a customer of the business as service information. The service information acquisition unit 51 may be configured to include, for example, a communication circuit (communication module).

[0092] The remote support request unit 52 determines whether to request remote support from the operator U1 based on the service information acquired from the service information acquisition unit 51. The remote support request unit 52 may determine whether to request remote support from the operator U1 based on, for example, the travel route of the robot 10 identified by the service information and information indicating sections on the travel route where autonomous travel is not possible. Note that the method by which the remote support request unit 52 determines whether to request remote support is not limited to the above, and any known method may be used. When the remote support request unit 52 determines that remote support is required, it transmits a remote support request to the remote support planning unit 23.

[0093] 2. Operation of Information Processing System Next, the operation of the information processing system 1 configured as described above will be described with reference to Figs. 3 to 8. Fig. 3 is a flowchart showing the operation (information processing method) of the information processing system 1 according to this embodiment. Fig. 3 shows the operation executed by the remote control server 20.

[0094] 3, the remote response planning unit 23 of the remote control server 20 receives new remote response requests for one or more robots 10 from one or more of the remote response request units 12, 22, and 52 (S11). The remote response planning unit 23 receives new remote response requests for one or more robots 10 out of the multiple robots 10 that are remotely controlled by the operator U1. Note that multiple new remote response requests may be received simultaneously. The remote response planning unit 23 functions as a first acquisition unit.

[0095] Next, the remote response planning unit 23 adds the received remote response request to the queue (S12). It can also be said that the remote response planning unit 23 inputs the remote response request to the queue.

[0096] Next, the remote response planning unit 23 acquires the required response time and the response completion grace period for each of the two or more unaddressed remote response requests in the queue (S13). "Unaddressed" means that the predetermined processing has not been performed for the remote response request (for example, the remote response request has not been notified to the operator U1). The remote response planning unit 23 acquires the required response time and the response completion grace period based on a second table in which the remote response content of the remote response request is associated with the required response time and the response completion grace period. Note that the required response time may be set for each operator U1. The response completion grace period may also be set based on the situation around the robot 10, etc.

[0097] The remote response planning unit 23 may acquire the required response time and the response completion grace time by performing a predetermined calculation based on the content of the remote response of the remote response request. The remote response planning unit 23 functions as a second acquisition unit.

[0098] Next, the remote response planning unit 23 determines whether all remote response requests in the queue can be handled by adjusting the response order (remote response order) (S14). Based on the required response time and the response completion grace time, the remote response planning unit 23 determines whether all remote response requests in the queue can be completed by their respective response completion grace times simply by changing the order in which all remote response requests in the queue are notified to the operator terminal 30 of the operator U1 (i.e., the order in which the operator U1 responds). In other words, the remote response planning unit 23 determines whether there is a remote response order that allows the remote response of each of two or more remote response requests in the queue to be completed by their respective response completion grace times. The remote response planning unit 23 functions as a determination unit.

[0099] Next, if the remote response planning unit 23 determines in step S14 that the request can be handled (yes in S14), it determines a plan in which the response order is adjusted (S15). The plan determined in step S15 may be a plan in which the response order of the multiple remote response requests is changed from the order in which they were received.

[0100] Furthermore, if the remote response planning unit 23 determines in step S14 that the request cannot be responded to (no in S14), it determines whether all remote response requests in the queue can be responded to, including vehicle control (S16). In step S16, the remote response planning unit 23 determines whether there is a remote response order that allows the remote response of each of the two or more remote response requests in the queue to be completed by the respective response completion grace times, assuming that predetermined vehicle control is to be performed on one or more robots 10 out of the multiple robots 10.

[0101] Based on the response time required and the response completion grace time, the remote response planning unit 23 determines whether all remote response requests in the queue can be completed by the response completion grace time by delaying the response completion grace time through vehicle control, or whether all remaining remote response requests can be completed by the response completion grace time by reducing the number of remote response requests that operator U1 responds to by performing remote responses to the remote response requests through vehicle control.

[0102] Next, if the remote response planning unit 23 determines in step S16 that it is possible to respond (yes in S16), it decides on a plan that includes vehicle control (S17), and if it determines in step S16 that it is not possible to respond (no in S16), it decides on a plan that includes on-site response by on-site staff U2 (S18).

[0103] Next, the control unit 24 executes a notification according to the plan determined in step S15, S17, or S18 (S19). For example, if it is determined that a remote response order exists, the control unit 24 executes a process of presenting notifications of two or more remote response requests on the operator terminal 30 of the operator U1 based on the remote response order. This process is an example of a predetermined process.

[0104] The following describes a method for determining the order of remote responses for each situation, with reference to FIGS. 4 to 8. FIG. 4 is a diagram for explaining a method (information processing method) for determining the order of remote responses when multiple remote responses can be completed by operator U1 alone. FIG. 4 describes a case where a "yes" determination is made in step S14. Also, in FIGS. 4 to 8, it is assumed that three remote response requests (remote response requests A, B, and C) are received simultaneously at time t=0, there are no remote response requests in the queue at time t=0, and operator U1 is not performing any remote operations.

[0105] 4, the remote response planning unit 23 receives remote response requests A, B, and C at time t=0 (S11) and inputs (adds) them to a queue (S12). Then, the remote response planning unit 23 acquires the required response time and the response completion allowable time for each remote response request A, B, and C based on the remote response content of each remote response request A and the second table (S13). In the example of FIG. 4, the remote response planning unit 23 acquires the required response time A and the response completion allowable time A corresponding to remote response request A, the required response time B and the response completion allowable time B corresponding to remote response request B, and the required response time C and the response completion allowable time C corresponding to remote response request C.

[0106] Next, the remote response planning unit 23 determines whether there is a remote response order (order of notification to operator U1) for remote response requests A, B, and C in which remote response request A is completed by response completion grace time A, remote response request B is completed by response completion grace time B, and remote response request C is completed by response completion grace time C. For example, the remote response planning unit 23 may determine whether the remote response for remote response request A can be completed by response completion grace time A based on a first time at which remote response request A can be notified and the required response time A, determine whether the remote response for remote response request B can be completed by response completion grace time B based on a second time at which remote response request B can be notified and the required response time B, and determine whether the remote response for remote response request C can be completed by response completion grace time C based on a third time at which remote response request C can be notified and the required response time C.

[0107] For example, if response completion grace period B is later than response completion grace period A and response completion grace period C is later than response completion grace period B, the second time may be set based on the first time and required response time A, and the third time may be set based on the second time and required response time B. The first time may be time t=0, or may be a time that takes into account the processing time of the remote control server 20 relative to time t=0. Furthermore, the second time may be the time when the remote response of remote response request A is completed, and the third time may be the time when the remote response of remote response request B is completed.

[0108] In the example of FIG. 4, the remote response planning unit 23 determines that all remote response requests can be handled by notifying the operator U1 of the remote response requests A, B, and C in that order (yes in S14).

[0109] In this case, the remote response planning unit 23 determines a plan that includes notifying multiple remote response requests in the order of remote response requests A, B, and C. Specifically, the remote response planning unit 23 determines a plan that includes notifying A of remote response request A at time t=ta, notifying B of remote response request B at time t=tb, and notifying C of remote response request C at time t=tc (S15).

[0110] Then, in step S19, the control unit 24 may send notification A to the operator terminal 30 via the network N at time t = ta, causing only notification A to be displayed on the display unit 31, send notification B to the operator terminal 30 via the network N at time t = tb after the remote response of remote response request A is completed, causing only notification B to be displayed on the display unit 31, and send notification C to the operator terminal 30 via the network N at time t = tc after the remote response of remote response request B is completed, causing only notification C to be displayed on the display unit 31.

[0111] Next, another example of the method for determining the order of remote responses will be described with reference to Fig. 5. Fig. 5 is a diagram for explaining the method for determining the order of remote responses (information processing method) when multiple remote response notifications are sent to operator U1 simultaneously. Fig. 5 explains the case where a "yes" determination is made in step S14. Note that a description of the same content as Fig. 4 will be omitted.

[0112] 5 , the remote response planning unit 23 acquires the required response times and the response completion allowance times for the remote response requests A, B, and C based on the remote operation details for each remote response request and the second table (S13). If the remote response planning unit 23 determines that a remote response order exists (yes in S14), it determines whether there are any remote response requests whose order in the remote response order can be changed. For example, if the time at which remote response requests A and B can be notified and the time based on the sum of the required response times A and B (in the example of FIG. 5 , the end time of required response time B) are earlier than the response completion allowance times A and B, respectively, the remote response planning unit 23 may determine that the order in which remote response requests A and B can be changed is possible.

[0113] 5 shows a case where the acquired response completion deferred times A and B are in the future than the times when the response required times A and B have elapsed from the current time (t=0). In other words, even if the response order of remote response requests A and B is swapped, the remote response of remote response request A can be completed by the response completion deferred time A, and the remote response of remote response request B can be completed by the response completion deferred time B. In the example of FIG. 5, the remote response planning unit 23 determines that the remote response order for remote response requests A and B can be swapped.

[0114] In this case, the remote response planning unit 23 determines a plan that includes simultaneously issuing notification A of remote response request A and notification B of remote response request B, and then issuing notification C of remote response request C. Specifically, the remote response planning unit 23 determines a plan that includes simultaneously issuing notifications A and B at time t = ta, and issuing notification C at time t = tc.

[0115] In step S19, the control unit 24 simultaneously sends notifications A and B at time t = ta, causing notifications A and B to be displayed simultaneously on the display unit 31, and sends notification C at time t = tc after the remote response of remote response requests A and B is completed, causing notification C to be simultaneously displayed on the display unit 31.

[0116] As a result, notifications for the remote response requests A and B are displayed simultaneously, allowing the operator U1 to decide which remote response request to perform first.

[0117] When simultaneously displaying notifications A and B on the display unit 31, the control unit 24 may also display information indicating the response priorities of notifications A and B. The response priority is information for assisting the operator U1 in determining which of notifications A and B to respond to first, and includes information indicating which notification the operator U1 should prioritize. The response priority may be determined in advance based on the category of the remote response request, or may be calculated based on the response completion grace period. For example, the shorter the response completion grace period, the higher the response priority.

[0118] Next, another example of a method for determining the remote response order will be described with reference to FIG. 6. FIG. 6 is a diagram for explaining an example of a method (information processing method) for determining the remote response order when vehicle control is performed. (a) of FIG. 6 shows the remote response order before vehicle control is performed, and (b) of FIG. 6 shows the remote response order after vehicle control is performed. FIG. 6 describes a case where some remote response requests are deleted from the queue by performing vehicle control, and a determination of yes is made in step S16. Note that a description of the same content as in FIG. 4 will be omitted.

[0119] In the case shown in (a) of Figure 6, the remote response for remote response request C is not completed by the response completion grace time C, so the remote response planning unit 23 determines that all remote response requests in the queue cannot be responded to by adjusting the remote response order, that is, there is no remote response order that can complete all remote responses by adjusting the remote response order alone (no in S14).

[0120] Next, the remote response planning unit 23 determines whether or not it is possible to plan including vehicle control (S16). For example, the remote response planning unit 23 extracts, from the remote control information, remote response requests A to C that allow vehicle control. For example, it is assumed that the remote response planning unit 23 acquires information from the remote control information that remote response request C can be made unnecessary by controlling the vehicle (for example, issuing a temporary stop instruction).

[0121] In this case, the remote response planning unit 23 determines that the robot 10 corresponding to remote response request C is capable of vehicle control to temporarily stop, and that temporary stopping by vehicle control makes remote response unnecessary, and deletes remote response request C from the queue. In other words, remote response request C is excluded from the remote response requests that are the targets for executing the specified process. As a result, the queue now contains only remote response requests A and B.

[0122] Then, the remote response planning unit 23 determines, based on the required response time and response completion grace period for the remote response requests of each of the one or more robots 10 other than the one or more robots 10 that are subject to vehicle control, whether there is a remote response order that can complete the remote response of the remote response requests of each of the other one or more robots 10 by their respective response completion grace periods.

[0123] 6B, the remote response planning unit 23 determines that all remote response requests can be handled (yes in S16) by notifying the operator U1 of the remote response requests in the order of remote response requests A and B. Note that in FIG. 6B, the exclusion of remote response request C is indicated by enclosing the required response time C with a dashed line.

[0124] The remote response planning unit 23 determines a plan including notifying multiple remote response requests in the order of remote response requests A and B, and issuing a vehicle control instruction (here, an instruction to temporarily halt) to execute vehicle control to the robot 10 corresponding to remote response request C. Specifically, the remote response planning unit 23 determines a plan including notifying A of remote response request A and issuing a vehicle control instruction to the robot 10 corresponding to remote response request C at time t = ta, and notifying B of remote response request B at time t = tb (S17).

[0125] As a result, in step S19, the control unit 24 simultaneously issues notification A and a vehicle control instruction at time t=ta, thereby displaying notification A on the display unit 31 and executing vehicle control of the robot 10 corresponding to remote response request C, and issues notification B at time t=tb after completion of the remote response to remote response request A, thereby displaying notification B on the display unit 31. Note that the vehicle control instruction is not limited to being issued at time t=ta.

[0126] Next, another example of a method for determining a remote response order will be described with reference to FIG. 7 . FIG. 7 is a diagram for explaining another example of a method (information processing method) for determining a remote response order when vehicle control is performed. FIG. 7 describes a case where a "yes" determination is made in step S16 by delaying the response completion grace time for the remote response request by performing vehicle control. It is assumed that the remote response order before vehicle control is performed is the same as (a) in FIG. 6 . Below, a description will be given of the processing that occurs after it is determined that there is no remote response order that can complete all remote responses using only the remote response order ("no" in S14).

[0127] The remote response requests A and B are requests from the robot 10 when it is stopped, such as a request to confirm safety before starting to move. In this case, an example of the remote response by the operator U1 is an instruction to start moving. The remote response request C is a request from the robot 10 when it is moving, such as a notification that it will pass a pedestrian on a narrow road. In this case, an example of the remote response by the operator U1 is an instruction to stop temporarily.

[0128] The remote response planning unit 23 determines whether or not the plan can include vehicle control (S16). The remote response planning unit 23 acquires, for example, vehicle control information from the remote control information, including vehicle control candidates corresponding to the remote response request and the priority order of the vehicle control for each vehicle control candidate, and makes the determination in step S16 based on the acquired vehicle control information.

[0129] Here, assume that vehicle control information is acquired that indicates that there are no vehicle control candidates for remote response requests A and B, and that there are vehicle control candidates for temporary stop or speed reduction for remote response request C. The absence of vehicle control candidates for remote response requests A and B is based on, for example, a judgment that automatically issuing an instruction to resume driving is risky. Also, assume that the implementation priority is "low" for temporary stop and "high" for speed reduction. This implementation priority is based on, for example, a judgment that stopping the vehicle is riskier.

[0130] The remote response planning unit 23 determines whether all remote responses can be completed by the response completion grace time if vehicle control is implemented, in descending order of implementation priority of the vehicle control candidates. When the remote response planning unit 23 acquires the above vehicle control information, it determines whether all remote response requests can be completed by the response completion grace time if a speed deceleration with a higher implementation priority is implemented for the remote response request C. If it determines that it can be implemented, the remote response planning unit 23 determines to execute speed deceleration as vehicle control. If it determines that it cannot be implemented, it further determines whether all remote response requests can be completed by the response completion grace time if a temporary stop with a lower implementation priority is implemented for the remote response request C. If it determines that it can be implemented, the remote response planning unit 23 determines to execute a temporary stop as vehicle control. If it determines that it cannot be implemented, it proceeds to step S18.

[0131] 7 shows an example in which speed deceleration is performed as vehicle control. When the remote response planning unit 23 performs speed deceleration as vehicle control for the robot 10 corresponding to the remote response request C, the remote response planning unit 23 further acquires a response completion grace period (time t = tca) for the robot 10 based on the vehicle control information, which is later than the response completion grace period when speed deceleration is not performed, and determines whether a remote response order exists based on the acquired response completion grace period for the robot 10. For example, the remote response planning unit 23 determines that the response completion grace period can be delayed from time t = tcb to time t = tca. Time t = tcb is the time when predetermined vehicle control is not performed, and time t = tca is the time when predetermined vehicle control is performed.

[0132] In FIG. 7, the fact that the response completion allowance time C has been delayed is shown by drawing a strikethrough line through the response completion allowance time C at time t=tcb.

[0133] The remote response planning unit 23 then makes a judgment in step S16 based on which of the end time of the required response time C and the time t = tca is later. If the end time of the required response time C is later, the remote response planning unit 23 judges "no" in step S16, and if the time t = tca is later, the remote response planning unit 23 judges "yes" in step S16. Figure 7 illustrates the case where the time t = tca is later.

[0134] The remote response planning unit 23 determines a plan including notifying the remote response requests corresponding to the remote response requests A, B, and C in that order, and issuing a vehicle control instruction (here, a deceleration instruction) for executing vehicle control to the robot 10 corresponding to the remote response request C. Specifically, the remote response planning unit 23 determines a plan including notifying A of the remote response request A and issuing a vehicle control instruction to the robot 10 corresponding to the remote response request C at time t = ta, notifying B of the remote response request B at time t = tb, and notifying C of the remote response request C at time t = tc (S17).

[0135] In step S19, the control unit 24 simultaneously executes notification A and a vehicle control instruction at time t=ta, executes notification B after completing the remote response for remote response request A, and executes notification C after completing the remote response for remote response request B. Note that the vehicle control instruction is not limited to being executed at time t=ta.

[0136] Next, another example of a method for determining the remote response order will be described with reference to FIG. 8 . FIG. 8 is a diagram for explaining a method (information processing method) for determining the remote response order when an emergency response request is made. FIG. 8 describes a case in which a request for emergency response is made, and some remote response requests are deleted from the queue, and a plan is determined in step S18. It is assumed that the response order before the emergency response request is made is the same as that shown in FIG. 6A. Below, a description will be given of the processing that occurs after it is determined that there is no remote response order that can complete all remote responses, even if vehicle control is included (no in S16).

[0137] The remote response requests A and B are requests from the stopped robot 10, such as a safety confirmation request to start traveling. In this case, an example of the remote response by the operator U1 is an instruction to start traveling. Furthermore, the remote response request C is a request from the robot 10 that is parked on the road and stuck. In this case, an example of the remote response by the operator U1 is remote operation (remote control).

[0138] If the answer to step S16 is no, the remote response planning unit 23 determines a plan that can handle all remote response requests by the response completion grace time, including on-site response. The remote response planning unit 23 acquires the response priority of each of the remote response requests A to C from the remote control information, for example.

[0139] Here, assume that information including the fact that the response priority for remote response requests A and B is "low" and the response priority for remote response request C is "high" is acquired from the remote control information. The response priority for remote response requests A and B is "low" based on, for example, a judgment that automatically issuing an instruction to resume driving is risky. Also, the response priority for remote response request C is "high" based on, for example, a judgment that there is a high possibility of causing a nuisance to other traffic.

[0140] The remote response planning unit 23 deletes the remote response requests from the queue by issuing an on-call response request in descending order of response priority until all remote response requests can be handled by the response completion grace time. For example, each time the remote response planning unit 23 switches to an on-call response request, it determines whether all remaining remote response requests can be handled by the remote response grace time.

[0141] In this case, the remote response planning unit 23 determines that remote response is unnecessary by requesting an on-site response from the robot 10 that corresponds to remote response request C, and deletes the remote response request from the queue. The example of Figure 8 shows a case where remote response request C is deleted from the queue by being switched to an on-site response request, and the queue contains only remote response requests A and B. As shown in Figure 8, the remote response planning unit 23 notifies the operator U1 of the remote response requests in the order of remote response requests A and B, and determines that all remote response requests except for the deleted remote response request C can be handled.

[0142] The remote response planning unit 23 determines a plan including notifying the remote response requests corresponding to the remote response requests A and B in that order, and making an emergency response request to dispatch the robot 10 corresponding to the remote response request C. Specifically, the remote response planning unit 23 determines a plan including notifying the staff terminal 40 of the remote response request A and making an emergency response request to the staff terminal 40 at time t=ta, and notifying the staff terminal 40 of the remote response request B at time t=tb (S18).

[0143] In step S19, the control unit 24 simultaneously executes notification A and a request for on-call response at time t = ta, and executes notification B at time t = tb after the remote response for remote response request A is completed. The control unit 24 transmits the request for on-call response to the staff terminal 40 via the network N at time t = ta. In this way, when it is assumed that predetermined vehicle control is to be performed on one or more robots 10 and it is determined that there is no remote response order, the remote response planning unit 23 transmits a notification requesting on-call response to the staff terminal 40 of the on-call response personnel as a predetermined process. Note that the request for on-call response is not limited to being made at time t = ta.

[0144] 4 to 8 illustrate examples in which a remote response order is determined so that the next remote response begins immediately after a certain remote response end time. However, this is not limiting. For example, a remote response order may be determined that includes a predetermined margin (time margin) between a certain remote response end time and the next remote response start time. Using FIG. 4 as an example, the remote response planning unit 23 may include a predetermined time margin between the response time A for remote response request A and the response time B for remote response request B executed after remote response request A, or between the response time B for remote response request B and the response time C for remote response request C executed after remote response request B. The remote response planning unit 23 may determine whether a remote response order including such a predetermined time margin exists, and if so, may adopt the remote response order including the predetermined time margin.

[0145] This reduces the possibility that the actual remote response will exceed the estimated response time, resulting in the inability to complete this remote response request or other remote response requests (subsequent remote response requests) by the response grace time. This also reduces the psychological burden on the operator U1, which increases when multiple remote responses are performed. The predetermined time margin may be, for example, a uniform time, or may be set according to the content of the remote response, or may be set according to the track record and skills of the operator U1.

[0146] (Variation of the embodiment) An information processing system according to this variation will be described below with reference to Figures 9 and 10. Note that the following description will focus on differences from the embodiment, and descriptions of content that is the same as or similar to the embodiment will be omitted or simplified. The configuration of the information processing system according to this variation may be the same as that of the information processing system 1 according to the embodiment, and description thereof will be omitted. Furthermore, the following description will use the reference numerals of the information processing system 1 according to the embodiment.

[0147] Fig. 9 is a flowchart showing the operation (information processing method) of the information processing system 1 according to this modification. Fig. 10 is a diagram for explaining a method (information processing method) for determining a readjusted remote response order.

[0148] In Fig. 9, the processes of steps S11 to S18 shown in Fig. 3 are omitted from illustration. For example, after steps S11 to S19 shown in Fig. 3 are executed, the process of step S21 shown in Fig. 9 is executed.

[0149] As shown in Fig. 9, after executing a notification according to the plan determined in step S15, 17, or 18 (see Fig. 10(a)), for example, while operator U1 is performing a remote response, the remote response planning unit 23 of the remote control server 20 receives a new remote response request (S21) and adds the new remote response request to the queue (S22). It can also be said that the remote response planning unit 23 inputs the new remote response request to the queue. Fig. 10 shows an example in which, at time t = t1 while operator U1 is performing a remote response for remote response request A, remote response request C is received and added to the queue.

[0150] Next, the remote response planning unit 23 acquires the response time required and the response completion grace time for the new remote response request (S23). Step S23 is the same process as step S13, and therefore a description thereof will be omitted.

[0151] Next, the remote response planning unit 23 readjusts the plan based on the required response times and the response completion grace periods (including the required response times and the response completion grace periods acquired in step S23) of one or more remote response requests included in the queue, excluding the remote response request currently being handled by the operator U1 (S24). If the remote response planning unit 23 can handle the requests by adjusting the response order (corresponding to yes in S14), it determines a plan with the adjusted response order (corresponding to S15). If the remote response planning unit 23 can handle the requests with vehicle control included (corresponding to yes in S16), it determines a plan with vehicle control included (corresponding to S17). If the remote response planning unit 23 cannot handle the requests with vehicle control included (corresponding to no in S16), it determines a plan with on-call response included (corresponding to S18).

[0152] 10 shows an example in which the response completion allowance time C acquired in step S23 is between the response completion allowance times A and B. If the remote response for remote response request C is scheduled after the remote response for remote response request B in FIG. 10(a), the response completion allowance time C cannot be achieved, so the order of remote response requests B and C is swapped, as shown in FIG. 10(b). The remote response planning unit 23 determines that all remote responses can be completed by the response completion allowance time by updating the plan to swap the order of remote response requests B and C (corresponding to "yes" in S16). The remote response planning unit 23 updates the remote response order based on the required response time B, the response completion allowance time B, the required response time C, and the response completion allowance time C.

[0153] The remote response planning unit 23 determines a plan that includes notifying remote response requests C and B in this order after remote response request A. Specifically, the remote response planning unit 23 determines a plan that includes notifying remote response request C at time t=tb1, and notifying remote response request B at time t=tb2 instead of time t=tb1. Note that in FIG. 10 , the fact that notification B has been delayed is indicated by illustrating notification B at time t=tb1 with a strikethrough.

[0154] Next, the control unit 24 determines whether the remote support request being handled has been completed (S25). In the example of Fig. 10, the control unit 24 determines whether the remote support request A has been completed. For example, the control unit 24 may receive an operation from the operator U1 indicating that the remote support request A has been completed.

[0155] When the control unit 24 determines that the remote support request being handled has been completed (yes in S25), it executes a notification according to the readjusted plan (S26). After the remote support request A is completed, the control unit 24 executes notification C at time t=tb1, and executes notification B at time t=tb2 after the remote support for remote support request A is completed. Furthermore, when the control unit 24 determines that the remote support request being handled has not been completed (no in S25), it returns to step S25 and waits until it is completed.

[0156] In this way, if a new remote response request is received while operator U1 is responding to a remote response request, the plan is readjusted, so that all remote response requests in the queue, including the new remote response request, can be completed by the response completion grace time.

[0157] (Display Examples for Operator) Variations of display examples of the operator terminal 30 in the information processing system 1 according to the above-described embodiment and modified embodiment will be described below with reference to Figures 11 to 13. Note that hereinafter, the embodiment and modified embodiment will also be referred to as the embodiment, etc.

[0158] First, a display example of information displayed on the operator terminal 30 in response to the notification in step S19 will be described with reference to FIG. 11 . FIG. 11 is a first diagram showing a display example of the operator terminal 30 (display unit 31) according to the present embodiment. FIG. 11 shows a remote response notification when the remote operation order shown in FIG. 4 has been determined. Also, FIG. 11 shows an example in which remote response notification A corresponding to remote response request A, remote response notification B corresponding to remote response request B, and remote response notification C corresponding to remote response request C are displayed side by side on one screen, but it is sufficient if at least remote response notification A, which is being handled, is displayed. The remote response notification is a notification for a remote response request that requires remote handling.

[0159] As shown in FIG. 11 , the control unit 24 may, for example, display on the operator terminal 30 the required response time corresponding to the remote response notification (e.g., the required response time corresponding to the remote response notification A currently being handled). This allows the operator U1 to understand the required response time estimated by the remote control server 20, and the operator U1 can use this required response time as a guide for the time required for the remote response. Furthermore, the control unit 24 may, for example, also display on the operator terminal 30 the response completion grace period corresponding to the remote response notification (e.g., the response completion grace period corresponding to the remote response notification A currently being handled). This allows the operator U1 to understand by what time the remote response should be completed so as not to impede the completion of the overall remote response.

[0160] Furthermore, the control unit 24 may display at least one of the required response time for each remote response notification (e.g., the required response time for each of the remote response notifications A to C) and the response completion grace period (e.g., the response completion grace period for each of the remote response notifications A to C) on the operator terminal 30. This allows the operator U1 to get a sense of the future schedule, thereby reducing the psychological burden on the operator U1, such as feeling that they need to respond unnecessarily in a hurry.

[0161] Furthermore, for example, when the response grace period completion time approaches during remote response, the control unit 24 may display on the operator terminal 30 that the grace period is shortening. This reduces the amount of information required for the operator U1 to understand when the remote response should be completed, so as not to impede the completion of the entire remote response.

[0162] The control unit 24 may display on the operator terminal 30 at least one of the required response time, the grace period for completing the response, and a message indicating that the grace period is shortening (for example, (xx seconds later) in Figure 11).

[0163] 11 , the control unit 24 may cause the operator terminal 30 to display a remote response notification before the remote response notification time determined according to the remote response order determined by the remote control server 20. For example, the control unit 24 may cause the operator terminal 30 to display not only the most recent remote response notification but also remote response notifications scheduled for the next and subsequent remote responses. This allows the operator U1 to know in advance the remote responses scheduled to be performed, thereby reducing the time required to understand the remote response details when the next remote response is performed.

[0164] Furthermore, the control unit 24 may simply display whether or not a remote response notification has been received, or the number of remote response notifications, on the operator terminal 30. This reduces the amount of information that the operator U1 must grasp, thereby reducing the burden on the operator U1 and allowing the operator U1 to easily grasp the future schedule.

[0165] Furthermore, when the presence or absence of a remote response notification or the number of remote response notifications is displayed, the control unit 24 may further display the contents on the operator terminal 30 through a detailed information display operation. For example, the control unit 24 may display on the operator terminal 30 that there is a remote response notification A, and when the control unit 24 receives a detailed information display operation via the operator terminal 30, it may further display at least one of the remote response request content, the response grace period, the required response time, the response completion grace period, and a message indicating that the grace period is shortening. This allows the amount of information to be switched to an appropriate amount according to the operator U1's preferences. The detailed information display operation is an operation performed by the operator U1 to display detailed information. The detailed information display operation may be, for example, the operator U1 clicking an icon for displaying detailed information or making an utterance indicating that detailed information should be displayed.

[0166] Furthermore, when the remote response order is updated as described above, the control unit 24 may notify the operator terminal 30 of the update. For example, the control unit 24 may provide a notification by highlighting the updated portion. This allows the operator U1 to easily understand that the future schedule he or she had in mind has changed, and also allows the operator U1 to change the content of his or her advance preparations. For example, the control unit 24 may notify the operator of the update only when the remote response order for the next response currently being handled has been updated. Since the operator U1 will most likely prepare for the next remote response order, the control unit 24 does not need to notify the operator when the remote response order after the next one has been updated. Furthermore, the control unit 24 may automatically switch to the response screen for the next remote response request according to the remote response order each time the response to a remote response request is completed.

[0167] The control unit 24 may also display additional information about the most recent task. An example of the additional information is the response completion grace period for the most recent task. This allows the operator U1 to know how much time is left to complete the currently running task while performing the task. The additional information is not limited to the response completion grace period. The control unit 24 may also display, as additional information, whether or not there is a task after the most recent task. This allows the operator U1 to know whether or not there is a task to be performed after the currently running task, allowing him or her to prepare for the next task before completing the currently running task. The control unit 24 may also display, as additional information, an alert notification when the response completion grace period for the most recent task is approaching. This allows for minimal notification only when the response grace period is approaching, thereby reducing the amount of information displayed.

[0168] Fig. 12 is a second diagram showing a display example of the operator terminal 30 (display unit 31) according to the present embodiment etc. Fig. 12 shows an example in which the schedule of the entire tasks is displayed when the response order shown in Fig. 4 is determined.

[0169] As shown in Fig. 12, after the remote response order is determined, the control unit 24 may display the entire task schedule on the operator terminal 30 so that the operator U1 can understand the order in which the tasks will be performed. In the example of Fig. 12, the currently-performing task (the task corresponding to the response time A in Fig. 12) and the next scheduled task (the tasks corresponding to the response times B and C in Fig. 12) are displayed. This allows the operator U1 to intuitively understand that after the current task, the task corresponding to the response time B and the task corresponding to the response time C must be performed in this order, and thus the operator U1 can prepare in advance for the next remote response request.

[0170] The control unit 24 may also display the schedule of the entire task on the operator terminal 30. This allows the operator U1 to know whether there are any remote support requests that need to be addressed after the remote support request currently being handled. Therefore, if there are any remote support requests that need to be addressed after the remote support request currently being handled, the operator U1 can be made aware of the need to complete the currently handled remote support request as quickly as possible. For example, because it is unclear whether there are any remote support requests that need to be addressed after the currently handled remote support request, it is possible to prevent the operator U1 from responding slowly and exceeding the expected response time. Note that the control unit 24 does not need to display the entire schedule. For example, the control unit 24 may display information on whether there are any remote support requests that need to be addressed after the currently handled remote support request, or may display a schedule for some of the remote support requests that need to be addressed. This reduces the burden on the operator U1 of understanding the entire task while achieving the above effects.

[0171] Fig. 13 is a third diagram showing a display example of the operator terminal 30 (display unit 31) according to the present embodiment etc. Fig. 13 shows another example in which the schedule of the entire tasks is displayed when the response order shown in Fig. 4 has been determined.

[0172] As shown in Figure 13, the control unit 24 does not need to display the response completion grace period because if the control unit 24 displays the response completion grace period, there is a possibility that the operator U1 will take his or her time to respond if there is sufficient time until the response completion grace period. Alternatively, the control unit 24 may display the response time required for the remote response (response time required for the remote response A to C shown in Figure 13) used when planning the remote response sequence. This allows the operator U1 to know that the response time required for the remote response should be used as a guideline for the response deadline (response time required for the remote response A to C shown in Figure 13) when remotely responding.

[0173] The response completion grace period may be displayed but may be viewable. For example, the control unit 24 may display the response completion grace period when receiving an operation to display detailed information from the operator U1. The display shown in FIG. 13 may be automatically switched to the next remote response screen according to the schedule each time a task is completed.

[0174] 11 to 13 have described an example in which information related to remote response is displayed on the operator terminal 30, but other information may be displayed on the operator terminal 30 along with the information related to remote response. For example, the control unit 24 may cause the operator terminal 30 to display the status of the vehicle currently undergoing remote response or that will require remote response next, along with the information related to remote response. The control unit 24 may display the information related to remote response and the vehicle status side by side on a single screen. The vehicle status may be, for example, information indicating the vehicle's driving status, such as its position and speed, or information indicating the vehicle's surrounding status, such as an image captured by an imaging device mounted on the vehicle or the detection results of a sensor.

[0175] This reduces the time required for the operator U1 to understand the condition of the vehicle before remotely responding.

[0176] (Other Embodiments) The present disclosure has been described above based on the embodiments and modified examples of the embodiments, but the present disclosure is not limited to the above-described embodiments, etc. As long as it does not deviate from the spirit of the present disclosure, various modifications that a person skilled in the art can conceive of to the present embodiments and embodiments constructed by combining components of different embodiments may also be included in the present disclosure.

[0177] For example, the remote response planning unit 23 in the above-described embodiment may calculate the required response time using at least one of the proficiency level and fatigue level of the operator U1. For example, when the remote response planning unit 23 determines that the proficiency level of the operator U1 is equal to or greater than a predetermined value, or when the remote response planning unit 23 determines that the fatigue level of the operator U1 is less than a predetermined value, the remote response planning unit 23 may extend the required response time acquired in step S13. When the remote response planning unit 23 determines that the proficiency level of the operator U1 is less than a predetermined value, or when the remote response planning unit 23 determines that the fatigue level of the operator U1 is equal to or greater than a predetermined value, the remote response planning unit 23 may shorten the required response time acquired in step S13. Furthermore, the remote response planning unit 23 may calculate the required response time using at least one of the road context and the surrounding conditions of the robot 10. For example, when the remote response planning unit 23 determines that the area around the robot 10 is congested based on at least one of the road context and the surrounding conditions of the robot 10, it may extend the required response time acquired in step S13, or when it determines that the area around the robot 10 is not congested based on at least one of the road context and the surrounding conditions of the robot 10, it may shorten the required response time acquired in step S13. In this way, for example, the remote response planning unit 23 may correct the required response time acquired in step S13 based on at least one of the proficiency level and fatigue level of the operator U1, the road context, and the surrounding conditions of the robot 10. This allows the required response time to be calculated more accurately, and therefore the remote response planning unit 23 can determine a more accurate plan.

[0178] Furthermore, the remote response planning unit 23 in the above-described embodiments and the like may calculate the response completion deferred time based on at least one of the road context and the surrounding conditions of the robot 10. For example, when the remote response planning unit 23 determines that there are many people around the robot 10 based on at least one of the road context and the surrounding conditions of the robot 10, the remote response planning unit 23 may shorten the response completion deferred time acquired in step S13, or when the remote response planning unit 23 determines that there are few people around the robot 10 based on at least one of the road context and the surrounding conditions of the robot 10, the remote response planning unit 23 may lengthen the response completion deferred time acquired in step S13. Furthermore, the remote response planning unit 23 may calculate the response completion deferred time using at least one of the robot 10's available driving time and planned driving route. The remote response planning unit 23 may extend the response completion grace period acquired in step S13 if it determines that the remaining driving time is equal to or greater than a predetermined value or that the planned driving distance based on the planned driving route is less than a predetermined value. Alternatively, the remote response planning unit 23 may shorten the response completion grace period acquired in step S13 if it determines that the remaining driving time is less than the predetermined value or that the planned driving distance based on the planned driving route is equal to or greater than a predetermined value. In this manner, the remote response planning unit 23 may correct the response completion grace period acquired in step S13 based on, for example, at least one of the road context, the surrounding conditions of the robot 10, the remaining driving time of the robot 10, and the planned driving route. This allows the response completion grace period to be calculated more accurately, thereby enabling the remote response planning unit 23 to determine a more accurate plan. The remaining driving time of the robot 10 can be calculated, for example, from the remaining battery charge and remaining fuel of the robot 10, and the planned driving route can be acquired from information indicating the driving route.

[0179] Furthermore, the remote response planning unit 23 in the above-described embodiment may display multiple notifications simultaneously on the display unit 31 at the timing of notifying a remote response request. This allows the operator U1 to determine the order of remote responses. The number of notifications displayed simultaneously may be changed depending on at least one of the proficiency level and fatigue level of the operator U1.

[0180] Furthermore, the remote response planning unit 23 in the above-described embodiments may update at least one of the required response time and the response completion grace period of the remote response requests accumulated in the queue based on the status of the robot 10 requiring remote response. The update may be performed periodically, for example. The remote response planning unit 23 may then readjust the plan based on at least one of the updated required response time and the response completion grace period. This allows for re-planning when the operating situation changes, thereby improving the accuracy of the plan.

[0181] Furthermore, in the above-described embodiment, when there are multiple operators U1, the remote response planning unit 23 may determine the operator terminal 30 to notify based on at least one of the proficiency and fatigue level of the operators U1. For example, the remote response planning unit 23 may prioritize notifying the operator terminal 30 of an operator U1 with a high proficiency level or a low fatigue level.

[0182] Furthermore, the remote response planning unit 23 in the above-described embodiment may be configured to respond to one remote response request by combining remote response, vehicle control, and on-site response, thereby enabling the vehicle control to be used to plan and shorten the response time and response completion grace period for the remote response request.

[0183] Furthermore, although the remote response planning unit 23 in the above-described embodiments can respond only in the order in which the remote response requests are notified, it may be configured to issue at least one of a vehicle control instruction and an on-site response request that extends the response completion grace period or deletes the remote response request from the queue when there is insufficient margin between the response completion grace period and the predicted response completion time (for example, less than a predetermined time). This makes it possible to prevent operational breakdowns even if the actual remote response exceeds the response time required in the planning stage.

[0184] Furthermore, the remote response planning unit 23 in the above embodiments can respond by controlling the vehicle and following the notification order of the remote response requests, but may also be configured to extend the response completion grace period or delete the remote response request from the queue if there is not enough time between the predicted response completion time and the response grace period. This prevents operational breakdowns even if the actual remote response exceeds the response time required in the planning stage.

[0185] Furthermore, in the above-described embodiments, the remote response planning unit 23 may be configured to prioritize the most effective remote response request (remote response request) to the operator terminal 30 of the operator U1 when multiple remote response requests, such as those shown in FIG. 5, can be simultaneously sent to the operator terminal 30 based on information about service quality, such as a delay in arrival at the delivery point. The remote response planning unit 23 may plan to prioritize the notification of the remote response request A or B, whichever remote response request results in a greater deterioration in service quality (e.g., a greater arrival delay), to the operator terminal 30 of the operator U1 first. The remote response planning unit 23 may also be configured to prioritize the selection of a vehicle control candidate for a robot 10 that has completed a service, such as a delivery service. The remote response planning unit 23 may, for example, calculate the arrival delay time when each vehicle control candidate is implemented and determine the vehicle control that minimizes the arrival delay time. This prevents operational breakdowns while minimizing degradation in service quality.

[0186] Furthermore, in the above-described embodiments, when remote responses are provided to multiple robots 10, the remote response planning unit 23 may be configured to select the most effective candidate (remote response request) based on information about service quality, such as a delay in arrival time at the delivery point, or may be configured to prioritize issuing vehicle control instructions or emergency response requests to robots 10 that have completed their service, such as a delivery service. The remote response planning unit 23 may, for example, calculate the arrival delay times of each candidate and select a robot 10 to which at least one of a vehicle control instruction and an emergency response request is to be issued so that the total is minimized. Furthermore, for multiple remote response requests, such as those shown as remote response requests A and B in FIG. 5 , the remote response planning unit 23 may acquire priorities based on information about service quality (e.g., delay times) and determine which remote response request to notify the operator terminal 30 of the operator U1 first based on the acquired priorities. Furthermore, for example, when there are two or more remote response requests with the same priority, the remote response planning unit 23 may determine that, among the robots 10 that are currently delivering and those that have already delivered, the remote response request of the robot 10 that is currently delivering should be prioritized and notified to the operator terminal 30 of the operator U1. This will prevent operational breakdowns while minimizing degradation of service quality.

[0187] Furthermore, in the above-described embodiment, the remote response planning unit 23 may determine the timing of notification depending on the status of the operator U1 when there is only one remote response request in the queue. When the operator U1 is currently remotely responding, the remote response planning unit 23 may notify the operator terminal 30 of the operator U1 of the remote response request accumulated in the queue after the remote response is completed, and when the operator U1 is not currently remotely responding, the remote response planning unit 23 may determine to immediately notify the operator terminal 30 of the operator U1 of the remote response request accumulated in the queue.

[0188] Furthermore, in the above-described embodiments, the remote response planning unit 23 may be configured to prioritize notification of remote response requests whose required response time is below a predetermined value when the required response time varies. For example, in a situation where a remote response is performed such that a departure operation is performed after traffic volume around the vehicle has decreased, the remote response planning unit 23 may calculate the required response time from traffic volume around the vehicle detected by a sensor or the like, and prioritize notification of remote response requests of vehicles whose calculated required response time is less than a predetermined value over remote response requests of other vehicles. For example, when traffic volume is low, the required response time may be less than the predetermined value. The predetermined value may be set in advance and stored in the storage unit 21. The predetermined value may be a uniform value, or may be set individually depending on the content of the remote response, or may be calculated based on the response completion grace time and required response time of the next remote response. In this way, by prioritizing the completion of remote responses for vehicles whose required response time is short, it is possible to reduce the occurrence of cases where other remote responses cannot be completed by the response completion grace time.

[0189] Furthermore, in the above-described embodiments, examples have been described in which remote response requests A and B are acquired simultaneously, but they may also be acquired at different times, for example, consecutively. For example, if remote response request A is acquired before remote response request B, and the remote response planning unit 23 determines that a remote response order exists, it may determine a plan to execute predetermined processes in the order of remote response requests A and B, or may determine a plan to execute predetermined processes in the order of remote response requests B and A, so that each remote response is completed by the remote response grace time. The remote response planning unit 23 may determine a plan to execute predetermined processes in the order in which the remote response requests were acquired, or may determine a plan to execute predetermined processes in an order different from the order in which the remote response requests were acquired.

[0190] Furthermore, in the above-described embodiments, an example has been described in which a remote response request is a request to an operator located remotely from the mobile object to remotely respond to the movement of the mobile object, but the present invention is not limited to this, and the remote response request may be a remote response request that is estimated to occur at a future time by estimating the future state of the vehicle based on past driving data, vehicle sensor information, etc. This makes it possible to reduce the occurrence of cases in which, when a new remote response request is input after the remote response order has been determined, it is not possible to determine the remote response order including the new remote response request.

[0191] Furthermore, in the above-described embodiments, an example in which a remote response request is obtained from a mobile body has been described, but the present invention is not limited to this. The remote response request may be obtained, for example, from a source other than the mobile body requiring remote response. The source other than the mobile body may be, for example, a server that requests a remote response from an operator, such as a service server or a remote operation server. Furthermore, the remote response request may be any request for an operation to be performed by a remote operator. For example, an example of a service server is a delivery service server in a product delivery service that requests a remote response, such as sending an arrival notification to a user who will receive the product after the product arrives at the delivery destination. This allows the order of remote responses to be determined taking into consideration all of the operations that the remote operator will handle.

[0192] Furthermore, in the information processing system according to the above-described embodiments, if the system configuration requires a pre-operation before the operator performs a remote response, the system may be configured to perform the pre-operation. Examples of the pre-operation include, but are not limited to, displaying an image of the vehicle on which the remote response is to be performed on a dedicated screen for remote response, or enlarging the image. The remote control server may automatically perform the pre-operation, for example, a predetermined time before the time when the operator will begin the remote response. This reduces the number of operations that the operator needs to perform. Note that performing the pre-operation is an example of executing a predetermined process.

[0193] 3, an example in which the determination in step S16 is executed when step S14 is determined to be "no" has been described, but the present invention is not limited to this, and step S18 may also be executed. That is, if the remote response planner 23 cannot address the issue by adjusting the response order ("no" in S14), the remote response planner 23 may determine a plan that includes an on-site response without including vehicle control (S18). For example, if the remote response planner 23 determines that no remote response order exists, the remote response planner 23 may, as a predetermined process, send a notification to the staff terminal 40 of the on-site staff member requesting an on-site response.

[0194] In the above embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0195] The order in which the steps in the flowchart are executed is merely an example for specifically explaining the present disclosure, and other orders may be used. Some of the steps may be executed simultaneously (in parallel) with other steps, or some of the steps may not be executed.

[0196] The division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as a single functional block, one functional block may be divided into multiple blocks, or some functions may be moved to another functional block.Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software.

[0197] Furthermore, the remote control server according to the above-described embodiments may be realized as a single device or may be realized by multiple devices. When the remote control server 20 is realized by multiple devices, the components of the remote control server 20 may be distributed among the multiple devices in any manner. When the remote control server 20 is realized by multiple devices, the communication method between the multiple devices is not particularly limited and may be wireless communication or wired communication. Furthermore, wireless communication and wired communication may be combined between the devices. Furthermore, at least one function of the remote control server 20 may be realized by the operator terminal 30 or a mobile object.

[0198] Furthermore, each component described in the above embodiments may be implemented as software or, typically, as an LSI, which is an integrated circuit. These components may be individually integrated into a single chip, or some or all of them may be integrated into a single chip. Here, the term "LSI" is used, but depending on the level of integration, it may also be referred to as an IC, system LSI, super LSI, or ultra LSI. Furthermore, the integrated circuit implementation method is not limited to LSI, and may be implemented using a dedicated circuit (a general-purpose circuit that executes a dedicated program) or a general-purpose processor. A field programmable gate array (FPGA), which can be programmed after LSI fabrication, or a reconfigurable processor, which can reconfigure the connections or settings of circuit cells within an LSI, may also be used. Furthermore, if an integrated circuit implementation technology that replaces LSI emerges due to advances in semiconductor technology or a derivative technology, that technology may naturally be used to integrate the components.

[0199] A system LSI is an ultra-multifunctional LSI manufactured by integrating multiple processing units on a single chip. Specifically, it is a computer system that includes a microprocessor, ROM (Read Only Memory), RAM (Random Access Memory), etc. Computer programs are stored in the ROM. The system LSI achieves its functions when the microprocessor operates in accordance with the computer program.

[0200] Furthermore, one aspect of the present disclosure may be a computer program that causes a computer to execute each of the characteristic steps included in the information processing method shown in either FIG. 3 or FIG.

[0201] Furthermore, for example, the program may be a program to be executed by a computer. Another aspect of the present disclosure may be a computer-readable non-transitory recording medium on which such a program is recorded. For example, such a program may be recorded on a recording medium and distributed or circulated. For example, the distributed program may be installed in a device having another processor, and the program may be executed by the processor, thereby causing the device to perform each of the above processes.

[0202] The present disclosure is applicable to an information processing method for an operator to remotely control a mobile object.

[0203] REFERENCE SIGNS LIST 1 Information processing system 10 Robot (mobile body) 11 Robot information acquisition unit 12, 22, 52 Remote response request unit 13 Vehicle control unit 20 Remote control server (information processing device) 21 Storage unit 23 Remote response planning unit (first acquisition unit, second acquisition unit, judgment unit) 24 Control unit 25 Remote response communication unit 30 Operator terminal (terminal device) 31 Display unit 32 Input unit 33 Interface control unit 40 Staff terminal (terminal device) 41 Emergency response notification unit 50 Service server 51 Service information acquisition unit N Network U1 Operator U2 Emergency staff

Claims

1. An information processing method comprising: acquiring a remote response request for each of a plurality of mobile bodies; acquiring a required response time and a response completion grace time for each of two or more unprocessed remote response requests; determining whether or not there is a remote response order that enables the remote response of each of the two or more remote response requests to be completed by the response completion grace time; and, if it is determined that there is a remote response order, executing a specified process for the two or more remote response requests based on the remote response order.

2. The information processing method of claim 1, wherein the two or more remote response requests include a first remote response request and a second remote response request, a first response required time and a first response completion grace time are obtained for the first remote response request, and a second response required time and a second response completion grace time are obtained for the second remote response request, determining whether the remote response of the first remote response request can be completed by the first response completion grace time based on a first time at which the first remote response request can be notified and the first response required time, and determining whether the remote response of the second remote response request can be completed by the second response completion grace time based on a second time at which the second remote response request can be notified and the second response required time.

3. The information processing method according to claim 2, wherein the second response completion grace period is a time later than the first response completion grace period, and the second time is a time determined based on the first time and the first response required time.

4. The information processing method according to claim 2, wherein the two or more remote response requests include a first remote response request and a second remote response request; when it is determined that the remote response order exists, it is determined whether there is a remote response request that can be moved up or down in the remote response order; and when it is determined that there is a remote response request that can be moved up or down, the specified processing is executed simultaneously for the remote response requests that can be moved up or down.

5. The information processing method of claim 4, wherein if the time at which the first remote response request and the second remote response request can be notified and the time based on the sum of the first response required time and the second response required time are earlier than the first response completion grace time and the second response completion grace time, respectively, it is determined that the first remote response request and the second remote response request can be handled in a different order.

6. An information processing method according to any one of claims 1 to 5, wherein, when it is determined that the remote response order does not exist, the predetermined process is to send a notification to a terminal device of a staff member requesting that the staff member respond on the spot.

7. An information processing method as described in any one of claims 1 to 5, further comprising: determining whether or not a remote response order exists when it is determined that the remote response order does not exist, assuming that a specified vehicle control is to be performed on one or more of the multiple moving bodies; and, when it is determined that the remote response order exists, executing the specified processing for the two or more remote response requests based on the remote response order, and causing the one or more moving bodies to perform the specified vehicle control.

8. The information processing method of claim 7, further comprising excluding the remote response requests obtained from the one or more mobile bodies from the remote response requests that are the subject of executing the specified processing, and determining, based on the required response time and the response completion grace time for the remote response request of each of the one or more other mobile bodies among the plurality of mobile bodies excluding the one or more mobile bodies, whether or not there is a remote response order that can complete the remote response of the remote response request of each of the other one or more mobile bodies by the response completion grace time for each of the other one or more mobile bodies.

9. The information processing method of claim 7, further acquiring the response completion grace period for each of the one or more moving bodies when the specified vehicle control is performed on the one or more moving bodies, the response completion grace period being later than when the specified vehicle control is not performed, and further determining whether or not the remote response order exists based on the acquired response completion grace period for each of the one or more moving bodies.

10. The information processing method according to claim 7, wherein the predetermined vehicle control includes at least one of a route change instruction, a speed change instruction, and a stop instruction to the one or more moving bodies.

11. The information processing method described in claim 7, wherein, when it is determined that the remote response order does not exist when the specified vehicle control is to be performed on the one or more moving bodies, the specified processing involves sending a notification to a terminal device of an on-site response personnel requesting on-site response.

12. An information processing method according to any one of claims 2 to 5, further comprising: when the order of the remote response is determined to be the first remote response request and then the second remote response request, acquiring a third remote response request after executing the specified processing for the first remote response request; acquiring a third response required time and a third response completion grace time for the third remote response request; and updating the remote response order based on the second response required time, the second response completion grace time, the third response required time, and the third response completion grace time.

13. An information processing method according to any one of claims 1 to 5, wherein the plurality of mobile entities includes a mobile entity capable of providing a specified service, and the two or more remote support requests include remote support requests obtained from a server device that manages the specified service.

14. The information processing method according to any one of claims 1 to 5, wherein the two or more remote response requests include a remote response request generated based on operation information of the plurality of moving bodies.

15. An information processing method according to any one of claims 1 to 5, wherein the predetermined processing includes processing for presenting a notification of each of the two or more remote response requests on an operator's terminal device based on the remote response order in accordance with the remote response order.

16. An information processing device comprising: a first acquisition unit that acquires a remote response request for each of a plurality of moving bodies; a second acquisition unit that acquires a required response time and a response completion grace time for each of two or more unprocessed remote response requests; a judgment unit that judges whether or not there is a remote response order that can complete the remote response for each of the two or more remote response requests by the respective response completion grace time; and a control unit that, when it is determined that there is a remote response order, executes a predetermined process for the two or more remote response requests based on the remote response order.

17. A program for causing a computer to execute the information processing method according to any one of claims 1 to 5.

Images