Information processing method, program, and information processing terminal

Abstract

This information processing method is executed by a computer, the method acquiring an operation mode of a mobile body while the mobile body is being remotely operated by an operator, and when the acquired operation mode is a specific mode (reverse mode) (S201: YES), outputting switching information that includes at least one of a switching area in which the operation mode can be switched from the specific mode to another mode (forward mode) and a switching method for switching the operation mode from the specific mode to the other mode (S202).

Description

Information processing method, program, and information processing terminal

[0001] The present disclosure relates to an information processing method, a program, and an information processing terminal for assisting an operator in remotely controlling a moving object.

[0002] Patent Literature 1 discloses a remote control system for an unmanned mobile vehicle. In this system, a handheld controller includes a forward / reverse switch for switching the travel mode of the unmanned mobile vehicle between outbound travel mode and inbound travel mode, and a joystick for indicating the travel direction of the unmanned mobile vehicle. In this system, when the travel mode of the unmanned mobile vehicle is switched from outbound travel mode to inbound travel mode, the image displayed on the display device is switched from the image captured by the camera capturing the front of the outbound journey to the image captured by the camera capturing the front of the inbound journey. In addition, in this system, the operation of the unmanned mobile vehicle in response to the operation of the handheld controller is maintained the same when switching to outbound travel mode and when switching to inbound travel mode.

[0003] JP 2013-114325 A

[0004] However, in the system disclosed in Patent Document 1, when an operator remotely controls an unmanned vehicle (mobile vehicle) in a driving mode that the operator finds difficult, there is no suggestion to switch the driving mode to another driving mode. As a result, the operator is basically forced to remotely control the vehicle in the driving mode that the operator finds difficult, which poses a problem.

[0005] Therefore, the present disclosure provides an information processing method and the like that makes it easier for an operator to remotely control a moving object.

[0006] An information processing method according to one aspect of the present disclosure is an information processing method executed by a computer, which acquires a driving mode of a moving body while the moving body is remotely operated by an operator, and if the acquired driving mode is a specific mode, outputs switching information including at least one of a switching area in which the driving mode can be switched from the specific mode to another mode, and a switching method for switching the driving mode from the specific mode to the other mode.

[0007] These comprehensive or specific aspects may be realized as a system, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, a method, an integrated circuit, a computer program, and a recording medium.

[0008] According to the information processing method and the like according to one aspect of the present disclosure, it becomes easier for an operator to remotely control a moving object.

[0009] FIG. 1 is an explanatory diagram illustrating reasons why an operator becomes unfamiliar with remotely operating a mobile object. FIG. 2 is an explanatory diagram illustrating an overview of an information processing system according to an embodiment. FIG. 3 is a block diagram illustrating an example of an overall configuration including an information processing system according to an embodiment. FIG. 4 is a schematic diagram illustrating parameters used when calculating a first proficiency level when a mobile object is present in a curved area. FIG. 5 is a schematic diagram illustrating parameters used when calculating a first proficiency level when a mobile object is present in a curved area. FIG. 6 is a schematic diagram illustrating parameters used when calculating a first proficiency level when a mobile object is present in a route other than a curved area. FIG. 7 is a schematic diagram illustrating parameters used when calculating a first proficiency level when a mobile object is present in a route other than a curved area. FIG. 8 is a schematic diagram illustrating parameters used when calculating a second proficiency level. FIG. 9 is a flowchart illustrating an example of a basic operation of an information processing system according to an embodiment. FIG. 10 is a flowchart illustrating an example of output of switching information of an information processing system according to an embodiment. FIG. 11 is a diagram illustrating an example of a curved area in a route of a mobile object. FIG. 12 is a diagram illustrating an example of a switching area in a route of a mobile object. FIG. 13 is a diagram illustrating an example of an image displayed on a display unit of an information processing terminal in the first example. FIG. 14 is a diagram illustrating an example of an image displayed on a display unit of an information processing terminal in the first example. FIG. 15 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the first example. FIG. 16 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the first example. FIG. 17 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the second example. FIG. 18 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the third example. FIG. 19 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the third example. FIG. 20 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the third example. FIG. 21 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the third example. FIG. 22 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the fourth example. FIG. 23 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the fourth example. FIG. 24 is a diagram showing an example of an image displayed on the display unit of the information processing terminal in the fourth example.FIG. 25 is a diagram showing an example of an image showing a procedure for changing direction, which is displayed on the display unit of the information processing terminal.

[0010] An information processing method according to a first aspect of the present disclosure is an information processing method executed by a computer, which acquires a driving mode of a moving body while the moving body is remotely operated by an operator, and if the acquired driving mode is a specific mode, outputs switching information including at least one of a switching area in which the driving mode can be switched from the specific mode to another mode, and a switching method for switching the driving mode from the specific mode to the other mode.

[0011] This has the advantage that the operator can easily switch the operation mode as needed by referring to the switching information, making it easier for the operator to remotely control the moving body.

[0012] Also, for example, in the information processing method according to the second aspect of the present disclosure, in the first aspect, the operator's proficiency in remote operation is further acquired, and if the acquired operating mode is the specific mode, it is determined whether or not to output the switching information based on the acquired proficiency.

[0013] This has the advantage that whether or not to output the switching information is determined depending on whether or not the operator is accustomed to remote operation, making it easier for the operator to refer to the switching information at an appropriate timing.

[0014] Also, for example, in the information processing method according to the third aspect of the present disclosure, in the second aspect, the proficiency level includes a first proficiency level calculated based on a current remote operation situation by the operator.

[0015] This has the advantage that whether or not to output switching information is determined in real time based on whether or not the operator is familiar with remote operation, making it easier for the operator to refer to the switching information at an appropriate time.

[0016] Also, for example, in the information processing method according to the fourth aspect of the present disclosure, in the second or third aspect, the proficiency level includes a second proficiency level calculated based on the operator's past remote operation performance.

[0017] This has the advantage that whether or not to output switching information is determined by determining whether or not the operator is familiar with remote operation based on past remote operation results, making it easier for the operator to refer to the switching information at an appropriate time.

[0018] Furthermore, for example, in an information processing method according to a fifth aspect of the present disclosure, in any one of the second to fourth aspects, a first evaluation value indicating the difficulty of movement of the moving body on a route to the destination of the moving body is further acquired, and if the acquired driving mode is the specific mode, a decision is made as to whether or not to output the switching information based further on the acquired first evaluation value.

[0019] This has the advantage that whether or not to output switching information is determined not only based on whether the operator is familiar with remote control, but also based on the difficulty of moving around on the mobile object, making it easier for the operator to refer to the switching information at the appropriate time.

[0020] Furthermore, for example, in an information processing method according to a sixth aspect of the present disclosure, in any one of the second to fifth aspects, a second evaluation value indicating the degree of execution of the service performed by the mobile body is further acquired, and if the acquired operating mode is the specific mode, it is determined whether or not to output the switching information based further on the acquired second evaluation value.

[0021] This has the advantage that whether or not to output switching information is determined not only based on whether the operator is familiar with remote operation, but also based on the degree of service execution, making it easier for the operator to refer to switching information at the appropriate time.

[0022] Furthermore, for example, in an information processing method according to a seventh aspect of the present disclosure, in any one of the first to sixth aspects, when the switching information including the switching area is output, it is determined whether a change of direction is possible at the current location of the moving body, and if it is determined that the change of direction is possible, the switching information is output with the current location of the moving body as the switching area, and if it is determined that the change of direction is not possible, the switching information is output with a location other than the current location of the moving body as the switching area.

[0023] This has the advantage that the operator can easily understand locations where direction changes are possible by referring to the switching information, making it easier for the operator to change the direction of the moving body by remote control.

[0024] Also, for example, a program according to an eighth aspect of the present disclosure causes a computer to execute the information processing method according to any one of the first to seventh aspects.

[0025] This has the advantage that the operator can easily switch the operation mode as needed by referring to the switching information, making it easier for the operator to remotely control the moving body.

[0026] Furthermore, for example, an information processing terminal according to a ninth aspect of the present disclosure includes a display unit, a communication unit, and a terminal-side processing unit. The communication unit acquires the switching information obtained by the information processing method according to any one of the first to seventh aspects. When the driving mode of the moving object is the specific mode while the moving object is being remotely operated by the operator, the terminal-side processing unit causes the display unit to display the switching information acquired by the communication unit.

[0027] This has the advantage that the operator can easily switch the operation mode as needed by referring to the switching information, making it easier for the operator to remotely control the moving body.

[0028] [1. Findings that Form the Basis of the Present Disclosure] First, the inventor's viewpoint will be explained below.

[0029] For example, a mobile unit 2 (see FIG. 1 , etc.), which is an autonomous mobile robot such as an automatic delivery robot, autonomously moves basically in a forward mode, moving forward. However, depending on the situation in which the mobile unit 2 is placed, the mobile unit 2 may autonomously move in a reverse mode, moving backward. For example, if the mobile unit 2 is an autonomous mobile forklift and moves through a relatively narrow warehouse or logistics center where packages to be delivered are densely packed, the mobile unit 2 may be forced to move in the reverse mode rather than the forward mode.

[0030] Here, since switching the moving body 2 back to forward mode after starting to move in reverse mode would likely result in a loss of time, the moving body 2 basically moves to its destination in reverse mode. For example, if the moving body 2 is placed in a situation where autonomous movement is difficult, an operator may intervene and remotely control the moving body 2. However, if an operator who is unfamiliar with remote control in reverse mode remotely controls the moving body 2, safety concerns may arise, such as the moving body 2 colliding with surrounding objects or people. Therefore, there is a need to quickly switch the driving mode of the moving body 2 from reverse mode to forward mode.

[0031] Reasons why an operator may become unfamiliar with remote control of the mobile body 2 in reverse mode are explained below with reference to FIG. 1 . FIG. 1 is an explanatory diagram of reasons why an operator may become unfamiliar with remote control of the mobile body 2. Specifically, when the mobile body 2 moves through a curved area, the inside wheel difference and the outside wheel difference are different between forward mode and reverse mode. However, one of the reasons why an operator may become unfamiliar with remote control is that it is difficult for the operator to intuitively understand these differences. For example, in the example shown in FIG. 1 , when the mobile body 2 moves through a curved area in forward mode (see (1) in FIG. 1 ), the front wheels (drive wheels) of the mobile body 2 pass on the inside of the rear wheels (steered wheels). Therefore, if the front wheels do not hit an obstacle, the rear wheels will not hit an obstacle either. On the other hand, when the mobile body 2 moves through a curved area in reverse mode (see (2) in FIG. 1 ), even if the rear wheels do not hit an obstacle, it does not necessarily mean that the front wheels will not hit an obstacle.

[0032] In view of the above, the inventors have come up with the present disclosure.

[0033] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0034] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, the arrangement and connection of the components, steps, and the order of steps shown in the following embodiments are merely examples and are not intended to limit the present disclosure.

[0035] (Embodiments) Hereinafter, an information processing system and an information processing method according to an embodiment will be described.

[0036] [2. Overview] First, an overview of an information processing system (information processing method) according to an embodiment will be described with reference to Fig. 2. Fig. 2 is an explanatory diagram of an overview of an information processing system according to an embodiment. Fig. 2(a) is a flowchart showing the flow of remote control of a moving object 2 when an information processing system according to a comparative example is used. Fig. 2(b) is a flowchart showing the flow of remote control of a moving object 2 when an information processing system according to an embodiment is used.

[0037] 2A, in the information processing system of the comparative example, when an operator intervenes and starts remote operation of the moving object 2, no information is provided to the operator. Therefore, the operator remotely controls the moving object 2 to move to the destination regardless of the driving mode (S101). Therefore, when using the information processing system of the comparative example, if the driving mode of the moving object 2 is reverse mode, the operator must remotely control the moving object 2 while keeping it in reverse mode.

[0038] 2(b), if the driving mode of the moving object 2 is the reverse mode (specific mode) at the time the operator intervenes and starts remote operation of the moving object 2 (S201: YES), the information processing system according to the embodiment presents switching information to the operator (S202). Here, the switching information is information including at least one of a switching area A2 (described later) in which the driving mode can be switched from the reverse mode (specific mode) to another mode (forward mode) and a switching method for switching the driving mode from the reverse mode to another mode.

[0039] For example, when the switching area A2 is presented as the switching information, the operator moves the mobile unit 2 in reverse mode to the switching area A2, then changes direction of the mobile unit 2 (S203), and performs an operation to switch the driving mode from reverse mode to forward mode. The operator then remotely controls the mobile unit 2 to move in forward mode to the destination (S204). Also, when the switching method is presented as the switching information, the operator moves the mobile unit 2 to an area where the direction of the mobile unit 2 can be changed, at his / her own discretion, then changes direction of the mobile unit 2 (S203), and performs an operation to switch the driving mode from reverse mode to forward mode in accordance with the presented switching method. The operator then remotely controls the mobile unit 2 to move in forward mode to the destination (S204).

[0040] As described above, in the information processing system (information processing method) according to the embodiment, when the driving mode of the moving body 2 is a specific mode (reverse mode), switching information is output to present the switching information to the operator. Therefore, in the information processing system (information processing method) according to the embodiment, the operator can switch the driving mode of the moving body 2 according to the presented switching information, making it easier for the operator to remotely control the moving body 2. For example, as described above, when an operator who is unfamiliar with remote control in reverse mode remotely controls the moving body 2, it becomes easier to quickly switch the driving mode of the moving body 2 from reverse mode to forward mode, and it is easier to dispel safety concerns such as the moving body 2 colliding with a surrounding object or person.

[0041] [3. Configuration] Next, a specific configuration of the information processing system 1 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of an overall configuration including the information processing system 1 according to the embodiment. The information processing system 1 is a system for outputting information related to a moving object 2.

[0042] The mobile object 2 includes an autonomous mobile robot, such as an automatic delivery robot. In addition to an autonomous mobile robot, the mobile object 2 may also include an autonomous mobile vehicle, such as an automobile or a motorcycle. In the embodiment, the mobile object 2 is not a completely autonomous mobile object, but is remotely monitored by an operator (user) via a remote control system. In some cases, the mobile object 2 may be moved by remote control by an operator via the remote control system. In the following description, the mobile object 2 is assumed to be an automatic delivery robot that automatically delivers packages and the like to a destination.

[0043] The mobile object 2 is not limited to the autonomous mobile object described above, but may be, for example, a manually operated object that can be manually operated by a driver. The work performed by the mobile object 2 is not limited to the delivery work described above, but may be other work such as cleaning, monitoring, sales, or advertising.

[0044] Here, the remote control system is a system that operates in a remote control center or server device where one or more operators are located, and allows each operator to remotely monitor and, in some cases, remotely control one or more mobile objects 2. Each operator uses an information processing terminal 3, such as a personal computer, to monitor the status of each mobile object 2 displayed on the display unit 34. Furthermore, each operator remotely controls the mobile object 2, for example, when a request for remote control is received from the mobile object 2 or when the operator determines that the mobile object 2 is in a state where it should be remotely controlled.

[0045] The information processing system 1 is an example of a computer that executes an information processing method. The components that make up the information processing system 1 may be provided in a single housing or may be distributed. When the components that make up the information processing system 1 are distributed, the information processing method may be executed by multiple computers. In the embodiment, the information processing system 1 is realized by a server device.

[0046] 3 , the mobile object 2 includes a communication unit 21, a position acquisition unit 22, a status acquisition unit 23, a surrounding information acquisition unit 24, a processing unit 25, a remote operation control unit 26, and an automatic driving control unit 27. The communication unit 21, the position acquisition unit 22, the status acquisition unit 23, the surrounding information acquisition unit 24, the processing unit 25, the remote operation control unit 26, and the automatic driving control unit 27 are realized by a processor that executes a program stored in memory, a communication interface, etc. The memory is a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and can store the program executed by the processor.

[0047] The communication unit 21 is a wireless communication interface for communicating with the information processing system 1 via a network N1 such as the Internet. The communication unit 21 transmits to the information processing system 1 location information relating to the location of the mobile object 2 acquired by the location acquisition unit 22 and status information relating to the status of the mobile object 2 acquired by the status acquisition unit 23. The communication unit 21 also transmits to the information processing system 1 surrounding information relating to the environment around the mobile object 2 acquired by the surrounding information acquisition unit 24. The communication unit 21 also receives an instruction signal indicating an instruction for remote control of the mobile object 2 transmitted from, for example, a remote control system.

[0048] The position acquisition unit 22 detects the position of the mobile object 2 to acquire information indicating the position of the mobile object 2. The position acquisition unit 22 detects the position of the mobile object 2 based on the positioning results of a positioning system such as GPS (Global Positioning System).

[0049] The status acquisition unit 23 acquires status information relating to the status of the mobile object 2 using various sensors such as a speed sensor, an acceleration sensor, other speed sensors, and a distance sensor. The status information includes, for example, information such as the speed, acceleration, and angular velocity of the mobile object 2, the distance between the mobile object 2 and an object, and the status of a system mounted on the mobile object 2. The status information also includes, for example, information indicating the communication status between the remote control system and the mobile object 2 and the degree of delay in an instruction signal transmitted from the remote control system.

[0050] Furthermore, the status information includes, for example, information indicating the current driving mode of the moving body 2. In the embodiment, the driving modes of the moving body 2 are two types: forward mode and reverse mode. Furthermore, in the embodiment, the reverse mode corresponds to the "specific mode", and the forward mode corresponds to the "other mode". Of course, the driving modes of the moving body 2 are not limited to the forward mode and the reverse mode, and may further include other driving modes. Furthermore, the "specific mode" is not limited to the reverse mode, but may be other driving modes. Similarly, the "other mode" is not limited to the forward mode, but may be other driving modes.

[0051] The surrounding information acquisition unit 24 acquires surrounding information about the environment around the mobile body 2 using sensors such as a camera, LiDAR (Light Detection and Ranging), or radar mounted on the mobile body 2. The surrounding information acquisition unit 24 also acquires, as surrounding information, information such as targets or grid maps obtained by processing the information acquired by the sensors using a machine learning model or signal processing, etc. The surrounding information includes, for example, an image captured in front of the mobile body 2, an image captured to the left of the mobile body 2, an image captured to the right of the mobile body 2, and an image captured behind the mobile body 2.

[0052] The processing unit 25 is a main controller of the moving object 2. The processing unit 25 manages and controls the entire moving object 2 by cooperating with each of the communication unit 21, the position acquisition unit 22, the status acquisition unit 23, the surrounding information acquisition unit 24, the remote operation control unit 26, and the automatic driving control unit 27.

[0053] The remote operation control unit 26 controls the mobile object 2 in accordance with instructions given to the mobile object 2 from the remote control system via the network N1, thereby causing the mobile object 2 to move autonomously. In this way, the mobile object 2 is remotely operated in accordance with instructions from an operator.

[0054] The automatic driving control unit 27 controls the mobile object 2 in accordance with instructions given to the mobile object 2 from the remote control system via the network N1, thereby causing the mobile object 2 to move autonomously. The instructions here differ from the instructions from the operator described above. For example, if the mobile object 2 is an automatic delivery robot, the automatic driving control unit 27 causes the mobile object 2 to move autonomously in accordance with instructions to deliver packages or the like to one or more destinations.

[0055] As shown in FIG. 3 , the information processing system 1 includes a communication unit (output unit) 11, an acquisition unit 12, and a processing unit 13. The communication unit 11, the acquisition unit 12, and the processing unit 13 are realized by a processor that executes a program stored in memory, a communication interface, and the like. The memory is a ROM, a RAM, or the like, and can store the program executed by the processor. The information processing system 1 also reads and references information stored in the storage device 4 by communicating with the storage device 4 via a network N1. The storage device 4 may be installed in the same server device as the information processing system 1.

[0056] The communication unit 11 is a wireless communication interface for communicating with each of one or more mobile bodies 2 targeted by the information processing system 1 via the network N1. The communication unit 11 receives, for each mobile body 2, the position information, status information, and surrounding information transmitted from the mobile body 2. Note that although only one mobile body 2 is shown in FIG. 3 , in the embodiment, there are multiple mobile bodies 2.

[0057] Furthermore, the communication unit 11 communicates with the information processing terminal 3 used by the operator via the network N1. The information processing terminal 3 is, for example, a smartphone, a tablet terminal, or a personal computer. In the following, the information processing terminal 3 is described as a personal computer. Although only one information processing terminal 3 is illustrated in FIG. 3 , if there are multiple operators, for example, there will be multiple information processing terminals 3 used by each of the multiple operators. In this case, the communication unit 11 communicates with each of the multiple information processing terminals 3.

[0058] In the embodiment, the communication unit 11 transmits the switching information to the information processing terminal 3 via the network N1. Details of the switching information will be described later. That is, the communication unit 11 also functions as an output unit that outputs the switching information.

[0059] 3, the information processing terminal 3 includes a communication unit 31, a processing unit (terminal-side processing unit) 32, an input unit 33, and a display unit 34. The communication unit 31 and the processing unit 32 are realized by a processor that executes a program stored in memory, a communication interface, etc. The memory is a ROM, a RAM, etc., and can store the program executed by the processor.

[0060] The communication unit 31 is a wireless communication interface for communicating with the information processing system 1 via the network N1. The communication unit 31 receives information about the mobile object 2, particularly switching information, transmitted from the information processing system 1.

[0061] The processing unit 32 is a main controller of the information processing terminal 3. The processing unit 32 manages and controls the entire information processing terminal 3 by cooperating with each of the communication unit 31, the input unit 33, and the display unit .

[0062] For example, when the processing unit 32 receives an input requesting a change in driving mode via the input unit 33, the processing unit 32 transmits a signal requesting a change in driving mode to the information processing system 1 via the communication unit 31. Furthermore, when the processing unit 32 receives an input indicating that the moving object 2 has completed changing direction via the input unit 33, the processing unit 32 transmits a signal indicating that the moving object 2 has completed changing direction to the information processing system 1 via the communication unit 31.

[0063] The input unit 33 is an input interface that accepts input from an operator. For example, the input unit 33 may include a pointing device such as a keyboard or a mouse. If the display unit 34 is a touch panel display, the display unit 34 also serves as the input unit 33. The input unit 33 accepts, for example, an input from the operator requesting switching of the driving mode. The input unit 33 also accepts, for example, an input from the operator indicating completion of a turn of the moving object 2. The input unit 33 also accepts, for example, an input from the operator to enlarge a map B3 (described later) and an input to restore the map to its normal size.

[0064] The display unit 34 is, for example, a liquid crystal display or the like, and displays information processed by the processing unit 32. The display unit 34 displays various information based on, for example, peripheral information transmitted from the information processing system 1. Specifically, the display unit 34 displays, for example, an image captured in front of the moving object 2, an image captured to the left of the moving object 2, an image captured to the right of the moving object 2, and an image captured behind the moving object 2, which are included in the peripheral information. The display unit 34 also displays information such as the speed and angular velocity of the moving object 2 and the status of a system installed in the moving object 2, based on, for example, status information transmitted from the information processing system 1. The display unit 34 also displays various information based on, for example, switching information transmitted from the information processing system 1. Specific examples of this various information will be described in detail in [6. UI (User Interface)] below.

[0065] Returning to the explanation of the information processing system 1, the acquisition unit 12 acquires various types of information via the communication unit 11 and the network N1. Specifically, the acquisition unit 12 acquires, for example, location information, status information, and surrounding information for each mobile object 2. In addition, the acquisition unit 12 acquires, for example, the driving mode of the mobile object 2 for each mobile object 2.

[0066] The processing unit 13 is a main controller of the information processing system 1. The processing unit 13 manages and controls the entire information processing system 1 by cooperating with each of the communication unit 11 and the acquisition unit 12. The processing unit 13 has a determination unit 131, a calculation unit 132, and a switching area determination unit 133. The determination unit 131, the calculation unit 132, and the switching area determination unit 133 are all realized as functions executed by the processing unit 13.

[0067] The determination unit 131 acquires the operator's remote operation proficiency calculated by the calculation unit 132, and when the driving mode acquired by the acquisition unit 12 is a specific mode (here, reverse mode), determines whether to output switching information based on the acquired proficiency. The operation of the determination unit 131 will be described in detail in "3. Examples of calculation and determination of proficiency."

[0068] The calculation unit 132 calculates the operator's proficiency in remote operation. In this embodiment, the calculation unit 132 calculates a first proficiency based on the current situation of the operator's remote operation. The calculation unit 132 also calculates a second proficiency based on the operator's past performance in remote operation. The operation of the calculation unit 132 will be described in detail in "4. Examples of calculation and determination of proficiency."

[0069] When outputting switching information including the switching area A2, the switching area determination unit 133 determines whether a direction change is possible at the current location of the moving object 2, based on the position information, status information, and surrounding information acquired by the acquisition unit 12, and switching area information stored in a switching area DB 42 (described later). If the switching area determination unit 133 determines that a direction change is possible, it determines the current location of the moving object 2 as the switching area A2. On the other hand, if the switching area determination unit 133 determines that a direction change is not possible, it determines a location other than the current location of the moving object 2 as the switching area A2. The operation of the switching area determination unit 133 will be described in detail in "2. Operation."

[0070] The storage device 4 stores a driving history DB (Database) 41 and a switching area DB 42 .

[0071] The driving history DB 41 stores a history of remote operations by the operator (i.e., past remote operation results) for each operator. The remote operation history is stored in the driving history DB 41 by, for example, acquiring remote operation information input by the operator via the information processing terminal 3. The remote operation history stored in the driving history DB 41 is referred to when the calculation unit 132 calculates the second proficiency level.

[0072] The switching area DB 42 stores information indicating all routes R1 (described later) that the moving object 2 can travel along, and switching area information indicating one or more switching areas A2 in which the moving object 2 can change direction along each of the routes R1. The switching area information may include, for example, information indicating the size of the switching area A2.

[0073] 4. Examples of Calculation and Determination of Skill Level An example of calculation of skill level by the calculation unit 132 and an example of determination by the determination unit 131 as to whether or not to output switching information will be described below.

[0074] [4-1. Example of Calculation of First Proficiency Level] As an example, when the moving object 2 is in a curved area, the calculation unit 132 calculates the first proficiency level based on the following formulas (1) and (2).

[0075]

[0076] In formula (1), "F １ " indicates the first proficiency level when the moving object 2 is in a curved area, and "X" indicates the unfamiliarity level when the moving object 2 is in a curved area. "X" is the unfamiliarity level "X" for each curved area that the moving object 2 has passed through in formula (2). １ " divided by the number of curve areas passed through.

[0077] 4 and 5 are schematic diagrams showing parameters used when calculating the first proficiency level when the moving object 2 is in a curved area. In each of FIGS. 4 and 5, the "influence of unfamiliarity level" indicates that the unfamiliarity level is high when "+" and low when "-". The number of "+" and "-" indicates the magnitude of the influence on the unfamiliarity level. These matters also apply to FIGS. 6 to 8, which will be described later.

[0078] "a" in formula (2) ０ ", ..., "a ７ " is a parameter used when calculating the first proficiency level when the moving object 2 is in a curved area, as shown in FIG. 4. １ ", ..., "m ４ " is a weighting coefficient used when calculating the first proficiency level when the moving object 2 is in a curved area, as shown in FIGS. 4 and 5. ０ ", ..., "A ７ " are "a ０ ", ..., "a ７ ” is a weighting coefficient for converting the “k １ ", ..., "k ５ " are all "a" in the stage before calculating the first proficiency. ０ ", ..., "a ７ " is a parameter that affects

[0079] The above-mentioned "a ０ ", ..., "a ７ "," "m １ " and "m ２ " can be acquired by referring to the status information acquired by the status acquisition unit 23 of the moving object 2. １ ", ..., "k ５ "," "m ３ " and "m ４ " can be acquired by referring to the surrounding information acquired by the surrounding information acquisition unit 24 of the moving object 2.

[0080] Furthermore, when the moving object 2 is on a route other than a curved area, the calculation unit 132 calculates the first proficiency level based on the following formulas (3) and (4).

[0081]

[0082] In formula (3), "F ２ " indicates the first proficiency level when the moving object 2 is on a route other than a curved area, and "Y" indicates the unfamiliarity level when the moving object 2 is on a route other than a curved area.

[0083] 6 and 7 are schematic diagrams showing parameters used when calculating the first proficiency level when the moving object 2 is present on a route other than a curved area. ０ ", ..., "b ９ " is a parameter used when calculating the first proficiency level when the moving object 2 is located outside a curved area, as shown in FIG. 6. ２ ", ..., "n ４ " is a weighting coefficient used when calculating the first proficiency level when the moving object 2 is on a route other than a curved area, as shown in FIGS. 6 and 7. ０ ", ..., "B ９ " are "b ０ ", ..., "b ９ " is a weighting coefficient for converting the " １ ", ..., "l ５ " are all "b ０ ", ..., "b ９ " is a parameter that affects

[0084] The above-mentioned "b ０ ", ..., "b ９ " and "n ２ " can be acquired by referring to the status information acquired by the status acquisition unit 23 of the moving object 2. １ ", ..., "l ５ "," "n ３ " and "n ４ " can be acquired by referring to the surrounding information acquired by the surrounding information acquisition unit 24 of the moving object 2.

[0085] The first proficiency level, whether it is when the moving body 2 is in a curved area or when the moving body 2 is on a route other than a curved area, indicates that the closer the level is to "1", the higher the proficiency level, i.e., the more familiar the operator is with remote operation in the specific mode (reverse mode), and indicates that the closer the level is to "0", the lower the proficiency level, i.e., the less familiar the operator is with remote operation in the specific mode.

[0086] [4-2. Example of Calculation of Second Proficiency Level] Furthermore, as an example, the calculation unit 132 calculates the second proficiency level based on the following formulas (5) and (6).

[0087]

[0088] In formula (5), "F ３ " indicates the second proficiency. Also, "c １ ", ..., "c ４ " is a parameter used when calculating the second proficiency level, as shown in FIG. 8. １ ", ..., "C ４ " are "c １ ", ..., "c ４ 8 is a schematic diagram showing parameters used when calculating the second proficiency level.

[0089] The above-mentioned "c １ ", ..., "c ４ " can be acquired by referring to the remote operation history for each operator stored in the operation history DB 41 of the storage device 4.

[0090] The closer the second proficiency level is to "1," the higher the proficiency level, i.e., the more familiar the operator is with remote operation in the specific mode (reverse mode), and the closer it is to "0," the lower the proficiency level, i.e., the less familiar the operator is with remote operation in the specific mode.

[0091] [4-3. Example of Determining Whether to Output Switching Information] Next, an example of determining whether to output switching information by the determining unit 131 will be described.

[0092] When determining whether to output switching information based on the first proficiency calculated by the calculation unit 132, the determination unit 131 compares the first proficiency with a preset first threshold. If the first proficiency exceeds the first threshold, the determination unit 131 determines not to output the switching information, and if the first proficiency is below the first threshold, the determination unit 131 determines to output the switching information.

[0093] Furthermore, when determining whether to output switching information based on the second proficiency level calculated by the calculation unit 132, the determination unit 131 compares the second proficiency level with a preset second threshold value. Then, when the second proficiency level exceeds the second threshold value, the determination unit 131 determines not to output the switching information, and when the second proficiency level is below the second threshold value, the determination unit 131 determines to output the switching information.

[0094] Here, when the calculation unit 132 further calculates (acquires) the first evaluation value, the determination unit 131 may further determine whether to output switching information based on the calculated (acquired) first evaluation value. The first evaluation value is a value indicating the difficulty of movement of the mobile object 2 on the route R1 to the destination of the mobile object 2. Specifically, the first evaluation value is calculated by the calculation unit 132 based on the following equation (7).

[0095]

[0096] In formula (7), "G" represents the first evaluation value, and "s １ " indicates the total distance traveled by the moving object 2 in all curve areas up to the destination, and "s ２ " indicates the total distance traveled by the mobile object 2 on all straight roads to the destination. ０ "," "B ０ "," "a ０ " and "b ０ " has already been explained, so the explanation will be omitted here.

[0097] The larger the first evaluation value, the more difficult it is for the mobile object 2 to travel from the current location to the destination. Note that the first evaluation value becomes smaller the further the mobile object 2 travels. For example, even if the first proficiency level exceeds the first threshold, the decision unit 131 may decide to output switching information if the first evaluation value exceeds a third threshold set in advance.

[0098] Furthermore, when the calculation unit 132 further calculates (acquires) a second evaluation value, the determination unit 131 may further determine whether to output switching information based on the calculated (acquired) second evaluation value. The second evaluation value is a value indicating the degree of execution of a service (here, automatic delivery) executed by the mobile object 2. Specifically, the second evaluation value is calculated by the calculation unit 132 based on the following equations (8) to (10).

[0099]

[0100] In equation (8), "T" represents the second evaluation value, and "k ｔ " is the coefficient, "t １ " is the predicted time required for the moving object 2 to travel from the current location to the destination when the driving mode is the forward mode (other mode), and "t ２ " indicates the predicted time required for the moving object 2 to move from the current location to the destination when the driving mode is the reverse mode (specific mode). ３ " indicates the time required for the moving body 2 to change direction.

[0101] In addition, in the formula (9), "a' ０ "," "a' １ " and "b' ２ " are "a" and "b" when the driving mode is forward mode. ０ "," "a １ " and "b ２ " corresponds to "s １ "," "s ２ "," "a ０ "," "a １ " and "b ２ " has already been explained, so the explanation will be omitted here.

[0102] The smaller the second evaluation value, the more effectively the service (here, automatic delivery) can be executed by remotely operating the vehicle in another driving mode (forward mode). For example, even if the first proficiency level exceeds the first threshold, the decision unit 131 may decide to output the switching information if the second evaluation value is lower than a fourth threshold set in advance.

[0103] 5. Operation The operation of the information processing system 1 according to the embodiment will be described below.

[0104] [5-1. Basic Operation] First, the basic operation of the information processing system 1 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the basic operation of the information processing system 1 according to the embodiment. Below, an example will be described in which the determination unit 131 determines whether to output switching information based on the first proficiency level or the second proficiency level calculated by the calculation unit 132.

[0105] First, the acquisition unit 12 of the information processing system 1 acquires the driving mode of the moving object 2 while the operator is intervening to remotely operate the moving object 2. Then, if the driving mode is the reverse mode (specific mode) (S301: YES), the calculation unit 132 of the processing unit 13 calculates the second proficiency level based on the operator's past remote operation performance (S302).

[0106] The decision unit 131 of the processing unit 13 determines whether to output switching information based on the second proficiency level calculated by the calculation unit 132. Here, the decision unit 131 first determines whether to suggest to the operator that the driving mode be switched. If the second proficiency level is low (i.e., the second proficiency level is below the second threshold value) (S303: YES), the decision unit 131 suggests to the operator that the driving mode be switched (S304). In step S304, as will be described in [6. UI] below, the decision unit 131 displays a character string or the like suggesting that the driving mode be switched on the display unit 34 of the information processing terminal 3.

[0107] On the other hand, if the second proficiency level is high (i.e., the second proficiency level exceeds the second threshold) (S303: NO), the calculation unit 132 calculates the first proficiency level based on the current remote operation status of the operator (S309). If the first proficiency level is low (i.e., the first proficiency level is below the first threshold) (S310: YES), the determination unit 131 suggests to the operator that the operation mode be switched (S304). On the other hand, if the first proficiency level is high (i.e., the first proficiency level exceeds the first threshold) (S310: NO), the communication unit 11 of the information processing system 1 does not output switching information (S311).

[0108] When the input unit 33 of the information processing terminal 3 receives an input from the operator selecting switching of the operation mode (S305: YES), the communication unit 11 outputs switching information (S306). As a result, the switching information is displayed on the display unit 34 of the information processing terminal 3. On the other hand, when the input unit 33 of the information processing terminal 3 does not receive an input from the operator selecting switching of the operation mode (S305: NO), the communication unit 11 does not output the switching information (S311).

[0109] Thereafter, the operator remotely controls the moving object 2 to move to the switching area A2 by referring to the switching information (S307: NO), and when the operator remotely controls the moving object 2 to change direction (S307: YES), switches the driving mode of the moving object 2 from the reverse mode (specific mode) to the forward mode (other mode) (S308).

[0110] [5-2. Example of Output of Switching Information] Next, an example of output of switching information of the information processing system 1 according to the embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of output of switching information of the information processing system 1 according to the embodiment. The output operation of the switching information shown in Fig. 10 is executed when the operator inputs to select switching of the operation mode in step S305 shown in Fig. 9.

[0111] First, when the switching area determination unit 133 of the processing unit 13 of the information processing system 1 receives a request to switch the driving mode from the information processing terminal 3 (S401), it analyzes the road surface conditions on which the mobile body 2 is moving and the communication conditions of the mobile body 2 based on the location information, status information, surrounding information, and switching area information acquired by the acquisition unit 12 (S402).

[0112] Here, an example of a specific process in step S402 will be described with reference to FIG. 11 . FIG. 11 is a diagram showing an example of a curved area A1 on the route R1 of the mobile object 2. For example, the switching area determination unit 133 analyzes that a U-turn is possible in the curved area A1 if (1) the current location of the mobile object 2 is in one of the curved areas A1, (2) the area around the mobile object 2 is clear, (3) there are no oncoming vehicles, (4) the degree of delay in the instruction signal is relatively small, and (5) a U-turn is possible in the curved area A1 according to traffic rules. On the other hand, if any one of the above conditions (1) to (5) is not satisfied, the switching area determination unit 133 analyzes that a U-turn is not possible.

[0113] If the analysis shows that a U-turn (i.e., a change of direction at the current location of the moving object 2) is possible (S403: YES), the switching area determination unit 133 determines the current location of the moving object 2 as the switching area A2 (see FIG. 12 ). Then, the communication unit 11 outputs switching information indicating that a U-turn is possible (i.e., indicating that the current location of the moving object 2 is the switching area A2) (S404). As a result, the display unit 34 of the information processing terminal 3 displays the switching information indicating that a U-turn is possible.

[0114] On the other hand, if a U-turn is not possible (S403: NO), the switching area determination unit 133 searches for a switching area A2 that is a location other than the current location of the moving object 2, and generates a route to the switching area A2 (S405). That is, the switching area determination unit 133 determines a location other than the current location of the moving object 2 as the switching area A2. Then, the communication unit 11 outputs switching information indicating the switching area A2 and the route (that is, indicating the location other than the current location of the moving object 2 as the switching area A2) (S406). As a result, the display unit 34 of the information processing terminal 3 displays switching information indicating the switching area A2 other than the current location of the moving object 2 and the route to the switching area A2.

[0115] If a U-turn is not possible (S403: NO), the switching area determination unit 133 may inquire of the operator as to whether or not to wait until a U-turn is possible. If the operator approves the inquiry, the switching area determination unit 133 decides to have the moving object 2 wait for a predetermined period of time. If a U-turn becomes possible within the predetermined period of time, the switching area determination unit 133 executes the same process as in the case of step S403: YES. On the other hand, if a U-turn is not possible within the predetermined period of time, the switching area determination unit 133 executes the same process as in the case of step S403: NO. If the operator does not approve the inquiry, the switching area determination unit 133 also executes the same process as in the case of step S403: NO.

[0116] Here, an example of a specific process in step S405 will be described with reference to FIG. 12 . FIG. 12 is a diagram showing an example of a switching area A2 on the route R1 of the mobile object 2. The switching area A2 may include, for example, a parking lot, a T-junction, or an empty lot near the route R1 of the mobile object 2. For example, the switching area determination unit 133 searches for a switching area A2 suitable for changing direction of the mobile object 2 based on (1) whether the switching area A2 is on the same side of the road as the mobile object 2 is traveling on, (2) the correlation between the vehicle width of the mobile object 2 and the width of the road, (3) the distance from the current location of the mobile object 2, (4) the presence or absence of a vehicle behind the mobile object 2 in the traveling direction, or (5) the degree of congestion on the route R1 of the mobile object 2.

[0117] Thereafter, the operator, for example, refers to the switching information and remotely controls the moving body 2 to change direction (S407: YES), and switches the driving mode of the moving body 2 from reverse mode (specific mode) to forward mode (other mode) (S408).

[0118] [6. UI] A specific aspect of information displayed on the display unit 34 of the information processing terminal 3, that is, a specific aspect of the UI, will be described below.

[0119] 13 is a diagram showing an example of an image P1 displayed on the display unit 34 of the information processing terminal 3 in the first example. As shown in FIG. 13, the display unit 34 displays an image P1 for remotely operating the moving object 2. Here, the image P1 is an image for remotely operating the moving object 2 in a reverse mode (specific mode). The image P1 includes an image P11 of an image in front of the moving object 2, an image P12 of an image behind the moving object 2, an image P13 of an image to the left of the moving object 2, and an image P14 of an image to the right of the moving object 2, all of which are included in the peripheral information received from the information processing system 1. Note that "in front of the moving object 2" here refers to the front of the moving object 2 when the moving object 2 is moving forward in reverse mode. In addition, the image P11 displays a route R1 to the destination of the moving object 2.

[0120] Image P1 also includes a box B1 in which a string of characters indicating information to be notified to the operator is displayed, an icon B2 for notifying the operator of information regarding the driving mode, and a map B3 showing the surroundings of the moving body 2.

[0121] In the first example, the information processing system 1 determines that the operator's proficiency level (first proficiency level or second proficiency level) is low. Therefore, as shown in Fig. 13, the box B1 displays the text "It has been detected that you are not familiar with the current operation mode," and the icon B2 flashes in the first pattern. In this way, in the first example, the information processing system 1 displays, on the display unit 34 of the information processing terminal 3, information indicating that the operator is not familiar with remote operation in the current operation mode.

[0122] Fig. 14 is a diagram showing an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the first example. Image P1 shown in Fig. 14 is displayed on the display unit 34 after image P1 shown in Fig. 13 is displayed on the display unit 34. As shown in Fig. 14, a character string saying "Do you want to switch the driving mode?" is displayed in box B1. In this way, in the first example, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information suggesting to the operator that the driving mode be switched.

[0123] Thereafter, when the operator selects icon B2 or performs other input to select a driving mode change, image P1 shown in FIG. 15 is displayed on the display unit 34 of the information processing terminal 3. FIG. 15 is a diagram illustrating an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the first example. As shown in FIG. 15, a character string "Turnable areas are displayed on a mini map" is displayed in box B1, and icon B2 changes to a color different from that when flashing in the first pattern. Furthermore, image P11 and map B3 display a route R11 from the current location to the turnable area (i.e., switching area A2). In the example shown in FIG. 15, map B3 further displays marker A3 indicating the position of the moving object 2, the distance to the destination (switching area A2), and the number of curves to be passed on the way to the destination. In this way, in the first example, the information processing system 1 displays switching information on the display unit 34 of the information processing terminal 3. Here, the display unit 34 displays, as switching information, a switching area A2, which is a location other than the current location of the moving object 2, and a route R11 to the switching area A2.

[0124] 15 is displayed on the display unit 34 of the information processing terminal 3, if the operator inputs to select map B3, an image P1 in which map B3 is enlarged will be displayed on the display unit 34 as shown in FIG. 16. FIG. 16 is a diagram showing an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the first example. Furthermore, if the operator inputs to select enlarged map B3, image P1 including map B3 in the normal size will be displayed on the display unit 34 as shown in FIG. In this way, the operator can change the size of map B3 as appropriate.

[0125] 17 is a diagram showing an example of an image P1 displayed on the display unit 34 of the information processing terminal 3 in the second example. In the second example, the information processing system 1 determines that the operator's proficiency level (first proficiency level and second proficiency level) is high. Therefore, as shown in FIG. 17, the icon B2 does not flash on the display unit 34. In other words, in the second example, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information indicating that the operator is accustomed to remote operation in the current operation mode.

[0126] Here, when the operator makes an input to select icon B2, the character string "Do you want to switch the driving mode?" is displayed in box B1. In this way, even if the operator is highly skilled, if the operator makes a predetermined input, similar to the first example, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information suggesting to the operator to switch the driving mode.

[0127] 18 is a diagram showing an example of an image P1 displayed on the display unit 34 of the information processing terminal 3 in the third example. In the third example, the information processing system 1 determines that a U-turn is possible (i.e., a change of direction is possible) at the current location of the mobile object 2. Therefore, as shown in FIG. 18, a character string saying "A U-turn is possible here" is displayed in the box B1, and the icon B2 flashes in a second pattern different from the first pattern. In this way, in the third example, the information processing system 1 displays switching information indicating that a U-turn is possible (i.e., the current location of the mobile object 2 is displayed as switching area A2) on the display unit 34 of the information processing terminal 3.

[0128] Thereafter, after the operator remotely controls the moving object 2 to change direction, when the operator performs an input to select icon B2, image P1 shown in Fig. 19 is displayed on the display unit 34 of the information processing terminal 3. Fig. 19 is a diagram showing an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the third example. As shown in Fig. 19, a character string saying "Are you done changing direction?" is displayed in box B1. In this way, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information confirming whether the change of direction has been completed.

[0129] When the operator performs an input to select icon B2, image P1 shown in Fig. 20 is displayed on the display unit 34. Fig. 20 is a diagram showing an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the third example. As shown in Fig. 20, a character string saying "Switching driving mode" is displayed in box B1. In this way, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information confirming that the driving mode will be switched.

[0130] Thereafter, when the operator performs an input to select icon B2, image P1 shown in FIG. 21 is displayed on the display unit 34. FIG. 21 is a diagram illustrating an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the third example. As shown in FIG. 21, the box B1 displays the text "Driving mode switched," and icon B2 changes to a color different from that when flashing in the second pattern. Furthermore, images P11, P12, P13, and P14 are switched to an image captured in front of the moving object 2, an image captured behind the moving object 2, an image captured to the left of the moving object 2, and an image captured to the right of the moving object 2, respectively, when the driving mode is the forward mode (another mode). In this way, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display image P1 for remotely operating the moving object 2 when the driving mode is the forward mode (another mode).

[0131] 22 is a diagram showing an example of an image P1 displayed on the display unit 34 of the information processing terminal 3 in the fourth example. In the fourth example, the information processing system 1 determines that the moving object 2 has entered the switching area A2. Therefore, as shown in FIG. 22, a character string saying "Entering a direction change area" is displayed in the box B1, and the icon B2 flashes in the second pattern. In this way, in the fourth example, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information notifying the operator that the moving object 2 has entered the switching area A2.

[0132] Thereafter, after the operator remotely controls the moving object 2 to change direction, when the operator performs an input to select icon B2, image P1 shown in Fig. 23 is displayed on the display unit 34. Fig. 23 is a diagram showing an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the fourth example. As shown in Fig. 23, a character string saying "Are you done changing direction?" is displayed in box B1. In this way, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information confirming whether the change of direction has been completed.

[0133] When the operator performs an input to select icon B2, image P1 shown in Fig. 24 is displayed on the display unit 34. Fig. 24 is a diagram showing an example of image P1 displayed on the display unit 34 of the information processing terminal 3 in the fourth example. As shown in Fig. 24, a character string saying "Switching driving mode" is displayed in box B1. In this way, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display information confirming that the driving mode will be switched.

[0134] Thereafter, when the operator performs an input to select icon B2, the same image P1 as the image P1 shown in Fig. 21 of the third example is displayed on the display unit 34. In this manner, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display the image P1 for remotely operating the moving object 2 whose driving mode is the forward mode (other mode).

[0135] [7. Advantages, etc.] As described above, the information processing method and information processing system 1 according to the embodiment output switching information when the driving mode is a specific mode (reverse mode in this case) while the moving object 2 is being remotely operated by an operator. The switching information includes at least one of a switching area A2 in which the driving mode can be switched from the specific mode to another mode (forward mode in this case) and a switching method for switching the driving mode from the specific mode to another mode.

[0136] Therefore, for example, if the operator is unfamiliar with remote operation in a specific mode, the operator can refer to the switching information and switch the operation mode from the specific mode to another mode in the switching area A2 or according to the switching method, thereby enabling remote operation in the operation mode that the operator is relatively familiar with. Also, for example, if the operator is not unfamiliar with remote operation in a specific mode but is skilled in remote operation in other modes, the operator can refer to the switching information and switch the operation mode from the specific mode to another mode in the switching area A2 or according to the switching method, thereby enabling remote operation in the operation mode that the operator is skilled in.

[0137] In this way, the information processing method and information processing system 1 according to the embodiment have the advantage that the operator can easily switch the driving mode as appropriate by referring to the switching information, making it easier for the operator to remotely control the moving body 2.

[0138] (Other Embodiments) While the information processing method and information processing system 1 according to one or more aspects of the present disclosure have been described above based on the embodiments, the present disclosure is not limited to the embodiments. As long as they do not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to the embodiments and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects of the present disclosure.

[0139] For example, in the embodiment, the decision unit 131 of the information processing system 1 decides whether to output the switching information based on the proficiency level calculated by the calculation unit 132, but this is not limited to this. For example, the decision unit 131 may decide to output the switching information when the driving mode acquired by the acquisition unit 12 is a specific mode (reverse mode). In this case, the calculation unit 132 of the information processing system 1 and the driving history DB 41 of the storage device 4 are not required.

[0140] <First Modification> Furthermore, for example, when the acquisition unit 12 acquires information indicating that the driving mode is a specific mode (reverse mode), the determination unit 131 may identify the current position of the mobile object 2 at that time, and calculate a first estimated arrival time when the mobile object 2 is caused to travel from the current position to the destination in the specific mode, and a second estimated arrival time when the mobile object 2 is caused to travel from the current position to the destination by switching to another mode (forward mode). Then, when the first estimated arrival time is later than the second estimated arrival time, the determination unit 131 may determine to output switching information.

[0141] Both the first estimated arrival time and the second estimated arrival time can be calculated using the history of remote operations by the operator, the history of remote operations by other operators, etc. For example, the first estimated arrival time can be calculated using the speed when traveling straight, the speed when turning right or left, and the time required to change direction when the mobile unit 2 is traveling in a specific mode (reverse mode). Also, for example, the second estimated arrival time can be calculated using the speed when traveling straight, the speed when turning right or left, and the time required to change direction when the mobile unit 2 is traveling in another mode (forward mode).

[0142] Furthermore, even if the first estimated arrival time is shorter than the second estimated arrival time, the decision unit 131 may decide to output switching information if the difference between the first estimated arrival time and the second estimated arrival time is within a predetermined time. This is because, since an operator is less accustomed to driving in the specific mode (reverse mode) than in other modes (forward modes), it is necessary to pay more attention when remotely operating the mobile object 2, which is likely to increase the burden on the operator. In other words, if there is almost no difference in the estimated arrival time between remotely operating the mobile object 2 in the specific mode and remotely operating the mobile object 2 by switching to another mode, suggesting to the operator that they switch to the other mode is more likely to reduce the burden on the operator.

[0143] Incidentally, the operator may momentarily switch from another mode (forward mode) to the specific mode (reverse mode). Even in such a case, if the switching information is output, the operator may feel annoyed. Therefore, the decision unit 131 may determine whether to output the switching information when the acquisition unit 12 acquires information indicating that the driving mode is the specific mode, rather than immediately determining whether to output the switching information. The decision unit 131 may determine whether to output the switching information when it detects that the specific mode has continued for a predetermined time from the time of acquisition, or when it detects that the specific mode has continued until the mobile object 2 has traveled a predetermined distance from the time of acquisition. In this case, the decision unit 131 may identify the current position of the mobile object 2 at the time of detection and calculate each of the first estimated arrival time and the second estimated arrival time.

[0144] <Second Modification> Furthermore, for example, the decision unit 131 may count the number of curves and the number of right or left turns on a route from the current position of the moving body 2 to the destination, and determine to output switching information when the counted number is equal to or greater than a predetermined number. When the moving body 2 is remotely operated in a specific mode (reverse mode), the difficulty of remote operation tends to increase when the moving body 2 is traveling around a curve and when the moving body 2 is turning right or left. Therefore, outputting switching information when there are a large number of curves and a large number of right or left turns can be expected to make it easier for the operator to remotely operate the moving body 2. On the other hand, when the number of curves and the number of right or left turns is zero or relatively small, that is, when the moving body 2 is mostly traveling straight, there is no significant difference in the difficulty of remotely operating the moving body 2 in the specific mode and the difficulty of remotely operating the moving body 2 in another mode (forward mode). Therefore, the decision unit 131 determines not to output switching information.

[0145] The determination unit 131 may determine whether to output switching information based not only on the number of curves and the number of right or left turns, but also on the curvature of the curves or the width of the road. For example, even if the number of curves and the number of right or left turns are less than a predetermined number, if the route includes a sharp curve, the determination unit 131 may determine to output switching information. Furthermore, even if the number of curves and the number of right or left turns are less than a predetermined number, if the route includes a very narrow section, the determination unit 131 may determine to output switching information.

[0146] <Third Modification> For example, in the embodiment, the information processing system 1 may determine the degree to which to recommend switching the driving mode depending on the proficiency of the operator. For example, the information processing system 1 may determine to recommend switching the driving mode more strongly to less skilled operators than to more skilled operators. Specifically, the information processing system 1 may display on the display unit 34 of the information processing terminal 3 a message or the like simply recommending switching the driving mode to more skilled operators, while displaying on the display unit 34 a message or the like instructing less skilled operators to switch the driving mode. That is, in the former case, switching the driving mode is left to the operator's discretion, while in the latter case, the operator is almost forced to switch the driving mode.

[0147] Furthermore, for example, in a first modified example, the information processing system 1 may determine the degree to which to recommend switching the operation mode depending on the difference between the first estimated arrival time and the second estimated arrival time. Specifically, when the first estimated arrival time is greater than the second estimated arrival time, the information processing system 1 may determine to strongly recommend to the operator to switch the operation mode compared to when the first estimated arrival time is less than the second estimated arrival time and the difference is within a predetermined time.

[0148] Also, for example, in the second modified example, the information processing system 1 may determine that the greater the counted number of curves and right or left turns, the more strongly to recommend to the operator that the driving mode be switched.

[0149] <Fourth Modification> For example, in the first example of the UI of the embodiment, when the operator inputs to select switching of the driving mode, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display a map B3 (see FIG. 15 ) indicating the distance to the destination (here, the switching area A2) and the number of curves to be passed on the way to the destination. However, this is not limited to this. For example, when inquiring of the operator about switching of the driving mode, the information processing system 1 may cause the display unit 34 to display the distance to the destination (here, the final destination of the mobile object 2, not the switching area A2) and the number of curves to be passed on the way to the destination. Furthermore, for example, when inquiring of the operator about switching of the driving mode, the information processing system 1 may cause the display unit 34 to display information about the switching area A2, the distance to the switching area A2, and the number of curves to be passed on the way to the switching area A2. Furthermore, for example, when inquiring of the operator about switching the driving mode, the information processing system 1 may display the estimated arrival time at the destination when the driving mode is switched and the estimated arrival time at the destination when the driving mode is not switched on the display unit 34, or may display the difference between these estimated arrival times on the display unit 34. Either of the above display modes makes it easier for the operator to determine whether or not to switch the driving mode.

[0150] <Fifth Modification> Furthermore, for example, in the first example of the UI of the embodiment, when the operator performs an input to select switching of the driving mode, the information processing system 1 causes the display unit 34 of the information processing terminal 3 to display a map B3 (see FIG. 15 ) indicating the switching area A2 and the route R11 to the switching area A2, but the display may be in a manner that allows for discrimination between when the switching area A2 is on the route R1 to the destination (here, not the switching area A2 but the final destination of the mobile object 2) and when it is in a location different from the route R1.

[0151] Also, for example, if there are multiple candidates for the switching area A2, the information processing system 1 may automatically determine the candidate for the switching area A2 with the earliest estimated arrival time as the switching area A2 to be displayed on the display unit 34.

[0152] Furthermore, for example, the information processing system 1 may display maps B3 of multiple candidate switching areas A2 on the display unit 34 and determine the candidate switching area A2 selected by the operator as the switching area A2. In this case, for example, the information processing system 1 may support the operator in selecting the switching area A2 by displaying the multiple candidate switching areas A2 on the display unit 34 in a manner that recommends the candidate switching area A2 with the earliest estimated arrival time. Furthermore, for example, the information processing system 1 may display the multiple candidate switching areas A2' on the display unit 34 in a manner that recommends the candidate switching area A2' with the shortest travel distance in a specific mode (reverse mode). Furthermore, for each candidate switching area A2', the information processing system 1 may display, together with the map B3, the estimated arrival time to the destination when the mobile object 2 changes direction at the candidate switching area A2' and the estimated arrival time to the switching area A2' on the display unit 34.

[0153] If the switching area A2 is located at a location different from the route R1, the route R1 needs to be changed, and the information processing system 1 may display the changed route R1 on the display unit 34.

[0154] <Sixth Modification> Furthermore, for example, in the embodiment, as shown in steps S407 and S408 of Fig. 10 , the operator remotely changes the direction of the moving object 2 and then switches the driving mode from a specific mode (reverse mode) to another mode (forward mode), but this is not limited to this. For example, the operator may remotely change the direction of the moving object 2 after switching the driving mode from a specific mode to another mode. This is because, in view of the high degree of difficulty in remotely controlling a change in direction in the specific mode, it is considered easier to reduce the burden on the operator by switching to another mode and then remotely controlling the change in direction.

[0155] Fig. 25 is a diagram showing an example of an image displayed on the display unit 34 of the information processing terminal 3, showing the procedure for changing direction. Fig. 25 shows the procedure for changing direction when turning the mobile object 2 right, as indicated by the thick arrow. Fig. 25(a) shows the procedure for changing direction after switching the driving mode from a specific mode (reverse mode) to another mode (forward mode) (hereinafter also referred to as "first procedure"). Fig. 25(b) shows the procedure for switching the driving mode from a specific mode to another mode after switching direction (hereinafter also referred to as "second procedure").

[0156] In the example shown in (a) of Figure 25, first, as indicated by the thin dashed arrow, the mobile unit 2 is driven straight to the location marked with a star in a specific mode (reverse mode), at which point the driving mode is switched from the specific mode to another mode (forward mode), and then, as indicated by the thin solid arrow, the mobile unit 2 is caused to change direction (here, turn left when viewed from the straight-ahead direction of the mobile unit 2).

[0157] On the other hand, in the example shown in (b) of Figure 25, first, as shown by the thin dashed arrow, the mobile unit 2 is caused to change direction in a specific mode (reverse mode) to the location marked with a star (here, a left turn as viewed from the straight-ahead direction of the mobile unit 2), and at the location marked with a star, the driving mode is switched from the specific mode to another mode (forward mode), and then the mobile unit 2 is caused to move straight, as shown by the thin solid arrow.

[0158] Considering the high degree of difficulty in remotely controlling a direction change in the specific mode as described above, the information processing system 1 may display the first procedure as switching information on the display unit 34 of the information processing terminal 3 in preference to the second procedure. Alternatively, the information processing system 1 may display both the first procedure and the second procedure as switching information on the display unit 34, leaving it up to the operator to select which procedure to adopt. For example, if the area in which the direction change is to be performed is relatively large, it is considered that it is easier to change direction in the specific mode (reverse mode), so the operator may adopt the second procedure.

[0159] In addition, the information processing system 1 may determine in advance whether to display the first procedure or the second procedure as switching information on the display unit 34 depending on the location or situation of the mobile object 2.

[0160] <Seventh Variation> Furthermore, for example, in an embodiment, when the information processing system 1 does not accept an input from the operator to select switching of the driving mode, that is, when remote operation in a specific mode (reverse mode) is continued, the information processing system 1 may search for a route with fewer curves and right or left turns than the current route R1, in other words, a route that is easier for the mobile body 2 to travel in the specific mode, and suggest to the operator that they change to the searched route.

[0161] In this case, the information processing system 1 may display differences between the route R1 before the change and the route after the change on the display unit 34 of the information processing terminal 3. The differences may include, for example, the estimated time of arrival at the destination, the number of curves, the number of right or left turns, the degree of curvature of the curve, or the width of the curve. Furthermore, the information processing system 1 may display, on the display unit 34, the difficulty of remotely controlling the curve and the right or left turn using the above differences as parameters.

[0162] For example, in the embodiment, the communication unit (output unit) 11 of the information processing system 1 may output only the switching method for switching the driving mode from a specific mode (reverse mode) to another mode (forward mode) as switching information. In this case, the switching area determination unit 133 of the information processing system 1 and the switching area DB 42 of the storage device 4 are not required.

[0163] For example, in the embodiment, as described in [6. UI], the switching information is presented to the operator by displaying the box B1 and the icon B2 on the display unit 34 of the information processing terminal 3, but this is not limiting. For example, the switching information may be presented to the operator by outputting various pieces of information by voice from a speaker provided in the information processing terminal 3.

[0164] For example, in the embodiment, the mobile object 2 may be at least remotely operable, and may include, in addition to an automatic delivery robot, a security robot, an elevator, a construction robot, etc. Of course, the security robot, elevator, construction robot, etc. may be capable of autonomous movement.

[0165] For example, the present disclosure can be realized as a program for causing a processor to execute steps included in an information processing method. Furthermore, the present disclosure can be realized as a non-transitory computer-readable recording medium, such as a CD-ROM, on which the program is recorded.

[0166] For example, when the present disclosure is realized as a program (software), each step is performed by running the program using hardware resources such as a computer's CPU (Central Processing Unit), memory, input / output circuits, etc. In other words, each step is performed by the CPU acquiring data from memory or input / output circuits, etc., performing calculations on the data, and outputting the calculation results to memory or input / output circuits, etc.

[0167] In the above embodiment, each component included in the information processing system 1 may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0168] Some or all of the functions of the information processing system 1 according to the above embodiment are typically realized as an LSI, which is an integrated circuit. These may be individually integrated into single chips, or may be integrated into a single chip that includes some or all of the functions. Furthermore, the integrated circuit is not limited to an LSI, and may be realized using a dedicated circuit or a general-purpose processor. It is also possible to use an FPGA (Field Programmable Gate Array), which can be programmed after LSI manufacturing, or a reconfigurable processor, which allows the connections and settings of circuit cells within the LSI to be reconfigured.

[0169] Furthermore, various modifications made to the embodiments of the present disclosure within the scope that would occur to a person skilled in the art are also included in the present disclosure, as long as they do not deviate from the gist of the present disclosure.

[0170] The present disclosure can be applied to a system that assists an operator in remotely controlling a moving object.

[0171] 2 Mobile object 3 Information processing terminal 31 Communication unit 32 Processing unit (terminal side processing unit) 34 Display unit A2 Switching area R1 Route

Claims

1. An information processing method executed by a computer, comprising: acquiring a driving mode of a moving body while the moving body is remotely operated by an operator; and, if the acquired driving mode is a specific mode, outputting switching information including at least one of a switching area in which the driving mode can be switched from the specific mode to another mode, and a switching method for switching the driving mode from the specific mode to the other mode.

2. The information processing method according to claim 1, further comprising acquiring a proficiency level of the operator in remote operation, and when the acquired operating mode is the specific mode, determining whether or not to output the switching information based on the acquired proficiency level.

3. The information processing method according to claim 2, wherein the proficiency level includes a first proficiency level calculated based on a current remote operation situation by the operator.

4. The information processing method according to claim 2, wherein the proficiency level includes a second proficiency level calculated based on past remote operation results by the operator.

5. The information processing method of claim 2, further comprising: acquiring a first evaluation value indicating the difficulty of moving the moving body on a route to a destination of the moving body; and, when the acquired driving mode is the specific mode, determining whether or not to output the switching information based further on the acquired first evaluation value.

6. The information processing method according to claim 2, further comprising: acquiring a second evaluation value indicating a degree of execution of a service performed by the mobile body; and, when the acquired driving mode is the specific mode, determining whether or not to output the switching information based further on the acquired second evaluation value.

7. An information processing method according to any one of claims 1 to 4, wherein, when outputting the switching information including the switching area, it is determined whether or not a change in direction is possible at the current location of the moving body, and if it is determined that the change in direction is possible, the switching information is output with the current location of the moving body as the switching area, and if it is determined that the change in direction is not possible, the switching information is output with a location other than the current location of the moving body as the switching area.

8. A program for causing a computer to execute the information processing method according to any one of claims 1 to 4.

9. An information processing terminal comprising: a display unit; a communication unit that acquires the switching information obtained by the information processing method described in any one of claims 1 to 4; and a terminal-side processing unit that displays the switching information acquired by the communication unit on the display unit when the driving mode of the moving body is the specific mode while the moving body is being remotely operated by the operator.

Images