Remote operator terminal

Abstract

The invention provides a remote operator terminal with a plurality of operating systems. A remote operator terminal used by a remote operator for remote operation of a target moving body is provided with a plurality of operating systems. The remote operator terminal also has a first user interface configured to cause a remote operator to select a first operating system of the plurality of operating systems used in the remote operation.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a remote operator terminal used by a remote operator for remote operation of a moving body. BACKGROUND

[0002] Patent Literature 1 discloses a remote driving device for remote driving of a vehicle. In a case where a state of equipment of the remote driving device is not in agreement with a state of devices of the vehicle, remote driving of the vehicle is prohibited. PRIOR ART DOCUMENT PATENT LITERATURE

[0003] Patent Literature 1: Japanese Patent Application Publication No. 2019-174993 SUMMARY PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] A remote operator terminal used by a remote operator for remote operation of a moving body is considered. In the past, a case where a plurality of operating systems are mounted on a single remote operator terminal has not been discussed. TECHNICAL MEANS FOR SOLVING THE PROBLEMS

[0005] A first aspect relates to a remote operator terminal used by a remote operator for remote operation of a moving body. The remote operator terminal includes: a plurality of operating systems; and a first user interface configured to cause the remote operator to select a first operating system used in the remote operation from among the plurality of operating systems. EFFECT OF THE INVENTION

[0006] According to the present disclosure, a single remote operator terminal includes a plurality of operating systems and a first user interface for causing the remote operator to select a first operating system used from among the plurality of operating systems. Therefore, the remote operator can freely select a preferred first operating system, a first operating system that is good at. In addition, the remote operator can also flexibly select the first operating system according to a kind of the moving body. This contributes to improvement in accuracy of remote operation of the moving body. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 is a conceptual diagram for explaining an outline of a remote operating system. Figure 2 is a conceptual diagram for explaining an example of a plurality of operating systems included in a remote operator terminal. Figure 3 is a block diagram showing a configuration example of a remote operator terminal. Figure 4is a block diagram showing a functional configuration example related to manual selection of a first operation system used in remote operation. Figure 5 is a diagram showing an example of specification information. Figure 6 is a flowchart showing an example of processing by the selection control section of the remote operator terminal. Figure 7 is a flowchart showing another example of processing by the selection control section of the remote operator terminal. Figure 8 is a block diagram showing a functional configuration example related to automatic selection of a first operation system used in remote operation. Figure 9 is a flowchart showing an example of processing by the automatic selection section of the remote operator terminal. Figure 10 is a flowchart showing another example of processing by the automatic selection section of the remote operator terminal. Figure 11 is a block diagram showing a configuration example related to calibration of an operation system. Figure 12 is a conceptual diagram for explaining an example of zero point setting of an operation system. Figure 13 is a conceptual diagram for explaining an example of maximum operation point setting of an operation system. Figure 14 is a block diagram showing a functional configuration example related to end point notification. Figure 15 is a conceptual diagram for explaining a first example of end point notification. Figure 16 is a conceptual diagram for explaining a second example of end point notification. Figure 17 is a conceptual diagram for explaining a third example of end point notification. Figure 18 is a block diagram showing a configuration example of a remote operator terminal. DETAILED DESCRIPTION

[0008] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0009] 1. Outline of Remote Operation System Figure 1 is a conceptual diagram for explaining an outline of a remote operation system 1 related to the present embodiment. The remote operation system 1 is a system for performing remote operation of a mobile body 100. Remote operation is a concept including remote driving. The remote operation system 1 includes the mobile body 100, a remote operator terminal 200, and a management system 300.

[0010] The mobile body 100 is a mobile body that can move. The mobile body 100 can also be manually operated by an operator who rides on the mobile body 100. The mobile body 100 can also have an autonomous moving function. In either case, the mobile body 100 is configured to be able to be remotely operated as needed. That is, the mobile body 100 is an object of remote operation by the remote operation system 1.

[0011] The kind of the mobile body 100 that is an object of remote operation is not limited to one, and can also be plural. For example, the mobile body 100 can also be a vehicle (for example: passenger car, truck, bus, MaaS vehicle, autonomous driving vehicle, and the like) that runs on a public road. As another example, the mobile body 100 can also be a vehicle (for example: forklift, factory cart, and the like) used within a factory. As still another example, the mobile body 100 can also be a special small vehicle (for example: golf course cart, personal mobility, electric wheelchair, and the like). As still another example, the mobile body 100 can also be a construction machine (for example: excavator, bulldozer, and the like). As still another example, the mobile body 100 can also be a robot (for example: logistics robot, work robot, and the like). As still another example, the mobile body 100 can also be a flying body (for example: drone, and the like). As still another example, the mobile body 100 can also be a ship (for example: small ship, large yacht, and the like).

[0012] The remote operator terminal 200 is a terminal device used when the remote operator O remotely operates the mobile body 100. That is, the remote operator terminal 200 is configured to be used by the remote operator O in order to perform remote operation of the mobile body 100. The remote operator terminal 200 can also be referred to as a remote operation HMI (Human Machine Interface) or a remote cockpit. A single remote operator terminal 200 can also be configured to be able to cope with remote operation of various kinds of mobile bodies 100. The combination of the remote operator O and the remote operator terminal 200 can be either predetermined or freely changed. That is, a single remote operator terminal 200 can be used only by a specific remote operator O, or can be used in order by various remote operators O.

[0013] The management system (manager) 300 performs management of the remote operation system 1. The management system 300 can also be constituted by a plurality of servers that perform distributed processing. For example, the management system 300 manages a plurality of remote operators O and a plurality of remote operator terminals 200. In addition, the management system 300 allocates the remote operator O and the remote operator terminal 200 to remote operation of the mobile body 100 in response to a remote operation request. In addition, the management system 300 can also manage the state of the mobile body 100 in remote operation.

[0014] The mobile body 100, the remote operator terminal 200, and the management system 300 can communicate with each other via a communication network. For example, the mobile body 100 can wirelessly communicate with the remote operator terminal 200 and the management system 300 via a wireless communication network. The remote operator terminal 200 and the management system 300 can communicate with each other via a wired communication network or a wireless communication network. The mobile body 100 and the remote operator terminal 200 can communicate via the management system 300 or directly without the management system 300.

[0015] The general flow of information in the process of remote operation of the mobile body 100 is as follows.

[0016] The mobile body 100 is equipped with various sensors including a camera. The camera photographs the situation around the mobile body 100. An image showing the situation around the mobile body 100 is obtained by the camera. The mobile body information MOV is information obtained by the various sensors, and at least contains the image photographed by the camera. The mobile body information MOV can also contain the position and state (e.g., speed, steering angle, etc.) of the mobile body 100. The mobile body 100 transmits the mobile body information MOV to the remote operator terminal 200.

[0017] The remote operator terminal 200 receives the mobile body information MOV transmitted from the mobile body 100. The remote operator terminal 200 presents the mobile body information MOV to the remote operator O. Specifically, the remote operator terminal 200 has a display device, and displays an image or the like on the display device. The remote operator O watches the displayed information, recognizes the situation around the mobile body 100, and performs remote operation of the mobile body 100. The remote operation information OPE is information related to the remote operation (steering operation, acceleration operation, deceleration operation, forward movement operation, lateral movement operation, etc.) performed by the remote operator O. For example, the remote operation information OPE contains the operation amount and operation content input by the remote operator O. The remote operation information OPE can be said to be information reflecting the degree of remote operation performed by the remote operator O. The remote operator terminal 200 transmits the remote operation information OPE to the mobile body 100.

[0018] The mobile body 100 receives the remote operation information OPE transmitted from the remote operator terminal 200. The mobile body 100 performs mobile body control in accordance with the received remote operation information OPE. In this way, remote operation of the mobile body 100 is realized.

[0019] 2. Multiple operation systems The "operation system" possessed by the remote operator terminal 200 will be discussed below. The operation system refers to a component or device used by the remote operator O to input an operation amount and operation content of remote operation of the mobile body 100. The remote operator O can input an operation amount and operation content of remote operation of the mobile body 100 by using the operation system of the remote operator terminal 200. Furthermore, remote operation information OPE corresponding to the operation amount and operation content input through the operation system is generated.

[0020] According to the present embodiment, a single remote operator terminal 200 is provided with a plurality of different operation systems 210-1 to 210-N. Here, N is an integer of two or more. Figure 2 Examples of various operation systems 210 are shown.

[0021] The full-spec operation system includes physical operation components such as a handle, a steering wheel, an accelerator pedal, and a brake pedal. The full-spec operation system can also include operation components such as a direction indicator and a switch. The operation components can be physically moved, and the position (displacement amount) and movement of the operation components are detected by sensors. As the sensors, a steering angle sensor, a steering torque sensor, an accelerator position sensor, a brake position sensor, and the like are exemplified. The position and movement of the operation components detected by the sensors correspond to the operation amount and operation content of remote operation of the mobile body 100.

[0022] The pedal-less operation system does not include an accelerator pedal and a brake pedal, and does not require operation with the feet. Therefore, the pedal-less operation system can be used by the remote operator O who is inconvenient with the feet.

[0023] For example, the pedal-less operation system includes one or more joysticks as operation components. The joystick can also be physically moved, and the position (displacement amount) and movement thereof are detected by sensors. As the sensors, a load sensor is exemplified. The position and movement of the joystick detected by the sensors correspond to the operation amount and operation content of remote operation of the mobile body 100. As an example, a case where the pedal-less operation system includes a left joystick and a right joystick is considered. For example, pulling the left joystick corresponds to "turning left", pushing the left joystick corresponds to "turning right", pulling the right joystick corresponds to "decelerating", and pushing the right joystick corresponds to "accelerating".

[0024] As another example, the pedal-less operation system can include a controller like that of a game machine. The controller is provided with a cross key and a plurality of buttons. The remote operator O can perform remote operation of the mobile body 100 with a sense of a game by using the controller.

[0025] A line-of-sight guided operation system implements remote operation based on movement of a remote operator O's line-of-sight (eyes). To this end, the line-of-sight guided operation system includes a line-of-sight measuring device (line-of-sight camera, eye tracker) that measures the direction of the remote operator O's line-of-sight. The line-of-sight measuring device is a well-known technology. For example, movement of the line-of-sight to the left direction corresponds to a "left turn", movement to the right direction corresponds to a "right turn", movement to the downward direction corresponds to a "deceleration", and movement to the upward direction corresponds to an "acceleration". Such a line-of-sight guided operation system can also be used for a remote operator O who is not physically capable. The line-of-sight guided operation system can also be implemented by a wearable device.

[0026] A portable terminal operation system implements remote operation by operation of a portable terminal. As the portable terminal, a smartphone, a tablet computer is exemplified. The portable terminal operation system can also be used for a remote operator O who is not physically capable.

[0027] For example, tilt (posture) of the portable terminal corresponds to an operation amount, operation content for remote operation. The tilt of the portable terminal is detected by, for example, a gyro sensor, an acceleration sensor mounted on the portable terminal. For example, tilting the portable terminal to the left corresponds to a "left turn", tilting to the right corresponds to a "right turn", tilting forward corresponds to a "deceleration", and tilting to the depth corresponds to an "acceleration".

[0028] As another example, a software operation system can also be displayed on a touch panel of the portable terminal. The software operation system, for example, simulates a controller having a cross key and a plurality of buttons. By touching the software operation system displayed on the touch panel, remote operation can be performed as in the case of the physical operation system.

[0029] Further, the plurality of different operation systems 210-1 to 210-N possessed by the single remote operator terminal 200 are not necessarily physically separated. For example, the operation systems 210-1 to 210-N can also be a plurality of different software operation systems. In this case, the software operation systems are switched and displayed on the same touch panel.

[0030] 3. Manual selection of the operation system by the remote operator 3-1. Summary In the following description, the mobile body 100 that is the object of remote operation will be referred to as "object mobile body 100-T". Hereinafter, the operation system 210 used by the remote operator O in the remote operation of the object mobile body 100-T will be referred to as "first operation system 210-A". As described above, the single remote operator terminal 200 is provided with a plurality of different operation systems 210-1 to 210-N. Therefore, the remote operator O can select a preferred first operation system 210-A or a first operation system 210-A that the remote operator O is good at from among the operation systems 210-1 to 210-N. In addition, the remote operator O can flexibly select the first operation system 210-A according to the kind of the object mobile body 100-T. Even if the object mobile body 100-T is the same, the remote operator O can change the first operation system 210-A according to the mood of the day. This contributes to improving the accuracy of the remote operation of the object mobile body 100-T.

[0031] 3-2. Configuration Example In order to facilitate the manual selection of the first operation system 210-A by the remote operator O, the remote operator terminal 200 can have the following configuration.

[0032] Figure 3 is a block diagram showing a configuration example of the remote operator terminal 200. The remote operator terminal 200 is provided with the operation systems 210-1 to 210-N, a first user interface 231, a second user interface 232, and a control device 250.

[0033] The first user interface 231 is a user interface (UI) for causing the remote operator O to select the first operation system 210-A. More specifically, the first user interface 231 is configured to prompt the remote operator O of one or more selectable operation systems 210-X from among the plurality of operation systems 210-1 to 210-N. The selectable operation system 210-X is an operation system 210 that the remote operator O can select, and is an option of the first operation system 210-A. Further, the first user interface 231 is configured to accept the selection of the first operation system 210-A from among the one or more selectable operation systems 210-X by the remote operator O.

[0034] For example, the first user interface 231 includes a first touch panel. The first touch panel can also be provided independently of the operation systems 210-1 to 210-N. One or more selectable operation systems 210-X are displayed on the first touch panel. The remote operator O touches the desired first operation system 210-A from among the one or more selectable operation systems 210-X displayed on the first touch panel. Thereby, the remote operator O can select the desired first operation system 210-A from among the one or more selectable operation systems 210-X.

[0035] As another example, the first user interface 231 can also include a plurality of selection sections provided in the plurality of operating systems 210-1 to 210-N respectively. For example, each selection section includes a lamp and a button. By causing the lamp provided in the selectable operating system 210-X to emit light, the remote operator O can recognize the selectable operating system 210-X. Then, the remote operator O presses the button provided in the desired first operating system 210-A among one or more selectable operating systems 210-X. Thereby, the remote operator O can select the desired first operating system 210-A from among one or more selectable operating systems 210-X.

[0036] As still another example, the first user interface 231 can also include a sound recognition section. The remote operator O expresses the desired first operating system 210-A by voice. The sound recognition section recognizes the first operating system 210-A by recognizing the voice of the remote operator O. Thereby, the remote operator O can select the desired first operating system 210-A from among one or more selectable operating systems 210-X.

[0037] The second user interface 232 is a user interface (UI) for causing the remote operator O to select the object mobile body 100-T that is the target of remote operation. In more detail, the second user interface 232 is configured to present one or more selectable mobile bodies 100-X to the remote operator O. The selectable mobile body 100-X is a mobile body 100 that the remote operator O can select, and is an option of the object mobile body 100-T. Further, the second user interface 232 is configured to accept selection of the object mobile body 100-T from among one or more selectable mobile bodies 100-X by the remote operator O.

[0038] For example, the second user interface 232 includes a second touch panel. The second touch panel can also be provided independently of the operating systems 210-1 to 210-N. One or more selectable mobile bodies 100-X are displayed on the second touch panel. The remote operator O touches the desired object mobile body 100-T among one or more selectable mobile bodies 100-X displayed on the second touch panel. Thereby, the remote operator O can select the desired object mobile body 100-T from among one or more selectable mobile bodies 100-X.

[0039] As another example, the second user interface 232 can also include a sound recognition section. The remote operator O expresses the desired object mobile body 100-T by voice. The sound recognition section recognizes the object mobile body 100-T by recognizing the voice of the remote operator O. Thereby, the remote operator O can select the desired object mobile body 100-T from among one or more selectable mobile bodies 100-X.

[0040] The first user interface 231 and the second user interface 232 can also be the same interface (e.g., a touch panel).

[0041] The control device 250 controls the remote operator terminal 200. For example, the control device 250 performs various processes in order to cause the remote operator O to select the first operation system 210-A through the first user interface 231. In addition, the control device 250 performs various processes in order to cause the remote operator O to select the object mobile body 100-T through the second user interface 232. Furthermore, the control device 250 can communicate with the object mobile body 100-T via a communication network. In addition, the control device 250 can communicate with the operation systems 210-1 to 210-N via wire or wireless. The control device 250 controls the exchange of data between the first operation system 210-A and the object mobile body 100-T in the course of remote operation of the object mobile body 100-T.

[0042] Figure 4 is a block diagram showing an example of the functional configuration of the control device 250. The control device 250 includes a plurality of operation system interfaces 251-1 to 251-N, a signal processing section 252, and a selection control section 253.

[0043] The operation system interface 251-i (i = 1 to N) is an interface for the operation system 210-i, and is connected to the operation system 210-i through wire or wireless. The operation system interface 251-i receives an operation signal indicating an operation amount, an operation content input by the remote operator O from the operation system 210-i. Then, the operation system interface 251-i outputs the received operation signal to the signal processing section 252. In addition, the operation system interface 251-i can also receive a control signal from the signal processing section 252, and transmit the control signal to the operation system 210-i.

[0044] The signal processing section 252 grasps the first operation system 210-A selected by the remote operator O (the selection of the first operation system 210-A will be described later). The signal processing section 252 receives an operation signal from the first operation system 210-A via the operation system interface 251-A. Then, the signal processing section 252 generates remote operation information OPE based on the received operation signal, and transmits the remote operation information OPE to the object mobile body 100-T.

[0045] The non-selected operation system 210-B is an operation system other than the first operation system 210-A among the operation systems 210-1 to 210-N. The remote operator O can also mistakenly touch the non-selected operation system 210-B during remote operation of the object mobile body 100-T. In this case, an operation signal from the non-selected operation system 210-B can be input to the signal processing section 252. However, the signal processing section 252 invalidates (ignores) the operation signal input from the non-selected operation system 210-B. In other words, the signal processing section 252 selects only the operation signal input from the first operation system 210-A via the operation system interface 251-A, and does not select the operation signal input from the non-selected operation system 210-B. The signal processing section 252 generates remote operation information OPE based on the selected operation signal, and transmits the remote operation information OPE to the object mobile body 100-T. Thus, even if the remote operator O mistakenly touches the non-selected operation system 210-B, it is possible to prevent a malfunction of the object mobile body 100-T.

[0046] In addition, the signal processing section 252 receives mobile body information MOV transmitted from the object mobile body 100-T. The signal processing section 252 outputs the received mobile body information MOV to the display system of the remote operator terminal 200. Alternatively, the signal processing section 252 can output the received mobile body information MOV to the first operation system 210-A via the operation system interface 251-A. However, the signal processing section 252 does not output the received mobile body information MOV to the non-selected operation system 210-B.

[0047] The selection control section 253 performs processing for causing the remote operator O to select the first operation system 210-A, the object mobile body 100-T. The selection control section 253 can also hold specification information SPEC showing the specifications of the respective operation systems 210-1 to 210-N.

[0048] Figure 5 is a diagram showing an example of the specification information SPEC. The specification information SPEC shows, for each operation system 210-i (i = 1 to N), one or more kinds of mobile bodies 100 that can be handled. In other words, the specification information SPEC shows, for each operation system 210-i, one or more kinds of mobile bodies 100 that can be remotely operated by the operation system 210-i. In other words, the specification information SPEC shows a correspondence relationship between the operation system 210-i and one or more kinds of selectable mobile bodies 100-X. As shown in FIG. 3, the specification information SPEC shows, for each operation system 210-i, one or more kinds of mobile bodies 100 that can be handled. In other words, the specification information SPEC shows, for each operation system 210-i, one or more kinds of mobile bodies 100 that can be remotely operated by the operation system 210-i. In other words, the specification information SPEC shows a correspondence relationship between the operation system 210-i and one or more kinds of selectable mobile bodies 100-X. As shown in FIG. 3, the specification information SPEC shows, for each operation system 210-i, one or more kinds of mobile bodies 100 that can be handled. In other words, the specification information SPEC shows, for each operation system 210-i, one or more kinds of mobile bodies 100 that can be remotely operated by the operation system 210-i. In other words, the specification information SPEC shows a correspondence relationship between the operation system 210-i and one or more kinds of selectable mobile bodies 100-X. Figure 5As shown, if the operating system 210-i is changed, the kind of mobile body 100 that can be coped with also changes. For example, the full-spec operating system is suitable for remote operation of various kinds of vehicles, and is also suitable for travel on public roads, long-distance driving. On the other hand, the portable terminal operating system is not necessarily suitable for travel on public roads, long-distance driving. The portable terminal operating system can be used for short-distance low-speed driving of vehicles in limited areas such as parking lots, automatic driving assistance such as shoulder retreat in emergency, remote operation of small vehicles in factories, and the like.

[0049] Figure 6 is a flowchart showing an example of processing performed by the selection control section 253 of the remote operator terminal 200.

[0050] In step Sll, the selection control section 253 prompts the remote operator O of the plurality of operating systems 210-1 to 210-N through the first user interface 231. In this example, the plurality of operating systems 210-1 to 210-N corresponds to the selectable operating system 210-X. The selectable operating system 210-X refers to an operating system 210 that the remote operator O can select, and is an option of the first operating system 210-A. The selection control section 253 causes the remote operator O to select the desired first operating system 210-A from among the operating systems 210-1 to 210-N (the selectable operating system 210-X) through the first user interface 231.

[0051] In step S12, the selection control section 253 accepts the selection of the first operating system 210-A by the remote operator O through the first user interface 231. The selection control section 253 can also explicitly show which first operating system 210-A is selected through the first user interface 231. Then, the selection control section 253 notifies the signal processing section 252 described above of the first operating system 210-A selected by the remote operator O. Thereby, the signal processing section 252 can grasp the first operating system 210-A selected by the remote operator O.

[0052] In step S13 after the selection of the first operating system 210-A, the selection control section 253 identifies one or more mobile bodies 100 that can be coped with by the selected first operating system 210-A, based on the specification information SPEC. In other words, the selection control section 253 identifies one or more mobile bodies 100 that can be subjected to remote operation by the selected first operating system 210-A, based on the specification information SPEC. The identified one or more mobile bodies 100 become one or more selectable mobile bodies 100-X.

[0053] In step S14, the selection control unit 253 presents one or more selectable mobile objects 100-X to the remote operator O via the second user interface 232. The selectable mobile objects 100-X are mobile objects 100 that the remote operator O can select and are options for the target mobile object 100-T. The selection control unit 253 prompts the remote operator O to select the desired target mobile object 100-T from the one or more selectable mobile objects 100-X via the second user interface 232.

[0054] In step S15, the selection control unit 253 accepts the selection of the target moving object 100-T by the remote operator O through the second user interface 232. The selection control unit 253 may also clearly indicate which target moving object 100-T has been selected through the second user interface 232. The selection control unit 253 then notifies the aforementioned signal processing unit 252 of the target moving object 100-T selected by the remote operator O. Thus, the signal processing unit 252 can grasp the target moving object 100-T selected by the remote operator O.

[0055] according to Figure 6 The processing flow shown can efficiently select the target moving object 100 -T suitable for the first operation system 210 -A selected by the remote operator O.

[0056] Figure 7 1 is a flowchart showing another example of the processing performed by the selection control unit 253 of the remote operator terminal 200 .

[0057] In step S21, the selection control unit 253 presents one or more selectable moving objects 100-X to the remote operator O via the second user interface 232. The selection control unit 253 prompts the remote operator O via the second user interface 232 to select a desired target moving object 100-T from among the one or more selectable moving objects 100-X.

[0058] In step S22, the selection control unit 253 accepts the selection of the target moving object 100-T by the remote operator O through the second user interface 232. The selection control unit 253 may also clearly indicate which target moving object 100-T has been selected through the second user interface 232. The selection control unit 253 then notifies the aforementioned signal processing unit 252 of the target moving object 100-T selected by the remote operator O. Thus, the signal processing unit 252 can grasp the target moving object 100-T selected by the remote operator O.

[0059] In step S23 after the selection of the subject mobile body 100-T, the selection control section 253 identifies one or more operation systems 210 capable of coping with the selected subject mobile body 100-T based on the specification information SPEC. In other words, the selection control section 253 identifies one or more operation systems 210 capable of being used in the remote operation of the selected subject mobile body 100-T based on the specification information SPEC. The identified one or more operation systems 210 become one or more selectable operation systems 210-X.

[0060] In step S24, the selection control section 253 prompts the remote operator O of the one or more selectable operation systems 210-X through the first user interface 231. The selection control section 253 causes the remote operator O to select a desired first operation system 210-A from among the one or more selectable operation systems 210-X through the first user interface 231.

[0061] In step S25, the selection control section 253 accepts the selection of the first operation system 210-A by the remote operator O through the first user interface 231. The selection control section 253 can also explicitly show which first operation system 210-A is selected through the first user interface 231. Then, the selection control section 253 notifies the signal processing section 252 described above of the first operation system 210-A selected by the remote operator O. Thereby, the signal processing section 252 can grasp the first operation system 210-A selected by the remote operator O.

[0062] According to Figure 7 the processing flow shown, the first operation system 210-A suitable for the subject mobile body 100-T selected by the remote operator O can be efficiently selected.

[0063] It is also envisaged that the remote operator O desires to change the first operation system 210-A after the start of the remote operation of the subject mobile body 100-T. However, the change of the first operation system 210-A is prohibited during the movement of the subject mobile body 100-T. The change of the first operation system 210-A is allowed when the subject mobile body 100-T is at a stop. For example, during the remote operation of the subject mobile body 100-T, the signal processing section 252 receives mobile body information MOV including speed information from the subject mobile body 100-T. The signal processing section 252 provides the speed information to the selection control section 253. The selection control section 253 prohibits the change of the first operation system 210-A during the movement of the subject mobile body 100-T and allows the change of the first operation system 210-A during the stop of the subject mobile body 100-T.

[0064] In a case where an abnormality occurs in the first operating system 210-A in use, the signal processing section 252 can also transmit a stop instruction to the target mobile body 100-T. Then, the selection control section 253 can also cause the remote operator O to change the first operating system 210-A in use through the first user interface 231.

[0065] 3-3. Effects As explained above, according to the present embodiment, the single remote operator terminal 200 is provided with a plurality of operating systems 210-1 to 210-N. Further, the remote operator terminal 200 is provided with the first user interface 231 for causing the remote operator O to select the first operating system 210-A in use from among the plurality of operating systems 210-1 to 210-N. Therefore, the remote operator O can freely select the preferred first operating system 210-A, the first operating system 210-A that the remote operator O is good at. In addition, the remote operator O can also flexibly select the first operating system 210-A in accordance with the kind of the target mobile body 100-T. This contributes to improvement in the accuracy of remote operation of the target mobile body 100-T.

[0066] 4. Automatic selection of operating system 4-1. Summary The remote operator terminal 200 can also be configured to automatically select the first operating system 210-A from among the plurality of operating systems 210-1 to 210-N. By this, the operation requested of the remote operator O becomes less, and the load imposed on the remote operator O is reduced.

[0067] 4-2. Configuration example The basic configuration of the remote operator terminal 200 is the same as in the case of the above-described Part 3. The description repeated from the above-described Part 3 is appropriately omitted.

[0068] Figure 8 is a block diagram showing a functional configuration example of the control device 250 of the remote operator terminal 200. The control device 250 includes a plurality of operating system interfaces 251-1 to 251-N, a signal processing section 252, and an automatic selection section 254.

[0069] The automatic selection section 254 performs an automatic selection process of automatically selecting the first operating system 210-A from among the plurality of operating systems 210-1 to 210-N. The automatic selection section 254 can also hold the above-described specification information SPEC (refer to Figure 5 ).

[0070] The automatic selection section 254 can also hold priority information PREF as a reference when automatically selecting the first operating system 210-A. For example, the priority information PREF includes history information showing a history of the first operating system 210-A that has been used in the past. As another example, the priority information PREF can also include operator preference information showing a preference of the remote operator O. The operator preference information, for example, shows an operating system 210 that is designated in advance by the remote operator O.

[0071] Figure 9 is a flowchart showing an example of processing by the automatic selection section 254 of the remote operator terminal 200.

[0072] In step S10, the automatic selection section 254 automatically selects the first operating system 210-A from among the plurality of operating systems 210-1 to 210-N based on the priority information PREF. For example, the automatic selection section 254 can automatically select the first operating system 210-A that was used last time as the first operating system 210-A this time based on the history information. As another example, the automatic selection section 254 can automatically select the operating system 210 that has been used the most frequently in the past as the first operating system 210-A this time based on the history information. As still another example, the automatic selection section 254 can automatically select an operating system that conforms to the preference of the remote operator O as the first operating system 210-A this time based on the operator preference information.

[0073] The automatic selection section 254 explicitly shows which first operating system 210-A is automatically selected through the first user interface 231. However, the possibility that the remote operator O does not like the automatically selected first operating system 210-A is also taken into account. Therefore, the automatic selection section 254 can also accept a request for changing the first operating system 210-A from the remote operator O through the first user interface 231. The selection of the first operating system 210-A by the remote operator O is as explained in Section 3 above. In the case where a request for changing the first operating system 210-A from the remote operator O is received through the first user interface 231, the automatic selection section 254 changes the first operating system 210-A in accordance with the request.

[0074] The automatic selection section 254 updates the history information included in the priority information PREF based on the result of the selection of the first operating system 210-A. In addition, the automatic selection section 254 notifies the signal processing section 252 of the selected first operating system 210-A. The signal processing section 252 grasps the selected first operating system 210-A.

[0075] The subsequent steps S13 to S15 are the same as in the case of Section 3 above.

[0076] According to Figure 9The processing flow shown enables efficient selection of the object moving body 100-T suitable for the first operating system 210-A automatically selected.

[0077] Figure 10 Fig. 23 is a flowchart showing another example of processing by the automatic selection section 254 of the remote operator terminal 200. Steps S21 to S23 are the same as in the case of the above-described Part 3.

[0078] In step S26 following step S23, the automatic selection section 254 automatically selects the first operating system 210-A from among one or more selectable operating systems 210-X based on the priority information PREF. For example, the automatic selection section 254 can automatically select the first operating system 210-A used last time as the first operating system 210-A this time based on the history information. As another example, the automatic selection section 254 can automatically select the operating system 210 with the highest frequency of use in the past as the first operating system 210-A this time based on the history information. As yet another example, the automatic selection section 254 can automatically select the operating system 210 that matches the preference of the remote operator O as the first operating system 210-A this time based on the operator preference information.

[0079] The automatic selection section 254 explicitly shows which first operating system 210-A is automatically selected through the first user interface 231. However, the possibility that the remote operator O does not like the first operating system 210-A automatically selected is also taken into account. Therefore, the automatic selection section 254 can also accept a request for change of the first operating system 210-A from the remote operator O through the first user interface 231. The selection of the first operating system 210-A by the remote operator O is as explained in the above-described Part 3. In the case where a request for change of the first operating system 210-A from the remote operator O is received through the first user interface 231, the automatic selection section 254 changes the first operating system 210-A in accordance with the request for change.

[0080] The automatic selection section 254 updates the history information included in the priority information PREF based on the result of selection of the first operating system 210-A. In addition, the automatic selection section 254 notifies the signal processing section 252 of the first operating system 210-A selected. The signal processing section 252 grasps the first operating system 210-A selected.

[0081] According to Figure 10 The processing flow shown enables efficient selection of the first operating system 210-A suitable for the object moving body 100-T selected by the remote operator O.

[0082] It is also assumed that the remote operator O desires to change the first operation system 210-A after the remote operation of the object moving body 100-T is started. However, the change of the first operation system 210-A is prohibited during the movement of the object moving body 100-T. The change of the first operation system 210-A is permitted when the object moving body 100-T is stopped. For example, during the remote operation of the object moving body 100-T, the signal processing section 252 receives the moving body information MOV including speed information from the object moving body 100-T. The signal processing section 252 supplies the speed information to the automatic selection section 254. The automatic selection section 254 does not accept the change request of the first operation system 210-A during the movement of the object moving body 100-T, and accepts the change request of the first operation system 210-A during the stop of the object moving body 100-T.

[0083] In the case where an abnormality occurs in the first operation system 210-A in use, the signal processing section 252 can also transmit the stop instruction to the object moving body 100-T. Then, the automatic selection section 254 can also automatically change the first operation system 210-A.

[0084] 4-3. Effects As explained above, the remote operator terminal 200 is configured to automatically select the first operation system 210-A from among the plurality of operation systems 210-1 to 210-N. Thereby, the operation requested of the remote operator O is less, and the load imposed on the remote operator O is reduced.

[0085] 5. Calibration of operation system 5-1. Summary Depending on the kind of the operation system 210, the zero point (N point) can not be uniquely determined. For example, in the case where the operation system 210 includes a joystick, the zero point of the joystick can not be uniquely determined. Also, since the strength of the arms, legs, the length of the arms, legs, the physique, and the like differ from person to person, the maximum operation point of the operation system 210 can also differ from person to person.

[0086] Therefore, the remote operator terminal 200 can also be configured to enable the desired calibration (customization) of the operation system 210 to be freely performed by the remote operator O. Thereby, the operation system 210 is more handy for the remote operator O to use.

[0087] Further, the timing of the calibration is not particularly limited and is arbitrary. For example, the remote operator O can also perform the calibration of each operation system 210 in advance before a request for remote operation is generated. As another example, the calibration of the first operation system 210-A can also be performed by the remote operator O after the first operation system 210-A is selected for the remote operation of the object moving body 100-T.

[0088] 5-2. Configuration Example Figure 11 is a block diagram showing a configuration example related to the calibration of the operating systems 210. The remote operator terminal 200 is provided with operating systems 210-1 to 210-N, a first user interface 231, a third user interface 233, and a control device 250. The operating systems 210-1 to 210-N and the first user interface 231 are the same as in the cases of the above-described Part 2 and Part 3.

[0089] The third user interface 233 is a user interface (UI) for the calibration of the operating systems 210. The third user interface 233 is configured so as to be able to give various notifications to the remote operator O. For example, the third user interface 233 can also include a display device that displays visual notifications or a touch panel. As another example, the third user interface 233 can also include a speaker that outputs sound notifications. The third user interface 233 can also be the same as the first user interface 231.

[0090] The control device 250 of the remote operator terminal 200 includes operating system interfaces 251-1 to 251-N and a calibration section 255. The operating system interfaces 251-1 to 251-N are the same as in the cases of the above-described Part 2 and Part 3. The calibration section 255 performs a calibration process.

[0091] Hereinafter, the operating system 210 that is an object of the calibration this time among the plurality of operating systems 210-1 to 210-N will be referred to as "object operating system 210-T". The calibration section 255 causes the remote operator O to select the object operating system 210-T through the first user interface 231. In more detail, the calibration section 255 causes the remote operator O to select a desired object operating system 210-T from among the operating systems 210-1 to 210-N through the first user interface 231. Then, the calibration section 255 accepts the selection of the object operating system 210-T by the remote operator O through the first user interface 231. The object operating system 210-T can also be the first operating system 210-A that is selected in order to perform remote operation of the object moving body 100-T (refer to the above-described Part 3 and Part 4). The calibration section 255 can also explicitly show which one is the selected object operating system 210-T through the first user interface 231.

[0092] After the object operating system 210-T is selected, the calibration section 255 performs calibration of the object operating system 210-T.

[0093] First, the calibration unit 255 prompts the remote operator O to perform a predetermined action on the target operating system 210-T. More specifically, the calibration unit 255 presents a notification prompting the remote operator O to perform the predetermined action on the target operating system 210-T via the third user interface 233. In response to the notification, the remote operator O performs the predetermined action on the target operating system 210-T.

[0094] The target operating system 210-T outputs an operation signal corresponding to a predetermined action performed by the remote operator O. The calibration unit 255 receives the operation signal from the target operating system 210-T via the operating system interface 251-T. The operation signal reflects the details of the predetermined action performed by the remote operator O. Furthermore, the operation signal reflects the state of the target operating system 210-T during the predetermined action. Based on the received operation signal, the calibration unit 255 sets at least one of the zero point and the maximum operation point of the target operating system 210-T.

[0095] 5-3. Example of zero point setting 5-3-1.The first example Figure 12 This is a conceptual diagram illustrating a first example of zero-point setting. The target operating system 210-T includes a physically movable operating member 211 (e.g., a joystick). The position of the operating member 211 is detected by a sensor 212. The position of the operating member 211 detected by the sensor 212 is reflected in the amount of operation during remote operation. The operation signal indicates the position of the operating member 211 detected by the sensor 212.

[0096] The first action is a predetermined action that the remote operator O is requested to perform when setting the zero point. Figure 12 In the example shown, the first action is, for example, “maintaining the position of the operating member 211 at a desired zero point for a predetermined time.” The predetermined time is, for example, several seconds.

[0097] The first notification is a notification that prompts the remote operator O to perform a first action on the target operating system 210-T. Figure 12 In the example shown, the first notification prompts the remote operator O to "maintain the position of the operating member 211 at the desired zero point for a predetermined period of time." The calibration unit 255 presents this first notification to the remote operator O via the third user interface 233. The first notification can be either visual or audible.

[0098] In response to the first notification, the remote operator O performs a first action on the target operating system 210-T. The target operating system 210-T outputs an operation signal corresponding to the first action performed by the remote operator O. The operation signal indicates the state of the target operating system 210-T during the first action.Figure 12 In the case of the example shown, the operation signal shows the position of the operation member 211 in the course of the first action. The calibration section 255 sets the zero point of the operation member 211 based on the position of the operation member 211 in the course of the first action. For example, the calibration section 255 sets the average position of the operation member 211 in the course of the first action as the zero point of the operation member 211.

[0099] 5-3-2. Second Example In the second example, the target operation system 210-T is a line-of-sight guidance type operation system. The line-of-sight guidance type operation system includes a line-of-sight measurement device (line-of-sight camera, eye tracker) that measures the line-of-sight direction of the remote operator O. Changes in the line-of-sight direction of the remote operator O are reflected in the operation amount during remote operation. The operation signal includes information on the line-of-sight direction of the remote operator O measured by the line-of-sight measurement device.

[0100] The first action is, for example, "maintain the line-of-sight direction at the desired zero point for a prescribed time". The prescribed time is, for example, several seconds. The first notification prompts the remote operator O to "maintain the line-of-sight direction at the desired zero point for a prescribed time". The calibration section 255 prompts the remote operator O of such a first notification through the third user interface 233. The first notification can be either a visual notification or an audible notification.

[0101] In response to the first notification, the remote operator O maintains the line-of-sight direction at the desired zero point for a prescribed time or more. For example, a plurality of gaze candidate points are displayed on the screen, and the remote operator O gazes at the gaze candidate point corresponding to the desired zero point for a prescribed time or more. The operation signal shows the line-of-sight direction of the remote operator O, that is, the content of the first action. The calibration section 255 sets the zero point of the line-of-sight measurement device based on the line-of-sight direction of the remote operator O in the course of the first action. For example, the calibration section 255 sets the average direction of the line-of-sight direction of the remote operator O in the course of the first action as the zero point of the line-of-sight measurement device.

[0102] 5-3-3. Third Example In the third example, the target operation system 210-T is a portable terminal. As the portable terminal, a smartphone, a tablet computer is exemplified. The inclination (posture) of the portable terminal is reflected in the operation amount during remote operation. The inclination of the portable terminal is detected by, for example, a gyro sensor, an acceleration sensor mounted on the portable terminal. The operation signal shows the inclination of the portable terminal detected by the sensor.

[0103] The first action is, for example, "hold the inclination of the portable terminal at the desired zero point and maintain it for a prescribed time." The prescribed time is, for example, several seconds. The first notification prompts the remote operator O to "hold the inclination of the portable terminal at the desired zero point and maintain it for a prescribed time." The calibration section 255 prompts such a first notification to the remote operator O through the third user interface 233. The first notification can be either a visual notification or an audio notification.

[0104] In response to the first notification, the remote operator O holds the inclination of the portable terminal at the desired zero point and maintains it for a prescribed time or more. The operation signal shows the state of the portable terminal in the course of the first action, that is, the inclination of the portable terminal. The calibration section 255 sets the zero point of the portable terminal based on the inclination of the portable terminal in the course of the first action. For example, the calibration section 255 sets the average inclination of the portable terminal in the course of the first action as the zero point of the portable terminal.

[0105] 5-4. Example of maximum operation point setting 5-4-1. First example Figure 13 is a conceptual diagram for explaining a first example of maximum operation point setting. The target operation system 210-T includes an operation member 211 (for example, a joystick) that can be physically moved. The position of the operation member 211 is detected by a sensor 212. The position of the operation member 211 detected by the sensor 212 is reflected in the operation amount at the time of remote operation. The operation signal shows the position of the operation member 211 detected by the sensor 212.

[0106] The second action is a prescribed action requested of the remote operator O at the time of maximum operation point setting. In the example shown in Figure 13 In the case of the example shown in, the second action is, for example, "move the position of the operation member 211 to the desired maximum operation point."

[0107] The second notification is a notification that prompts the remote operator O to perform the second action on the target operation system 210-T. In the case of the example shown in, the second notification prompts the remote operator O to "move the position of the operation member 211 to the desired maximum operation point." The calibration section 255 prompts such a second notification to the remote operator O through the third user interface 233. The second notification can be either a visual notification or an audio notification. Figure 13

[0108] In response to the second notification, the remote operator O performs the second action on the target operation system 210-T. The target operation system 210-T outputs an operation signal corresponding to the second action performed by the remote operator O. The operation signal shows the state of the target operation system 210-T in the course of the second action. In the case of the example shown in, the operation signal shows the position of the operation member 211 in the course of the second action. Figure 13 ​In the case of the illustrated example, the operation signal shows the position of the operation member 211 during the second operation. The calibration section 255 sets the maximum operation point of the operation member 211 based on the maximum displacement of the operation member 211 during the second operation. That is, the calibration section 255 sets the maximum displacement point of the operation member 211 during the second operation as the maximum operation point.

[0109] Further, the maximum operation point can be set for the case of pulling the operation member 211 and the case of pushing the operation member 211, respectively.

[0110] 5-4-2. Second Example In the second example, the target operation system 210-T is a line-of-sight guidance type operation system. The line-of-sight guidance type operation system includes a line-of-sight measurement device (line-of-sight camera, eye tracker) that measures the line-of-sight direction of the remote operator O. A change in the line-of-sight direction of the remote operator O is reflected in the operation amount during remote operation. The operation signal includes information on the line-of-sight direction of the remote operator O measured by the line-of-sight measurement device.

[0111] The second operation is, for example, "move the line-of-sight direction to the desired maximum operation point". The second notification prompts the remote operator O to "move the line-of-sight direction to the desired maximum operation point". The calibration section 255 prompts the remote operator O of such a second notification through the third user interface 233. The second notification can be a visual notification or an audio notification.

[0112] In response to the second notification, the remote operator O moves the line-of-sight direction to the desired maximum operation point. The operation signal shows the content of the second operation, that is, the line-of-sight direction of the remote operator O. The calibration section 255 sets the maximum operation point of the line-of-sight measurement device based on the maximum displacement of the line-of-sight direction of the remote operator O during the second operation. That is, the calibration section 255 sets the maximum displacement point of the line-of-sight direction of the remote operator O during the second operation as the maximum operation point.

[0113] 5-4-3. Third Example In the third example, the target operation system 210-T is a portable terminal. As the portable terminal, a smartphone, a tablet computer is exemplified. The inclination (posture) of the portable terminal is reflected in the operation amount during remote operation. The inclination of the portable terminal is detected by, for example, a gyro sensor, an acceleration sensor mounted on the portable terminal. The operation signal shows the inclination of the portable terminal detected by the sensor.

[0114] The second action is, for example, "change the tilt of the portable terminal to the desired maximum operating point." The second notification prompts the remote operator O to "change the tilt of the portable terminal to the desired maximum operating point." The calibration unit 255 presents this second notification to the remote operator O via the third user interface 233. The second notification can be either visual or audible.

[0115] In response to the second notification, remote operator O changes the tilt of the portable terminal to the desired maximum operating point. The operation signal indicates the state of the portable terminal during the second operation, that is, the tilt of the portable terminal. The calibration unit 255 sets the maximum operating point of the portable terminal based on the maximum change in the tilt of the portable terminal during the second operation. For example, the calibration unit 255 sets the point of maximum change in the tilt of the portable terminal during the second operation as the maximum operating point of the portable terminal.

[0116] 5-5. Setting information like Figure 11 As shown, the calibration unit 255 stores setting information CONF. Setting information CONF indicates the zero point and maximum operating point of each operating system 210-i (i = 1 to N). Initially, setting information CONF may also indicate default values ​​(initial values) for the zero point and maximum operating point of each operating system 210-i. Through the calibration process described above, the zero point and maximum operating point of the target operating system 210-T in setting information CONF are updated. Calibration can also be performed for all of the multiple operating systems 210-1 to 210-N.

[0117] The calibration unit 255 provides the latest setting information CONF to the signal processing unit 252. The signal processing unit 252 may also generate remote operation information OPE related to the first operating system 210-A based on the zero point and maximum operation point of the first operating system 210-A indicated by the setting information CONF.

[0118] 5-6. Effect As described above, the remote operator terminal 200 is configured so that the remote operator O can perform calibration (customization) of the desired target operating system 210 -T.

[0119] 6. Endpoint Notification 6-1. Overview Depending on the type of first operating system 210-A used for remote operation of the target mobile object 100-T, there may be no mechanical end point (e.g., joystick). In this case, the remote operator O may not be able to recognize that the current operation amount has reached the maximum operation amount of the first operating system 210-A.

[0120] Therefore, the remote operator terminal 200 can also be configured to notify the remote operator O of the fact that the current operation amount input by the remote operator O has reached the maximum operation amount of the first operation system 210-A. Alternatively, the remote operator terminal 200 can also be configured to notify the remote operator O of the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A. This is because, from the relationship between the maximum operation amount and the current operation amount, it is possible to recognize the fact that the current operation amount has reached the maximum operation amount. Since the remote operator O can recognize the fact that the current operation amount has reached the maximum operation amount, it is easy to more accurately remotely operate the object mobile body 100-T. That is, the precision of the remote operation of the object mobile body 100-T is improved.

[0121] 6-2. Configuration Example Figure 14 is a block diagram showing a configuration example related to end point notification. The remote operator terminal 200 is provided with operation systems 210-1 to 210-N, a control device 250, and a notification device 260. The operation systems 210-1 to 210-N are the same as in the case of the above-described Part 2, Part 3. The notification device 260 is configured to be able to notify various information to the remote operator O. Specific examples of the notification device 260 will be described later.

[0122] The control device 250 of the remote operator terminal 200 includes operation system interfaces 251-1 to 251-N, a signal processing section 252, and a notification control section 256. The operation system interfaces 251-1 to 251-N are the same as in the case of the above-described Part 2, Part 3.

[0123] The signal processing section 252 holds the setting information CONF. As described above, the setting information CONF shows the zero point and the maximum operation point of each operation system 210-i (i = 1 to N). During the remote operation of the object mobile body 100-T, the signal processing section 252 receives an operation signal from the first operation system 210-A via the operation system interface 251-A. The operation signal shows the current operation amount and the current operation content of the first operation system 210-A. The signal processing section 252 can generate remote operation information OPE related to the first operation system 210-A based on the operation signal received from the first operation system 210-A and the setting information CONF related to the first operation system 210-A.

[0124] Further, the signal processing section 252 supplies an operation signal and setting information CONF related to the first operation system 210-A to the notification control section 256. The operation signal shows a current operation amount, a current operation content input by the remote operator O. The maximum operation point shown by the setting information CONF corresponds to the maximum operation amount. Further, the signal processing section 252 receives the mobile body information MOV from the object mobile body 100-T, and supplies the mobile body information MOV to the notification control section 256.

[0125] The notification control section 256 notifies various information to the remote operator O by controlling the notification device 260. For example, the notification control section 256 notifies the remote operator O of the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A, based on the operation signal and the setting information CONF related to the first operation system 210-A. As another example, the notification control section 256 can also notify the remote operator O of the case where the current operation amount has reached the maximum operation amount.

[0126] 6-3. Various Examples Hereinafter, various examples of the notification device 260 and the end point notification will be described.

[0127] 6-3-1. First Example Figure 15 is a conceptual diagram for explaining the first example of the end point notification. In the first example, the notification device 260 includes a display device 261 that displays visual information.

[0128] The notification control section 256 controls the display device 261 to display visual information that shows the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A. That is, the display device 261 displays visual information that shows the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A. In the example shown in Figure 15 In the example shown in FIG. 6, the display device 261 displays visual information that shows the relationship between the current driving force and the maximum driving force. The maximum driving force is shown by the first rectangle of a predetermined height. The second rectangle that shows the current driving force is included in the first rectangle. The height of the second rectangle increases as the current driving force becomes larger.

[0129] Thus, the remote operator O can intuitively grasp to what extent the current operation amount is with respect to the maximum operation amount. Further, the remote operator O can also grasp the case where the current operation amount has reached the maximum operation amount.

[0130] The notification control section 256, that is, the display device 261 can also dynamically change at least one of the color, brightness, chroma, and size of the visual information as the current operation amount approaches the maximum operation amount. For example, the color of the visual information (for example, the second rectangle) can gradually change from a blue color to a red color as the current driving force approaches the maximum driving force.

[0131] The notification control section 256, that is, the display device 261 can also dynamically change the display position of the visual information in accordance with the steering operation by the remote operator O. Information related to the steering operation by the remote operator O (steering direction, steering amount) is obtained in accordance with the operation signal from the first operation system 210-A. For example, when straight traveling, the visual information is displayed near the center of the screen of the display device 261. Also, the display position of the visual information moves in a manner consistent with the steering direction. That is, when making a left turn, the visual information is displayed on the left side in the screen of the display device 261. On the other hand, when making a right turn, the visual information is displayed on the right side in the screen of the display device 261. It can also be that the amount of movement of the display position becomes larger as the steering amount becomes larger. Since the line-of-sight direction of the remote operator O is highly likely to coincide with the steering direction, by dynamically changing the display position of the visual information in accordance with the steering operation as described above, the remote operator O is more likely to view the visual information. That is, the remote operator O is more likely to grasp the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A.

[0132] 6-3-2. Second Example Figure 16 is a conceptual diagram for explaining a second example of end-point notification. In the second example, the notification device 260 includes a stimulus generating device 262 that applies force or vibration to the remote operator O. As the stimulus generating device 262, a haptic device, a vibration generating device, a massaging device, or the like is exemplified. For example, the stimulus generating device 262 is embedded in a seat on which the remote operator O sits.

[0133] The notification control section 256 controls the stimulus generating device 262 to generate force or vibration. That is, the stimulus generating device 262 applies force or vibration to the remote operator O. The notification control section 256, that is, the stimulus generating device 262 applies a specific force or a specific vibration to the remote operator O when the current operation amount reaches the maximum operation amount. For example, the notification control section 256, that is, the stimulus generating device 262 applies force or vibration having a characteristic pattern to the remote operator O when the current operation amount reaches the maximum operation amount. Thereby, the remote operator O can recognize that the current operation amount has reached the maximum operation amount.

[0134] The notification control section 256, that is, the stimulation generating device 262 can also increase the magnitude of the force or vibration applied to the remote operator O as the current operation amount approaches the maximum operation amount.

[0135] The notification control section 256, that is, the stimulation generating device 262 can also dynamically change the position at which the force or vibration is generated in accordance with the steering operation by the remote operator O. Information related to the steering operation by the remote operator O (steering direction, steering amount) is obtained from the operation signal from the first operation system 210-A. For example, when going straight, the force or vibration is generated near the center position of the seat. Also, the position at which the force or vibration is generated moves in a manner consistent with the steering direction. That is, when turning left, the force or vibration is generated on the left side of the seat. On the other hand, when turning right, the force or vibration is generated on the right side of the seat. It can also be that the amount of movement of the position at which the force or vibration is generated increases as the steering amount increases.

[0136] The notification control section 256, that is, the stimulation generating device 262 can also dynamically change the position at which the force or vibration is generated in accordance with the speed of the target mobile body 100-T. The speed of the target mobile body 100-T is obtained from the mobile body information MOV. For example, it can also be that the position at which the force or vibration is generated moves forward of the seat as the speed of the target mobile body 100-T increases.

[0137] 6-3-3. Third Example Figure 17 is a conceptual diagram for explaining a third example of end-point notification. In the third example, the notification device 260 includes a speaker 263 that outputs sound. For example, the speaker 263 is provided around the seat on which the remote operator O is seated. A plurality of speakers 263 can also be provided.

[0138] The notification control section 256 controls the speaker 263 to emit sound. The notification control section 256, that is, the speaker 263 outputs a specific sound (for example, "Maximum operation amount reached") when the current operation amount reaches the maximum operation amount. Thereby, the remote operator O can recognize the situation in which the current operation amount has reached the maximum operation amount.

[0139] The notification control section 256, that is, the speaker 263 can also increase the intensity of the sound output as the current operation amount approaches the maximum operation amount.

[0140] The notification control section 256, that is, the speaker 263 can also dynamically change the output position of the sound in accordance with the steering operation by the remote operator O. Information related to the steering operation by the remote operator O (steering direction, steering amount) is obtained in accordance with the operation signal from the first operation system 210-A. For example, when going straight, the sound is output from above the head of the remote operator O. Also, the output position of the sound moves in a manner consistent with the steering direction. That is, when making a left turn, the sound is output from the left side of the remote operator O. On the other hand, when making a right turn, the sound is output from the right side of the remote operator O. It can also be that, as the steering amount becomes larger, the amount of movement of the output position of the sound becomes larger.

[0141] The notification control section 256, that is, the speaker 263 can also dynamically change the output position of the sound in accordance with the speed of the object moving body 100-T. The speed of the object moving body 100-T is obtained in accordance with the moving body information MOV. For example, it can also be that, as the speed of the object moving body 100-T becomes faster, the output position of the sound moves forward.

[0142] 6-4. Effects As explained above, the remote operator terminal 200 is configured to notify the remote operator O of the case where the current operation amount has reached the maximum operation amount of the first operation system 210-A. Alternatively, the remote operator terminal 200 can also be configured to notify the remote operator O of the relationship between the maximum operation amount of the first operation system 210-A and the current operation amount. In either case, the remote operator O is able to recognize the case where the current operation amount has reached the maximum operation amount. Therefore, the remote operator O is able to more accurately remotely operate the object moving body 100-T. That is, the precision of the remote operation of the object moving body 100-T is improved.

[0143] 7. Combinations Two or more of the features explained in Sections 3 to 6 above can also be combined.

[0144] 8. Configuration example of remote operator terminal Figure 18 is a block diagram showing a configuration example of the remote operator terminal 200. The remote operator terminal 200 is provided with a plurality of different operation systems 210-1 to 210-N, a display system 220, various user interfaces 231, 232, 233, a communication device 240, a control device 250, and a notification device 260. The operation systems 210-1 to 210-N, the user interfaces 231, 232, 233, and the notification device 260 are as described above.

[0145] The display system 220 displays various information to the remote operator O who performs remote operation. In other words, the display system 220 prompts various information to the remote operator O by displaying various information. The display system 220 includes a display device (monitor) such as a liquid crystal display, an organic EL display, and a touch panel. The display system 220 (touch panel) can also include the first user interface 231. The display system 220 (touch panel) can also include the second user interface 232. The display system 220 (touch panel) can also include the third user interface 233. The display system 220 can also include the display device 261 of the notification device 260 illustrated in FIG. 6 as the display device. Figure 15

[0146] The communication device 240 communicates with the object mobile body 100-T and the management system 300 via a communication network.

[0147] The control device 250 controls the remote operator terminal 200. The control device 250 can also include one or a plurality of processors 257 (hereinafter, simply referred to as processor 257) and one or a plurality of storage devices 258 (hereinafter, simply referred to as storage device 258). The processor 257 performs various processing. Examples of the processor 257 include a general-purpose processor, a special-purpose processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), an integrated circuit, a conventional circuit, and / or a combination thereof. The processor 257 can also be referred to as a circuitry or a processing circuitry. The circuitry is hardware that is programmed in such a manner as to achieve the functions described or hardware that performs the functions. The storage device 258 stores various information. As the storage device 258, a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like are exemplified.

[0148] The control program PROG is a computer program that is executed by the processor 257. The functions of the control device 250 can also be achieved by cooperation between the processor 257 and the storage device 258 that execute the control program PROG. The control program PROG is stored in the storage device 258. Alternatively, the control program PROG can also be recorded in a computer-readable recording medium. The control program PROG can also be provided via a network.​

[0149] The control device 250 has necessary functions among the functions of the above-described operation system interface 251, the signal processing section 252, the selection control section 253, the automatic selection section 254, the calibration section 255, the notification control section 256, and the like. The specification information SPEC, the priority information PREF, the setting information CONF, and the like are stored in the storage device 258.

[0150] Further, in the remote operation of the object mobile body 100-T, the control device 250 communicates with the object mobile body 100-T via the communication device 240. The control device 250 receives the mobile body information MOV transmitted from the object mobile body 100-T. The control device 250 presents the mobile body information MOV including the image to the remote operator O through the display system 220. The remote operator O can recognize the state of the object mobile body 100-T and the situation around the object mobile body 100-T based on the mobile body information MOV presented through the display system 220.

[0151] The remote operator O inputs an operation amount and an operation content for the remote operation of the object mobile body 100-T by using the first operation system 210-A. The operation amount and the operation content for the remote operation are detected by the sensor provided to the first operation system 210-A. The control device 250 generates remote operation information OPE reflecting the operation amount and the operation content input by the remote operator O. The remote operation information OPE can be said to be information reflecting the degree of the driving operation by the remote operator O. The control device 250 transmits the remote operation information OPE to the object mobile body 100-T via the communication device 240. Explanation of Reference Numerals

[0152] 1 Remote operation system 100 Mobile body 200 Remote operator terminal 210 Operation system 231 First user interface 232 Second user interface 233 Third user interface 250 Control device 260 Notification device 300 Management system

Claims

1. A remote operator terminal used by a remote operator to remotely operate a target moving object, comprising: Multiple operating systems; and A first user interface is configured to enable the remote operator to select a first operating system from among the plurality of operating systems to be used in the remote operation.

2. The remote operator terminal according to claim 1, wherein: The first user interface is configured to: Prompting the remote operator with one or more selectable operating systems from the plurality of operating systems, A selection of the first operating system from among the one or more selectable operating systems is accepted by the remote operator.

3. The remote operator terminal according to claim 2, wherein: further comprising a second user interface configured to allow the remote operator to select the target moving object from among one or more selectable moving objects, After the remote operator selects the first operating system, the second user interface presents the remote operator with one or more movable bodies that can be remotely operated by the selected first operating system as the one or more selectable movable bodies.

4. The remote operator terminal according to claim 2, wherein: further comprising a second user interface configured to allow the remote operator to select the target moving object from among one or more selectable moving objects, After the remote operator selects the target moving object, the first user interface presents one or more operating systems, among the plurality of operating systems, that can be used for the remote operation of the selected target moving object to the remote operator as the one or more selectable operating systems.

5. The remote operator terminal according to any one of claims 1 to 4, wherein: The first user interface prohibits the change of the first operating system during the movement of the target moving object, and allows the change of the first operating system during the stop of the target moving object.

6. The remote operator terminal according to any one of claims 1 to 4, wherein: further comprising a control device capable of communicating with the target moving object via a communication network, During the remote operation of the target moving object, the control device controls the exchange of data between the first operating system and the target moving object.

7. The remote operator terminal according to claim 6, wherein: The non-selected operating system is an operating system other than the first operating system among the plurality of operating systems. During the remote operation of the target moving object, the control device disables input from the non-selected operation system.

Images