Remote operator terminal

Abstract

To provide a remote operator terminal including a plurality of types of operation systems.SOLUTION: A remote operator terminal used by a remote operator for remotely operating a target mobility includes: a plurality of types of operation systems; a user interface; and a control device configured to perform calibration of a target operation system among the plurality of types of operation systems. During calibration, the control device presents, through the user interface, a notification prompting the remote operator to perform an action on the target operation system to the remote operator. The control device thereafter sets one of a zero point and a maximum operation point of the target operation system on the basis of the content of the action or the state of the target operation system during the action.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present disclosure relates to a remote operator terminal used by a remote operator for remote operation of mobility. [Background technology]

[0002] Patent Document 1 discloses a remote driving device for remotely driving a vehicle. If the equipment status of the remote driving device does not match the equipment status of the vehicle, remote driving of the vehicle is prohibited. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-174993 Summary of the Invention [Problem to be solved by the invention]

[0004] Consider the remote operator terminal used by a remote operator to remotely control the target mobility. Cases where multiple types of operation systems are installed in a single remote operator terminal have not been considered in the past. Furthermore, depending on the type of operation system, there is a possibility that the zero point (N point) cannot be uniquely determined. [Means for solving the problem]

[0005] A first aspect relates to a remote operator terminal used by a remote operator for remote control of a target mobility. The remote operator terminal Multiple types of operation systems, A user interface; a control device configured to calibrate a target operation system among a plurality of types of operation systems; Equipped with. In the calibration, the control device presents a notification to the remote operator through the user interface prompting the remote operator to perform an action on the target operation system, and the control device then sets the zero point or maximum operation point of the target operation system based on the content of the action or the state of the target operation system during the action. [Effects of the Invention]

[0006] According to the present disclosure, a single remote operator terminal is equipped with multiple types of operation systems. Furthermore, the remote operator terminal is configured to allow the remote operator to calibrate (customize) the desired target operation system. This improves the usability of the operation system for the remote operator. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a conceptual diagram for explaining an overview of a remote control system. [Figure 2] 10 is a conceptual diagram for explaining an example of a plurality of types of operation systems provided in a remote operator terminal. FIG. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of a remote operator terminal. [Figure 4] 10 is a block diagram showing an example of a functional configuration related to manual selection of a first operation system used for remote operation. FIG. [Figure 5] FIG. 10 is a diagram illustrating an example of specification information. [Figure 6] 10 is a flowchart illustrating an example of processing by a selection control unit of a remote operator terminal. [Figure 7] 10 is a flowchart illustrating another example of processing by the selection control unit of the remote operator terminal. [Figure 8] FIG. 2 is a block diagram showing an example of a functional configuration related to automatic selection of a first operation system to be used for remote operation. [Figure 9] 10 is a flowchart illustrating an example of processing by an automatic selection unit of a remote operator terminal. [Figure 10]10 is a flowchart illustrating another example of processing by the automatic selection unit of the remote operator terminal. [Figure 11] FIG. 2 is a block diagram showing an example of a configuration related to calibration of an operation system. [Figure 12] FIG. 10 is a conceptual diagram for explaining an example of setting a zero point of an operating system. [Figure 13] FIG. 10 is a conceptual diagram for explaining an example of setting a maximum operation point of an operation system. [Figure 14] FIG. 10 is a block diagram illustrating an example of a functional configuration related to an end notification. [Figure 15] FIG. 10 is a conceptual diagram for explaining a first example of an end notification. [Figure 16] FIG. 10 is a conceptual diagram illustrating a second example of an end notification. [Figure 17] FIG. 10 is a conceptual diagram illustrating a third example of an end notification. [Figure 18] FIG. 2 is a block diagram showing an example of the configuration of a remote operator terminal. DETAILED DESCRIPTION OF THE INVENTION

[0008] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0009] 1. Overview of the remote control system 1 is a conceptual diagram for explaining an overview of a remote control system 1 according to this embodiment. The remote control system 1 is a system for remotely controlling a mobility 100. Remote control is a concept that includes remote driving. The remote control system 1 includes the mobility 100, a remote operator terminal 200, and a management system 300.

[0010] Mobility 100 is a mobile body that can move. Mobility 100 may be manually operated by an operator on board mobility 100. Mobility 100 may also have an autonomous movement function. In either case, mobility 100 is configured so that it can be remotely operated as needed. In other words, mobility 100 is the target of remote operation by remote operation system 1.

[0011] The type of mobility 100 to be remotely controlled is not limited to one, and may be multiple. For example, the mobility 100 may be a vehicle that travels on public roads (e.g., a passenger car, a truck, a bus, a MaaS vehicle, an autonomous vehicle, etc.). As another example, the mobility 100 may be a vehicle used in a factory (e.g., a forklift, a factory cart, etc.). As yet another example, the mobility 100 may be a specialized small vehicle (e.g., a golf course cart, a personal mobility vehicle, an electric wheelchair, etc.). As yet another example, the mobility 100 may be construction machinery (e.g., a power shovel, a bulldozer, etc.). As yet another example, the mobility 100 may be a robot (e.g., a logistics robot, a work robot, etc.). As yet another example, the mobility 100 may be an aerial vehicle (e.g., a drone, etc.). As yet another example, the mobility 100 may be a ship (e.g., a small boat, a large cruiser, etc.).

[0012] The remote operator terminal 200 is a terminal device used by the remote operator O when remotely operating the mobility 100. In other words, the remote operator terminal 200 is configured to be used by the remote operator O to remotely operate the mobility 100. The remote operator terminal 200 can also be called a remote operation HMI (Human Machine Interface) or a remote cockpit. A single remote operator terminal 200 may be configured to be capable of remotely operating various types of mobility 100. The combination of the remote operator O and the remote operator terminal 200 may be predetermined or may be freely changeable. In other words, a single remote operator terminal 200 may be used only by a specific remote operator O, or may be used in turn by various remote operators O.

[0013] The management system (manager) 300 manages the remote operation system 1. The management system 300 may be configured with multiple servers that perform distributed processing. For example, the management system 300 manages multiple remote operators O and multiple remote operator terminals 200. In addition, in response to a remote operation request, the management system 300 assigns the remote operators O and the remote operator terminals 200 to remotely operate the mobility 100. The management system 300 may also manage the state of the mobility 100 during remote operation.

[0014] The mobility 100, the remote operator terminal 200, and the management system 300 can communicate with each other via a communication network. For example, the mobility 100 can wirelessly communicate with the remote operator terminal 200 and the management system 300 via a wireless communication network. The remote operator terminal 200 and the management system 300 can communicate with each other via a wired communication network or a wireless communication network. The mobility 100 and the remote operator terminal 200 may communicate with each other via the management system 300, or may communicate with each other directly without going through the management system 300.

[0015] The general flow of information during remote operation of Mobility 100 is as follows:

[0016] The mobility 100 is equipped with various sensors including a camera. The camera captures the situation around the mobility 100. The camera obtains images showing the situation around the mobility 100. The mobility information MOV is information obtained by the various sensors and includes at least images captured by the camera. The mobility information MOV may include the position and status of the mobility 100 (e.g., speed, steering angle, etc.). The mobility 100 transmits the mobility information MOV to the remote operator terminal 200.

[0017] The remote operator terminal 200 receives the mobility information MOV transmitted from the mobility 100. The remote operator terminal 200 presents the mobility information MOV to the remote operator O. Specifically, the remote operator terminal 200 is equipped with a display device and displays images, etc. on the display device. The remote operator O looks at the displayed information, recognizes the situation around the mobility 100, and remotely controls the mobility 100. The remote operation information OPE is information related to remote operation by the remote operator O (steering operation, acceleration operation, deceleration operation, forward / backward movement operation, lateral movement operation, etc.). For example, the remote operation information OPE includes the amount of operation and operation content input by the remote operator O. It can be said that the remote operation information OPE is information reflecting the degree of remote operation by the remote operator O. The remote operator terminal 200 transmits the remote operation information OPE to the mobility 100.

[0018] The mobility 100 receives the remote operation information OPE transmitted from the remote operator terminal 200. The mobility 100 performs mobility control in accordance with the received remote operation information OPE. In this way, the mobility 100 is remotely controlled.

[0019] 2. Multiple types of controls Below, we will consider the "operation system" provided in the remote operator terminal 200. The operation system is a component or device used by the remote operator O to input the operation amount and operation content for remotely operating the mobility 100. By using the operation system of the remote operator terminal 200, the remote operator O can input the operation amount and operation content for remotely operating the mobility 100. Then, remote operation information OPE is generated according to the operation amount and operation content input through the operation system.

[0020] According to this embodiment, a single remote operator terminal 200 is provided with a plurality of different types of operation systems 210-1 to 210-N, where N is an integer equal to or greater than 2. Figure 2 shows examples of various operation systems 210.

[0021] The full-spec operation system includes physical operation members such as a steering wheel, accelerator pedal, and brake pedal. The full-spec operation system may further include operation members such as turn signals and switches. The operation members can be physically moved, and the position (displacement) and movement of the operation members are detected by sensors. Examples of sensors include a steering angle sensor, a steering torque sensor, an accelerator position sensor, and a brake position sensor. The position and movement of the operation members detected by the sensors correspond to the operation amount and operation content for remotely controlling the mobility 100.

[0022] The pedal-less operation system does not include an accelerator pedal or a brake pedal and does not require foot operation, so that even a remote operator O who has difficulty walking can use the pedal-less operation system.

[0023] For example, a pedal-less operation system includes one or more joysticks as operation members. The joysticks can also be physically moved, and their position (displacement) and movement are detected by sensors. An example of a sensor is a load sensor. The position and movement of the joystick detected by the sensor correspond to the operation amount and operation content for remotely controlling the mobility 100. As an example, consider a case where the pedal-less operation system includes a left joystick and a right joystick. For example, pulling the left joystick corresponds to "turn left," pushing the left joystick corresponds to "turn right," pulling the right joystick corresponds to "deceleration," and pushing the right joystick corresponds to "acceleration."

[0024] As another example, the pedal-less operation system may include a controller similar to that used in a game console. The controller may include a cross key and multiple buttons. By using the controller, the remote operator O can remotely control the mobility 100 in a game-like manner.

[0025] A gaze-guided operation system realizes remote operation by the movement of the gaze (eyes) of the remote operator O. To this end, the gaze-guided operation system includes a gaze measurement device (gaze camera, eye tracker) that measures the gaze direction of the remote operator O. Gaze measurement devices are publicly known technology. For example, gaze movement to the left corresponds to "turn left," movement to the right corresponds to "turn right," movement downward corresponds to "deceleration," and movement upward corresponds to "acceleration." Such a gaze-guided operation system can be used even by a remote operator O who has disabilities in their legs or hands. The gaze-guided operation system may be realized by a wearable device.

[0026] The mobile terminal operation system realizes remote operation by operating a mobile terminal. Examples of the mobile terminal include a smartphone and a tablet. The mobile terminal operation system can be used even by a remote operator O who has difficulty walking.

[0027] For example, the tilt (posture) of the mobile device is associated with the amount of operation and operation content for remote control. The tilt of the mobile device is detected, for example, by a gyro sensor or acceleration sensor mounted on the mobile device. For example, tilting the mobile device to the left is associated with "left turn," tilting it to the right is associated with "right turn," tilting it toward you is associated with "deceleration," and tilting it away from you is associated with "acceleration."

[0028] As another example, a software operation system may be displayed on the touch panel of a mobile terminal. The software operation system may be, for example, modeled after a controller having a cross key and multiple buttons. By touching the software operation system displayed on the touch panel, remote control can be performed in the same way as with a physical operation system.

[0029] The different types of operation systems 210-1 to 210-N provided in a single remote operator terminal 200 do not necessarily have to be physically separate. For example, the operation systems 210-1 to 210-N may be different types of software operation systems. In this case, the software operation systems are displayed on the same touch panel in a switchable manner.

[0030] 3.Manual selection of operating system by remote operator Overview In the following description, the mobility 100 that is the target of remote operation is referred to as the "target mobility 100-T." The operation system 210 that the remote operator O uses to remotely operate the target mobility 100-T is hereinafter referred to as the "first operation system 210-A." As described above, a single remote operator terminal 200 is equipped with multiple different types of operation systems 210-1 to 210-N. Therefore, the remote operator O can select a preferred first operation system 210-A or a first operation system 210-A that the remote operator O is good at from the operation systems 210-1 to 210-N. Furthermore, the remote operator O may flexibly select the first operation system 210-A depending on the type of the target mobility 100-T. Even if the target mobility 100-T is the same, the remote operator O may change the first operation system 210-A depending on his / her mood that day. These contribute to improving the accuracy of the remote operation of the target mobility 100-T.

[0031] 3-2.Configuration example To facilitate the remote operator O's manual selection of the first operation system 210-A, the remote operator terminal 200 may have the following configuration.

[0032] 3 is a block diagram showing an example of the configuration of the remote operator terminal 200. The remote operator terminal 200 comprises operation systems 210-1 to 210-N, a first user interface 231, a second user interface 232, and a control device 250.

[0033] The first user interface 231 is a user interface (UI) for allowing the remote operator O to select the first operation system 210-A. More specifically, the first user interface 231 is configured to present one or more selectable operation systems 210-X from among the multiple types of operation systems 210-1 to 210-N to the remote operator O. The selectable operation system 210-X is an operation system 210 that can be selected by the remote operator O, and is an option for the first operation system 210-A. Furthermore, the first user interface 231 is configured to accept the selection of the first operation system 210-A by the remote operator O from the one or more selectable operation systems 210-X.

[0034] For example, the first user interface 231 includes a first touch panel. The first touch panel may be provided separately from the operation systems 210-1 to 210-N. One or more selectable operation systems 210-X are displayed on the first touch panel. The remote operator O touches a desired first operation system 210-A from the one or more selectable operation systems 210-X displayed on the first touch panel. This allows the remote operator O to select the desired first operation system 210-A from the one or more selectable operation systems 210-X.

[0035] As another example, the first user interface 231 may include a plurality of selection units provided for each of the plurality of types of operation systems 210-1 to 210-N. For example, each selection unit includes a lamp and a button. The remote operator O can recognize the selectable operation system 210-X when the lamp provided for the selectable operation system 210-X lights up. Then, the remote operator O presses the button provided for the desired first operation system 210-A among the one or more selectable operation systems 210-X. In this way, the remote operator O can select the desired first operation system 210-A from the one or more selectable operation systems 210-X.

[0036] As yet another example, the first user interface 231 may include a voice recognition unit. The remote operator O vocally utters the desired first operation system 210-A. The voice recognition unit recognizes the first operation system 210-A by recognizing the speech of the remote operator O. This allows the remote operator O to select the desired first operation system 210-A from one or more selectable operation systems 210-X.

[0037] The second user interface 232 is a user interface (UI) that allows the remote operator O to select a target mobility 100-T that is the target of remote operation. More specifically, the second user interface 232 is configured to present one or more selectable mobility 100-X to the remote operator O. The selectable mobility 100-X is a mobility 100 that the remote operator O can select, and is an option for the target mobility 100-T. Furthermore, the second user interface 232 is configured to accept the remote operator O's selection of the target mobility 100-T from among the one or more selectable mobility 100-X.

[0038] For example, the second user interface 232 includes a second touch panel. The second touch panel may be provided separately from the operation systems 210-1 to 210-N. One or more selectable mobility 100-X are displayed on the second touch panel. The remote operator O touches a desired target mobility 100-T from the one or more selectable mobility 100-X displayed on the second touch panel. This allows the remote operator O to select the desired target mobility 100-T from the one or more selectable mobility 100-X.

[0039] As another example, the second user interface 232 may include a voice recognition unit. The remote operator O vocally utters the desired target mobility 100-T. The voice recognition unit recognizes the target mobility 100-T by recognizing the speech of the remote operator O. This allows the remote operator O to select the desired target mobility 100-T from one or more selectable mobilities 100-X.

[0040] The first user interface 231 and the second user interface 232 may be the same interface (for example, a touch panel).

[0041] The control device 250 controls the remote operator terminal 200. For example, the control device 250 executes various processes to allow the remote operator O to select the first operation system 210-A through the first user interface 231. The control device 250 also executes various processes to allow the remote operator O to select the target mobility 100-T through the second user interface 232. Furthermore, the control device 250 is capable of communicating with the target mobility 100-T via a communication network. The control device 250 is also capable of communicating with the operation systems 210-1 to 210-N via wired or wireless communication. The control device 250 controls the exchange of data between the first operation system 210-A and the target mobility 100-T during remote operation of the target mobility 100-T.

[0042] 4 is a block diagram showing an example of the functional configuration of the control device 250. The control device 250 includes a plurality of operation system interfaces 251-1 to 251-N, a signal processing unit 252, and a selection control unit 253.

[0043] The operation system interface 251-i (i = 1 to N) is an interface for the operation system 210-i, and is connected to the operation system 210-i by wire or wirelessly. The operation system interface 251-i receives an operation signal indicating the amount of operation or the operation content input by the remote operator O from the operation system 210-i. Then, the operation system interface 251-i outputs the received operation signal to the signal processing unit 252. The operation system interface 251-i may also receive a control signal from the signal processing unit 252 and transmit the control signal to the operation system 210-i.

[0044] The signal processing unit 252 grasps the first operation system 210-A selected by the remote operator O (the selection of the first operation system 210-A will be described later). The signal processing unit 252 receives an operation signal from the first operation system 210-A via the operation system interface 251-A. Then, the signal processing unit 252 generates remote operation information OPE based on the received operation signal, and transmits the remote operation information OPE to the target mobility 100-T.

[0045] The unselected operation system 210-B is one of the operation systems 210-1 to 210-N other than the first operation system 210-A. While remotely operating the target mobility 100-T, the remote operator O may accidentally touch the unselected operation system 210-B. In that case, there is a risk that an operation signal from the unselected operation system 210-B will be input to the signal processing unit 252. However, the signal processing unit 252 invalidates (ignores) the operation signal input from the unselected operation system 210-B. In other words, the signal processing unit 252 selects only the operation signal input from the first operation system 210-A via the operation system interface 251-A, and does not select the operation signal input from the unselected operation system 210-B. The signal processing unit 252 generates remote operation information OPE based on the selected operation signal and transmits the remote operation information OPE to the target mobility 100-T. This makes it possible to prevent malfunction of the target mobility 100-T even if the remote operator O accidentally touches the non-selected operation system 210-B.

[0046] The signal processing unit 252 also receives mobility information MOV transmitted from the target mobility 100-T. The signal processing unit 252 outputs the received mobility information MOV to the display system of the remote operator terminal 200. Alternatively, the signal processing unit 252 may output the received mobility information MOV to the first operation system 210-A via the operation system interface 251-A. However, the signal processing unit 252 does not output the received mobility information MOV to the non-selected operation system 210-B.

[0047] The selection control unit 253 executes a process for allowing the remote operator O to select the first operation system 210-A and the target mobility 100-T. The selection control unit 253 may hold specification information SPEC indicating the specifications of each of the operation systems 210-1 to 210-N.

[0048] FIG. 5 is a diagram showing an example of the specification information SPEC. The specification information SPEC indicates one or more types of mobility 100 that can be supported for each operation system 210-i (i = 1 to N). In other words, the specification information SPEC indicates, for each operation system 210-i, one or more types of mobility 100 that can be remotely controlled by that operation system 210-i. In other words, the specification information SPEC indicates the correspondence between the operation system 210-i and one or more types of selectable mobility 100-X. As shown in FIG. 5, the type of compatible mobility 100 changes depending on the operation system 210-i. For example, a full-spec operation system is suitable for remotely controlling various types of vehicles and is also suitable for driving on public roads and long-distance driving. On the other hand, a mobile terminal operation system is not necessarily suitable for driving on public roads or long-distance driving. A mobile terminal operation system can be used for short-distance, low-speed driving of a vehicle in a limited area such as a parking lot, autonomous driving assistance such as pulling over to the shoulder in an emergency, remote control of a small vehicle in a factory, etc.

[0049] FIG. 6 is a flowchart showing an example of processing by the selection control unit 253 of the remote operator terminal 200.

[0050] In step S11, the selection control unit 253 presents a plurality of types of operation systems 210-1 to 210-N to the remote operator O through the first user interface 231. In this example, the plurality of types of operation systems 210-1 to 210-N correspond to the selectable operation systems 210-X. The selectable operation systems 210-X are operation systems 210 that the remote operator O can select, and are options for the first operation system 210-A. The selection control unit 253 prompts the remote operator O through the first user interface 231 to select a desired first operation system 210-A from the operation systems 210-1 to 210-N (selectable operation systems 210-X).

[0051] In step S12, the selection control unit 253 accepts the selection of the first operation system 210-A by the remote operator O through the first user interface 231. The selection control unit 253 may clearly indicate which first operation system 210-A has been selected through the first user interface 231. Then, the selection control unit 253 notifies the signal processing unit 252 of the first operation system 210-A selected by the remote operator O. This allows the signal processing unit 252 to know which first operation system 210-A has been selected by the remote operator O.

[0052] In step S13 after the first operation system 210-A is selected, the selection control unit 253 recognizes one or more mobility entities 100 that can be handled by the selected first operation system 210-A based on the specification information SPEC. In other words, the selection control unit 253 recognizes one or more mobility entities 100 that can be remotely operated by the selected first operation system 210-A based on the specification information SPEC. The recognized one or more mobility entities 100 become one or more selectable mobility entities 100-X.

[0053] In step S14, the selection control unit 253 presents one or more selectable mobility 100-X to the remote operator O through the second user interface 232. The selectable mobility 100-X is a mobility 100 that the remote operator O can select, and is an option for the target mobility 100-T. The selection control unit 253 prompts the remote operator O through the second user interface 232 to select a desired target mobility 100-T from among the one or more selectable mobility 100-X.

[0054] In step S15, the selection control unit 253 accepts the selection of the target mobility 100-T by the remote operator O through the second user interface 232. The selection control unit 253 may clearly indicate which target mobility 100-T has been selected through the second user interface 232. Then, the selection control unit 253 notifies the signal processing unit 252 of the target mobility 100-T selected by the remote operator O. This allows the signal processing unit 252 to know the target mobility 100-T selected by the remote operator O.

[0055] According to the processing flow shown in FIG. 6, it is possible to efficiently select the target mobility 100-T that is suitable for the first operation system 210-A selected by the remote operator O.

[0056] FIG. 7 is a flowchart showing another example of the processing by the selection control unit 253 of the remote operator terminal 200.

[0057] In step S21, the selection control unit 253 presents one or more selectable mobilities 100-X to the remote operator O through the second user interface 232. The selection control unit 253 prompts the remote operator O through the second user interface 232 to select a desired target mobility 100-T from among the one or more selectable mobilities 100-X.

[0058] In step S22, the selection control unit 253 accepts the selection of the target mobility 100-T by the remote operator O through the second user interface 232. The selection control unit 253 may clearly indicate which target mobility 100-T has been selected through the second user interface 232. Then, the selection control unit 253 notifies the signal processing unit 252 of the target mobility 100-T selected by the remote operator O. This allows the signal processing unit 252 to know the target mobility 100-T selected by the remote operator O.

[0059] In step S23 after the target mobility 100-T is selected, the selection control unit 253 recognizes one or more operation systems 210 compatible with the selected target mobility 100-T based on the specification information SPEC. In other words, the selection control unit 253 recognizes one or more operation systems 210 that can be used to remotely control the selected target mobility 100-T based on the specification information SPEC. The recognized one or more operation systems 210 become one or more selectable operation systems 210-X.

[0060] In step S24, the selection control unit 253 presents one or more selectable operation systems 210-X to the remote operator O through the first user interface 231. The selection control unit 253 prompts the remote operator O through the first user interface 231 to select a desired first operation system 210-A from the one or more selectable operation systems 210-X.

[0061] In step S25, the selection control unit 253 accepts the selection of the first operation system 210-A by the remote operator O through the first user interface 231. The selection control unit 253 may clearly indicate which first operation system 210-A has been selected through the first user interface 231. Then, the selection control unit 253 notifies the signal processing unit 252 of the first operation system 210-A selected by the remote operator O. This allows the signal processing unit 252 to know which first operation system 210-A has been selected by the remote operator O.

[0062] According to the processing flow shown in FIG. 7, it is possible to efficiently select the first operation system 210-A suitable for the target mobility 100-T selected by the remote operator O.

[0063] It is also assumed that the remote operator O may wish to change the first operation system 210-A after remote operation of the target mobility 100-T has begun. However, changing the first operation system 210-A is prohibited while the target mobility 100-T is moving. Changing the first operation system 210-A is permitted when the target mobility 100-T is stopped. For example, while remotely operating the target mobility 100-T, the signal processing unit 252 receives mobility information MOV including speed information from the target mobility 100-T. The signal processing unit 252 provides the speed information to the selection control unit 253. The selection control unit 253 prohibits changing the first operation system 210-A while the target mobility 100-T is moving, and permits changing the first operation system 210-A when the target mobility 100-T is stopped.

[0064] If an abnormality occurs in the first operation system 210-A that is in use, the signal processing unit 252 may transmit a stop instruction to the target mobility 100-T. Then, the selection control unit 253 may prompt the remote operator O to change the first operation system 210-A that is being used through the first user interface 231.

[0065] Effects As described above, according to this embodiment, a single remote operator terminal 200 is provided with a plurality of types of operation systems 210-1 to 210-N. Furthermore, the remote operator terminal 200 is provided with a first user interface 231 that allows the remote operator O to select a first operation system 210-A to use from the plurality of types of operation systems 210-1 to 210-N. Therefore, the remote operator O can freely select a preferred first operation system 210-A or a first operation system 210-A that he or she is good at. Furthermore, the remote operator O may flexibly select a first operation system 210-A depending on the type of the target mobility 100-T. These contribute to improving the accuracy of remote operation of the target mobility 100-T.

[0066] 4. Automatic selection of control system Overview The remote operator terminal 200 may be configured to automatically select the first operation system 210-A from among a plurality of types of operation systems 210-1 to 210-N. This reduces the number of operations required of the remote operator O, and reduces the burden on the remote operator O.

[0067] 4-2.Configuration example The basic configuration of the remote operator terminal 200 is the same as that described above in Section 3. Explanations that overlap with those in Section 3 above will be omitted as appropriate.

[0068] 8 is a block diagram showing an example of the functional configuration of the control device 250 of the remote operator terminal 200. The control device 250 includes a plurality of operation system interfaces 251-1 to 251-N, a signal processing unit 252, and an automatic selection unit 254.

[0069] The automatic selection unit 254 executes an automatic selection process to automatically select the first operation system 210-A from among the multiple types of operation systems 210-1 to 210-N. The automatic selection unit 254 may hold the above-mentioned specification information SPEC (see FIG. 5).

[0070] The automatic selection unit 254 may hold priority information PREF that serves as a criterion for automatically selecting the first operation system 210-A. For example, the priority information PREF includes history information that indicates a history of first operation systems 210-A that have been used in the past. As another example, the priority information PREF may include operator preference information that indicates the preferences of the remote operator O. The operator preference information indicates, for example, an operation system 210 that has been designated in advance by the remote operator O.

[0071] FIG. 9 is a flowchart showing an example of processing by the automatic selection unit 254 of the remote operator terminal 200.

[0072] In step S10, the automatic selection unit 254 automatically selects the first operation system 210-A from among the multiple types of operation systems 210-1 to 210-N based on the priority information PREF. For example, the automatic selection unit 254 may automatically select the first operation system 210-A that was used last time as the current first operation system 210-A based on history information. As another example, the automatic selection unit 254 may automatically select the operation system 210 that has been used most frequently in the past as the current first operation system 210-A based on history information. As yet another example, the automatic selection unit 254 may automatically select an operation system that matches the preferences of the remote operator O as the current first operation system 210-A based on operator preference information.

[0073] The automatic selection unit 254 clearly indicates which first operation system 210-A has been automatically selected through the first user interface 231. However, it is possible that the remote operator O may not like the automatically selected first operation system 210-A. Therefore, the automatic selection unit 254 may receive a change request for the first operation system 210-A from the remote operator O through the first user interface 231. The selection of the first operation system 210-A by the remote operator O is as described in Section 3 above. When a change request for the first operation system 210-A from the remote operator O is received through the first user interface 231, the automatic selection unit 254 changes the first operation system 210-A in accordance with the change request.

[0074] The automatic selection unit 254 updates the history information included in the priority information PREF based on the selection result of the first operation system 210-A. Furthermore, the automatic selection unit 254 notifies the signal processing unit 252 of the selected first operation system 210-A. The signal processing unit 252 grasps the selected first operation system 210-A.

[0075] The subsequent steps S13 to S15 are the same as those in Section 3 above.

[0076] According to the processing flow shown in FIG. 9, it is possible to efficiently select the target mobility 100-T that is suitable for the automatically selected first operation system 210-A.

[0077] 10 is a flowchart showing another example of the processing by the automatic selection unit 254 of the remote operator terminal 200. Steps S21 to S23 are the same as those in Section 3 above.

[0078] In step S26 following step S23, the automatic selection unit 254 automatically selects a first operation system 210-A from one or more selectable operation systems 210-X based on the priority information PREF. For example, the automatic selection unit 254 may automatically select the first operation system 210-A that was used last time as the current first operation system 210-A based on history information. As another example, the automatic selection unit 254 may automatically select the operation system 210 that has been used most frequently in the past as the current first operation system 210-A based on history information. As yet another example, the automatic selection unit 254 may automatically select an operation system that matches the preferences of the remote operator O as the current first operation system 210-A based on operator preference information.

[0079] The automatic selection unit 254 clearly indicates which first operation system 210-A has been automatically selected through the first user interface 231. However, it is possible that the remote operator O may not like the automatically selected first operation system 210-A. Therefore, the automatic selection unit 254 may receive a change request for the first operation system 210-A from the remote operator O through the first user interface 231. The selection of the first operation system 210-A by the remote operator O is as described in Section 3 above. When a change request for the first operation system 210-A from the remote operator O is received through the first user interface 231, the automatic selection unit 254 changes the first operation system 210-A in accordance with the change request.

[0080] The automatic selection unit 254 updates the history information included in the priority information PREF based on the selection result of the first operation system 210-A. Furthermore, the automatic selection unit 254 notifies the signal processing unit 252 of the selected first operation system 210-A. The signal processing unit 252 grasps the selected first operation system 210-A.

[0081] According to the processing flow shown in FIG. 10, it is possible to efficiently select the first operation system 210-A suitable for the target mobility 100-T selected by the remote operator O.

[0082] It is also assumed that the remote operator O may wish to change the first operation system 210-A after remote operation of the target mobility 100-T has begun. However, changing the first operation system 210-A is prohibited while the target mobility 100-T is moving. Changing the first operation system 210-A is permitted when the target mobility 100-T is stopped. For example, while remotely operating the target mobility 100-T, the signal processing unit 252 receives mobility information MOV including speed information from the target mobility 100-T. The signal processing unit 252 provides the speed information to the automatic selection unit 254. The automatic selection unit 254 does not accept a request to change the first operation system 210-A while the target mobility 100-T is moving, but accepts a request to change the first operation system 210-A while the target mobility 100-T is stopped.

[0083] If an abnormality occurs in the first operation system 210-A that is in use, the signal processing unit 252 may transmit a stop instruction to the target mobility 100-T. Then, the automatic selection unit 254 may automatically change the first operation system 210-A.

[0084] Effects As described above, the remote operator terminal 200 is configured to automatically select the first operation system 210-A from among the multiple types of operation systems 210-1 to 210-N. This reduces the number of operations required of the remote operator O, and reduces the burden on the remote operator O.

[0085] 5. Calibration of the control system Overview The zero point (N point) may not be uniquely determined depending on the type of operation system 210. For example, if the operation system 210 includes a joystick, the zero point of the joystick may not be uniquely determined. Furthermore, since arm and leg strength, arm and leg length, physique, etc. vary from person to person, the maximum operation point of the operation system 210 may also vary from person to person.

[0086] Therefore, the remote operator terminal 200 may be configured so that the remote operator O can freely perform a desired calibration (customization) of the operation system 210. This makes the operation system 210 easier for the remote operator O to use.

[0087] The timing of the calibration is not particularly limited and is arbitrary. For example, the remote operator O may calibrate each operation system 210 in advance before a request for remote operation is made. As another example, after the first operation system 210-A is selected for remote operation of the target mobility 100-T, the remote operator O may calibrate the first operation system 210-A.

[0088] 5-2.Configuration example 11 is a block diagram showing an example of a configuration related to the calibration of operation system 210. Remote operator terminal 200 includes operation systems 210-1 to 210-N, a first user interface 231, a third user interface 233, and a control device 250. Operation systems 210-1 to 210-N and first user interface 231 are similar to those in Section 2 and Section 3 described above.

[0089] The third user interface 233 is a user interface (UI) for calibrating the operation system 210. The third user interface 233 is configured to be able to present various notifications to the remote operator O. For example, the third user interface 233 may include a display device or a touch panel that displays visual notifications. As another example, the third user interface 233 may include a speaker that outputs audio notifications. The third user interface 233 may be the same as the first user interface 231.

[0090] The control device 250 of the remote operator terminal 200 includes operation system interfaces 251-1 to 251-N and a calibration unit 255. The operation system interfaces 251-1 to 251-N are the same as those in the above-mentioned section 2 and section 3. The calibration unit 255 performs calibration processing.

[0091] The operation system 210 to be calibrated this time among the multiple types of operation systems 210-1 to 210-N will be referred to as the "target operation system 210-T" hereinafter. The calibration unit 255 prompts the remote operator O to select the target operation system 210-T through the first user interface 231. More specifically, the calibration unit 255 prompts the remote operator O to select a desired target operation system 210-T from the operation systems 210-1 to 210-N through the first user interface 231. The calibration unit 255 then accepts the selection of the target operation system 210-T by the remote operator O through the first user interface 231. The target operation system 210-T may be the first operation system 210-A selected for remote operation of the target mobility 100-T (see Sections 3 and 4 above). The calibration unit 255 may clearly indicate the selected target operation system 210-T through the first user interface 231.

[0092] After the target operation system 210-T is selected, the calibration unit 255 calibrates the target operation system 210-T.

[0093] First, the calibration unit 255 prompts the remote operator O to perform a predetermined action on the target operation system 210-T. More specifically, the calibration unit 255 presents a notification prompting the remote operator O to perform a predetermined action on the target operation system 210-T through the third user interface 233. In response to the notification, the remote operator O performs the predetermined action on the target operation system 210-T.

[0094] The target operation system 210-T outputs an operation signal corresponding to a predetermined action performed by the remote operator O. The calibration unit 255 receives the operation signal from the target operation system 210-T through the operation system interface 251-T. The operation signal reflects the content of the predetermined action performed by the remote operator O. The operation signal also reflects the state of the target operation system 210-T during the predetermined action. The calibration unit 255 sets at least one of the zero point and the maximum operation point of the target operation system 210-T based on the received operation signal.

[0095] 5-3. Example of zero point setting 5-3-1. First example 12 is a conceptual diagram for explaining a first example of zero point setting. The target operating system 210-T includes an operating member 211 (e.g., a joystick) that can be physically moved. The position of the operating member 211 is detected by a sensor 212. The position of the operating member 211 detected by the sensor 212 is reflected in the amount of operation during remote control. The operation signal indicates the position of the operating member 211 detected by the sensor 212.

[0096] The first action is a predetermined action required of the remote operator O when setting the zero point. In the example shown in Fig. 12, the first action is, for example, "maintaining the position of the operating member 211 at the desired zero point for a predetermined time." The predetermined time is, for example, several seconds.

[0097] The first notification is a notification that prompts the remote operator O to perform a first action on the target operation system 210-T. In the example shown in FIG. 12, the first notification prompts the remote operator O to "keep the position of the operation member 211 at the desired zero point for a predetermined period of time." The calibration unit 255 presents such a first notification to the remote operator O through the third user interface 233. The first notification may be a visual notification or an audio notification.

[0098] In response to the first notification, the remote operator O performs a first action on the target operating system 210-T. The target operating system 210-T outputs an operation signal corresponding to the first action performed by the remote operator O. The operation signal indicates the state of the target operating system 210-T during the first action. In the example shown in FIG. 12, the operation signal indicates the position of the operating member 211 during the first action. The calibration unit 255 sets a zero point of the operating member 211 based on the position of the operating member 211 during the first action. For example, the calibration unit 255 sets the average position of the operating member 211 during the first action as the zero point of the operating member 211.

[0099] 5-3-2. Second example In the second example, the target operation system 210-T is a gaze-guided operation system. The gaze-guided operation system includes a gaze measurement device (gaze camera, eye tracker) that measures the gaze direction of the remote operator O. Changes in the gaze direction of the remote operator O are reflected in the amount of operation during remote operation. The operation signal includes information on the gaze direction of the remote operator O measured by the gaze measurement device.

[0100] The first action is, for example, "maintaining the line of sight direction at the desired zero point for a predetermined time." The predetermined time is, for example, several seconds. The first notification prompts the remote operator O to "maintain the line of sight direction at the desired zero point for a predetermined time." The calibration unit 255 presents such a first notification to the remote operator O through the third user interface 233. The first notification may be a visual notification or an audio notification.

[0101] In response to the first notification, the remote operator O maintains the gaze direction at the desired zero point for a predetermined time or more. For example, a plurality of gaze candidate points are displayed on the screen, and the remote operator O gazes at the gaze candidate point corresponding to the desired zero point for a predetermined time or more. The operation signal indicates the content of the first action, i.e., the gaze direction of the remote operator O. The calibration unit 255 sets the zero point of the gaze measurement device based on the gaze direction of the remote operator O during the first action. For example, the calibration unit 255 sets the average direction of the gaze directions of the remote operator O during the first action as the zero point of the gaze measurement device.

[0102] 5-3-3.Third example In a third example, the target operating system 210-T is a mobile terminal. Examples of mobile terminals include a smartphone and a tablet. The tilt (posture) of the mobile terminal is reflected in the amount of operation during remote control. The tilt of the mobile terminal is detected, for example, by a gyro sensor or acceleration sensor mounted on the mobile terminal. The operation signal indicates the tilt of the mobile terminal detected by the sensor.

[0103] The first action is, for example, "maintaining the tilt of the mobile device at the desired zero point for a predetermined time." The predetermined time is, for example, several seconds. The first notification prompts the remote operator O to "maintain the tilt of the mobile device at the desired zero point for a predetermined time." The calibration unit 255 presents such a first notification to the remote operator O through the third user interface 233. The first notification may be a visual notification or an audio notification.

[0104] In response to the first notification, the remote operator O maintains the tilt of the portable terminal at the desired zero point for a predetermined period of time or more. The operation signal indicates the state of the portable terminal during the first action, i.e., the tilt of the portable terminal. The calibration unit 255 sets the zero point of the portable terminal based on the tilt of the portable terminal during the first action. For example, the calibration unit 255 sets the average tilt of the portable terminal during the first action as the zero point of the operation member 211.

[0105] 5-4. Maximum operating point setting example 5-4-1. First example 13 is a conceptual diagram for explaining a first example of maximum operation point setting. The target operation system 210-T includes an operation member 211 (e.g., a joystick) that can be physically moved. The position of the operation member 211 is detected by a sensor 212. The position of the operation member 211 detected by the sensor 212 is reflected in the amount of operation during remote control. The operation signal indicates the position of the operation member 211 detected by the sensor 212.

[0106] The second action is a predetermined action that is required of the remote operator O when setting the maximum operation point. In the example shown in Fig. 13, the second action is, for example, "moving the position of the operation member 211 to the desired maximum operation point."

[0107] The second notification is a notification that prompts the remote operator O to perform a second action on the target operation system 210-T. In the example shown in FIG. 13, the second notification prompts the remote operator O to "move the position of the operation member 211 to the desired maximum operation point." The calibration unit 255 presents such a second notification to the remote operator O through the third user interface 233. The second notification may be a visual notification or an audio notification.

[0108] In response to the second notification, the remote operator O performs a second action on the target operating system 210-T. The target operating system 210-T outputs an operation signal corresponding to the second action performed by the remote operator O. The operation signal indicates the state of the target operating system 210-T during the second action. In the example shown in FIG. 13, the operation signal indicates the position of the operating member 211 during the second action. The calibration unit 255 sets the maximum operation point of the operating member 211 based on the maximum displacement of the operating member 211 during the second action. In other words, the calibration unit 255 sets the maximum displacement point of the operating member 211 during the second action as the maximum operation point.

[0109] It should be noted that maximum operation points may be set for both the case where the operating member 211 is pulled and the case where it is pushed.

[0110] 5-4-2. Second example In the second example, the target operation system 210-T is a gaze-guided operation system. The gaze-guided operation system includes a gaze measurement device (gaze camera, eye tracker) that measures the gaze direction of the remote operator O. Changes in the gaze direction of the remote operator O are reflected in the amount of operation during remote operation. The operation signal includes information on the gaze direction of the remote operator O measured by the gaze measurement device.

[0111] The second action is, for example, "moving the gaze direction to the desired maximum operation point." The second notification prompts the remote operator O to "move the gaze direction to the desired maximum operation point." The calibration unit 255 presents such a second notification to the remote operator O through the third user interface 233. The second notification may be a visual notification or an audio notification.

[0112] In response to the second notification, the remote operator O moves the gaze direction to the desired maximum operation point. The operation signal indicates the content of the second action, i.e., the gaze direction of the remote operator O. The calibration unit 255 sets the maximum operation point of the gaze measurement device based on the maximum displacement of the gaze direction of the remote operator O during the second action. In other words, the calibration unit 255 sets the maximum displacement point of the gaze direction of the remote operator O during the second action as the maximum operation point.

[0113] 5-4-3.Third example In a third example, the target operating system 210-T is a mobile terminal. Examples of mobile terminals include a smartphone and a tablet. The tilt (posture) of the mobile terminal is reflected in the amount of operation during remote control. The tilt of the mobile terminal is detected, for example, by a gyro sensor or acceleration sensor mounted on the mobile terminal. The operation signal indicates the tilt of the mobile terminal detected by the sensor.

[0114] The second action is, for example, "changing the tilt of the mobile terminal to the desired maximum operation point." The second notification prompts the remote operator O to "changing the tilt of the mobile terminal to the desired maximum operation point." The calibration unit 255 presents such a second notification to the remote operator O through the third user interface 233. The second notification may be a visual notification or an audio notification.

[0115] In response to the second notification, the remote operator O changes the tilt of the mobile device to a desired maximum operation point. The operation signal indicates the state of the mobile device during the second action, i.e., the tilt of the mobile device. The calibration unit 255 sets the maximum operation point of the mobile device based on the maximum change in the tilt of the mobile device during the second action. For example, the calibration unit 255 sets the maximum change point of the tilt of the mobile device during the second action as the maximum operation point of the mobile device.

[0116] 5-5.Settings information As shown in Fig. 11, the calibration unit 255 holds setting information CONF. The setting information CONF indicates the zero point and maximum operation point of each operation system 210-i (i = 1 to N). Initially, the setting information CONF may indicate default values ​​(initial values) of the zero point and maximum operation point of each operation system 210-i. The zero point and maximum operation point of the target operation system 210-T in the setting information CONF are updated by the above-mentioned calibration process. Calibration may be performed for all of the multiple types of operation systems 210-1 to 210-N.

[0117] The calibration unit 255 provides the latest setting information CONF to the above-mentioned signal processing unit 252. The signal processing unit 252 may generate remote operation information OPE regarding the first operation system 210-A based on the zero point and the maximum operation point of the first operation system 210-A indicated in the setting information CONF.

[0118] Effects As described above, the remote operator terminal 200 is configured to allow the remote operator O to perform calibration (customization) of the desired target operation system 210-T. This makes the operation system 210 easier for the remote operator O to use.

[0119] 6. End Notification Overview Depending on the type of the first operation system 210-A used to remotely operate the target mobility 100-T, there may be cases where a mechanical end does not exist (e.g., a joystick). In such cases, the remote operator O may not be able to recognize that the current operation amount has reached the maximum operation amount of the first operation system 210-A.

[0120] Therefore, the remote operator terminal 200 may be configured to notify the remote operator O that the current operation amount input by the remote operator O has reached the maximum operation amount of the first operation system 210-A. Alternatively, the remote operator terminal 200 may be configured to notify the remote operator O of the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A. This is because the relationship between the maximum operation amount and the current operation amount allows the remote operator O to recognize that the current operation amount has reached the maximum operation amount. Since the remote operator O can recognize that the current operation amount has reached the maximum operation amount, it becomes easier to remotely operate the target mobility 100-T more accurately. In other words, the accuracy of the remote operation of the target mobility 100-T is improved.

[0121] 6-2.Configuration example 14 is a block diagram showing an example of a configuration related to end notification. The remote operator terminal 200 comprises operation systems 210-1 to 210-N, a control device 250, and a notification device 260. The operation systems 210-1 to 210-N are the same as those in the above-mentioned section 2 and section 3. The notification device 260 is configured to be able to notify the remote operator O of various information. A specific example of the notification device 260 will be described later.

[0122] The control device 250 of the remote operator terminal 200 includes operation system interfaces 251-1 to 251-N, a signal processing unit 252, and a notification control unit 256. The operation system interfaces 251-1 to 251-N are the same as those in Section 2 and Section 3 described above.

[0123] The signal processing unit 252 holds setting information CONF. As described above, the setting information CONF indicates the zero point and maximum operation point of each operation system 210-i (i = 1 to N). During remote operation of the target mobility 100-T, the signal processing unit 252 receives an operation signal from the first operation system 210-A via the operation system interface 251-A. The operation signal indicates the current operation amount and current operation content of the first operation system 210-A. The signal processing unit 252 can generate remote operation information OPE for the first operation system 210-A based on the operation signal received from the first operation system 210-A and the setting information CONF for the first operation system 210-A.

[0124] Furthermore, the signal processing unit 252 provides an operation signal and setting information CONF related to the first operation system 210-A to the notification control unit 256. The operation signal indicates the current operation amount and current operation content input by the remote operator O. The maximum operation point indicated by the setting information CONF corresponds to the maximum operation amount. Furthermore, the signal processing unit 252 receives mobility information MOV from the target mobility 100-T and provides the mobility information MOV to the notification control unit 256.

[0125] The notification control unit 256 notifies the remote operator O of various information by controlling the notification device 260. For example, the notification control unit 256 notifies the remote operator O of the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A based on the operation signal related to the first operation system 210-A and the setting information CONF. As another example, the notification control unit 256 may notify the remote operator O that the current operation amount has reached the maximum operation amount.

[0126] 6-3.Various examples Various examples of notification device 260 and end notifications are described below.

[0127] 6-3-1. First example 15 is a conceptual diagram for explaining a first example of the end notification. In the first example, a notification device 260 includes a display device 261 that displays visual information.

[0128] The notification control unit 256 controls the display device 261 to display visual information indicating the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A. That is, the display device 261 displays visual information indicating the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A. In the example shown in FIG. 15, the display device 261 displays visual information indicating the relationship between the current driving force and the maximum driving force. The maximum driving force is represented by a first rectangle of a predetermined height. A second rectangle representing the current driving force is contained within the first rectangle. The height of the second rectangle increases as the current driving force increases.

[0129] This allows the remote operator O to intuitively grasp how much the current operation amount is compared to the maximum operation amount, and also allows the remote operator O to grasp that the current operation amount has reached the maximum operation amount.

[0130] The notification control unit 256, i.e., the display device 261, may dynamically change at least one of the color, brightness, saturation, and size of the visual information as the current operation amount approaches the maximum operation amount. For example, as the current driving force approaches the maximum driving force, the color of the visual information (e.g., the second rectangle) may gradually change from a bluish color to a reddish color.

[0131] The notification control unit 256, i.e., the display device 261, may dynamically change the display position of the visual information in accordance with the steering operation by the remote operator O. Information regarding the steering operation by the remote operator O (steering direction, steering amount) is obtained from an operation signal from the first operation system 210-A. For example, when traveling straight, the visual information is displayed near the center of the screen of the display device 261. Then, the display position of the visual information moves to match the steering direction. That is, when turning left, the visual information is displayed on the left side of the screen of the display device 261. On the other hand, when turning right, the visual information is displayed on the right side of the screen of the display device 261. The movement amount of the display position may increase as the steering amount increases. Because the line of sight of the remote operator O is likely to match the steering direction, dynamically changing the display position of the visual information in accordance with the steering operation as described above makes it easier for the remote operator O to see the visual information. That is, the remote operator O can more easily grasp the relationship between the maximum operation amount and the current operation amount of the first operation system 210-A.

[0132] 6-3-2. Second example 16 is a conceptual diagram illustrating a second example of the end notification. In the second example, the notification device 260 includes a stimulus generator 262 that applies force or vibration to the remote operator O. Examples of the stimulus generator 262 include a haptic device, a vibration generator, a massage device, etc. For example, the stimulus generator 262 is embedded in the seat on which the remote operator O sits.

[0133] The notification control unit 256 controls the stimulus generator 262 to generate a force or a vibration. That is, the stimulus generator 262 applies the force or the vibration to the remote operator O. When the current operation amount reaches the maximum operation amount, the notification control unit 256, that is, the stimulus generator 262 applies a specific force or a specific vibration to the remote operator O. For example, when the current operation amount reaches the maximum operation amount, the notification control unit 256, that is, the stimulus generator 262 applies a force or a vibration with a characteristic pattern to the remote operator O. This enables the remote operator O to recognize that the current operation amount has reached the maximum operation amount.

[0134] The notification control unit 256, i.e., the stimulus generating device 262, may increase the magnitude of the force or vibration applied to the remote operator O as the current amount of operation approaches the maximum amount of operation.

[0135] The notification control unit 256, i.e., the stimulus generation device 262, may dynamically change the position where the force or vibration is generated in accordance with the steering operation by the remote operator O. Information about the steering operation by the remote operator O (steering direction, steering amount) is obtained from an operation signal from the first operation system 210-A. For example, when traveling straight, the force or vibration is generated near the center position of the seat. Then, the position where the force or vibration is generated moves to match the steering direction. In other words, when turning left, the force or vibration is generated on the left side of the seat. On the other hand, when turning right, the force or vibration is generated on the right side of the seat. The movement of the generation position may increase as the steering amount increases.

[0136] The notification control unit 256, i.e., the stimulus generating device 262, may dynamically change the position where the force or vibration is generated depending on the speed of the target mobility 100-T. The speed of the target mobility 100-T is obtained from the mobility information MOV. For example, as the speed of the target mobility 100-T increases, the position where the force or vibration is generated may move toward the front of the seat.

[0137] 6-3-3.Third example 17 is a conceptual diagram for explaining a third example of end notification. In the third example, notification device 260 includes speaker 263 that outputs sound. For example, speaker 263 is installed around the seat where remote operator O sits. Multiple speakers 263 may be installed.

[0138] The notification control unit 256 controls the speaker 263 to generate a sound. When the current operation amount reaches the maximum operation amount, the notification control unit 256, i.e., the speaker 263, outputs a specific sound (e.g., "The maximum operation amount has been reached"). This allows the remote operator O to recognize that the current operation amount has reached the maximum operation amount.

[0139] The notification control unit 256, that is, the speaker 263, may increase the strength of the sound to be output as the current operation amount approaches the maximum operation amount.

[0140] The notification control unit 256, i.e., the speaker 263, may dynamically change the output position of the audio in accordance with the steering operation by the remote operator O. Information regarding the steering operation by the remote operator O (steering direction, steering amount) is obtained from an operation signal from the first operation system 210-A. For example, when traveling straight, audio is output from above the head of the remote operator O. Then, the output position of the audio moves to match the steering direction. In other words, when turning left, audio is output from the left side of the remote operator O. On the other hand, when turning right, audio is output from the right side of the remote operator O. The amount of movement of the output position of the audio may increase as the steering amount increases.

[0141] The notification control unit 256, i.e., the speaker 263, may dynamically change the audio output position depending on the speed of the target mobility 100-T. The speed of the target mobility 100-T is obtained from the mobility information MOV. For example, as the speed of the target mobility 100-T increases, the audio output position may move forward.

[0142] Effects As described above, the remote operator terminal 200 is configured to notify the remote operator O that the current operation amount has reached the maximum operation amount of the first operation system 210-A. Alternatively, the remote operator terminal 200 may be configured to notify the remote operator O of the relationship between the maximum operation amount of the first operation system 210-A and the current operation amount. In either case, the remote operator O can recognize that the current operation amount has reached the maximum operation amount. This makes it easier for the remote operator O to remotely operate the target mobility 100-T more accurately. In other words, the accuracy of the remote operation of the target mobility 100-T is improved.

[0143] 7. Combination Combinations of two or more of the features described in sections 3-6 above are also possible.

[0144] 8. Example of remote operator terminal configuration 18 is a block diagram showing an example configuration of a remote operator terminal 200. The remote operator terminal 200 includes a plurality of different types of operation systems 210-1 to 210-N, a display system 220, various user interfaces 231, 232, 233, a communication device 240, a control device 250, and a notification device 260. The operation systems 210-1 to 210-N, the user interfaces 231, 232, 233, and the notification device 260 are as described above.

[0145] The display system 220 displays various types of information to the remote operator O who performs remote operation. In other words, the display system 220 presents various types of information to the remote operator O by displaying them. The display system 220 includes a display device (monitor) such as a liquid crystal display, an organic EL display, or a touch panel. The display system 220 (touch panel) may include a first user interface 231. The display system 220 (touch panel) may include a second user interface 232. The display system 220 (touch panel) may include a third user interface 233. The display system 220 may include a display device 261 as the notification device 260 shown in FIG. 15.

[0146] The communication device 240 communicates with the target mobility 100-T and the management system 300 via a communication network.

[0147] The control device 250 controls the remote operator terminal 200. The control device 250 may include one or more processors 257 (hereinafter simply referred to as processors 257) and one or more storage devices 258 (hereinafter simply referred to as storage devices 258). The processor 257 performs various processes. Examples of the processor 257 include a general-purpose processor, a special-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), an integrated circuit, a conventional circuit, and / or a combination thereof. The processor 257 may also be referred to as circuitry or processing circuitry. Circuitry is hardware programmed to realize the described functions or hardware that executes the functions. The storage device 258 stores various information. Examples of the storage device 258 include volatile memory, non-volatile memory, a hard disk drive (HDD), a solid-state drive (SSD), etc.

[0148] The control program PROG is a computer program executed by the processor 257. The functions of the control device 250 may be realized by cooperation between the processor 257, which executes the control program PROG, and the storage device 258. The control program PROG is stored in the storage device 258. Alternatively, the control program PROG may be recorded on a computer-readable recording medium. The control program PROG may be provided via a network.

[0149] The control device 250 includes necessary functions among the above-mentioned operation system interface 251, signal processing unit 252, selection control unit 253, automatic selection unit 254, calibration unit 255, notification control unit 256, etc. Specification information SPEC, priority information PREF, setting information CONF, etc. are stored in a storage device 258.

[0150] Furthermore, during remote operation of the target mobility 100-T, the control device 250 communicates with the target mobility 100-T via the communication device 240. The control device 250 receives mobility information MOV transmitted from the target mobility 100-T. The control device 250 presents the mobility information MOV, including video, to the remote operator O via the display system 220. Based on the mobility information MOV presented via the display system 220, the remote operator O can recognize the state of the target mobility 100-T and the surrounding circumstances.

[0151] The remote operator O uses the first operation system 210-A to input operation amounts and operation contents for remotely operating the target mobility 100-T. The operation amounts and operation contents for remote operation are detected by sensors installed in the first operation system 210-A. The control device 250 generates remote operation information OPE that reflects the operation amounts and operation contents input by the remote operator O. It can be said that the remote operation information OPE is information that reflects the degree of driving operation by the remote operator O. The control device 250 transmits the remote operation information OPE to the target mobility 100-T via the communication device 240. [Explanation of symbols]

[0152] 1. Remote control system 100 Mobility 200 Remote Operator Terminal 210 Operation system 231 First User Interface 232 Second User Interface 233 Third User Interface 250 control device 260 Notification device 300 Management System

Claims

1. A remote operator terminal used by a remote operator for remote operation of a target mobility, Multiple types of operation systems, A user interface; a control device configured to calibrate a target operation system among the plurality of types of operation systems; Equipped with In the calibration, the control device presenting a first notification to the remote operator through the user interface, the first notification prompting the remote operator to perform a first action on the target operating system; A zero point of the target operation system is set based on the content of the first action or the state of the target operation system during the first action. It was configured as Remote operator terminal.

2. 2. A remote operator terminal according to claim 1, the target operation system includes an operation member that can be physically moved, The position of the operating member is reflected in the amount of operation during the remote operation, the first action is to maintain the position of the operating member at a desired zero point for a predetermined time; The control device sets the zero point of the operating member based on the position of the operating member during the first action. Remote operator terminal.

3. 2. A remote operator terminal according to claim 1, the target operation system includes a line-of-sight measurement device that measures the line-of-sight direction of the remote operator, a change in the line of sight of the remote operator is reflected in an operation amount during the remote operation, the first action is to maintain the line of sight direction at a desired zero point for a predetermined time; The control device sets the zero point of the gaze measurement device based on the gaze direction during the first action. Remote operator terminal.

4. 2. A remote operator terminal according to claim 1, the target operation system includes a mobile terminal, The tilt of the mobile terminal is reflected in the amount of operation during the remote operation, the first action is to maintain the tilt of the mobile terminal at a desired zero point for a predetermined time; The control device sets the zero point of the portable terminal based on the tilt of the portable terminal during the first action. Remote operator terminal.

5. 2. A remote operator terminal according to claim 1, In the calibration, the control device further presenting a second notification to the remote operator through the user interface, the second notification prompting the remote operator to perform a second action on the target operating system; A maximum operation point of the target operation system is set based on the content of the second action or the state of the target operation system during the second action. It was configured as Remote operator terminal.

6. 6. A remote operator terminal according to claim 5, the target operation system includes an operation member that can be physically moved, The position of the operating member is reflected in the amount of operation during the remote operation, the second action is to move the position of the operating member to a desired maximum operating point; The control device sets the maximum operation point of the operating member based on a maximum displacement of the operating member during the second action. Remote operator terminal.

7. 6. A remote operator terminal according to claim 5, the target operation system includes a line-of-sight measurement device that measures the line-of-sight direction of the remote operator, a change in the line of sight of the remote operator is reflected in an operation amount during the remote operation, the second action is to move the line of sight to a desired maximum manipulation point; The control device sets the maximum operation point of the gaze measurement device based on a maximum displacement of the gaze direction during the second action. Remote operator terminal.

8. 6. A remote operator terminal according to claim 5, the target operation system includes a mobile terminal, The tilt of the mobile terminal is reflected in the amount of operation during the remote operation, the second action is to change the tilt of the mobile terminal to a desired maximum operation point; The control device sets the maximum operation point of the mobile device based on a maximum change in the tilt of the mobile device during the second action. Remote operator terminal.

9. A remote operator terminal according to any one of claims 1 to 8, comprising: The target operation system is a first operation system selected for the remote operation from among the plurality of types of operation systems. Remote operator terminal.

10. A remote operator terminal used by a remote operator for remote operation of a target mobility, Multiple types of operation systems, A user interface; a control device configured to calibrate a target operation system among the plurality of types of operation systems; Equipped with In the calibration, the control device presenting a second notification to the remote operator through the user interface, the second notification prompting the remote operator to perform a second action on the target operating system; A maximum operation point of the target operation system is set based on the content of the second action or the state of the target operation system during the second action. It was configured as Remote operator terminal.

Images