Remote operation system, remote operator terminal, and remote operation management method

Abstract

The invention relates to a remote operation system, a remote operator terminal and a remote operation management method. A remote operation system for remote operation of a moving body includes one or more processors. The one or more processors are configured to acquire terminal state information indicating a state of a remote operator terminal configured to be used by a remote operator to perform the remote operation; based on the terminal state information, a terminal score indicating the suitability of the remote operator terminal for the remote operation is calculated. The terminal score is used to select a first remote operator terminal assigned to the remote operation of the target moving body.

Description

Technical Field

[0001] The present disclosure relates to a remote operation system, a remote operator terminal, and a remote operation management method. Background Art

[0002] U.S. Patent Application Publication No. 2020 / 0062267 discloses a system related to remote operation of a vehicle. The system assigns an appropriate remote operator for remote operation of the vehicle based on the skills and experience values of the remote operator. Summary of the Invention

[0003] Consider the case where a remote operator remotely operates a mobility (mobility, travel, mobile tool). Regarding the remote operator terminal used by the remote operator for remote operation, there may be a state suitable for remote operation and a state that is not. If a remote operator terminal in a state unsuitable for remote operation is used, the accuracy of remote operation will decrease. In U.S. Patent Application Publication No. 2020 / 0062267, the state of the remote operator terminal is not considered.

[0004] The present disclosure provides a technology capable of ensuring the accuracy of remote operation of a mobility.

[0005] The remote operation system for a mobility according to the first aspect of the present disclosure includes one or more processors configured to obtain terminal state information indicating the state of a remote operator terminal, the remote operator terminal being configured to be used by a remote operator to perform the remote operation; and based on the terminal state information, calculate a terminal score indicating the suitability of the remote operator terminal for the remote operation, the terminal score being used to select a first remote operator terminal assigned to the remote operation of an object mobility.

[0006] In the remote operation system according to the first aspect of the present disclosure, alternatively, the state of the remote operator terminal includes the communication state of the remote operator terminal, the terminal score includes a first terminal score, and the better the communication state of the remote operator terminal, the higher the first terminal score.

[0007] In the remote operation system according to the first aspect of the present disclosure, alternatively, the state of the remote operator terminal includes the abnormality degree of the control device of the remote operator terminal, the terminal score includes a second terminal score, and the higher the abnormality degree, the lower the second terminal score.

[0008] In the remote operation system according to the first aspect of the present disclosure, alternatively, the one or more processors are configured to acquire the terminal status information in real time and calculate the terminal score in real time.

[0009] In the remote operation system according to the first aspect of the present disclosure, alternatively, the one or more processors are configured to acquire operator status information indicating the status of the remote operator; based on the operator status information, calculate an operator score indicating the suitability of the remote operator for the remote operation, and the operator score is used to select the first remote operator assigned to the remote operation of the target moving body.

[0010] In the remote operation system according to the first aspect of the present disclosure, alternatively, the status of the remote operator includes the health status of the remote operator, the operator score includes a first operator score, and the better the health status of the remote operator, the higher the first operator score.

[0011] In the remote operation system according to the first aspect of the present disclosure, alternatively, the status of the remote operator includes the working hours of the remote operator engaged in the remote operation within a certain period in the past, the operator score includes a second operator score, and the longer the working hours, the lower the second operator score.

[0012] In the remote operation system according to the first aspect of the present disclosure, alternatively, the status of the remote operator includes the hourly wage of the remote operator, the operator score includes a third operator score, and the lower the hourly wage, the higher the third operator score.

[0013] In the remote operation system according to the first aspect of the present disclosure, alternatively, the one or more processors are configured to acquire the operator status information in real time and calculate the operator score in real time.

[0014] In the remote operation system according to the first aspect of the present disclosure, alternatively, the one or more processors are configured to calculate a comprehensive score by combining the terminal score and the operator score, and the comprehensive score represents the suitability of the combination of the remote operator terminal and the remote operator for the remote operation, and the comprehensive score is used to select the combination of the first remote operator and the first remote operator terminal assigned to the remote operation of the target moving body.

[0015] In the remote operation system according to the first aspect of the present disclosure, alternatively, the one or more processors are configured to select a combination of the first remote operator and the first remote operator terminal assigned to the remote operation of the target moving body based on the terminal score and the operator score.

[0016] In the remote operation system according to the first aspect of the present disclosure, alternatively, the one or more processors are configured to obtain terminal specification information indicating the specifications of each of a plurality of remote operator terminals, obtain requirement ability information indicating required terminal specifications required for the remote operation of the target moving body, and select the combination of the first remote operator and the first remote operator terminal having the required terminal specifications based on the terminal specification information, the required terminal specifications, the terminal score, and the operator score.

[0017] In the remote operation system according to the first aspect of the present disclosure, alternatively, there are multiple types of the moving bodies, and the one or more processors are configured to obtain operator license information indicating licenses that each of a plurality of remote operators has for operations on the multiple types of moving bodies, obtain requirement ability information indicating required licenses required for the remote operation of the target moving body, and select the combination of the first remote operator having the required license and the first remote operator terminal based on the operator license information, the required license, the terminal score, and the operator score.

[0018] In the remote operation system according to the first aspect of the present disclosure, alternatively, the requirement ability information indicates required terminal specifications required for the remote operation of the target moving body, and the one or more processors are configured to obtain terminal specification information indicating the specifications of each of a plurality of remote operator terminals, and select the combination of the first remote operator having the required license and the first remote operator terminal having the required terminal specifications based on the operator license information, the terminal specification information, the requirement ability information, the terminal score, and the operator score.

[0019] The remote operator terminal for remotely operating a mobile body used by a remote operator according to the second aspect of the present disclosure includes one or more processors configured to obtain terminal state information indicating the state of the remote operator terminal, calculate a terminal score indicating the suitability of the remote operator terminal for the remote operation based on the terminal state information, and send information on the terminal score or information on a comprehensive score reflecting the terminal score to a management system, where the management system is a system for selecting a first remote operator and a first remote operator terminal assigned to the remote operation of a target mobile body.

[0020] In the remote operator terminal according to the second aspect of the present disclosure, alternatively, the one or more processors may be configured to obtain operator state information indicating the state of the remote operator, calculate an operator score indicating the suitability of the remote operator for the remote operation based on the operator state information, and send information on the operator score or information on the comprehensive score reflecting the terminal score and the operator score to the management system.

[0021] In the remote operator terminal according to the second aspect of the present disclosure, alternatively, the one or more processors may be configured to calculate the comprehensive score indicating the suitability of the combination of the remote operator terminal and the remote operator for the remote operation by integrating the terminal score and the operator score, and send information on the comprehensive score to the management system.

[0022] The third aspect of the present disclosure is a remote operation management method for a computer to manage the remote operation of a mobile body. The remote operation method includes: obtaining terminal state information indicating the state of a remote operator terminal, where the remote operator terminal is configured to be used by a remote operator to perform the remote operation; calculating a terminal score indicating the suitability of the remote operator terminal for the remote operation based on the terminal state information; and selecting a first remote operator terminal assigned to the remote operation of a target mobile body based on the terminal score.

[0023] According to the first aspect, the second aspect, and the third aspect of the present disclosure, a terminal score indicating the suitability of a remote operator terminal for a remote operation is calculated based on the state of the remote operator terminal. Moreover, based on the terminal score, a first remote operator terminal assigned to the remote operation of a target mobile body is selected. That is, considering the suitability, an appropriate first remote operator terminal is assigned to perform the remote operation of the target mobile body. Thereby, the accuracy of the remote operation of the target mobile body is ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Hereinafter, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings. In the drawings, the same reference numerals denote the same elements, and:

[0025] Figure 1 is a conceptual diagram for explaining the outline of a remote operation system.

[0026] Figure 2 is a diagram showing examples of the content of various remote operations requested for various mobile bodies.

[0027] Figure 3 is a conceptual diagram showing an example of the configuration (structure) of a remote operation system.

[0028] Figure 4 is a block diagram showing an example of the configuration of a mobile body.

[0029] Figure 5 is a block diagram showing an example of the configuration of a remote operator terminal.

[0030] Figure 6 is a block diagram showing an example of the configuration of a management system.

[0031] Figure 7 is a conceptual diagram for explaining the outline of the allocation process performed by the management system.

[0032] Figure 8 is a block diagram showing an example of a functional structure related to score calculation.

[0033] Figure 9 is a conceptual diagram showing an example of an operator state and an operator score.

[0034] Figure 10 is a conceptual diagram showing an example of a terminal state and a terminal score.

[0035] Figure 11 is a block diagram showing another example of a functional structure related to score calculation.

[0036] Figure 12 is a block diagram showing an example of a functional structure related to the allocation process from the first perspective.

[0037] Figure 13 is a conceptual diagram showing an example of operator license information.

[0038] Figure 14 is a block diagram showing an example of a functional structure related to the allocation process from the second perspective.

[0039] Figure 15 is a conceptual diagram showing an example of terminal specification information.

[0040] Figure 16It is a block diagram showing an example of a functional structure related to the allocation process under the third viewpoint.

[0041] Figure 17 It is a conceptual diagram showing an example of terminal specification information related to the operating system and the required terminal specifications.

[0042] Figure 18 It is a conceptual diagram showing an example of terminal specification information related to the display system and the required terminal specifications. Detailed implementation manners

[0043] With reference to the accompanying drawings, the embodiments of the present disclosure will be described.

[0044] 1. Outline of the remote operation system

[0045] Figure 1 It is a conceptual diagram for explaining the outline of the remote operation system 1 related to this embodiment. The remote operation system 1 is a system for remotely operating the mobile body 100. Remote operation includes the concept of remote driving. The remote operation system 1 includes a mobile body 100, a remote operator terminal 200, and a management system 300.

[0046] The mobile body 100 is a movable body (moving object) that can move. The mobile body 100 can also be manually operated by an operator riding on the mobile body 100. The mobile body 100 can also have an autonomous movement function. In any case, the mobile body 100 is configured to be remotely operable as needed. That is, the mobile body 100 is the object of remote operation of the remote operation system 1.

[0047] The type of the mobile body 100 that becomes the object of remote operation is not limited to one kind, and can be multiple kinds. For example, the mobile body 100 is a vehicle traveling on a road (example: sedan, truck, bus, MaaS (Mobility as a Service) vehicle, autonomous driving vehicle, etc.). As another example, the mobile body 100 can also be a vehicle used in a factory (example: forklift, factory delivery truck, etc.). As still another example, the mobile body 100 can also be a special small vehicle (example: golf course cart, personal mobility means, electric wheelchair, etc.). As still another example, the mobile body 100 can also be construction machinery (example: power excavator, bulldozer, etc.). As still another example, the mobile body 100 can also be a robot (example: logistics robot, work robot, etc.). As still another example, the mobile body 100 can also be a flying object (example: drone, etc.). As still another example, the mobile body 100 can also be a ship (example: small ship, large cruiser, etc.). As still another example, the mobile body 100 can also be a ride in an amusement park (example: trolley, amusement facility, etc.).

[0048] Figure 2 An example of the content of various remote operations requested for various mobile bodies 100 is shown. AsFigure 2 As exemplified, the mobile body 100 varies in a wide variety, and the content of the requested remote operation also varies. In addition, the remote operation can be requested either by the mobile body 100 itself or by a service provider cooperating with the remote operation system 1. In the latter case, the content of the remote operation can also be said to be the content of the service provided by the service provider.

[0049] The remote operator terminal 200 is a terminal device used by the remote operator O to remotely operate the mobile body 100. That is, the remote operator terminal 200 is configured for the remote operator O to use for remotely operating the mobile body 100. Examples of the remote operator terminal 200 include a cockpit-type terminal, a PC, a tablet computer, a smart phone, etc. A single remote operator terminal 200 can also be configured to be capable of handling the remote operations of various mobile bodies 100. Or, a single remote operator terminal 200 can also be a dedicated terminal for the remote operation of a specific mobile body 100. In addition, the combination of the remote operator O and the remote operator terminal 200 can be predetermined or can be freely changed. That is, a single remote operator terminal 200 can be used only by a specific remote operator O, or can be used by various remote operators O in turn.

[0050] The management system (manager) 300 manages the remote operation system 1. The management system 300 can also be composed of multiple servers performing distributed processing. For example, the management system 300 manages multiple remote operators O and multiple remote operator terminals 200. In addition, the management system 300 responds to a remote operation request and allocates a remote operator O and a remote operator terminal 200 to perform the remote operation of the mobile body 100. In addition, the management system 300 can also manage the state of the mobile body 100 during the remote operation. Details of the management system 300 will be described later.

[0051] The mobile body 100, the remote operator terminal 200, and the management system 300 can communicate with each other via a communication network. For example, the mobile body 100 can perform wireless communication with the remote operator terminal 200 and / or the management system 300 via a wireless communication network. The remote operator terminal 200 and the management system 300 can communicate with each other via a wired communication network or a wireless communication network. The mobile body 100 and the remote operator terminal 200 can communicate either via the management system 300 or directly without passing through the management system 300.

[0052] The general information flow during the remote operation of the mobile body 100 is as follows.

[0053] The mobile body 100 is equipped with various sensors including a camera. The camera captures the conditions around the mobile body 100. Through the camera, an image representing the conditions around the mobile body 100 is obtained. The mobile body information MOV is the information obtained by various sensors and at least includes the image captured by the camera. The mobile body information MOV may also include the position and state of the mobile body 100 (e.g., speed, steering angle, etc.). The mobile body 100 sends the mobile body information MOV to the remote operator terminal 200.

[0054] The remote operator terminal 200 receives the mobile body information MOV sent from the mobile body 100. The remote operator terminal 200 presents the mobile body information MOV to the remote operator O. Specifically, the remote operator terminal 200 has a display device and displays the image, etc. on the display device. The remote operator O views the displayed information, identifies the conditions around the mobile body 100, and performs the remote operation of the mobile body 100. The remote operation information OPE is the information related to the remote operation (steering operation, acceleration operation, deceleration operation, forward and backward movement operation, lateral movement operation, etc.) performed by the remote operator O. For example, the remote operation information OPE includes the operation amount input by the remote operator O. The remote operation information OPE can be said to be the information reflecting the degree of the remote operation performed by the remote operator O. The remote operator terminal 200 sends the remote operation information OPE to the mobile body 100.

[0055] The mobile body 100 receives the remote operation information OPE sent from the remote operator terminal 200. The mobile body 100 performs mobile body control according to the received remote operation information OPE. In this way, the remote operation of the mobile body 100 is realized.

[0056] Figure 3 It is a conceptual diagram showing a configuration example of the remote operation system 1. In Figure 3 the example shown, the remote operation management center is set in a predetermined site or a predetermined building. Moreover, the management system 300 and multiple remote operator terminals 200 (200-1 to 200-N, where N is an integer of 2 or more) are set in the remote operation management center. In addition, multiple remote operators O work in the remote operation management center. The management system 300 monitors and manages the states of the multiple remote operator terminals 200 in the remote operation management center. In addition, the management system 300 monitors and manages the states of the multiple remote operators O in the remote operation management center.

[0057] 2. Configuration Example

[0058] 2-1. Configuration Example of Mobile Body

[0059] Figure 4It is a block diagram showing a configuration example of the moving body 100. The moving body 100 includes a communication device 110, a sensor group 120, one or more actuators 130, and a control device 150.

[0060] The communication device 110 performs wireless communication with the outside of the moving body 100. For example, the communication device 110 performs wireless communication with the remote operator terminal 200 and / or the management system 300.

[0061] The sensor group 120 includes an identification sensor, a moving body state sensor, a position sensor, etc. The identification sensor identifies (detects) the situation around the moving body 100. Examples of the identification sensor include a camera C, LIDAR (Laser Imaging Detection and Ranging), and radar. The moving body state sensor detects the state of the moving body 100. The moving body state sensor includes a speed sensor, an acceleration sensor, a yaw rate sensor, and a rudder angle sensor, etc. The position sensor detects the position and orientation of the moving body 100. For example, the position sensor includes GNSS (Global Navigation Satellite System).

[0062] The actuator 130 moves the moving body 100. For example, the actuator 130 includes a forward and backward moving actuator for moving the moving body 100 forward and backward (accelerating, decelerating). As another example, the actuator 130 may also include a lateral moving actuator for moving the moving body 100 laterally. As still another example, when the moving body 100 has an arm, the actuator 130 may also include an arm actuator for moving the arm (motion).

[0063] For example, when the moving body 100 is an ordinary vehicle, the actuator 130 includes a steering device, a driving device, and a braking device. The steering device steers the wheels. For example, the steering device includes an electric power steering (EPS) device. The driving device is a power source that generates a driving force. Examples of the driving device include an engine, an electric motor, and a wheel hub motor. The braking device generates a braking force.

[0064] The control device 150 is a computer that controls the mobile body 100. The control device 150 includes one or more processors 160 (hereinafter simply referred to as the processor 160) and one or more storage devices 170 (hereinafter simply referred to as the storage device 170). The processor 160 performs various processes. Examples of the processor 160 include a general-purpose processor, a special-purpose processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), an integrated circuit, a conventional circuit, and / or a combination thereof. The processor 160 may also be referred to as circuitry or processing circuitry. Circuitry is hardware programmed to implement the described functions or hardware that performs the functions. The storage device 170 stores various information. Examples of the storage device 170 include a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), and an SSD (Solid State Drive).

[0065] The control program PROG1 is a computer program executed by the processor 160. The functions of the control device 150 can also be implemented through the cooperation of the processor 160 that executes the control program PROG1 and the storage device 170. The control program PROG1 is stored in the storage device 170. Alternatively, the control program PROG1 may also be recorded on a computer-readable recording medium.

[0066] The control device 150 acquires driving environment information ENV indicating the driving (operation) environment of the mobile body 100. The driving environment information ENV is stored in the storage device 170.

[0067] The driving environment information ENV includes surrounding condition information representing the recognition results of recognition sensors. For example, the surrounding condition information includes the images captured by the camera C. Additionally, the surrounding condition information may also include object information related to the objects around the moving body 100. Examples of the objects around the moving body 100 include pedestrians, other vehicles (front vehicles, parked vehicles, etc.), white lines, stop lines, signal lights (traffic lights), signs, roadside structures (buildings), etc. The object information represents the relative position and relative speed of the object with respect to the moving body 100. For example, by analyzing the images obtained by the camera, the object can be recognized and the relative position of the object can be calculated. Additionally, based on the point cloud information obtained by the LIDAR, the object can also be recognized and the relative position and relative speed of the object can be obtained.

[0068] In addition, the driving environment information ENV may also include moving body state information representing the detection results of the moving body state sensors. The moving body state information represents the speed, acceleration (longitudinal acceleration, lateral acceleration), yaw rate, steering angle, etc. of the moving body 100.

[0069] Furthermore, the driving environment information ENV may also include position information representing the position and moving direction (azimuth) of the moving body 100. The position information is obtained by the position sensor. High-precision position information can also be obtained by performing self-position estimation processing (localization) using map information and surrounding condition information (object information).

[0070] The control device 150 performs movement control for controlling the movement of the moving body 100. The movement control includes longitudinal movement control and lateral movement control. The control device 150 performs movement control by controlling the actuator 130.

[0071] The control device 150 may also perform autonomous movement control based on the driving environment information ENV. More specifically, the control device 150 generates a movement plan for the moving body 100 based on the driving environment information ENV. Furthermore, the control device 150 generates a target trajectory required for the moving body 100 to travel according to the movement plan based on the driving environment information ENV. The target trajectory includes a target position and a target speed. Moreover, the control device 150 performs movement control to make the moving body 100 follow the target trajectory.

[0072] In the case of remotely operating the moving body 100, the control device 150 communicates with the remote operator terminal 200 via the communication device 110.

[0073] The control device 150 sends the moving body information MOV to the remote operator terminal 200. The moving body information MOV is the information required for the remote operation of the moving body 100 by the remote operator O, and includes at least a part of the above-described driving environment information ENV. In particular, the moving body information MOV includes the image captured by the camera C. The moving body information MOV may also include other surrounding condition information. The moving body information MOV may also include the moving body state information. The moving body information MOV may also include the position information.

[0074] In addition, the control device 150 receives the remote operation information OPE from the remote operator terminal 200. The remote operation information OPE is information related to the remote operation (steering operation, acceleration operation, deceleration operation, forward and backward movement operation, lateral movement operation, etc.) of the remote operator O. For example, the remote operation information OPE includes the operation amount input by the remote operator O. The control device 150 performs movement control according to the received remote operation information OPE.

[0075] 2-2. Configuration Example of Remote Operator Terminal

[0076] Figure 5 It is a block diagram showing a configuration example of the remote operator terminal 200. The remote operator terminal 200 includes a communication device 210, a display device 220, an input device 230, an operator sensor 240, and a control device 250.

[0077] The communication device 210 communicates with the moving body 100 and the management system 300.

[0078] The display device 220 displays various information to the remote operator O who performs the remote operation. In other words, the display device 220 presents various information to the remote operator O by displaying various information. Typically, the display device 220 is a display (monitor) such as a liquid crystal display or an organic EL display. The display device 220 may also be a touch panel.

[0079] The input device 230 accepts the input of the remote operator O. For example, the input device 230 includes a remote operation component that the remote operator O operates when remotely operating the moving body 100. As the remote operation component, a steering wheel (rudder wheel), an accelerator pedal, a brake pedal, a direction indicator, a joystick, a cross key, and a switch can be exemplified. The remote operation component may also be a touch panel. The input device 230 may also include a keyboard, a mouse, a touch panel, etc. other than the remote operation component.

[0080] The operator sensor 240 is a sensor for monitoring the state of the remote operator O. For example, the operator sensor 240 includes a biological sensor that detects biological information of the remote operator O. Examples of the biological information include body temperature, heart rate, blood pressure, and sweating amount.

[0081] The control device 250 controls the remote operator terminal 200. The control device 250 includes one or more processors 260 (hereinafter simply referred to as the processor 260) and one or more storage devices 270 (hereinafter simply referred to as the storage device 270). The processor 260 performs various processes. Examples of the processor 260 include a CPU, a GPU, an ASIC, an FPGA, etc. The storage device 270 stores various information. Examples of the storage device 270 include a volatile memory, a non-volatile memory, an HDD, an SSD, etc. The processor 260 performs various processes. Examples of the processor 260 include a general-purpose processor, a special-purpose processor, a CPU, a GPU, an ASIC, an FPGA, an integrated circuit, a conventional circuit, and / or a combination thereof. The processor 260 may also be referred to as circuitry or processing circuitry. Circuitry is hardware programmed to implement the described functions or hardware that performs the functions. The storage device 270 stores various information. Examples of the storage device 270 include a volatile memory, a non-volatile memory, an HDD, an SSD, etc.

[0082] The control program PROG2 is a computer program executed by the processor 260. The functions of the control device 250 can also be realized through the cooperation of the processor 260 and the storage device 270 that execute the control program PROG2. The control program PROG2 is stored in the storage device 270. Alternatively, the control program PROG2 can also be recorded on a computer-readable recording medium. The control program PROG2 can also be provided via a network.

[0083] The control device 250 communicates with the mobile body 100 via the communication device 210. The control device 250 receives the mobile body information MOV sent from the mobile body 100. The control device 250 presents the mobile body information MOV to the remote operator O by displaying the mobile body information MOV including an image on the display device 220. The remote operator O can identify the state of the mobile body 100 and / or the surrounding conditions based on the mobile body information MOV displayed on the display device 220.

[0084] The remote operator O operates the remote operation part of the input device 230. The operation amount of the remote operation part is detected by a sensor provided in the remote operation part. The control device 250 generates remote operation information OPE that reflects the operation amount of the remote operation part by the remote operator O. The remote operation information OPE can be said to be information that reflects the degree of remote operation by the remote operator O. Further, the control device 250 transmits the remote operation information OPE to the mobile body 100 via the communication device 210.

[0085] 2-3. Configuration example of management system

[0086] Figure 6 FIG. is a block diagram showing a configuration example of the management system 300. The management system 300 includes a communication device 310 and a control device 350.

[0087] The communication device 310 communicates with the mobile body 100 and the remote operator terminal 200.

[0088] The control device 350 controls the management system 300. The control device 350 includes one or more processors 360 (hereinafter simply referred to as the processor 360) and one or more storage devices 370 (hereinafter simply referred to as the storage device 370). The processor 360 performs various processes. Examples of the processor 360 include a general-purpose processor, a special-purpose processor, a CPU, a GPU, an ASIC, an FPGA, an integrated circuit, a conventional circuit, and / or a combination thereof. The processor 360 may also be referred to as circuitry or processing circuitry. Circuitry is hardware programmed to implement the described functions or hardware that executes the functions. The storage device 370 stores various information. Examples of the storage device 370 include a volatile memory, a non-volatile memory, an HDD, an SSD, etc.

[0089] The management program PROG3 is a computer program executed by the processor 360. The functions of the control device 350 can also be realized by the cooperation of the processor 360 that executes the management program PROG3 and the storage device 370. The management program PROG3 is stored in the storage device 370. Alternatively, the management program PROG3 may also be recorded on a computer-readable recording medium. The management program PROG3 may also be provided via a network.

[0090] The storage device 370 also stores operator management information MGT-O and terminal management information MGT-T. The operator management information MGT-O is information for managing multiple remote operators O. For example, the operator management information MGT-O represents, for each remote operator O, an operator ID, a license, an availability, an allocation status, and a labor (work) history, etc. The terminal management information MGT-T is information for managing multiple remote operator terminals 200. For example, the terminal management information MGT-T represents, for each remote operator terminal 200, a terminal ID, specifications, an availability, an allocation status, and an operation (work) history, etc.

[0091] The control device 350 communicates with the mobile body 100 and the remote operator terminal 200 via the communication device 310. The control device 350 can also relay the communication between the mobile body 100 and the remote operator terminal 200. That is, the control device 350 can also relay the mobile body information MOV and the remote operation information OPE between the mobile body 100 and the remote operator terminal 200.

[0092] Before starting the remote operation of the mobile body 100, the control device 350 executes an "allocation process" of allocating an appropriate remote operator O and an appropriate remote operator terminal 200 to the remote operation of the mobile body 100. Hereinafter, the allocation process performed by the management system 300 (control device 350) will be described in further detail.

[0093] 3. Allocation Process

[0094] Figure 7 It is a conceptual diagram for explaining the outline of the allocation process performed by the management system 300. Hereinafter, the mobile body 100 that is the object of the remote operation will be referred to as the "object mobile body 100-X", the remote operator O allocated to the remote operation of the object mobile body 100-X will be referred to as the "first remote operator O-X", and the remote operator terminal 200 allocated to the remote operation of the object mobile body 100-X will be referred to as the "first remote operator terminal 200-X".

[0095] The management system 300 (control device 350) includes an allocation processing unit 400 that executes the allocation process. First, the allocation processing unit 400 receives a remote operation request REQ related to the object mobile body 100-X. For example, the remote operation request REQ is issued from the object mobile body 100-X itself. In this case, the object mobile body 100-X wirelessly communicates the remote operation request REQ to the management system 300. As another example, the remote operation request REQ can also be sent from a service operator cooperating with the remote operation system 1 to the management system 300.

[0096] The remote operation request REQ indicates the type of the object mobile body 100-X and the content of the desired remote operation. In response to the received remote operation request REQ, the allocation processing unit 400 allocates the first remote operator O-X and the first remote operator terminal 200-X to perform the remote operation of the object mobile body 100-X. More specifically, the allocation processing unit 400 allocates an appropriate first remote operator O-X among the multiple remote operators O to perform the remote operation of the object mobile body 100-X based on the remote operation request REQ and the operator management information MGT-O. In addition, the allocation processing unit 400 allocates an appropriate first remote operator terminal 200-X among the multiple remote operator terminals 200 to perform the remote operation of the object mobile body 100-X based on the remote operation request REQ and the terminal management information MGT-T.

[0097] In order to achieve appropriate allocation processing, the following three viewpoints are studied.

[0098] [Viewpoint 1] The status of each remote operator O and the status of each remote operator terminal 200.

[0099] [Viewpoint 2] The licenses held by each remote operator O and the required licenses for the remote operation of the object mobile body 100-X.

[0100] [Viewpoint 3] The terminal specifications of each remote operator terminal 200 and the required terminal specifications for the remote operation of the object mobile body 100-X.

[0101] The allocation processing unit 400 performs the allocation processing considering at least one of Viewpoint 1 to Viewpoint 3. The allocation processing unit 400 may also perform the allocation processing considering two or more of Viewpoint 1 to Viewpoint 3. Hereinafter, Viewpoint 1, Viewpoint 2, and Viewpoint 3 will be described in detail in Sections 4, 5, and 6 respectively.

[0102] 4. [Viewpoint 1] Allocation Processing Considering Operator Status and Terminal Status

[0103] Regarding the remote operator O, there may be a state suitable for remote operation and a state that is not. Assuming that the first remote operator O-X in a state not suitable for remote operation is selected, the accuracy of the remote operation will be reduced. Similarly, regarding the remote operator terminal 200, there may be a state suitable for remote operation and a state that is not. Assuming that the first remote operator terminal 200-X in a state not suitable for remote operation is used, the accuracy of the remote operation will be reduced.

[0104] Thus, in the first aspect, the allocation process is performed in consideration of the states of each remote operator O and each remote operator terminal 200. For this purpose, the states of each remote operator O and each remote operator terminal 200 are quantified in the form of "scores". A score is a quantitative parameter indicating the suitability (adaptability) for remote operation. For example, the score is calculated within the range of 0 to 100. The higher the score, the higher the suitability for remote operation. Then, based on this score, an appropriate first remote operator O-X and an appropriate first remote operator terminal 200-X are selected and assigned to the remote operation of the target moving body 100-X. Hereinafter, the first aspect will be described in more detail.

[0105] 4-1. Example of score

[0106] Figure 8 It is a block diagram showing an example of a functional structure related to score calculation.

[0107] 4-1-1. Example of operator score

[0108] The operator state acquisition unit 510 acquires operator state information STA-O indicating the state of the remote operator O. The operator score calculation unit 520 calculates an operator score SCR-O based on the operator state information STA-O. The operator score SCR-O indicates the suitability of the remote operator O for remote operation.

[0109] Figure 9 An example of the operator state and the operator score SCR-O is shown.

[0110] In the first example, the operator state acquisition unit 510 acquires the health state of the remote operator O. That is, the state of the remote operator O includes the health state of the remote operator O. For example, the remote operator O reports his / her own health state using the remote operator terminal 200 or his / her own terminal (e.g., smartphone). As another example, in order to detect the health state of the remote operator O, the operator sensor 240 can also be used. The operator sensor 240 can be included in the remote operator terminal 200 (refer to Figure 5) may also be included in the terminal (e.g., smartphone) owned by the remote operator O. The operator sensor 240 includes a biological sensor that detects the biological information of the remote operator O. Examples of the biological information include body temperature, heart rate, blood pressure, sweating amount, etc. The operator state acquisition unit 510 acquires the biological information detected by the operator sensor 240. Moreover, the operator state acquisition unit 510 acquires the health state of the remote operator O based on the biological information of the remote operator O. For example, the health state is obtained from the biological information by using a machine learning model. The operator score calculation unit 520 calculates the first operator score based on the health state of the remote operator O. As Figure 9 shown, the better the health state of the remote operator O, the higher the first operator score.

[0111] In the second example, the operator state acquisition unit 510 acquires the working hours of the remote operator O engaged in remote operation during a certain period in the past. That is to say, the state of the remote operator O includes the working hours of the remote operator O engaged in remote operation during a certain period in the past. This working hours can be obtained, for example, from the operation log recorded by the remote operator terminal 200 that the remote operator O has operated in the past. Or, the working hours can be obtained from the labor history records of each remote operator O included in the operator management information MGT-O. The operator score calculation unit 520 calculates the second operator score based on the working hours of the remote operator O. As Figure 9 shown, the longer the working hours of the remote operator O, the lower the second operator score.

[0112] In the third example, the operator state acquisition unit 510 acquires the hourly wage of the remote operator O. That is to say, the state of the remote operator O includes the hourly wage of the remote operator O. For example, the remote operator O uses the remote operator terminal 200 or his own terminal (e.g., smartphone) to register his own hourly wage. The hourly wage of each remote operator O can also be pre-registered in the operator management information MGT-O and can be obtained from the operator management information MGT-O. The operator score calculation unit 520 calculates the third operator score based on the hourly wage of the remote operator O. As Figure 9 shown, the lower the hourly wage of the remote operator O, the higher the third operator score.

[0113] Two or more of the above first to third examples can also be combined. In this case, the respective scores are added together.

[0114] Typically, the operator state acquisition unit 510 acquires the operator state information STA-O indicating the state of the remote operator O in real time. Moreover, the operator score calculation unit 520 calculates the operator score SCR-O based on the operator state information STA-O in real time.

[0115] The operator status acquisition unit 510 and the operator score calculation unit 520 may be included in the remote operator terminal 200, may be included in the management system 300, or may be distributed between the remote operator terminal 200 and the management system 300. When the operator status acquisition unit 510 is included in the management system 300, the remote operator terminal 200 sends the biological information of the remote operator O, information on working hours, etc. to the management system 300. When the operator status acquisition unit 510 and the operator score calculation unit 520 are included in the remote operator terminal 200, the remote operator terminal 200 only needs to send the information on the operator score SCR-O to the management system 300. In this case, since there is no need to send the biological information of the remote operator O, information on working hours, etc. to the management system 300, the communication volume between the remote operator terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0116] 4-1-2. Examples of Terminal Scores

[0117] The terminal status acquisition unit 530 acquires terminal status information STA-T indicating the status of the remote operator terminal 200. The terminal score calculation unit 540 calculates a terminal score SCR-T based on the terminal status information STA-T. The terminal score SCR-T indicates the suitability of the remote operator terminal 200 for remote operation.

[0118] Figure 10 Examples of the terminal status and the terminal score SCR-T are shown.

[0119] In the first example, the terminal status acquisition unit 530 acquires the communication status of the remote operator terminal 200. That is, the status of the remote operator terminal 200 includes the communication status of the remote operator terminal 200. As the communication status, communication speed (throughput), communication delay, etc. can be exemplified. The remote operator terminal 200 can measure the communication speed, communication delay, etc. based on the reception status of the data received from the communication partner. As another example, the remote operator terminal 200 measures the communication speed, communication delay, etc. based on the data sent to the communication partner and the feedback from the communication partner. The terminal status acquisition unit 530 acquires the communication status measured by the remote operator terminal 200. The terminal score calculation unit 540 calculates the first terminal score based on the communication status of the remote operator terminal 200. As Figure 10 shown, the better the communication status of the remote operator terminal 200, the higher the first terminal score.

[0120] In the second example, the terminal status acquisition unit 530 acquires the control device 250 of the remote operator terminal 200 (refer to Figure 5)The degree of abnormality. That is, the state of the remote operator terminal 200 includes the degree of abnormality of the control device 250 of the remote operator terminal 200. For example, the control device 250 has a self-diagnosis function. The terminal state acquisition unit 530 acquires the self-diagnosis result (normal, warning, abnormal) obtained by the control device 250 using the self-diagnosis function. The terminal score calculation unit 540 calculates the second terminal score based on the degree of abnormality of the control device 250 of the remote operator terminal 200. As Figure 10 shown, the higher the degree of abnormality of the control device 250 of the remote operator terminal 200, the lower the second terminal score.

[0121] It is also possible to combine the above first example and the second example. In this case, the respective scores are added together.

[0122] Typically, the terminal state acquisition unit 530 acquires the terminal state information STA-T indicating the state of the remote operator terminal 200 in real time. Moreover, the terminal score calculation unit 540 calculates the terminal score SCR-T based on the terminal state information STA-T in real time.

[0123] The terminal state acquisition unit 530 and the terminal score calculation unit 540 may be included in the remote operator terminal 200, may be included in the management system 300, or may be distributed between the remote operator terminal 200 and the management system 300. When the terminal state acquisition unit 530 is included in the management system 300, the remote operator terminal 200 sends information on the communication state and / or the degree of abnormality of the control device 250 to the management system 300. When the terminal state acquisition unit 530 and the terminal score calculation unit 540 are included in the remote operator terminal 200, the remote operator terminal 200 only needs to send information on the terminal score SCR-T to the management system 300. In this case, since there is no need to send information on the communication state and / or the degree of abnormality of the control device 250 of the remote operator terminal 200 to the management system 300, the communication volume between the remote operator terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0124] 4-1-3. Comprehensive Score

[0125] Figure 11 is a block diagram showing another example of the functional structure related to score calculation. The score integration unit 550 calculates the comprehensive score SCR by integrating the operator score SCR-O and the terminal score SCR-T. That is, the comprehensive score SCR reflects the operator score SCR-O and the terminal score SCR-T. This comprehensive score SCR indicates the suitability of the combination of the remote operator O and the remote operator terminal 200 for remote operation. The higher the comprehensive score SCR, the higher the suitability.

[0126] For example, the score integration unit 550 calculates the average of the operator score SCR-O and the terminal score SCR-T. When the average is less than 100, this average is used as the integrated score SCR. On the other hand, when the average is 100 or more, the integrated score SCR is uniformly set to 100.

[0127] The score integration unit 550 may be included in the remote operator terminal 200 or in the management system 300. It is also possible to include all of the operator status acquisition unit 510, the operator score calculation unit 520, the terminal status acquisition unit 530, the terminal score calculation unit 540, and the score integration unit 550 in the remote operator terminal 200. In this case, the remote operator terminal 200 only needs to send the information of the integrated score SCR to the management system 300. Therefore, the communication volume between the remote operator terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0128] 4-2. Assignment processing

[0129] Figure 12 It is a block diagram showing an example of a functional structure related to the assignment processing under the first aspect. The assignment processing unit 400 of the management system 300 includes a score acquisition unit 410 and a selection unit 450.

[0130] The score acquisition unit 410 acquires the operator score SCR-O from the operator score calculation unit 520. More specifically, the score acquisition unit 410 acquires the operator scores SCR-O of the respective multiple remote operators O represented by the operator management information MGT-O. The score acquisition unit 410 may also register the operator scores SCR-O of the respective multiple remote operators O in the operator management information MGT-O. Typically, the score acquisition unit 410 acquires the operator scores SCR-O of the respective multiple remote operators O in real time.

[0131] The selection unit 450 selects an appropriate first remote operator O-X from among a plurality of remote operators O based on the operator score SCR-O and the operator management information MGT-O. More specifically, the selection unit 450 extracts available (idle) remote operators O from among the plurality of remote operators O based on the operator management information MGT-O. Further, the selection unit 450 selects the first remote operator O-X from among the available remote operators O based on the operator score SCR-O. For example, the selection unit 450 preferentially selects a remote operator O with a high operator score SCR-O as the first remote operator O-X. That is, the operator score SCR-O is used as the operator priority. In the case where there are a plurality of candidates with the same operator score SCR-O, the selection unit 450 may also preferentially select a remote operator O with a longer waiting time as the first remote operator O-X. Moreover, the selected first remote operator O-X is assigned to perform the remote operation of the target moving body 100-X.

[0132] The score acquisition unit 410 acquires the terminal score SCR-T from the terminal score calculation unit 540. More specifically, the score acquisition unit 410 acquires the terminal score SCR-T of each of the plurality of remote operator terminals 200 indicated by the terminal management information MGT-T. The score acquisition unit 410 may also register the terminal score SCR-T of each of the plurality of remote operator terminals 200 in the terminal management information MGT-T. Typically, the score acquisition unit 410 acquires the terminal score SCR-T of each of the plurality of remote operator terminals 200 in real time.

[0133] The selection unit 450 selects an appropriate first remote operator terminal 200-X from among the plurality of remote operator terminals 200 based on the terminal score SCR-T and the terminal management information MGT-T. More specifically, the selection unit 450 extracts available (idle) remote operator terminals 200 from among the plurality of remote operator terminals 200 based on the terminal management information MGT-T. Further, the selection unit 450 selects the first remote operator terminal 200-X from among the available remote operator terminals 200 based on the terminal score SCR-T. For example, the selection unit 450 preferentially selects a remote operator terminal 200 with a high terminal score SCR-T as the first remote operator terminal 200-X. That is, the terminal score SCR-T is used as the terminal priority. In the case where there are a plurality of candidates with the same terminal score SCR-T, the selection unit 450 may also preferentially select a remote operator terminal 200 with a longer waiting time as the first remote operator terminal 200-X. Moreover, the selected first remote operator terminal 200-X is assigned to perform the remote operation of the target moving body 100-X.

[0134] Or, in Figure 11In the case of the example shown, the score acquisition unit 410 acquires the comprehensive score SCR from the score integration unit 550. More specifically, the score acquisition unit 410 acquires the comprehensive score SCR for each of multiple combinations of the remote operator O and the remote operator terminal 200. Typically, the score acquisition unit 410 acquires the comprehensive score SCR for each of multiple combinations of the remote operator O and the remote operator terminal 200 in real time.

[0135] The selection unit 450 selects an appropriate combination of the first remote operator O-X and the first remote operator terminal 200-X based on the comprehensive score SCR, the operator management information MGT-O, and the terminal management information MGT-T. More specifically, the selection unit 450 extracts available (idle) remote operators O from the multiple remote operators O based on the operator management information MGT-O. In addition, the selection unit 450 extracts available (idle) remote operator terminals 200 from the multiple remote operator terminals 200 based on the terminal management information MGT-T. Further, the selection unit 450 selects the first remote operator O-X from the available remote operators O and selects the first remote operator terminal 200-X from the available remote operator terminals 200 based on the comprehensive score SCR. For example, the selection unit 450 preferentially selects a combination of a remote operator O and a remote operator terminal 200 with a high comprehensive score SCR as the combination of the first remote operator O-X and the first remote operator terminal 200-X. That is, the comprehensive score SCR is used as the priority. In the case where there are multiple candidates with the same comprehensive score SCR, the selection unit 450 may also preferentially select a combination including a remote operator O with a longer waiting time. Moreover, the selected combination of the first remote operator O-X and the first remote operator terminal 200-X is assigned to perform the remote operation of the target moving body 100-X.

[0136] 4-3. Effects

[0137] As described above, according to the first aspect, based on the state of the remote operator O, the operator score SCR-O indicating the suitability of the remote operator O for the remote operation is calculated. Moreover, based on this operator score SCR-O, the first remote operator O-X assigned to the remote operation of the target moving body 100-X is selected. That is, considering the suitability, an appropriate first remote operator O-X is assigned to perform the remote operation of the target moving body 100-X. Thereby, the accuracy of the remote operation of the target moving body 100-X is ensured.

[0138] In addition, based on the state of the remote operator terminal 200, a terminal score SCR-T representing the suitability of the remote operator terminal 200 for remote operation is calculated. Moreover, based on this terminal score SCR-T, the first remote operator terminal 200-X assigned to the remote operation of the target moving body 100-X is selected. That is, considering the suitability, an appropriate first remote operator terminal 200-X is assigned to perform the remote operation of the target moving body 100-X. Thereby, the accuracy of the remote operation of the target moving body 100-X is ensured.

[0139] The score calculation can also be performed by the remote operator terminal 200. In this case, the remote operator terminal 200 only needs to send the score information to the management system 300. Therefore, the communication volume between the remote operator terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0140] 5. [Second perspective] Considering the allocation process of operator licenses

[0141] Consider the case where there are multiple moving bodies 100 as the objects of remote operation (refer to Figure 2 ). The remote operator O does not necessarily hold licenses (qualifications, licenses) related to all types of moving bodies 100. When a remote operator O who does not have a license to operate the target moving body 100-X is wrongly assigned to perform the remote operation of the target moving body 100-X, the allocation process needs to be restarted from the beginning, which is inefficient.

[0142] Therefore, in the second perspective, the allocation process is considered in view of the licenses held by each of the multiple remote operators O. More specifically, the first remote operator O-X who has the required license for the remote operation of the target moving body 100-X is assigned to perform the remote operation of the target moving body 100-X. Hereinafter, the second perspective will be described in more detail.

[0143] 5-1. Operator license information

[0144] Figure 13 An example of the operator license information LIC is shown. The operator license information LIC represents the licenses (qualifications, licenses) that each of the multiple remote operators O has for the operation (driving) of various moving bodies 100. In other words, the operator license information LIC represents the licenses held by each remote operator O. The license for a certain moving body 100 is a certificate officially permitting the operation of that moving body 100. Each remote operator O holds licenses for one or more moving bodies 100. It is also possible for a remote operator O to hold licenses for two or more moving bodies 100. The licenses held may be different for each remote operator O.

[0145] The operator license information LIC is included in the operator management information MGT-O. For example, each remote operator O uses a remote operator terminal 200 or their own terminal to declare the licenses they possess. The management system 300 pre-generates the operator license information LIC by collecting the license information held by each remote operator O, and updates it as needed.

[0146] 5-2. Allocation Processing

[0147] Figure 14 It is a block diagram showing an example of the functional structure related to the allocation processing under the second viewpoint. The allocation processing unit 400 of the management system 300 includes a required license acquisition unit 420 and a selection unit 450.

[0148] The required license acquisition unit 420 receives a remote operation request REQ. The remote operation request REQ indicates the type of the target moving body 100-X. The required license acquisition unit 420 can identify the "required license REQ-LIC" required for the remote operation of the target moving body 100-X based on the type of the target moving body 100-X.

[0149] The required ability information REQ-ABL represents the abilities required for the remote operation of the target moving body 100-X. The required ability information REQ-ABL includes at least the information of the required license REQ-LIC.

[0150] The selection unit 450 includes an operator selection unit 450-O. The operator selection unit 450-O selects an appropriate first remote operator O-X from multiple remote operators O based on the operator management information MGT-O and the required ability information REQ-ABL. More specifically, the operator selection unit 450-O extracts the available (idle) remote operators O from the multiple remote operators O based on the operator management information MGT-O. Furthermore, the operator selection unit 450-O obtains the operator license information LIC included in the operator management information MGT-O and the information of the required license REQ-LIC included in the required ability information REQ-ABL. Moreover, the operator selection unit 450-O selects the first remote operator O-X with the required license REQ-LIC from the available remote operators O based on the operator license information LIC and the required license REQ-LIC. The selected first remote operator O-X is assigned to perform the remote operation of the target moving body 100-X.

[0151] In the case where there are multiple remote operators O with a license requirement REQ-LIC, the operator selection unit 450-O may also consider the operator score SCR-O described in the above subsection 4 to select the first remote operator O-X. More specifically, the operator selection unit 450-O may also preferentially select a remote operator O with a high operator score SCR-O as the first remote operator O-X. In the case where there are multiple candidates with the same operator score SCR-O, the operator selection unit 450-O may also preferentially select a remote operator O with a longer waiting time as the first remote operator O-X.

[0152] As another example, in the case where there are multiple remote operators O with a license requirement REQ-LIC, the operator selection unit 450-O may also select a remote operator O with the fewest types of licenses held as the first remote operator O-X for this time. In this case, remote operators O with licenses for a greater variety of mobile bodies 100 will be retained. As a result, the probability of being able to appropriately assign a remote operator O to the target mobile body 100-X that requests remote operation is increased next.

[0153] 5-3. Effects

[0154] As described above, according to the second aspect, the license possessed by each remote operator O and the license requirement REQ-LIC required for the remote operation of the target mobile body 100-X are considered. Moreover, the first remote operator O-X with the license requirement REQ-LIC is assigned to perform the remote operation of the target mobile body 100-X. Thereby, the accuracy of the remote operation of the target mobile body 100-X is ensured. In addition, the situation where a remote operator O without the license requirement REQ-LIC is erroneously assigned to the target mobile body 100-X is prevented in advance. Therefore, there is no need to start the allocation process from scratch, and an efficient allocation process can be achieved.

[0155] 6. [Third Aspect] Allocation Process Considering Terminal Specifications

[0156] The specifications required by the remote operator terminal 200 may vary depending on the type of the target mobile body 100-X. In addition, the specifications required by the remote operator terminal 200 may also vary depending on the content of the requested remote operation. If a remote operator terminal 200 that does not meet the specifications required for the remote operation of the target mobile body 100-X is assigned to perform the remote operation of the target mobile body 100-X, the accuracy of the remote operation will be reduced. In addition, it is inefficient to start the allocation process from scratch.

[0157] Therefore, in the third aspect, the allocation process is performed in consideration of the specifications of each of the plurality of remote operator terminals 200. More specifically, the first remote operator terminal 200-X that meets the required terminal specifications for the remote operation of the target moving body 100-X is allocated to perform the remote operation of the target moving body 100-X. Hereinafter, the third aspect will be described in more detail.

[0158] 6-1. First Example

[0159] Figure 15 An example of the terminal specification information SPC is shown. The terminal specification information SPC represents the specifications of each of the plurality of remote operator terminals 200. In Figure 15 the example shown, the terminal specification information SPC represents, for each remote operator terminal 200, the type of the remote operator terminal 200, the type of the moving body 100 that can be handled (capable of handling), and the content of the remote operation (service) that can be handled.

[0160] As the type of the remote operator terminal 200, a cockpit-type terminal, a PC, a tablet computer, a smartphone, etc. can be exemplified. If the type of the remote operator terminal 200 changes, the type of the moving body 100 that can be handled and the content of the remote operation that can be handled also change accordingly. For example, a cockpit-type terminal is suitable for the remote operation of various types of vehicles and is also suitable for road driving and long-distance driving. On the other hand, a tablet computer is not necessarily suitable for road driving and long-distance driving. A tablet computer can be used for short-distance driving of a vehicle in a limited area such as a parking lot, automatic driving assistance such as shoulder evacuation (transfer to the roadside) in an emergency, remote operation of a small vehicle in a factory, etc.

[0161] The terminal specification information SPC is included in the terminal management information MGT-T. For example, the management system 300 collects information on its type from each of the plurality of remote operator terminals 200. Moreover, the management system 300 pre-generates Figure 15 the exemplified terminal specification information SPC based on the type of each remote operator terminal 200, and updates it as needed.

[0162] Figure 16 is a block diagram showing an example of a functional structure related to the allocation process in the third aspect. The allocation processing unit 400 of the management system 300 includes a required terminal specification acquisition unit 430 and a selection unit 450.

[0163] The required terminal specification acquisition unit 430 receives a remote operation request REQ. The remote operation request REQ represents at least one of "the type of the object moving body 100-X" and "the content of the desired remote operation of the object moving body 100-X". The required terminal specification acquisition unit 430 can identify the "required terminal specification REQ-SPC" required for the remote operation of the object moving body 100-X based on the remote operation request REQ. In this example, the remote operation request REQ is directly used as the required terminal specification REQ-SPC.

[0164] The required ability information REQ-ABL represents the ability required for the remote operation of the object moving body 100-X. The required ability information REQ-ABL includes at least the information of the required terminal specification REQ-SPC.

[0165] The selection unit 450 includes a terminal selection unit 450-T. The terminal selection unit 450-T selects an appropriate first remote operator terminal 200-X from multiple remote operator terminals 200 based on the terminal management information MGT-T and the required ability information REQ-ABL. More specifically, the terminal selection unit 450-T extracts the available (idle) remote operator terminals 200 from the multiple remote operator terminals 200 based on the terminal management information MGT-T. Furthermore, the terminal selection unit 450-T obtains the terminal specification information SPC included in the terminal management information MGT-T and the information of the required terminal specification REQ-SPC included in the required ability information REQ-ABL. Moreover, the terminal selection unit 450-T selects the first remote operator terminal 200-X that meets the required terminal specification REQ-SPC from the available remote operator terminals 200. The selected first remote operator terminal 200-X is assigned to perform the remote operation of the object moving body 100-X.

[0166] In the case where there are multiple remote operator terminals 200 that meet the required terminal specification REQ-SPC, the terminal selection unit 450-T may also consider the terminal score SCR-T described in the above subsection 4 to select the first remote operator terminal 200-X. More specifically, the terminal selection unit 450-T may preferentially select the remote operator terminal 200 with a higher terminal score SCR-T as the first remote operator terminal 200-X. In the case where there are multiple candidates with the same terminal score SCR-T, the terminal selection unit 450-T may also preferably select the remote operator terminal 200 with a longer waiting time as the first remote operator terminal 200-X.

[0167] 6-2. Second example

[0168] It is also possible to consider preparing dedicated remote operator terminals 200 (remote cockpits) for each type of moving body 100. On the other hand, if it is possible to remotely operate various types of moving bodies 100 with a single remote operator terminal 200, it is preferable from the viewpoints of cost reduction, reduction of the installation area, efficiency, convenience, etc. Thus, in the second example, a study is made on remotely operating various types of moving bodies 100 with a single remote operator terminal 200. For this purpose, the specifications (equipment) of the remote operator terminal 200 are re-captured more abstractly. More specifically, as described below, the specifications (equipment) of the remote operator terminal 200 are defined by "operating system capabilities" and "display system capabilities". In addition, descriptions that overlap with the above first example are appropriately omitted.

[0169] 6-2-1. Operating System Capabilities

[0170] The input device 230 of the remote operator terminal 200 (refer to Figure 5 ) includes remote operation components that are operated by the remote operator O when remotely operating the moving body 100. As remote operation components, a steering wheel (rudder wheel), an accelerator pedal, a brake pedal, a joystick, a cross key, a touch panel, etc. can be exemplified. Even if the physical structures of the remote operation components are different, the uses of the remote operation components are common. That is, the remote operation components are used for the forward and backward operations and the lateral operation of the target moving body 100-X. Therefore, a certain remote operation component can be replaced with another remote operation component.

[0171] For example, the forward operation, backward operation, and lateral operation of the joystick respectively correspond to the operation of the accelerator pedal, the operation of the brake pedal, and the operation of the steering wheel. As another example, it is also possible to reproduce the operating systems of various types of moving bodies 100 through a user interface (UI) displayed on the touch panel. In any case, the operation amount of a certain remote operation component can be converted into the operation amount of another remote operation component. Therefore, a certain remote operation component can be replaced with another remote operation component.

[0172] Based on the above viewpoints, the operating system capabilities of the remote operator terminal 200 are defined. Specifically, the operating system capabilities of the remote operator terminal 200 include "the number of inputs for the forward and backward operations of the moving body 100" and "the number of inputs for the lateral operation of the moving body 100". The operating system capabilities of the remote operator terminal 200 may also include "the presence or absence of operation feedback". "The presence or absence of operation feedback" means whether the remote operator terminal 200 has an operation reaction force mechanism that can generate an operation reaction force on the remote operator O.

[0173] Figure 17 It is a diagram showing an example of terminal specification information SPC related to the operating system and required terminal specifications REQ-SPC. AsFigure 17 As shown, the terminal specification information SPC includes operating system information representing the operating system capabilities of each of the multiple remote operator terminals 200. For example, the number of front and rear operation inputs of the remote operator terminal 200 equipped with an accelerator pedal and a brake pedal is "2". As another example, the number of front and rear operation inputs of the remote operator terminal 200 equipped with a joystick is "2". In the case of a tablet computer capable of displaying an arbitrary user interface on a touch panel, the operating system capabilities can be freely set.

[0174] The required terminal specification REQ-SPC includes the operating system capabilities required for the remote operation of the object moving body 100-X. As Figure 17 shown, if the type of the object moving body 100-X (e.g., vehicle, ship, construction machinery) is different, the required operating system capabilities also change. In addition, even if the type of the object moving body 100-X is the same, the required operating system capabilities may be different depending on the content of the desired remote operation (e.g., long-distance driving, shoulder avoidance).

[0175] The required terminal specification acquisition unit 430 receives the remote operation request REQ. The remote operation request REQ represents at least one of "the type of the object moving body 100-X" and "the content of the desired remote operation of the object moving body 100-X". The required terminal specification acquisition unit 430 can identify the required terminal specification REQ-SPC including the operating system capabilities required for the remote operation of the object moving body 100-X based on the remote operation request REQ.

[0176] For example, a conversion table 435 representing the correspondence between the remote operation request REQ and the required terminal specification REQ-SPC is prepared in advance (refer to Figure 16 ). The conversion table 435 is stored in advance in the storage device 370 of the management system 300. The required terminal specification acquisition unit 430 can identify the required terminal specification REQ-SPC corresponding to the remote operation request REQ by referring to the conversion table 435.

[0177] The terminal selection unit 450-T acquires the information of the terminal specification information SPC and the required terminal specification REQ-SPC. Moreover, the terminal selection unit 450-T selects the first remote operator terminal 200-X that meets the required terminal specification REQ-SPC from the available remote operator terminals 200 based on the terminal specification information SPC and the required terminal specification REQ-SPC. More specifically, the terminal selection unit 450-T selects the first remote operator terminal 200-X that at least has the required operating system capabilities based on the terminal specification information SPC (operating system information) and the required terminal specification REQ-SPC. The selected first remote operator terminal 200-X is assigned to perform the remote operation of the object moving body 100-X.

[0178] In the case where there are multiple remote operator terminals 200 having the required operating system capabilities, the terminal selection unit 450-T may also consider the terminal score SCR-T described in the above subsection 4 to select the first remote operator terminal 200-X. More specifically, the terminal selection unit 450-T may also preferentially select the remote operator terminal 200 with a high terminal score SCR-T as the first remote operator terminal 200-X. In the case where there are multiple candidates with the same terminal score SCR-T, the terminal selection unit 450-T may also preferentially select the remote operator terminal 200 with a longer waiting time as the first remote operator terminal 200-X.

[0179] As another example, in the case where there are multiple remote operator terminals 200 having the required operating system capabilities, the terminal selection unit 450-T may also select the remote operator terminal 200 having the minimum operating system capabilities as the first remote operator terminal 200-X for this time. In this case, the remote operator terminals 200 with higher operating system capabilities will be reserved. As a result, the probability of being able to allocate an appropriate remote operator terminal 200 is increased even when a higher operating system capability is required for the remote operation of the next target moving body 100-X.

[0180] As a variant, in a specific situation, the terminal selection unit 450-T may also select the remote operator terminal 200 that slightly fails to meet the required operating system capabilities as the first remote operator terminal 200-X. As a specific situation, the case where there are no other options in an emergency can be considered. For example, although operation feedback is preferably desired, in an emergency, the remote operator terminal 200 without operation feedback may also be selected as the first remote operator terminal 200-X. However, in this case, the terminal selection unit 450-T instructs function restrictions on the target moving body 100-X and the first remote operator terminal 200-X. As function restrictions, restrictions such as reducing the upper limit speed to a limit value lower than the default value and narrowing the steering range to a narrower range than the default range can be considered. The instructions for the target moving body 100-X and the first remote operator terminal 200-X are carried out through communication.

[0181] 6-2-2. Display system capabilities

[0182] The display device 220 of the remote operator terminal 200 (refer to Figure 5) There are also various types. For example, the number of monitors included in the display device 220 may vary for each remote operator terminal 200. Additionally, the resolution (size) of each monitor also varies. If the resolution (size) of the monitor is large, it is also possible to simultaneously display multiple types of images within one monitor. Additionally, when the control device 250 of the remote operator terminal 200 has a Picture-in-Picture function, it is also possible to embed other images within one image. Based on the above viewpoints, the display system capabilities of the remote operator terminal 200 are defined. Specifically, the display system capabilities of the remote operator terminal 200 include "the number of monitors" and "the number of images that can be simultaneously displayed".

[0183] Figure 18 It is a diagram showing an example of the terminal specification information SPC related to the display system and the required terminal specification REQ-SPC. As Figure 18 shown, the terminal specification information SPC includes display system information representing the display system capabilities of each of the multiple remote operator terminals 200.

[0184] The required terminal specification REQ-SPC includes the display system capabilities required for the remote operation of the object moving body 100-X. As Figure 18 shown, if the type of the object moving body 100-X (e.g., vehicle, construction machinery) is different, the required display system capabilities also change. Additionally, even if the type of the object moving body 100-X is the same, the required display system capabilities may be different depending on the content of the desired remote operation (e.g., chauffeur service, valet parking).

[0185] The required terminal specification acquisition unit 430 receives a remote operation request REQ. The remote operation request REQ indicates at least one of "the type of the object moving body 100-X" and "the content of the desired remote operation of the object moving body 100-X". The required terminal specification acquisition unit 430 can identify the required terminal specification REQ-SPC including the display system capabilities required for the remote operation of the object moving body 100-X based on the remote operation request REQ.

[0186] For example, a conversion table 435 showing the correspondence between the remote operation request REQ and the required terminal specification REQ-SPC is prepared in advance (refer to Figure 16 ). The conversion table 435 is pre-stored in the storage device 370 of the management system 300. The required terminal specification acquisition unit 430 can identify the required terminal specification REQ-SPC corresponding to the remote operation request REQ by referring to the conversion table 435.

[0187] The terminal selection unit 450-T acquires the terminal specification information SPC and the information of the required terminal specification REQ-SPC. Moreover, based on the terminal specification information SPC and the required terminal specification REQ-SPC, the terminal selection unit 450-T selects the first remote operator terminal 200-X that meets the required terminal specification REQ-SPC from the available remote operator terminals 200. More specifically, the terminal selection unit 450-T selects the first remote operator terminal 200-X that has at least the required display system capabilities based on the terminal specification information SPC (display system information) and the required terminal specification REQ-SPC. The selected first remote operator terminal 200-X is assigned to remotely operate the target moving body 100-X.

[0188] In the case where there are multiple remote operator terminals 200 that have the required display system capabilities, the terminal selection unit 450-T may also consider the terminal score SCR-T described in the above subsection 4 to select the first remote operator terminal 200-X. More specifically, the terminal selection unit 450-T may preferentially select the remote operator terminal 200 with a high terminal score SCR-T as the first remote operator terminal 200-X. In the case where there are multiple candidates with the same terminal score SCR-T, the terminal selection unit 450-T may also preferentially select the remote operator terminal 200 with a longer waiting time as the first remote operator terminal 200-X.

[0189] As another example, in the case where there are multiple remote operator terminals 200 that meet the required display system capabilities, the terminal selection unit 450-T may also select the remote operator terminal 200 with the minimum display system capabilities as the first remote operator terminal 200-X for this time. In this case, the remote operator terminals 200 with higher display system capabilities will be reserved. As a result, the probability of being able to assign an appropriate remote operator terminal 200 is increased even when a higher display system capability is required for the remote operation of the next target moving body 100-X.

[0190] As a modification example, in a specific situation, the terminal selection unit 450-T may also select a remote operator terminal 200 that slightly fails to meet the required display system capabilities as the first remote operator terminal 200-X. As a specific situation, a situation where there are no other options in an emergency can be considered. For example, although it is preferable to display two types of auxiliary images in addition to the three types of necessary images, in an emergency, a remote operator terminal 200 that can only display three types of images may be selected as the first remote operator terminal 200-X. However, in this case, the terminal selection unit 450-T instructs the object moving body 100-X and the first remote operator terminal 200-X of function restrictions. As function restrictions, a restriction value that reduces the upper limit speed to a value lower than the default value, a restricted range that narrows the steering range to a range narrower than the default range, etc. can be considered. The instruction to the object moving body 100-X and the first remote operator terminal 200-X is performed through communication.

[0191] In addition, the remote operator O can freely customize the display method of the image according to the content of the remote operation request REQ. The display method of the image includes layout, image display position, image size, and presence or absence of image display, etc. The remote operator O can also freely set the presence or absence of display of the auxiliary image. As the auxiliary image, it includes vehicle width lines, trajectory lines, maximum turning lines, and safe parking positions, etc.

[0192] 6-3. Effects

[0193] As described above, according to the third aspect, the specifications of each remote operator terminal 200 and the required terminal specifications REQ-SPC required for the remote operation of the object moving body 100-X are considered. Moreover, the first remote operator terminal 200-X that meets the required terminal specifications REQ-SPC is allocated to perform the remote operation of the object moving body 100-X. Thereby, the accuracy of the remote operation of the object moving body 100-X is ensured. In addition, the situation where a remote operator terminal 200 that does not have the required terminal specifications REQ-SPC is erroneously allocated to the object moving body 100-X is prevented in advance. Therefore, there is no need to re-perform the allocation process from the beginning, and an efficient allocation process can be achieved.

Claims

1. A remote operation system is a system for remote operation of a mobile object, characterized in that: comprising one or more processors, wherein the one or more processors are configured to: acquiring terminal status information indicating a status of a remote operator terminal configured to be used by a remote operator to perform remote operation; calculating a terminal score representing suitability of the remote operator terminal for the remote operation based on the terminal status information, The terminal score is used to select a first remote operator terminal to be assigned to the remote operation of the target moving object.

2. The remote operation system according to claim 1, characterized in that: the status of the remote operator terminal comprises a communication status of the remote operator terminal, The terminal score includes a first terminal score, and the first terminal score is higher as the communication state of the remote operator terminal is better.

3. The remote operation system according to claim 1, characterized in that: The state of the remote operator terminal includes a degree of abnormality of a control device of the remote operator terminal, The terminal score includes a second terminal score, and the second terminal score is lower as the abnormality degree is higher.

4. The remote operation system according to claim 1, characterized in that: The one or more processors are configured to: obtaining the terminal status information in real time, The terminal score is calculated in real time.

5. The remote operation system according to any one of claims 1 to 4, characterized in that: The one or more processors are configured to: obtaining operator status information indicating a status of the remote operator; calculating an operator score representing suitability of the remote operator for the remote operation based on the operator status information, The operator score is used to select a first remote operator to be assigned to the remote operation of the target moving object.

6. The remote operation system according to claim 5, characterized in that: The status of the remote operator includes a health status of the remote operator, The operator score includes a first operator score. The better the health status of the remote operator is, the higher the first operator score is.

7. The remote operation system according to claim 5, characterized in that: The state of the remote operator includes the working time of the remote operator in the remote operation in a certain period in the past, The operator score includes a second operator score, and the second operator score is lower as the working time is longer.

8. The remote operation system according to claim 5, characterized in that: The status of the remote operator includes an hourly wage of the remote operator, The operator score includes a third operator score, and the lower the hourly wage, the higher the third operator score.

9. The remote operation system according to claim 5, characterized in that: The one or more processors are configured to: Obtain the operator status information in real time, The operator score is calculated in real time.

10. The remote operation system according to claim 5, characterized in that: The one or more processors are configured to calculate a comprehensive score by combining the terminal score and the operator score, the comprehensive score represents the suitability of the combination of the remote operator terminal and the remote operator for the remote operation, The comprehensive score is used to select a combination of the first remote operator and the first remote operator terminal to be assigned to the remote operation of the target moving object.

11. The remote operation system according to claim 5, characterized in that: The one or more processors are configured to select a combination of the first remote operator and the first remote operator terminal to be assigned to the remote operation of the target moving object based on the terminal score and the operator score.

12. The remote operation system according to claim 11, characterized in that: The one or more processors are configured to: acquiring terminal specification information indicating specifications of each of the plurality of remote operator terminals, acquiring requested capability information indicating requested terminal specifications required for the remote operation of the target moving object, The combination of the first remote operator and the first remote operator terminal having the required terminal specification is selected based on the terminal specification information, the required terminal specification, the terminal score, and the operator score.

13. The remote operation system according to claim 11, characterized in that: There are multiple types of said moving bodies. The one or more processors are configured to: acquiring operator license information indicating the licenses that each of the plurality of remote operators has for operations on the plurality of types of mobile objects, acquiring requested capability information indicating a requested license required for the remote operation of the target moving object, The combination of the first remote operator having the required license and the first remote operator terminal is selected based on the operator license information, the required license, the terminal score, and the operator score.

14. The remote operation system according to claim 13, characterized in that: The required capability information indicates a required terminal specification required for the remote operation of the target moving object. The one or more processors are configured to: acquiring terminal specification information indicating specifications of each of the plurality of remote operator terminals, The combination of the first remote operator having the required license and the first remote operator terminal having the required terminal specification is selected based on the operator license information, the terminal specification information, the required capability information, the terminal score, and the operator score.

15. A remote operator terminal is a terminal used by a remote operator to remotely operate a mobile object, characterized in that: comprising one or more processors, wherein the one or more processors are configured to: obtaining terminal status information indicating the status of the remote operator terminal, calculating a terminal score representing suitability of the remote operator terminal for the remote operation based on the terminal status information, Information on the terminal score or information on the integrated score reflecting the terminal score is transmitted to a management system that selects a first remote operator and a first remote operator terminal assigned to the remote operation of the target moving object.

16. The remote operator terminal according to claim 15, characterized in that The one or more processors are configured to: obtaining operator status information indicating a status of the remote operator, calculating an operator score representing suitability of the remote operator for the remote operation based on the operator status information, The information of the operator score or the information of the comprehensive score reflecting the terminal score and the operator score is transmitted to the management system.

17. The remote operator terminal according to claim 16, characterized in that The one or more processors are configured to: calculating the comprehensive score indicating suitability of the combination of the remote operator terminal and the remote operator for the remote operation by combining the terminal score and the operator score, The information of the comprehensive score is sent to the management system.

18. A remote operation management method is a method for a computer to manage the remote operation of a mobile object, characterized in that: include: acquiring terminal status information indicating a status of a remote operator terminal configured to be used by a remote operator to perform the remote operation; calculating, based on the terminal status information, a terminal score representing suitability of the remote operator terminal for the remote operation; Based on the terminal score, a first remote operator terminal assigned to the remote operation of the target moving object is selected.

Images