Remote operation system, remote operation management method, and management program

Abstract

The invention provides a remote operation system, a remote operation management method and a management program. A remote operation system for remote operation of a moving body includes one or more processors configured to: acquire terminal specification information indicating a specification of each of a plurality of remote operation terminals; acquiring request capability information indicating at least a request terminal specification required for the remote operation of the target moving body; and assigning a first remote operation terminal satisfying the required terminal specification among the plurality of remote operation terminals to the remote operation of the target moving body on the basis of the terminal specification information and the required terminal specification.

Description

Technical Field

[0001] The present disclosure relates to a remote operation system, a remote operation management method, and a management program. Background Art

[0002] U.S. Patent Application Publication No. 2020 / 0062267 discloses a system related to remote operation of a vehicle. The system allocates a suitable remote operator to the remote operation of the vehicle based on the skills and experience values of the remote operator. Summary of the Invention

[0003] Consider the case where a remote operator remotely operates a moving body. The specifications required for the remote operation terminal used by the remote operator may vary depending on the type of the target moving body and the content of the required remote operation. If a remote operation terminal that does not meet the specifications required for the remote operation of the target moving body is allocated to the remote operation of the target moving body, the accuracy of the remote operation will be reduced.

[0004] The present disclosure provides a technology capable of ensuring the accuracy of remote operation of a moving body.

[0005] The remote operation system for remote operation of a moving body according to the first aspect of the present disclosure includes one or more processors configured to: obtain terminal specification information indicating the specifications of each of a plurality of remote operation terminals; obtain requirement ability information indicating at least a requirement terminal specification that is the specification required for the remote operation of the target moving body; and allocate, based on the terminal specification information and the requirement terminal specification, a first remote operation terminal that meets the requirement terminal specification among the plurality of remote operation terminals to the remote operation of the target moving body.

[0006] The remote operation management method according to the second aspect of the present disclosure is a remote operation management method for a computer to manage the remote operation of a moving body, including: obtaining terminal specification information indicating the specifications of each of a plurality of remote operation terminals; obtaining requirement ability information indicating at least the requirement terminal specification required for the remote operation of the target moving body; and allocating, based on the terminal specification information and the requirement terminal specification, a first remote operation terminal that meets the requirement terminal specification among the plurality of remote operation terminals to the remote operation of the target moving body.

[0007] The management program according to the third aspect of the present disclosure is executed by a computer to manage remote operations of a moving body, and includes: acquiring terminal specification information indicating the specifications of each of a plurality of remote operation terminals; acquiring requirement ability information indicating at least the required terminal specifications for the remote operation of the target moving body; and based on the terminal specification information and the required terminal specifications, allocating a first remote operation terminal that satisfies the required terminal specifications among the plurality of remote operation terminals to the remote operation of the target moving body.

[0008] According to the present disclosure, the specifications of each remote operation terminal and the required terminal specifications for the remote operation of the target moving body are considered. Moreover, a first remote operation terminal that satisfies the required terminal specifications is allocated to the remote operation of the target moving body. Thereby, the accuracy of the remote operation of the target moving body can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Hereinafter, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the drawings, in which like reference numerals denote like elements, and in which:

[0010] Figure 1 is a conceptual diagram for explaining the outline of the remote operation system.

[0011] Figure 2 is a diagram showing an example of the content of various remote operations required for various moving bodies.

[0012] Figure 3 is a conceptual diagram showing a configuration example of the remote operation system.

[0013] Figure 4 is a block diagram showing a configuration example of the moving body.

[0014] Figure 5 is a block diagram showing a configuration example of the remote operation terminal.

[0015] Figure 6 is a block diagram showing a configuration example of the management system.

[0016] Figure 7 is a conceptual diagram for explaining the outline of the allocation process performed by the management system.

[0017] Figure 8 is a block diagram showing an example of a functional configuration associated with score calculation.

[0018] Figure 9 is a conceptual diagram showing an example of the operator state and the operator score.

[0019] Figure 10 is a conceptual diagram showing an example of the terminal state and the terminal score.

[0020] Figure 11 It is a block diagram showing another example of the functional configuration associated with score calculation.

[0021] Figure 12 It is a block diagram showing an example of the functional configuration associated with the allocation process from the first perspective.

[0022] Figure 13 It is a conceptual diagram showing an example of operator license information.

[0023] Figure 14 It is a block diagram showing an example of the functional configuration associated with the allocation process from the second perspective.

[0024] Figure 15 It is a conceptual diagram showing an example of terminal specification information.

[0025] Figure 16 It is a block diagram showing an example of the functional configuration associated with the allocation process from the third perspective.

[0026] Figure 17 It is a conceptual diagram showing an example of terminal specification information related to the operating system and the required terminal specifications.

[0027] Figure 18 It is a conceptual diagram showing an example of terminal specification information related to the display system and the required terminal specifications. Detailed implementation mode

[0028] With reference to the accompanying drawings, the implementation mode of the present disclosure will be described.

[0029] 1. Outline of the remote operation system

[0030] Figure 1 It is a conceptual diagram for explaining the outline of the remote operation system 1 according to the present implementation mode. The remote operation system 1 is a system for remotely operating the moving body 100. The remote operation includes the concept of remote driving. The remote operation system 1 includes the moving body 100, the remote operation terminal 200, and the management system 300.

[0031] The moving body 100 is a movable moving body. The moving body 100 can also be manually operated by an operator riding on the moving body 100. The moving body 100 can also have an autonomous movement function. In any case, the moving body 100 is configured to be able to be remotely operated as needed. That is, the moving body 100 is the object of remote operation based on the remote operation system 1.

[0032] The types of the moving body 100 that becomes the object of remote operation are not limited to one, and may also be multiple. For example, the moving body 100 is a vehicle traveling on a public road (e.g., a passenger car, a truck, a bus, a MaaS vehicle, an autonomous vehicle, etc.). As another example, the moving body 100 may also be a vehicle used in a factory (e.g., a forklift, a factory cart, etc.). As still another example, the moving body 100 may also be a special small vehicle (e.g., a golf course cart, a personal mobility device, an electric wheelchair, etc.). As still another example, the moving body 100 may also be a construction machine (e.g., a power shovel, a bulldozer, etc.). As still another example, the moving body 100 may also be a robot (e.g., a logistics robot, a work robot, etc.). As still another example, the moving body 100 may also be an aircraft (e.g., a drone, etc.). As still another example, the moving body 100 may also be a ship (e.g., a small ship, a large yacht, etc.). As still another example, the moving body 100 may also be a ride in an amusement park (e.g., a kart, a moving body of an amusement facility, etc.).

[0033] Figure 2 Examples of the content of various remote operations required (requested) for various moving bodies 100 are shown. As Figure 2 illustrated, the moving body 100 is diverse, and the content of the required remote operation is also diverse. In addition, the remote operation can be requested either from the moving body 100 itself or from a service operator cooperating with the remote operation system 1. In the latter case, the content of the remote operation can also be said to be the content of the service provided by the service operator.

[0034] The remote operation terminal 200 is a terminal device used by the remote operator O when remotely operating the moving body 100. That is, the remote operation terminal 200 is configured to be used by the remote operator O for remotely operating the moving body 100. As the remote operation terminal 200, a cockpit-type terminal, a PC, a tablet computer, a smartphone, etc. can be exemplified. A single remote operation terminal 200 may also be configured to be able to handle remote operations of various types of moving bodies 100. Or, a single remote operation terminal 200 may also be dedicated to the remote operation of a specific moving body 100. In addition, the combination of the remote operator O and the remote operation terminal 200 may be predetermined or can be freely changed. That is, a single remote operation terminal 200 may be used only by a specific remote operator O, or may be used sequentially by various remote operators O.

[0035] The management system (manager) 300 manages the remote operation of the operating system 1. The management system 300 may also be composed of multiple servers that perform distributed processing. For example, the management system 300 manages multiple remote operators O and multiple remote operation terminals 200. In addition, in response to a remote operation request, the management system 300 assigns a remote operator O and a remote operation terminal 200 to the remote operation of the moving body 100. In addition, the management system 300 may also manage the state of the moving body 100 during remote operation. Details of the management system 300 will be described later.

[0036] The moving body 100, the remote operation terminal 200, and the management system 300 can communicate with each other via a communication network. For example, the moving body 100 can perform wireless communication with the remote operation terminal 200 and the management system 300 via a wireless communication network. The remote operation terminal 200 and the management system 300 can communicate with each other via a wired communication network or a wireless communication network. The moving body 100 and the remote operation terminal 200 can communicate either via the management system 300 or directly without passing through the management system 300.

[0037] The general flow of information during the remote operation of the moving body 100 is as follows.

[0038] Various sensors including a camera are mounted on the moving body 100. The camera captures an image of the surrounding conditions of the moving body 100. Through the camera, an image representing the surrounding conditions of the moving body 100 can be obtained. The moving body information MOV is the information obtained by various sensors and at least includes the image captured by the camera. The moving body information MOV may also include the position and state of the moving body 100 (for example: speed, steering angle, etc.). The moving body 100 sends the moving body information MOV to the remote operation terminal 200.

[0039] The remote operation terminal 200 acquires (receives) the moving body information MOV sent from the moving body 100. The remote operation terminal 200 presents the moving body information MOV to the remote operator O. Specifically, the remote operation terminal 200 is equipped with a display device and displays images and the like on the display device. The remote operator O observes the displayed information, identifies the surrounding conditions of the moving body 100, and performs the remote operation of the moving body 100. The remote operation information OPE is information related to the remote operation (steering operation, acceleration operation, deceleration operation, forward and backward movement operation, lateral movement operation, etc.) of the remote operator O. For example, the remote operation information OPE includes the operation amount input by the remote operator O. The remote operation information OPE can be said to be information reflecting the degree of the remote operation of the remote operator O. The remote operation terminal 200 sends the remote operation information OPE to the moving body 100.

[0040] The mobile body 100 acquires the remote operation information OPE sent from the remote operation terminal 200. The mobile body 100 performs mobile body control according to the acquired remote operation information OPE. In this way, the remote operation of the mobile body 100 can be realized.

[0041] Figure 3 It is a conceptual diagram showing a configuration example of the remote operation system 1. In Figure 3 the example shown, the remote operation management center is set in a predetermined site or a predetermined building. And, the management system 300 and a plurality of remote operation terminals 200 (200-1 to 200-N: N is an integer of 2 or more) are set in the remote operation management center. In addition, a plurality of remote operators O work in the remote operation management center. The management system 300 monitors and manages the states of the plurality of remote operation terminals 200 in the remote operation management center. In addition, the management system 300 monitors and manages the states of the plurality of remote operators O in the remote operation management center.

[0042] 2. Configuration

[0043] 2-1. Configuration example of the mobile body

[0044] Figure 4 It is a block diagram showing a configuration example of the mobile body 100. The mobile body 100 includes a communication device 110, a sensor group 120, one or more actuators 130, and a control device 150.

[0045] The communication device 110 performs wireless communication with the outside of the mobile body 100. For example, the communication device 110 performs wireless communication with the remote operation terminal 200 and the management system 300.

[0046] The sensor group 120 includes an identification sensor, a mobile body state sensor, a position sensor, etc. The identification sensor identifies (detects) the situation around the mobile body 100. As the identification sensor, a camera C, LIDAR (Laser Imaging Detection and Ranging), a radar, etc. can be exemplified. The mobile body state sensor detects the state of the mobile body 100. The mobile body state sensor includes a speed sensor, an acceleration sensor, a yaw rate sensor, a rudder angle sensor, etc. The position sensor detects the position and orientation of the mobile body 100. For example, the position sensor includes GNSS (Global Navigation Satellite System).

[0047] The actuator 130 moves the moving body 100. For example, the actuator 130 includes a longitudinal movement actuator for moving the moving body 100 forward and backward (accelerating and decelerating). As another example, the actuator 130 may also include a lateral movement actuator for moving the moving body 100 laterally. As yet another example, when the moving body 100 is provided with an arm, the actuator 130 may also include an arm actuator for moving the arm.

[0048] For example, when the moving body 100 is a normal vehicle, the actuator 130 includes a steering device, a driving device, and a braking device. The steering device steers the wheels (turns the steering). For example, the steering device includes an electric power steering (EPS) device. The driving device is a power source that generates a driving force. Examples of the driving device include an engine, an electric motor, an in-wheel motor, etc. The braking device generates a braking force.

[0049] The control device 150 is a computer that controls the moving body 100. The control device 150 includes one or more processors 160 (hereinafter, simply referred to as the processor 160) and one or more storage devices 170 (hereinafter, simply referred to as the storage device 170). The processor 160 performs various processes. Examples of the processor 160 include a general-purpose processor, a special-purpose processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), an integrated circuit, an existing type of circuit, and / or a combination thereof. The processor 160 may also be referred to as circuitry or processing circuitry. The circuitry is hardware that has been programmed to implement the described functions or hardware that executes the functions. The storage device 170 stores various information. Examples of the storage device 170 include a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc.

[0050] The control program PROG1 is a computer program executed by the processor 160. The functions of the control device 150 can be realized through the cooperation of the processor 160 that executes the control program PROG1 and the storage device 170. The control program PROG1 is stored in the storage device 170. Alternatively, the control program PROG1 may also be recorded on a computer-readable recording medium.

[0051] The control device 150 acquires driving environment information ENV showing the driving environment of the moving body 100. The driving environment information ENV is stored in the storage device 170.

[0052] The driving environment information ENV includes surrounding condition information indicating the recognition result of the recognition sensor. For example, the surrounding condition information includes the image captured by the camera C. In addition, the surrounding condition information may also include object information related to the objects around the moving body 100. Examples of the objects around the moving body 100 include pedestrians, other vehicles (preceding vehicles, parked vehicles, etc.), white lines, stop lines, traffic lights, signs, roadside structures, etc. The object information represents the relative position and relative speed of the object with respect to the moving body 100. For example, it is possible to recognize an object by analyzing the image obtained by the camera and calculate the relative position of the object. In addition, it is also possible to recognize an object based on the point cloud information obtained by the LIDAR and obtain the relative position and relative speed of the object.

[0053] In addition, the driving environment information ENV may also include moving body state information indicating the detection result of the moving body state sensor. The moving body state information represents the speed, acceleration (longitudinal acceleration, lateral acceleration), yaw rate, steering angle, etc. of the moving body 100.

[0054] Moreover, the driving environment information ENV may also include position information indicating the position and moving direction (azimuth) of the moving body 100. The position information is obtained by the position sensor. It is also possible to obtain highly accurate position information through self-position estimation processing (Localization) using map information and surrounding condition information (object information).

[0055] The control device 150 executes movement control for controlling the movement of the moving body 100. The movement control includes forward and backward movement control and lateral movement control. The control device 150 executes movement control by controlling the actuator 130.

[0056] The control device 150 may also execute autonomous movement control based on the driving environment information ENV. More specifically, the control device 150 generates a movement plan for the moving body 100 based on the driving environment information ENV. Moreover, the control device 150 generates a target trajectory required for the moving body 100 to travel according to the movement plan based on the driving environment information ENV. The target trajectory includes a target position and a target speed. And the control device 150 performs movement control in such a way that the moving body 100 follows the target trajectory.

[0057] In the case of remotely operating the moving body 100, the control device 150 communicates with the remote operation terminal 200 via the communication device 110.

[0058] The control device 150 transmits the moving body information MOV to the remote operation terminal 200. The moving body information MOV is information required for the remote operation of the moving body 100 by the remote operator O, and includes at least a part of the above-described driving environment information ENV. In particular, the moving body information MOV includes the image captured by the camera C. The moving body information MOV may also include other surrounding condition information. The moving body information MOV may also include moving body state information. The moving body information MOV may also include position information.

[0059] In addition, the control device 150 receives the remote operation information OPE from the remote operation terminal 200. The remote operation information OPE is information related to the remote operation (steering operation, acceleration operation, deceleration operation, forward and backward movement operation, lateral movement operation, etc.) of the remote operator O. For example, the remote operation information OPE includes the operation amount input by the remote operator O. The control device 150 performs movement control according to the received remote operation information OPE.

[0060] 2-2. Configuration example of remote operation terminal

[0061] Figure 5 is a block diagram showing a configuration example of the remote operation terminal 200. The remote operation terminal 200 includes a communication device 210, a display device 220, an input device 230, an operator sensor 240, and a control device 250.

[0062] The communication device 210 communicates with the moving body 100 and the management system 300.

[0063] The display device 220 displays various information to the remote operator O who performs the remote operation. In other words, the display device 220 presents various information to the remote operator O by displaying various information. Typically, the display device 220 is a display (monitor) such as a liquid crystal display or an organic EL display. The display device 220 may also be a touch panel.

[0064] The input device 230 accepts the input from the remote operator O. For example, the input device 230 includes remote operation components that are operated when the remote operator O remotely operates the moving body 100. As remote operation components, a steering wheel, an accelerator pedal, a brake pedal, a direction indicator, a joystick, a cross key, a switch, etc. can be exemplified. The remote operation component may also be a touch panel. The input device 230 may also include a keyboard, a mouse, a touch panel, etc. other than the remote operation components.

[0065] The operator sensor 240 is a sensor for monitoring the state of the remote operator O. For example, the operator sensor 240 includes a biological sensor that detects biological information of the remote operator O. As the biological information, body temperature, heart rate, blood pressure, sweating amount, etc. can be exemplified.

[0066] The control device 250 controls the remote operation terminal 200. The control device 250 includes one or more processors 260 (hereinafter simply referred to as the processor 260) and one or more storage devices 270 (hereinafter simply referred to as the storage device 270). The processor 260 performs various processes. As the processor 260, a CPU, GPU, ASIC, FPGA, etc. can be exemplified. Examples of the processor 260 include a general-purpose processor, a special-purpose processor, a CPU, a GPU, an ASIC, an FPGA, an integrated circuit, an existing type of circuit, and / or a combination thereof. The processor 260 may also be referred to as a circuit or a processing circuit. A circuit is hardware that has been programmed to implement the described functions or hardware that executes functions. The storage device 270 stores various information. As the storage device 270, a volatile memory, a non-volatile memory, an HDD, an SSD, etc. can be exemplified.

[0067] The control program PROG2 is a computer program executed by the processor 260. The functions of the control device 250 can be realized by the cooperation of the processor 260 that executes the control program PROG2 and the storage device 270. The control program PROG2 is stored in the storage device 270. Alternatively, the control program PROG2 may also be recorded on a computer-readable recording medium. The control program PROG2 may also be provided via a network.

[0068] The control device 250 communicates with the mobile body 100 via the communication device 210. The control device 250 receives the mobile body information MOV sent from the mobile body 100. The control device 250 presents the mobile body information MOV including an image to the remote operator O by displaying it on the display device 220. The remote operator O can identify the state of the mobile body 100 and the surrounding conditions based on the mobile body information MOV displayed on the display device 220.

[0069] The remote operator O operates the remote operation part of the input device 230. The operation amount of the remote operation part is detected by a sensor provided on the remote operation part. The control device 250 generates remote operation information OPE that reflects the operation amount of the remote operation part of the remote operator O. The remote operation information OPE can be said to be information that reflects the degree of driving operation performed by the remote operator O. And, the control device 250 sends the remote operation information OPE to the mobile body 100 via the communication device 210.

[0070] 2-3. Configuration example of the management system

[0071] Figure 6 It is a block diagram showing a configuration example of the management system 300. The management system 300 includes a communication device 310 and a control device 350.

[0072] The communication device 310 communicates with the moving body 100 and the remote operation terminal 200.

[0073] The control device 350 controls the management system 300. The control device 350 includes one or more processors 360 (hereinafter simply referred to as the processor 360) and one or more storage devices 370 (hereinafter simply referred to as the storage device 370). The processor 360 performs various processes. Examples of the processor 360 include a general-purpose processor, a special-purpose processor, a CPU, a GPU, an ASIC, an FPGA, an integrated circuit, an existing type of circuit, and / or a combination thereof. The processor 360 may also be referred to as a circuit or a processing circuit. A circuit is hardware that has been programmed to implement the described functions or hardware that executes functions. The storage device 370 stores various information. Examples of the storage device 370 include a volatile memory, a non-volatile memory, an HDD, an SSD, etc.

[0074] The management program PROG3 is a computer program executed by the processor 360. The functions of the control device 350 can be realized through the cooperation of the processor 360 that executes the management program PROG3 and the storage device 370. The management program PROG3 is stored in the storage device 370. Alternatively, the management program PROG3 may also be recorded on a computer-readable recording medium. The management program PROG3 may also be provided via a network.

[0075] The operator management information MGT-O and the terminal management information MGT-T are also stored in the storage device 370. The operator management information MGT-O is information for managing a plurality of remote operators O. For example, the operator management information MGT-O shows the operator ID, the held license, the usage status (availability), the allocation status, the work history, etc. for each remote operator O. The terminal management information MGT-T is information for managing a plurality of remote operation terminals 200. For example, the terminal management information MGT-T shows the terminal ID, the specifications, the usage status (availability), the allocation status, the operation history, etc. for each remote operation terminal 200.

[0076] The control device 350 communicates with the moving body 100 and the remote operation terminal 200 via the communication device 310. The control device 350 may also relay the communication between the moving body 100 and the remote operation terminal 200. That is, the control device 350 may relay the moving body information MOV and the remote operation information OPE between the moving body 100 and the remote operation terminal 200.

[0077] Before starting the remote operation of the moving body 100, the control device 350 executes an "assignment process" for assigning a remote operator O suitable for the remote operation of the moving body 100 and a suitable remote operation terminal 200. Hereinafter, the assignment process performed by the management system 300 (control device 350) will be described in further detail.

[0078] 3. Assignment Process

[0079] Figure 7 It is a conceptual diagram for explaining the outline of the assignment process performed by the management system 300. Hereinafter, the moving body 100 that is the object of remote operation will be referred to as the "object moving body 100-X". Hereinafter, the remote operator O assigned to the remote operation of the object moving body 100-X will be referred to as the "first remote operator O-X". Hereinafter, the remote operation terminal 200 assigned to the remote operation of the object moving body 100-X will be referred to as the "first remote operation terminal 200-X".

[0080] The management system 300 (control device 350) includes an assignment processing unit 400 that executes the assignment process. First, the assignment processing unit 400 acquires a remote operation request REQ related to the object moving body 100-X. For example, the remote operation request REQ is issued from the object moving body 100-X itself. In this case, the object moving body 100-X transmits the remote operation request REQ to the management system 300 via wireless communication. As another example, the remote operation request REQ may also be transmitted from a service provider cooperating with the remote operation system 1 to the management system 300.

[0081] The remote operation request REQ shows the type of the object moving body 100-X and the content of the desired remote operation. In response to the acquired remote operation request REQ, the assignment processing unit 400 assigns the first remote operator O-X and the first remote operation terminal 200-X to the remote operation of the object moving body 100-X. More specifically, the assignment processing unit 400 assigns a suitable first remote operator O-X among the multiple remote operators O to the remote operation of the object moving body 100-X based on the remote operation request REQ and the operator management information MGT-O. In addition, the assignment processing unit 400 assigns a suitable first remote operation terminal 200-X among the multiple remote operation terminals 200 to the remote operation of the object moving body 100-X based on the remote operation request REQ and the terminal management information MGT-T.

[0082] In order to achieve a suitable assignment process, the following three viewpoints are studied.

[0083] [First Viewpoint] The status of each remote operator O and the status of each remote operation terminal 200.

[0084] [Second Viewpoint] Permits held by each remote operator O and required permits for remote operation of the object moving body 100-X.

[0085] [Third Viewpoint] Terminal specifications of each remote operation terminal 200 and required terminal specifications for remote operation of the object moving body 100-X.

[0086] The allocation processing unit 400 performs allocation processing considering at least one of the first to third viewpoints. The allocation processing unit 400 may also perform allocation processing considering two or more of the first to third viewpoints. Hereinafter, the first, second, and third viewpoints will be described in detail in Sections 4, 5, and 6, respectively.

[0087] 4. [First Viewpoint] Allocation Processing Considering Operator State and Terminal State

[0088] Regarding the remote operator O, there may be a state suitable for remote operation and a state unsuitable for remote operation. If we assume the first remote operator O-X in a state unsuitable for remote operation, the accuracy of remote operation will decrease. Similarly, regarding the remote operation terminal 200, there may also be a state suitable for remote operation and a state unsuitable for remote operation. If we assume using the first remote operation terminal 200-X in a state unsuitable for remote operation, the accuracy of remote operation will decrease.

[0089] Therefore, in the first viewpoint, allocation processing is performed considering the state of each remote operator O and the state of each remote operation terminal 200. For this purpose, the state of each remote operator O and the state of each remote operation terminal 200 are quantified in the form of "scores". A score is a quantitative parameter indicating the suitability for remote operation. For example, the score is calculated within the range of 0 to 100. The higher the score, the higher the suitability for remote operation. And based on this score, a suitable first remote operator O and a suitable first remote operation terminal 200-X are selected and allocated to the remote operation of the object moving body 100-X. Hereinafter, the first viewpoint will be described in more detail.

[0090] 4-1. Examples of Scores

[0091] Figure 8 It is a block diagram showing an example of the functional configuration associated with score calculation.

[0092] 4-1-1. Examples of Operator Scores

[0093] The operator status acquisition unit 510 acquires operator status information STA-O indicating the status of the remote operator O. The operator score calculation unit 520 calculates an operator score SCR-O based on the operator status information STA-O. The operator score SCR-O indicates the suitability of the remote operator O for remote operation.

[0094] Figure 9 An example of the operator status and the operator score SCR-O is shown.

[0095] In the first example, the operator status acquisition unit 510 acquires the health status of the remote operator O. That is, the status of the remote operator O includes the health status of the remote operator O. For example, the remote operator O uses the remote operation terminal 200 or their own terminal (e.g., a smartphone) to report their own health status. As another example, in order to detect the health status of the remote operator O, the operator sensor 240 can also be used. The operator sensor 240 can be included either in the remote operation terminal 200 (refer to Figure 5 ) or in the terminal (e.g., a smartphone) held by the remote operator O. The operator sensor 240 includes a biological sensor that detects biological information of the remote operator O. Examples of the biological information include body temperature, heart rate, blood pressure, sweating amount, etc. The operator status acquisition unit 510 acquires the biological information detected by the operator sensor 240. And, the operator status acquisition unit 510 acquires the health status of the remote operator O based on the biological information of the remote operator O. For example, the health status is obtained from the biological information by using a machine learning model. The operator score calculation unit 520 calculates the first operator score based on the health status of the remote operator O. As Figure 9 shown, the better the health status of the remote operator O, the higher the first operator score.

[0096] In the second example, the operator status acquisition unit 510 acquires the working hours of the remote operator O engaged in remote operation during a certain past period. That is, the status of the remote operator O includes the working hours of the remote operator O engaged in remote operation during a certain past period. This working hours can be obtained, for example, from the operation log recorded in the remote operation terminal 200 that the remote operator O operated in the past. Or, the working hours are obtained from the labor history records of each remote operator O included in the operator management information MGT-O. The operator score calculation unit 520 calculates the second operator score based on the working hours of the remote operator O. As Figure 9 shown, the longer the working hours of the remote operator O, the lower the second operator score.

[0097] In the third example, the operator status acquisition unit 510 acquires the hourly wage of the remote operator O. That is, the status of the remote operator O includes the hourly wage of the remote operator O. For example, the remote operator O uses the remote operation terminal 200 or their own terminal (e.g., a smartphone) to register their hourly wage. The hourly wage of each remote operator O can also be pre-registered in the operator management information MGT-O and acquired from the operator management information MGT-O. The operator score calculation unit 520 calculates the third operator score based on the hourly wage of the remote operator O. As Figure 9 shown, the lower the hourly wage of the remote operator O, the higher the third operator score.

[0098] Combinations of two or more of the above first to third examples can also be performed. In this case, the respective scores are added.

[0099] Typically, the operator status acquisition unit 510 acquires the operator status information STA-O indicating the status of the remote operator O in real time. And the operator score calculation unit 520 calculates the operator score SCR-O based on the operator status information STA-O in real time.

[0100] The operator status acquisition unit 510 and the operator score calculation unit 520 can be included in the remote operation terminal 200, can be included in the management system 300, or can be dispersed between the remote operation terminal 200 and the management system 300. When the operator status acquisition unit 510 is included in the management system 300, the remote operation terminal 200 sends information such as the biological information and working hours information of the remote operator O to the management system 300. When the operator status acquisition unit 510 and the operator score calculation unit 520 are included in the remote operation terminal 200, the remote operation terminal 200 only needs to send the information of the operator score SCR-O to the management system 300. In this case, since there is no need to send information such as the biological information and working hours information of the remote operator O to the management system 300, the communication volume between the remote operation terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0101] 4-1-2. Examples of Terminal Scores

[0102] The terminal status acquisition unit 530 acquires the terminal status information STA-T indicating the status of the remote operation terminal 200. The terminal score calculation unit 540 calculates the terminal score SCR-T based on the terminal status information STA-T. The terminal score SCR-T indicates the suitability of the remote operation terminal 200 for remote operation.

[0103] Figure 10 Examples of the terminal status and the terminal score SCR-T are shown.

[0104] In the first example, the terminal state acquisition unit 530 acquires the communication state of the remote operation terminal 200. That is, the state of the remote operation terminal 200 includes the communication state of the remote operation terminal 200. As the communication state, communication speed (throughput), communication delay, etc. can be exemplified. The remote operation terminal 200 can measure the communication speed, communication delay, etc. based on the reception state of the data received from the communication object. As another example, the remote operation terminal 200 measures the communication speed, communication delay, etc. based on the data sent to the communication object and the feedback from the communication object. The terminal state acquisition unit 530 acquires the communication state measured by the remote operation terminal 200. The terminal score calculation unit 540 calculates the first terminal score based on the communication state of the remote operation terminal 200. As Figure 10 shown, the better the communication state of the remote operation terminal 200, the higher the first terminal score.

[0105] In the second example, the terminal state acquisition unit 530 acquires the abnormality degree of the control device 250 (refer to Figure 5 ) of the remote operation terminal 200. That is, the state of the remote operation terminal 200 includes the abnormality degree of the control device 250 of the remote operation terminal 200. For example, the control device 250 has a self-diagnosis function. The terminal state acquisition unit 530 acquires the self-diagnosis result (normal, warning, abnormal) based on the self-diagnosis function of the control device 250. The terminal score calculation unit 540 calculates the second terminal score based on the abnormality degree of the control device 250 of the remote operation terminal 200. As Figure 10 shown, the higher the abnormality degree of the control device 250 of the remote operation terminal 200, the lower the second terminal score.

[0106] It is also possible to perform a combination of the first example and the second example above. In this case, the respective scores are added.

[0107] Typically, the terminal state acquisition unit 530 acquires the terminal state information STA-T indicating the state of the remote operation terminal 200 in real time. And the terminal score calculation unit 540 calculates the terminal score SCR-T in real time based on the terminal state information STA-T.

[0108] The terminal status acquisition unit 530 and the terminal score calculation unit 540 can be included in the remote operation terminal 200, can be included in the management system 300, or can be dispersed between the remote operation terminal 200 and the management system 300. When the terminal status acquisition unit 530 is included in the management system 300, the remote operation terminal 200 sends information on the communication status and the abnormality degree of the control device 250 to the management system 300. When the terminal status acquisition unit 530 and the terminal score calculation unit 540 are included in the remote operation terminal 200, the remote operation terminal 200 only needs to send information on the terminal score SCR-T to the management system 300. In this case, since there is no need to send information on the communication status of the remote operation terminal 200 and the abnormality degree of the control device 250 to the management system 300, the communication volume between the remote operation terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0109] 4-1-3. Integrated Score

[0110] Figure 11 It is a block diagram showing another example of the functional configuration related to score calculation. The score integration unit 550 calculates the integrated score SCR by integrating the operator score SCR-O and the terminal score SCR-T. That is, the operator score SCR-O and the terminal score SCR-T are reflected in the integrated score SCR. This integrated score SCR represents the suitability of the combination of the remote operator O and the remote operation terminal 200 for remote operation. The higher the integrated score SCR, the higher the suitability.

[0111] For example, the score integration unit 550 calculates the average value of the operator score SCR-O and the terminal score SCR-T. When the average value is less than 100, this average value is used as the integrated score SCR. On the other hand, when the average value is 100 or more, the integrated score SCR is uniformly set to 100.

[0112] The score integration unit 550 can be included in the remote operation terminal 200 or can be included in the management system 300. It can also be that the operator status acquisition unit 510, the operator score calculation unit 520, the terminal status acquisition unit 530, the terminal score calculation unit 540, and the score integration unit 550 are all included in the remote operation terminal 200. In this case, the remote operation terminal 200 only needs to send information on the integrated score SCR to the management system 300. Therefore, the communication volume between the remote operation terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0113] 4-2. Allocation Processing

[0114] Figure 12It is a block diagram showing a functional configuration example related to the allocation process under the first viewpoint. The allocation processing unit 400 of the management system 300 includes a score acquisition unit 410 and a selection unit 450.

[0115] The score acquisition unit 410 acquires the operator score SCR-O from the operator score calculation unit 520. More specifically, the score acquisition unit 410 acquires the operator score SCR-O of each of the multiple remote operators O shown by the operator management information MGT-O. The score acquisition unit 410 may also register the operator score SCR-O of each of the multiple remote operators O in the operator management information MGT-O. Typically, the score acquisition unit 410 acquires the operator score SCR-O of each of the multiple remote operators O in real time.

[0116] The selection unit 450 selects a suitable first remote operator O-X from the multiple remote operators O based on the operator score SCR-O and the operator management information MGT-O. More specifically, the selection unit 450 extracts the available (idle) remote operators O from the multiple remote operators O based on the operator management information MGT-O. Moreover, the selection unit 450 selects the first remote operator O-X from the available remote operators O based on the operator score SCR-O. For example, the selection unit 450 preferentially selects the remote operator O with a high operator score SCR-O as the first remote operator O-X. That is, the operator score SCR-O is used as the operator priority. In the case where there are multiple candidates with the same operator score SCR-O, the selection unit 450 may also preferentially select the remote operator O with a longer waiting time as the first remote operator O-X. And, the selected first remote operator O-X is assigned to the remote operation of the target moving body 100-X.

[0117] The score acquisition unit 410 acquires the terminal score SCR-T from the terminal score calculation unit 540. More specifically, the score acquisition unit 410 acquires the terminal score SCR-T of each of the multiple remote operation terminals 200 shown by the terminal management information MGT-T. The score acquisition unit 410 may also register the terminal score SCR-T of each of the multiple remote operation terminals 200 in the terminal management information MGT-T. Typically, the score acquisition unit 410 acquires the terminal score SCR-T of each of the multiple remote operation terminals 200 in real time.

[0118] The selection unit 450 selects a suitable first remote operation terminal 200-X from multiple remote operation terminals 200 based on the terminal score SCR-T and the terminal management information MGT-T. More specifically, the selection unit 450 extracts the available (idle) remote operation terminals 200 from the multiple remote operation terminals 200 based on the terminal management information MGT-T. Moreover, the selection unit 450 selects the first remote operation terminal 200-X from the available remote operation terminals 200 based on the terminal score SCR-T. For example, the selection unit 450 preferentially selects the remote operation terminal 200 with a high terminal score SCR-T as the first remote operation terminal 200-X. That is to say, the terminal score SCR-T is used as the terminal priority. In the case where there are multiple candidates with the same terminal score SCR-T, the selection unit 450 may also preferentially select the remote operation terminal 200 with a longer waiting time as the first remote operation terminal 200-X. And, the selected first remote operation terminal 200-X is assigned to the remote operation of the target moving body 100-X.

[0119] Alternatively, in Figure 11 In the case of the example shown in, the score acquisition unit 410 acquires the integrated score SCR from the score integration unit 550. More specifically, the score acquisition unit 410 acquires the integrated score SCR for each of the multiple combinations of the remote operator O and the remote operation terminal 200. Typically, the score acquisition unit 410 acquires the integrated score SCR for each of the multiple combinations of the remote operator O and the remote operation terminal 200 in real time.

[0120] The selection unit 450 selects a suitable combination of the first remote operator O-X and the first remote operation terminal 200-X based on the integrated score SCR, the operator management information MGT-O, and the terminal management information MGT-T. More specifically, the selection unit 450 extracts available (idle) remote operators O from among the multiple remote operators O based on the operator management information MGT-O. In addition, the selection unit 450 extracts available (idle) remote operation terminals 200 from among the multiple remote operation terminals 200 based on the terminal management information MGT-T. Furthermore, the selection unit 450 selects the first remote operator O-X from the available remote operators O and selects the first remote operation terminal 200-X from the available remote operation terminals 200 based on the integrated score SCR. For example, the selection unit 450 preferentially selects a combination of a remote operator O and a remote operation terminal 200 with a high integrated score SCR as the combination of the first remote operator O-X and the first remote operation terminal 200-X. That is, the integrated score SCR is used as the priority. In the case where there are multiple candidates with the same integrated score SCR, the selection unit 450 may also preferentially select a combination including a remote operator O with a longer waiting time. And the selected combination of the first remote operator O-X and the first remote operation terminal 200-X is assigned to the remote operation of the target moving body 100-X.

[0121] 4-3. Effects

[0122] As described above, according to the first aspect, based on the state of the remote operator O, the operator score SCR-O indicating the suitability of the remote operator O for remote operation is calculated. And based on this operator score SCR-O, the first remote operator O-X assigned to the remote operation of the target moving body 100-X is selected. That is, considering the suitability, a suitable first remote operator O-X is assigned to the remote operation of the target moving body 100-X. Thereby, the accuracy of the remote operation of the target moving body 100-X can be ensured.

[0123] In addition, based on the state of the remote operation terminal 200, the terminal score SCR-T indicating the suitability of the remote operation terminal 200 for remote operation is calculated. And based on this terminal score SCR-T, the first remote operation terminal 200-X assigned to the remote operation of the target moving body 100-X is selected. That is, considering the suitability, a suitable first remote operation terminal 200-X is assigned to the remote operation of the target moving body 100-X. Thereby, the accuracy of the remote operation of the target moving body 100-X can be ensured.

[0124] The score calculation can be performed by the remote operation terminal 200. In this case, the remote operation terminal 200 only needs to send score information to the management system 300. Therefore, the communication volume between the remote operation terminal 200 and the management system 300 can be reduced. This helps to reduce the communication resources used.

[0125] 5. [Second perspective] Considered the allocation process of operator licenses

[0126] Consider the case where there are multiple types of mobile bodies 100 as the objects of remote operation (refer to Figure 2 ). The remote operator O does not necessarily hold licenses (qualifications, licenses) for all types of mobile bodies 100. If the remote operator O without the license to operate the target mobile body 100-X is wrongly assigned to the remote operation of the target mobile body 100-X, the allocation process needs to be restarted from the beginning, resulting in inefficiency.

[0127] Therefore, in the second perspective, the allocation process is considered based on the licenses held by each of the multiple remote operators O. More specifically, the first remote operator O-X with the required license for the remote operation of the target mobile body 100-X is assigned to the remote operation of the target mobile body 100-X. Hereinafter, the second perspective will be described in more detail.

[0128] 5-1. Operator license information

[0129] Figure 13 An example of the operator license information LIC is shown. The operator license information LIC shows the licenses (qualifications, licenses) held by each of the multiple remote operators O for the operation (driving) of multiple types of mobile bodies 100. In other words, the operator license information LIC shows the licenses held by each remote operator O. The license for a certain mobile body 100 is a license officially permitting the operation of that mobile body 100. Each remote operator O holds licenses for one or more types of mobile bodies 100. It is also possible that one remote operator O holds licenses for two or more types of mobile bodies 100. The licenses held may vary for each remote operator O.

[0130] The operator license information LIC is included in the operator management information MGT-O. For example, each remote operator O uses the remote operation terminal 200 or their own terminal to report the licenses they hold. The management system 300 pre-generates the operator license information LIC by collecting the information on the licenses held by each remote operator O and updates it as needed.

[0131] 5-2. Allocation process

[0132] Figure 14It is a block diagram showing a functional configuration example related to the allocation process under the second viewpoint. The allocation processing unit 400 of the management system 300 includes a required license acquisition unit 420 and a selection unit 450.

[0133] The required license acquisition unit 420 acquires a remote operation request REQ. The remote operation request REQ indicates the type of the target moving body 100-X. The required license acquisition unit 420 can identify the "required license REQ-LIC" required for the remote operation of the target moving body 100-X based on the type of the target moving body 100-X.

[0134] The required ability information REQ-ABL indicates the abilities required for the remote operation of the target moving body 100-X. The required ability information REQ-ABL includes at least the information of the required license REQ-LIC.

[0135] The selection unit 450 includes an operator selection unit 450-O. The operator selection unit 450-O selects a suitable first remote operator O-X from multiple remote operators O based on the operator management information MGT-O and the required ability information REQ-ABL. More specifically, the operator selection unit 450-O extracts the available (idle) remote operators O from the multiple remote operators O based on the operator management information MGT-O. Furthermore, the operator selection unit 450-O acquires the operator license information LIC included in the operator management information MGT-O and the information of the required license REQ-LIC included in the required ability information REQ-ABL. And, the operator selection unit 450-O selects the first remote operator O-X having the required license REQ-LIC from the available remote operators O. The selected first remote operator O-X is assigned to the remote operation of the target moving body 100-X.

[0136] In the case where there are multiple remote operators O having the required license REQ-LIC, the operator selection unit 450-O may consider the operator score SCR-O described in Section 4 above to select the first remote operator O-X. More specifically, the operator selection unit 450-O may preferentially select the remote operator O with a higher operator score SCR-O as the first remote operator O-X. In the case where there are multiple candidates with the same operator score SCR-O, the operator selection unit 450-O may preferentially select the remote operator O with a longer waiting time as the first remote operator O-X.

[0137] As another example, in the case where there are multiple remote operators O who have a license requirement REQ-LIC, the operator selection unit 450-O can select the remote operator O with the fewest types of licenses held as the first remote operator O-X for this time. In this case, the remote operator O with more types of licenses for the mobile body 100 can be retained. As a result, the probability of appropriately allocating a suitable remote operator O to the target mobile body 100-X that requests remote operation next is increased.

[0138] 5-3. Effects

[0139] As described above, according to the second aspect, the license held by each remote operator O and the license requirement REQ-LIC required for the remote operation of the target mobile body 100-X are considered. And, the first remote operator O-X having the license requirement REQ-LIC is assigned to the remote operation of the target mobile body 100-X. Thereby, the accuracy of the remote operation of the target mobile body 100-X can be ensured. In addition, a situation where a remote operator O who does not have the license requirement REQ-LIC is erroneously assigned to the target mobile body 100-X can be prevented. Thereby, it is not necessary to re-perform the allocation process from the beginning, and an efficient allocation process can be achieved.

[0140] 6. [Third Aspect] Allocation Process Considering Terminal Specifications

[0141] The specifications required by the remote operation terminal 200 may vary depending on the type of the target mobile body 100-X. In addition, the specifications required by the remote operation terminal 200 may vary depending on the content of the requested remote operation. If a remote operation terminal 200 that does not satisfy the specifications required for the remote operation of the target mobile body 100-X is assigned to the remote operation of the target mobile body 100-X, the accuracy of the remote operation will decrease. In addition, it is inefficient (low efficiency) to re-perform the allocation process from the beginning.

[0142] Therefore, in the third aspect, the allocation process is performed considering the specifications of each of the multiple remote operation terminals 200. More specifically, the first remote operation terminal 200-X that satisfies the required terminal specifications required for the remote operation of the target mobile body 100-X is assigned to the remote operation of the target mobile body 100-X. Hereinafter, the third aspect will be described in more detail.

[0143] 6-1. First Example

[0144] Figure 15 An example of the terminal specification information SPC is shown. The terminal specification information SPC shows the specifications of each of the multiple remote operation terminals 200. In Figure 15In the example shown, regarding the terminal specification information SPC, for each remote operation terminal 200, the type of the remote operation terminal 200, the types of mobile bodies 100 that can be handled, and the content of the remote operations (services) that can be handled are shown.

[0145] As the types of the remote operation terminal 200, a cockpit-type terminal, a PC, a tablet computer, a smart phone, etc. can be exemplified. If the type of the remote operation terminal 200 changes, the types of mobile bodies 100 that can be handled and the content of the remote operations that can be handled also change. For example, a vehicle cockpit is suitable for remote operations of various types of vehicles and is also suitable for driving on public roads and long-distance driving. On the other hand, a tablet computer is not necessarily suitable for driving on public roads and long-distance driving. A tablet computer can be used for short-distance driving of vehicles in limited areas such as parking lots, automatic driving assistance such as shoulder retreat in case of emergency, and remote operations of small vehicles in factories.

[0146] The terminal specification information SPC is included in the terminal management information MGT-T. For example, the management system 300 collects information on its type from each of the multiple remote operation terminals 200. And, the management system 300 pre-generates Figure 15 the exemplified terminal specification information SPC, and updates it as needed.

[0147] Figure 16 It is a block diagram showing an example of a functional configuration related to the allocation process from the third perspective. The allocation processing unit 400 of the management system 300 includes a required terminal specification acquisition unit 430 and a selection unit 450.

[0148] The required terminal specification acquisition unit 430 acquires a remote operation request REQ. The remote operation request REQ shows at least one of "the type of the target mobile body 100-X" and "the content of the desired remote operation of the target mobile body 100-X". The required terminal specification acquisition unit 430 can identify the "required terminal specification REQ-SPC" required for the remote operation of the target mobile body 100-X based on the remote operation request REQ. In this example, the remote operation request REQ is directly used as the required terminal specification REQ-SPC.

[0149] The required ability information REQ-ABL shows the abilities required for the remote operation of the target mobile body 100-X. The required ability information REQ-ABL includes at least information on the required terminal specification REQ-SPC.

[0150] The selection unit 450 includes a terminal selection unit 450-T. The terminal selection unit 450-T selects a suitable first remote operation terminal 200-X from multiple remote operation terminals 200 based on the terminal management information MGT-T and the required capability information REQ-ABL. More specifically, the terminal selection unit 450-T extracts the available (idle) remote operation terminals 200 from the multiple remote operation terminals 200 based on the terminal management information MGT-T. Furthermore, the terminal selection unit 450-T obtains the terminal specification information SPC included in the terminal management information MGT-T and the information of the required terminal specification REQ-SPC included in the required capability information REQ-ABL. And the terminal selection unit 450-T selects the first remote operation terminal 200-X that meets the required terminal specification REQ-SPC from the available remote operation terminals 200. The selected first remote operation terminal 200-X is assigned to the remote operation of the target moving body 100-X.

[0151] In the case where there are multiple remote operation terminals 200 that meet the required terminal specification REQ-SPC, the terminal selection unit 450-T can consider the terminal score SCR-T described in the above Section 4 to select the first remote operation terminal 200-X. More specifically, the terminal selection unit 450-T can preferentially select the remote operation terminal 200 with a high terminal score SCR-T as the first remote operation terminal 200-X. In the case where there are multiple candidates with the same terminal score SCR-T, the terminal selection unit 450-T can preferentially select the remote operation terminal 200 with a longer waiting time as the first remote operation terminal 200-X.

[0152] 6-2. Second Example

[0153] It is also possible to consider preparing dedicated remote operation terminals 200 (remote cockpits) for each type of moving body 100. On the other hand, if it is possible to remotely operate various types of moving bodies 100 with a single remote operation terminal 200, it is preferable from the viewpoints of cost reduction, reduction of installation area, efficiency, convenience, etc. Therefore, in the second example, the case of remotely operating various types of moving bodies 100 with a single remote operation terminal 200 is studied. For this purpose, the specifications (equipment) of the remote operation terminal 200 are further abstracted and re-formulated. More specifically, as described below, the specifications (equipment) of the remote operation terminal 200 are defined by "operating system capability" and "display system capability". In addition, the description that duplicates the above first example is appropriately omitted.

[0154] 6-2-1. Operating System Capability

[0155] The input device 230 of the remote operation terminal 200 (refer to Figure 5 ) includes a remote operation component that is operated by a remote operator O when remotely operating the moving body 100. As the remote operation component, a steering wheel, an accelerator pedal, a brake pedal, a joystick, a cross key, a touch panel, etc. can be exemplified. Even if the physical configuration of the remote operation component is different, the uses of the remote operation components are common. That is, the remote operation component is used for the forward and backward operations and the lateral operation of the target moving body 100-X. Thus, a certain remote operation component can be replaced by another remote operation component.

[0156] For example, the forward operation, backward operation, and lateral operation of the joystick are respectively associated with the operation of the accelerator pedal, the operation of the brake pedal, and the operation of the steering wheel. As another example, through the user interface displayed on the touch panel, the operating systems of various types of moving bodies 100 can also be reproduced. In any case, the operation amount of a certain remote operation component can be converted into the operation amount of another remote operation component. Thus, a certain remote operation component can be replaced by another remote operation component.

[0157] The operating system capabilities of the remote operation terminal 200 are defined from the above viewpoints. Specifically, the operating system capabilities of the remote operation terminal 200 include "the number of inputs for the forward and backward operations of the moving body 100" and "the number of inputs for the lateral operation of the moving body 100". The operating system capabilities of the remote operation terminal 200 may also include "the presence or absence of operation feedback". "The presence or absence of operation feedback" means "whether the remote operation terminal 200 has an operation reaction force mechanism capable of generating an operation reaction force on the remote operator O".

[0158] Figure 17 It is a diagram showing an example of the terminal specification information SPC related to the operating system and the required terminal specification REQ-SPC. As Figure 17 shown, the terminal specification information SPC includes operating system information showing the operating system capabilities of each of the multiple remote operation terminals 200. For example, the number of forward and backward operation inputs of the remote operation terminal 200 equipped with an accelerator pedal and a brake pedal is "2". As another example, the number of forward and backward operation inputs of the remote operation terminal 200 equipped with a joystick is "2". In the case of a tablet computer capable of displaying an arbitrary user interface (user interface) on the touch panel, the operating system capabilities can be freely set.

[0159] The required terminal specification REQ-SPC includes the operating system capabilities required for the remote operation of the target moving body 100-X. As Figure 17As shown, if the type of the object moving body 100-X (e.g., vehicle, ship, construction machinery) is different, the required operating system capabilities will also change. In addition, even if the types of the object moving bodies 100-X are the same, the required operating system capabilities may be different depending on the content of the desired remote operation (e.g., long-distance driving, shoulder avoidance).

[0160] The required terminal specification acquisition unit 430 acquires a remote operation request REQ. The remote operation request REQ shows at least one of "the type of the object moving body 100-X" and "the content of the desired remote operation of the object moving body 100-X". The required terminal specification acquisition unit 430 can identify the required terminal specification REQ-SPC including the operating system capabilities required for the remote operation of the object moving body 100-X based on the remote operation request REQ.

[0161] For example, a conversion table 435 showing the correspondence between the remote operation request REQ and the required terminal specification REQ-SPC is prepared in advance (refer to Figure 16 ). The conversion table 435 is stored in advance in the storage device 370 of the management system 300. The required terminal specification acquisition unit 430 can identify the required terminal specification REQ-SPC corresponding to the remote operation request REQ by referring to the conversion table 435.

[0162] The terminal selection unit 450-T acquires the terminal specification information SPC and the information of the required terminal specification REQ-SPC. Then, the terminal selection unit 450-T selects the first remote operation terminal 200-X that meets the required terminal specification REQ-SPC from the available remote operation terminals 200 based on the terminal specification information SPC and the required terminal specification REQ-SPC. More specifically, the terminal selection unit 450-T selects the first remote operation terminal 200-X that at least has the required operating system capabilities based on the terminal specification information SPC (operating system information) and the required terminal specification REQ-SPC. The selected first remote operation terminal 200-X is assigned to the remote operation of the object moving body 100-X.

[0163] When there are multiple remote operation terminals 200 that have the required operating system capabilities, the terminal selection unit 450-T can consider the terminal score SCR-T described in Section 4 above to select the first remote operation terminal 200-X. More specifically, the terminal selection unit 450-T can preferentially select the remote operation terminal 200 with a higher terminal score SCR-T as the first remote operation terminal 200-X. When there are multiple candidates with the same terminal score SCR-T, the terminal selection unit 450-T can preferentially select the remote operation terminal 200 with a longer waiting time as the first remote operation terminal 200-X.

[0164] As another example, in the case where there are multiple remote operation terminals 200 that satisfy the required operating system capabilities, the terminal selection unit 450-T may also select a remote operation terminal 200 with the minimum operating system capabilities as the first remote operation terminal 200-X for this time. In this case, the remote operation terminals 200 with higher operating system capabilities are reserved. As a result, even in the case where a high operating system capability is required for the next object moving body 100-X, the probability of being able to allocate an appropriate remote operation terminal 200 is increased.

[0165] As a modification, in a specific situation, the terminal selection unit 450-T may also select a remote operation terminal 200 that slightly fails to meet the required operating system capabilities as the first remote operation terminal 200-X. As a specific situation, the case where there are no other options in an emergency can be considered. For example, although it is preferably desired to have operation feedback, in an emergency, a remote operation terminal 200 without operation feedback may also be selected as the first remote operation terminal 200-X. However, in this case, the terminal selection unit 450-T instructs the object moving body 100-X and the first remote operation terminal 200-X of "function limitation". As "function limitation", "limiting the upper speed to a limit value lower than the default value", "narrowing the steering range to a narrower range than the default range", etc. can be considered. The instructions for the object moving body 100-X and the first remote operation terminal 200-X are performed through communication.

[0166] 6-2-2. Display system capabilities

[0167] The display device 220 of the remote operation terminal 200 (refer to Figure 5 ) is also diverse. For example, the number of monitors included in the display device 220 may vary for each remote operation terminal 200. In addition, the resolution (size) of each monitor is also diverse. If the resolution (size) of the monitor is large, it is also possible to simultaneously display multiple types of images within one monitor. In addition, in the case where the control device 250 of the remote operation terminal 200 has a Picture-in-Picture function, it is also possible to embed other images within one image. The display system capabilities of the remote operation terminal 200 are defined from the above viewpoints. Specifically, the display system capabilities of the remote operation terminal 200 include "the number of monitors" and "the number of images that can be simultaneously displayed".

[0168] Figure 18 is a diagram showing examples of the terminal specification information SPC related to the display system and the required terminal specification REQ-SPC. As Figure 18As shown, the terminal specification information SPC includes display system information showing the display system capabilities of each of the plurality of remote operation terminals 200.

[0169] The required terminal specification REQ - SPC includes the display system capabilities required for the remote operation of the object moving body 100 - X. As Figure 18 shown, if the type of the object moving body 100 - X (e.g., vehicle, construction machinery) is different, the required display system capabilities will also change. In addition, even if the type of the object moving body 100 - X is the same, the required display system capabilities may be different depending on the content of the desired remote operation (e.g., driving agent, parking agent).

[0170] The required terminal specification acquisition unit 430 acquires the remote operation request REQ. The remote operation request REQ shows at least one of "the type of the object moving body 100 - X" and "the content of the desired remote operation of the object moving body 100 - X". The required terminal specification acquisition unit 430 can identify the required terminal specification REQ - SPC including the display system capabilities required for the remote operation of the object moving body 100 - X based on the remote operation request REQ.

[0171] For example, a conversion table 435 showing the correspondence between the remote operation request REQ and the required terminal specification REQ - SPC is prepared in advance (refer to Figure 16 ). The conversion table 435 is pre - stored in the storage device 370 of the management system 300. The required terminal specification acquisition unit 430 can identify the required terminal specification REQ - SPC corresponding to the remote operation request REQ by referring to the conversion table 435.

[0172] The terminal selection unit 450 - T acquires the information of the terminal specification information SPC and the required terminal specification REQ - SPC. And, the terminal selection unit 450 - T selects the first remote operation terminal 200 - X that meets the required terminal specification REQ - SPC from the available remote operation terminals 200 based on the terminal specification information SPC and the required terminal specification REQ - SPC. More specifically, the terminal selection unit 450 - T selects the first remote operation terminal 200 - X that at least has the required display system capabilities based on the terminal specification information SPC (display system information) and the required terminal specification REQ - SPC. The remote operation of the object moving body 100 - X is assigned to the selected first remote operation terminal 200 - X.

[0173] In the case where there are multiple remote operation terminals 200 with the required display system capabilities, the terminal selection unit 450-T can select the first remote operation terminal 200-X by considering the terminal score SCR-T described in Section 4 above. More specifically, the terminal selection unit 450-T can preferentially select a remote operation terminal 200 with a high terminal score SCR-T as the first remote operation terminal 200-X. In the case where there are multiple candidates with the same terminal score SCR-T, the terminal selection unit 450-T can preferentially select a remote operation terminal 200 with a longer waiting time as the first remote operation terminal 200-X.

[0174] As another example, in the case where there are multiple remote operation terminals 200 that satisfy the required display system capabilities, the terminal selection unit 450-T can also select a remote operation terminal 200 with the minimum display system capabilities as the first remote operation terminal 200-X for this time. In this case, the remote operation terminals 200 with higher display system capabilities are reserved. As a result, the probability of being able to allocate a suitable remote operation terminal 200 is increased even in the case where a high display system capability is required for the remote operation of the next object moving body 100-X.

[0175] As a variant, in a specific situation, the terminal selection unit 450-T can also select a remote operation terminal 200 that slightly fails to meet the required display system capabilities as the first remote operation terminal 200-X. As a specific situation, the case where there are no other options in an emergency can be considered. For example, although it is preferable to display two types of auxiliary images in addition to three types of necessary images, in an emergency, a remote operation terminal 200 that can only display three types of images can be selected as the first remote operation terminal 200-X. However, in this case, the terminal selection unit 450-T instructs the object moving body 100-X and the first remote operation terminal 200-X of "function limitation". As "function limitation", "limiting the upper speed to a limit value lower than the default value", "narrowing the steering range to a narrower range than the default range", etc. can be considered. The instructions for the object moving body 100-X and the first remote operation terminal 200-X are carried out through communication.

[0176] In addition, the remote operator O can freely customize the display form of the image according to the content of the remote operation request REQ. The display form of the image includes layout, image display position, image size, presence or absence of image display, etc. The remote operator O can also freely set the presence or absence of the display of the auxiliary image. As the auxiliary image, it includes vehicle width lines, trajectory lines, maximum turning lines, safe parking positions, etc.

[0177] 6-3. Effects

[0178] As described above, according to the third viewpoint, the specifications of each remote operation terminal 200 and the required terminal specifications REQ-SPC required for the remote operation of the object moving body 100-X are considered. And, the first remote operation terminal 200-X that satisfies the required terminal specifications REQ-SPC is assigned to the remote operation of the object moving body 100-X. Thereby, the accuracy of the remote operation of the object moving body 100-X can be ensured. In addition, it is possible to prevent the situation where a remote operation terminal 200 that does not have the required terminal specifications REQ-SPC is erroneously assigned to the object moving body 100-X. Thus, there is no need to re-perform the allocation process from the beginning, and an efficient allocation process can be achieved.

Claims

1. A remote operation system for remote operation of a mobile object, characterized in that: comprising one or more processors, The one or more processors are configured as: acquiring terminal specification information indicating specifications of each of the plurality of remote operation terminals; acquiring required capability information indicating at least required terminal specifications, the required terminal specifications being specifications required for the remote operation of the target mobile object; Based on the terminal specification information and the requested terminal specification, a first remote operation terminal satisfying the requested terminal specification among the plurality of remote operation terminals is allocated to the remote operation of the target moving object.

2. The remote operation system according to claim 1, characterized in that: The one or more processors are further configured to: acquiring a remote operation request indicating at least one of a type of the target moving object and a content of the remote operation of the target moving object; The required terminal specification is identified based on the remote operation request.

3. The remote operation system according to claim 2, characterized in that: Also includes one or more memories, The one or more memories store a table indicating a correspondence relationship between the remote operation request and the required terminal specification, The one or more processors are configured to refer to the table to identify the requested terminal specification corresponding to the remote operation request.

4. The remote operating system according to any one of claims 1 to 3, characterized in that: The terminal specification information includes operating system information indicating operating system capabilities related to operating systems of the respective remote operation terminals. The required terminal specifications include operating system capabilities required for the remote operation of the target moving object, The one or more processors are configured to assign the first remote operation terminal having at least the required operating system capability to the remote operation of the target moving object based on the operating system information and the required terminal specification.

5. The remote operation system according to claim 4, characterized in that: The operating system capability includes the number of inputs for forward and backward operations and the number of inputs for horizontal operations.

6. The remote operation system according to claim 5, characterized in that: The operating system capability also includes the presence or absence of operation feedback.

7. The remote operation system according to any one of claims 1 to 3, characterized in that: The terminal specification information includes display system information indicating display system capabilities related to display systems of the respective remote operation terminals. The required terminal specifications include display system capabilities required for the remote operation of the target moving object, The one or more processors are configured to assign the first remote operation terminal having at least the requested display system capability to the remote operation of the target moving object based on the display system information and the requested terminal specification.

8. The remote operation system according to claim 7, characterized in that: The display system capabilities include the number of monitors and the number of images displayed simultaneously.

9. A remote operation management method, wherein a computer manages remote operation of a mobile object, characterized in that: include: acquiring terminal specification information indicating specifications of each of the plurality of remote operation terminals; acquiring requested capability information indicating at least requested terminal specifications required for the remote operation of the target mobile object; as well as Based on the terminal specification information and the requested terminal specification, a first remote operation terminal satisfying the requested terminal specification among the plurality of remote operation terminals is allocated to the remote operation of the target moving object.

10. A management program, executed by a computer, for managing remote operations of a mobile object, characterized in that: include: acquiring terminal specification information indicating specifications of each of the plurality of remote operation terminals; acquiring requested capability information indicating at least requested terminal specifications required for the remote operation of the target mobile object; as well as Based on the terminal specification information and the requested terminal specification, a first remote operation terminal satisfying the requested terminal specification among the plurality of remote operation terminals is allocated to the remote operation of the target moving object.

Images