Remote assistance system and management device

Abstract

The invention relates to a remote assistance system and a management device. A remote assistance system for remote assistance with respect to a mobile body is provided with a management device and a relay device different from the management device. The management device receives an assistance request transmitted from a mobile body and selects an assistance resource required for remote assistance. The relay device relays data communication between the mobile body and the remote assistance terminal during the period of remote assistance. As a result, a technique is provided that makes it possible to flexibly change a configuration relating to communication between the vehicle and the remote assistance terminal in accordance with the scale of the remote assistance system.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a remote support system for a mobile body and a management device for performing remote support for a mobile body. BACKGROUND

[0002] In Patent Literature 1, a method of remotely operating a vehicle is disclosed. The method disclosed in Patent Literature 1 coordinates remote operation between a dispatcher who receives an assistance request from a vehicle and a control center that allocates resources required for remote operation.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2020-205037 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] Consideration is given to communication between a vehicle that accepts remote support and a remote support terminal in a remote support system. In Patent Literature 1, there is no mention of a method in which a vehicle and a terminal for remote operation communicate during execution of remote operation.

[0008] An object of the present disclosure is to provide a technology capable of flexibly changing a configuration related to communication between a vehicle and a remote support terminal according to the scale of a remote support system.

[0009] TECHNICAL SOLUTION FOR SOLVING THE PROBLEM

[0010] A first aspect relates to a remote support system for a mobile body. The remote support system includes a management device and one or more relay devices different from the management device. The management device receives an assistance request transmitted from the mobile body, and selects an assistance resource required for remote support. The one or more relay devices relay data communication between the mobile body and a remote support terminal during remote support.

[0011] A second aspect relates to a management device for performing remote support for a mobile body. The management device receives an assistance request transmitted from the mobile body, and selects an assistance resource required for remote support. The management device selects one or more relay devices different from the management device and relaying data communication between the mobile body and a remote support terminal from among a plurality of relay device candidates.

[0012] EFFECT OF THE INVENTION

[0013] According to the first aspect, the remote assistance system has management devices and relay devices that differ in function. That is, the management devices perform transmission and reception of assistance requests and selection of assistance resources, while the relay devices relay communication between the mobile body and the remote assistance terminal. With this feature, in the case where the scale of the remote assistance system changes, it is possible to add or reduce either of the management devices and the relay devices according to the required function. This can be said to improve the scalability with respect to changes in the scale of the remote assistance system.

[0014] According to the second aspect, the management devices select one or more relay devices from among a plurality of relay device candidates. With this, even in the case where a part of the plurality of relay device candidates cannot operate normally due to a failure or maintenance, it is possible to select a relay device from among the relay device candidates that are operating normally. Thus, the function of the remote assistance system 1 as a whole is maintained, and as a result, the function continuity of the remote assistance system is improved. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a diagram showing an outline of remote assistance for a vehicle.

[0016] Figure 2 is a diagram showing an example of data recording processing.

[0017] Figure 3 is a diagram showing an example of relay device selection.

[0018] Figure 4 is a diagram for explaining the priority of relay device selection.

[0019] Figure 5 is a block diagram showing an example of the configuration of a vehicle.

[0020] Figure 6 is a block diagram showing an example of the configuration of a management device.

[0021] Figure 7 is a block diagram showing an example of the configuration of a relay device.

[0022] Figure 8 is a block diagram showing an example of the configuration of a remote assistance terminal.

[0023] Figure 9 is a diagram explaining an embodiment of the large scale of the remote assistance system.

[0024] BRIEF DESCRIPTION OF DRAWINGS

[0025] 1: Remote assistance system

[0026] 10: Assistance requester

[0027] 100: Vehicle

[0028] 100R: registered vehicle

[0029] 200: remote assistance terminal

[0030] 200c: terminal candidate

[0031] 300: management device

[0032] 300c: management device candidate

[0033] 400: relay device

[0034] 400c: relay device candidate

[0035] 500: external storage device

[0036] INS: supporter indication

[0037] MDB: management device database

[0038] RD: record data

[0039] RDB: resource database

[0040] REQ: assistance request

[0041] S: remote assistance supporter

[0042] Sc: supporter candidate DETAILED DESCRIPTION

[0043] Embodiments of the present disclosure will be described with reference to the accompanying drawings. In the present embodiment, an example in which "mobile body" is "vehicle" will be described. The technology of the present disclosure is applicable even if the following described "vehicle" is replaced (generalized) as "mobile body". Mobile body includes not only vehicle but also drone, robot, ship, and the like.

[0044] 1. Remote assistance system

[0045] Figure 1 is a diagram showing an outline of remote assistance for vehicle 100. Remote assistance is performed by remote assistance system 1. Remote assistance system 1 includes vehicle 100, remote assistance terminal 200, management device 300, and one or more relay devices 400. Vehicle 100, remote assistance terminal 200, management device 300, and one or more relay devices 400 are capable of communicating with each other via a communication network. Hereinafter, one or more relay devices 400 will be simply referred to as "relay device 400".

[0046] The vehicle 100 is a subject of remote assistance. The vehicle 100 can also be an autonomous vehicle. As the autonomous driving here, autonomous driving premised on that a driver does not necessarily have to concentrate 100% on driving (so-called Level 3 or more of autonomous driving) is assumed. The vehicle 100 can also be a Level 4 or more of autonomous driving vehicle that does not require a driver. The vehicle 100 is equipped with various sensors including a camera. For example, the camera captures the situation around the vehicle 100, and acquires an image representing the situation around the vehicle 100. The vehicle 100 performs autonomous driving using the various sensors.

[0047] Remote assistance refers to all of the intervention for the vehicle 100 by the remote assistant S located away from the vehicle 100 via communication. Remote assistance is a concept including remote monitoring, remote assistance, and remote driving. Remote monitoring includes monitoring of the environment around the vehicle 100, the vehicle state of the vehicle 100, the state of the occupant of the vehicle 100, and the like. Remote assistance refers to the instruction by the remote assistant S on the travel of the vehicle 100 upon request from the vehicle 100. For example, the vehicle 100 as an autonomous vehicle transmits an assistance request to the remote assistance system 1 in a case where it cannot judge the appropriate timing of starting, lane changing, left and right turns, and the like. The remote assistant S who receives the assistance request instructs the appropriate timing to the autonomous vehicle. Remote driving refers to the control of the travel of the vehicle 100 based on the operation input by the remote assistant S to the remote assistance terminal 200.

[0048] The remote assistance terminal 200 is a terminal device used by the remote assistant S when performing remote assistance for the vehicle 100. The remote assistance terminal 200 can also be referred to as a remote assistance HMI (Human Machine Interface).

[0049] The management device 300 performs management of remote assistance. The management device 300 can communicate with the vehicle 100, the remote assistance terminal 200, and the relay device 400 via a communication network. Typically, the management device 300 is a management server on the cloud. The management server can also be constituted by a plurality of servers that perform distributed processing.

[0050] The relay device 400 relays data communication between the vehicle 100 and the remote assistance terminal 200 during remote assistance for the vehicle 100.

[0051] In a case where the vehicle 100 is an autonomous vehicle, the vehicle 100 performs various vehicle processes. As representative vehicle processes in autonomous driving, the following processes can be listed.

[0052] (1) Recognition processing: The vehicle 100 recognizes the situation around the vehicle 100 using a recognition sensor. For example, the vehicle 100 recognizes the signal display (example: green signal, yellow signal, red signal, right turn signal, and the like) of a signal (traffic light) using a camera.

[0053] (2) Action judgment processing: The vehicle 100 judges whether to perform an action on the basis of the result of the recognition processing. As the action, examples include starting, stopping, right turn, left turn, lane change, and the like.

[0054] (3) Timing judgment processing: The vehicle 100 judges the timing of performing the above-mentioned action.

[0055] Typically, the situation that needs remote assistance by the remote assistant S is a situation in which automatic driving is difficult. For example, in a case where the camera of the vehicle 100 cannot recognize the color of a signal at an intersection, or in a case where the vehicle 100 cannot judge the timing of performing lane change, remote assistance can be needed.

[0056] The vehicle 100 requests remote assistance in a case where it is judged that remote assistance is needed. Specifically, the vehicle 100 transmits an assistance request REQ to the management device 300.

[0057] Further, the vehicle 100 related to the present embodiment is not limited to an automatic driving vehicle. In a case where the driver, occupant of the vehicle 100 who is driving (manually) needs remote assistance, the remote assistance system 1 is also applicable. In association therewith, the expression "the vehicle 100 requests remote assistance" in the present specification also includes a case where the driver, occupant who is riding on the vehicle 100 requests remote assistance. Hereinafter, the driver, occupant who is riding on the vehicle 100 and requests remote assistance will be referred to as an "assistance requester 10".

[0058] The management device 300 selects an assistance resource needed for remote assistance for the vehicle 100 after receiving the assistance request REQ transmitted from the vehicle 100. The assistance resource means a device, human resource needed for remote assistance, including the remote assistant S, the remote assistance terminal 200, and the relay device 400. Hereinafter, the processing of selecting an assistance resource for remote assistance will be referred to as "resource selection".

[0059] The management device 300 transmits a notification including the assistance request REQ and / or the information of the vehicle 100 to the remote assistance terminal 200. The remote assistance terminal 200 prompts the notification to the remote assistant S. The remote assistant S recognizes that he or she is assigned to the vehicle 100 and starts to perform remote assistance for the vehicle 100.

[0060] During the remote assistance, the vehicle 100 and the remote assistance terminal 200 communicate various data via the relay device 400. During the remote assistance, the vehicle 100 and the remote assistance terminal 200 can also communicate various data via the relay device 400 without going through the management device 300. The vehicle 100 transmits vehicle information VCL to the relay device 400. The vehicle information VCL includes the state of the vehicle 100, an image of the surroundings of the vehicle 100 obtained by a camera, a result of vehicle processing, and the like. The relay device 400 transmits the received vehicle information VCL to the remote assistance terminal 200. The remote assistance terminal 200 prompts the vehicle information VCL received from the vehicle 100 to the remote assistance person S.

[0061] The remote assistance person S confirms the prompted vehicle information VCL, identifies the situation around the vehicle 100, and performs remote assistance for the vehicle 100. For example, the remote assistance assists at least one of the aforementioned identification processing, action judgment processing, and timing judgment processing. The assistance instruction INS is an instruction to the vehicle 100 input by the remote assistance person S. For example, the assistance instruction INS instructs the vehicle 100 to start. The remote assistance terminal 200 receives input of the assistance instruction INS from the remote assistance person S, and transmits the input assistance instruction INS to the relay device 400. The relay device 400 transmits the received assistance instruction INS to the vehicle 100. The vehicle 100 performs vehicle control in accordance with the assistance instruction INS received from the relay device 400.

[0062] Relay of data communication by the relay device 400 is effective for maintaining high-speed communication in remote assistance. Communication between the vehicle 100 and the remote assistance terminal 200 requires high speed. Because vehicle information VCL and assistance instruction INS such information are constantly transmitted and received between the vehicle 100 and the remote assistance terminal 200 during remote assistance. For example, the remote assistance person S views the image output at the remote assistance terminal 200 and decides the content of the assistance instruction INS. In a case where the assistance instruction INS cannot be communicated at an appropriate timing due to communication delay or the like, it can lead to great trouble (malfunction). In conjunction therewith, the relay device 400 preferably has high processing performance. Because the relay device 400 needs to quickly transmit the received information. On the other hand, the management device 300, which is responsible for reception and transmission of the assistance request REQ and resource selection, does not need to have as high processing performance as the relay device 400 because the processing load is not as high as the relay device 400.

[0063] 2. Relay device

[0064] 2-1. Recording by the relay device

[0065] The relay device 400 can also at least temporarily record at least a portion of the data received via the data communication in addition to relaying the data communication between the vehicle 100 and the remote assistance terminal 200 during remote assistance. Hereinafter, the recording of data by the relay device 400 is referred to as a "data recording process", and the data that is the subject of the data recording process is referred to as "recording data RD". The recording data RD includes at least a portion of the vehicle information VCL and the assistant instruction INS. The recording data RD is stored, for example, in a storage region (for example, a storage device 470 described later) provided in the relay device 400.

[0066] Figure 2 is a schematic diagram that represents an example of the data recording process. The relay device 400 transmits (forwards) the recording data RD to one or more external storage devices 500 (hereinafter, simply referred to as "external storage devices 500") other than the relay device 400. The recording data RD is stored in the external storage devices 500. The relay device 400 deletes the recording data RD within the relay device 400 after transmitting the recording data RD.

[0067] 2-2. Relay device selection outline

[0068] Consider a case in which the remote assistance system 1 is provided with a plurality of relay device candidates 400c. The plurality of relay device candidates 400c each have the same configuration and functions as the aforementioned relay device 400. In this case, the management device 300 selects the relay device 400 that is actually used for remote assistance for the vehicle 100 from among the plurality of relay device candidates 400c. Hereinafter, this process is referred to as "relay device selection".

[0069] Figure 3 is a schematic diagram that represents an example of the relay device selection. In Figure 3 (A) of, the management device 300 selects the relay device 400 from among the plurality of relay device candidates 400c by the relay device selection. As Figure 3 (B) of, the management device 300 can select a plurality of relay devices 400 by the relay device selection. In Figure 3 (B) of, the first relay device 400-1 and the second relay device 400-2 are selected. In this case, for example, the following holds: data from the vehicle 100 to the remote assistance terminal 200 (for example, the vehicle information VCL) is relayed by the first relay device 400-1, and, conversely, data from the remote assistance terminal 200 to the vehicle 100 (for example, the assistant instruction INS) is relayed by the second relay device 400-2. That is, the plurality of relay devices 400 can be appropriately used in division. Details of the process involved in the relay device selection are described later.

[0070] 2-3. Effects

[0071] The remote assistance system 1 has the management device 300 and the relay device 400 that have different roles (functions). That is, the management device 300 performs the transmission and reception of the assistance request REQ and the resource selection, and on the other hand, the relay device 400 relays the communication between the vehicle 100 and the remote assistance terminal 200. This feature improves the scalability of the remote assistance system 1 when the scale of the remote assistance system 1 changes. For example, consider a case where the roles of the management device 300 and the relay device 400 are integrated into one device (a management relay device). In this case, as the number of vehicles that use remote assistance increases, the management relay device itself needs to be added. On the other hand, in the remote assistance system 1 of the present disclosure, the number of management devices 300, which have a small processing load, remains unchanged, and it is possible to cope by adding the relay devices 400, which have a large processing load. An example of the case where the scale of the remote assistance system 1 is expanded will be described in Section 5.

[0072] In addition, in the case where the relay device 400 performs the data recording process, it is useful for the recorded data for recording RD to be "fair" recording. That is, the data for recording RD is the information itself that is transmitted and received during remote assistance, and is not provided at the discretion of the assistance requester 10 and / or the remote assistance S. For example, in the case where trouble occurs in remote assistance, the data for recording RD can be used as evidence (images of the vehicle-mounted camera, transmitted signals from the remote assistance terminal 200, and the like) provided from a third party. In addition, since the relay device 400 and the external storage device 500 each have different roles in the case where the relay device 400 transmits the data for recording RD to the external storage device 500, the scalability is further improved. In addition, in the case where the relay device 400 transmits the data for recording RD to the external storage device 500, it is possible to prevent, in advance, the situation where the storage capacity of the relay device 400 is exhausted and new data for recording RD cannot be recorded.

[0073] Further, in the case where the management device 300 performs the relay device selection, it is possible to improve the functional continuity of the remote assistance system 1. For example, even in the case where a part of the plurality of relay device candidates 400c cannot normally operate due to a failure, maintenance, and the like, it is possible to select the relay device 400 from the plurality of relay device candidates 400c that normally operate. As a whole of the remote assistance system 1, the function is maintained.

[0074] 3. Details of Relay Device Selection

[0075] The relay device selection can also be performed based on the priority level set for each of the plurality of relay device candidates 400c. Figure 4is a schematic diagram for illustrating the priority selected by the relay device. The management device 300 stores relay device information RDI as information associated with each of the plurality of relay device candidates 400c. The relay device information RDI includes position information, a generated processing load, a communication load (load), and the like of each of the plurality of relay device candidates 400c.

[0076] The priority of each of the plurality of relay device candidates 400c can also be set based on a communication distance between the vehicle 100 and the remote assistance terminal via the relay device candidate 400c. In this case, the management device 300 holds position information of each of the vehicle 100, the remote assistance terminal 200, and the plurality of relay device candidates 400c. The position information of the remote assistance terminal 200 and the plurality of relay device candidates 400c is known information and is given to the management device 300 in advance. The position information of the vehicle 100 is acquired by a GNSS (Global Navigation Satellite System) or the like mounted on the vehicle 100 and is provided from the vehicle 100 to the management device 300. The position information of the vehicle 100 can be periodically provided or can be included in the REQ at the time of assistance request.

[0077] The management device 300 calculates a geographical distance (relay distance) from the vehicle 100 to the remote assistance terminal 200 via each of the plurality of relay device candidates 400c. The shorter the obtained relay distance, the higher the priority is set. That is, the management device 300 selects the relay device 400 from the plurality of relay device candidates 400c in such a manner that the communication distance is shortened. In addition, as the communication distance, a distance on a network can also be referred to.

[0078] In addition, the priority can also be set based on a load generated on a network (network load) used by each of the plurality of relay device candidates 400c. For example, the management device 300 calculates a frequency band usage rate with respect to a network bandwidth used by each of the plurality of relay device candidates 400c. The frequency band usage rate is used as one of scales indicating the network load. The smaller the obtained network load, the higher the priority is set. That is, the management device 300 preferentially selects one of the plurality of relay device candidates 400c having a smaller network load as the relay device 400.

[0079] Further, the priority can also be set based on processing load (relay device load) generated by each of the plurality of relay device candidates 400c. For example, the management device 300 calculates a proportion of time (CPU usage rate) in which a CPU (Central Processing Unit) possessed by each of the plurality of relay device candidates 400c performs certain processing in a unit time. Alternatively, the CPU usage rate can also be transmitted from each of the plurality of relay device candidates 400c to the management device 300. The CPU usage rate is used as one of scales indicating the relay device load. The smaller the relay device load, the higher the priority is set. That is, the management device 300 preferentially selects one of the plurality of relay device candidates 400c having a smaller processing load as the relay device 400.

[0080] <Effects>

[0081] In a case where the relay device selection is performed with priority given to the shortness of the communication distance, communication delay is less likely to occur. In addition, the possibility of network failure occurring between the vehicle 100 and the remote assistance terminal 200 is reduced. In addition, in a case where the relay device selection is performed with priority given to the network load and / or the relay device load, the relay device 400 is selected in such a manner that the network load and / or the relay device load is reduced. As a result, the load distribution of the entire remote assistance system 1 is balanced.

[0082] In addition, the management device 300 can also calculate evaluation scores of combinations of the priorities exemplified in this section, and perform the relay device selection based on the evaluation scores.

[0083] 4. Configuration Example

[0084] 4-1. Vehicle

[0085] Figure 5 is a block diagram showing a configuration example of the vehicle 100. The vehicle 100 is provided with a communication device 110, a sensor group 120, a travel device 130, and a control device 150.

[0086] The communication device 110 communicates with the outside of the vehicle 100. For example, the communication device 110 communicates with the management device 300 and the relay device 400.

[0087] The sensor group 120 includes an identification sensor, a vehicle state sensor, a position sensor, and the like. The identification sensor identifies (detects) a situation around the vehicle 100. As the identification sensor, a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, and the like can be exemplified. The vehicle state sensor detects a state of the vehicle 100. The vehicle state sensor includes a speed sensor, an acceleration sensor, a yaw rate sensor, a rudder angle sensor, and the like. The position sensor detects a position and an orientation of the vehicle 100. For example, the position sensor includes a GNSS.

[0088] The traveling device 130 includes a steering device, a driving device, and a braking device. The steering device steers the vehicle. For example, the steering device includes an electric power steering (EPS) device. The driving device is a power source that generates a driving force. As the driving device, an engine, an electric motor, a wheel hub motor, and the like can be exemplified. The braking device generates a braking force.

[0089] The control device 150 is a computer that controls the vehicle 100. The control device 150 includes one or a plurality of processors 160 (hereinafter, simply referred to as a processor 160) and one or a plurality of storage devices 170 (hereinafter, simply referred to as a storage device 170). The processor 160 can also be referred to as a processing circuit. The processor 160 performs various processes. For example, the processor 160 includes a CPU. The storage device 170 stores various information required for the processes performed by the processor 160. As the storage device 170, a volatile memory, a non-volatile memory, an HDD, an SSD, and the like can be exemplified. The control device 150 can also include one or a plurality of electronic control units (ECUs).

[0090] The vehicle control program PROG1 is a computer program that is executed by the processor 160. By the processor 160 executing the vehicle control program PROG1, the function of the control device 150 is realized. The vehicle control program PROG1 is stored in the storage device 170. Alternatively, the vehicle control program PROG1 can also be recorded in a recording medium that is readable by a computer.

[0091] The control device 150 acquires driving environment information ENV that indicates a driving environment of the vehicle 100 using the sensor group 120. The driving environment information ENV is stored in the storage device 170.

[0092] Driving environment information ENV includes surrounding condition information representing the recognition results obtained by the recognition sensor. For example, surrounding condition information includes images captured by a camera. Surrounding condition information may also include object information regarding objects surrounding vehicle 100. Examples of objects surrounding vehicle 100 include pedestrians, other vehicles (such as the preceding vehicle and parked vehicles), white lines, signals, signs, and roadside structures. Object information indicates the relative position and speed of the object relative to vehicle 100.

[0093] Furthermore, the driving environment information ENV includes vehicle state information indicating the vehicle state detected by the vehicle state sensor.

[0094] Furthermore, the driving environment information ENV includes vehicle position information indicating the position and orientation of the vehicle 100. The vehicle position information is obtained by a position sensor. High-precision vehicle position information can also be obtained through localization using map information and surrounding situation information (object information).

[0095] The control device 150 performs vehicle travel control for controlling the travel of the vehicle 100. Vehicle travel control includes steering control, drive control, and brake control. The control device 150 performs vehicle travel control by controlling the travel device 130 (steering device, drive device, and brake device).

[0096] The control device 150 can also perform autonomous driving control based on the driving environment information ENV. More specifically, the control device 150 generates a driving plan for the vehicle 100 based on the driving environment information ENV. Furthermore, the control device 150 generates a target trajectory (trajectory) for the vehicle 100 to travel according to the driving plan based on the driving environment information ENV. The target trajectory includes a target position and a target speed. Furthermore, the control device 150 controls vehicle driving so that the vehicle 100 follows the target trajectory.

[0097] The control device 150 transmits an assistance request REQ to the management device 300 as needed. During the remote assistance, the control device 150 communicates with the remote assistance terminal 200 via the relay device 400 .

[0098] 4-2. Management Device

[0099] Figure 6 3 is a block diagram showing a configuration example of the management device 300. The management device 300 includes a communication device 310 and a control device 350.

[0100] The communication device 310 communicates with the vehicle 100 , the remote assistance terminal 200 , and the relay device 400 .

[0101] The control device 350 controls the management device 300. The control device 350 includes one or more processors 360 (hereinafter, simply referred to as processor 360) and one or more storage devices 370 (hereinafter, simply referred to as storage device 370). The processor 360 can also be referred to as processing circuitry. The processor 360 performs various processes. For example, the processor 360 includes a CPU (Central Processing Unit). The storage device 370 stores various information required for the processes performed by the processor 360. As the storage device 370, a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like can be exemplified.

[0102] The management program PROG3 is a computer program executed by the processor 360. By the processor 360 executing the management program PROG3, the function of the control device 350 is realized. The management program PROG3 is stored in the storage device 370. Alternatively, the management program PROG3 can also be recorded on a recording medium readable by a computer. The management program PROG3 can also be provided via a network.

[0103] The storage device 370 also contains a resource database RDB. The resource database RDB stores and manages information on assistance resources required for remote assistance. The resource database RDB contains, as information on the remote assistant S, assistant information SI, and, as information on the remote assistance terminal 200, terminal information TI. As the assistant information SI, an idle status, an amount of experience in performing remote assistance, license information, and the like can be exemplified. In addition, as the terminal information TI, an idle status, the number of displays possessed, performance of input / output devices, and the like information are contained. In addition, in a case where the management device 300 performs relay device selection, relay device information RDI is contained in the resource database RDB.

[0104] The control device 350 communicates with the vehicle 100, the remote assistance terminal 200, and the relay device 400 via the communication device 310.

[0105] The control device 350 receives the assistance request REQ transmitted from the vehicle 100. The assistance request REQ is stored in the storage device 370. The control device 350 refers to the resource database RDB, and performs resource selection of allocating a specific assistance resource to the vehicle 100.

[0106] The control device 350 transmits an allocation notification including the assistance request REQ and / or information of the vehicle 100 to the remote assistance terminal 200. In response to the allocation notification, the remote assistant S starts performing remote assistance on the vehicle 100.

[0107] 4-3. Relay Device

[0108] Figure 7 is a block diagram representing a configuration example of the relay device 400. The relay device 400 includes a communication device 410 and a control device 450.

[0109] The communication device 410 communicates with the vehicle 100 and the remote assistance terminal 200. In addition, in a case where the relay device 400 performs the data recording processing, the communication device 410 can also communicate with the external storage device 500.

[0110] The control device 450 controls the relay device 400. The control device 450 includes one or more processors 460 (hereinafter, simply referred to as processors 460) and one or more storage devices 470 (hereinafter, simply referred to as storage devices 470). The processor 460 can also be referred to as processing circuitry. The processor 460 performs various processing. For example, the processor 460 includes a CPU (Central Processing Unit). The storage device 470 stores various information required for the processing performed by the processor 460. As the storage device 470, a volatile memory, a non-volatile memory, an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like can be exemplified.

[0111] During remote assistance, the control device 450 receives the vehicle information VCL from the vehicle 100. The control device 450 transmits the received vehicle information VCL to the remote assistance terminal 200. The control device 450 receives the assistant instruction INS transmitted from the remote assistance terminal 200. The control device 450 transmits the received assistant instruction INS to the vehicle 100.

[0112] In a case where the aforementioned data recording processing is performed, the control device 450 can also store the recording data RD in the storage device 470 within the control device 450. In addition, the control device 450 can also transmit the recording data RD to the external storage device 500 via the communication device 410. The external storage device 500 stores the received recording data RD in the storage area.

[0113] 4-4. Remote Assistance Terminal

[0114] Figure 8 is a block diagram representing a configuration example of the remote assistance terminal 200. The remote assistance terminal 200 includes a communication device 210, an output device 220, an input device 230, and a control device 250.

[0115] The communication device 210 communicates with the vehicle 100, the management device 300, and the relay device 400. For example, the communication device 210 receives a notification (assignment notification) indicating that the remote assistance has been assigned from the communication device 310. The remote assistant S confirms the assignment notification displayed on the output device 220 in a case where the remote assistant S has his or her own remote assistance terminal 200, and recognizes that the remote assistant S is the remote assistant S.

[0116] The assignment notification to the remote assistant S can also be performed as a process separate from the remote assistance terminal 200. In this case, the remote assistant S receives the assignment notification from the communication device 310 or the communication device 210. Specifically, a communication device (smartphone, tablet, PC, or the like) possessed by the remote assistant S receives the assignment notification. The assignment notification includes information (for example, a management number) related to the remote assistance terminal 200, and enables the remote assistant S to access the remote assistance terminal 200.

[0117] The output device 220 outputs various information. For example, the output device 220 includes a display device. The display device prompts the remote assistant S by displaying various information. As another example, the output device 220 can also include a speaker.

[0118] The input device 230 receives an input from the remote assistant S. For example, the input device 230 includes a touch panel, a keyboard, a mouse, a button, or the like. The remote assistant S can input an assistant instruction INS using the input device 230.

[0119] The control device 250 controls the remote assistance terminal 200. The control device 250 includes one or more processors 260 (hereinafter, simply referred to as processors 260) and one or more storage devices 270 (hereinafter, simply referred to as storage devices 270). The processors 260 perform various processes. The processors 260 can also be referred to as processing circuits. For example, the processors 260 include a CPU. The storage devices 270 store various information required for the processes performed by the processors 260. As the storage devices 270, a volatile memory, a non-volatile memory, an HDD, an SSD, or the like can be exemplified.

[0120] The remote assistance program PROG2 is a computer program executed by the processors 260. The functions of the control device 250 are realized by the processors 260 executing the remote assistance program PROG2. The remote assistance program PROG2 is stored in the storage devices 270. Alternatively, the remote assistance program PROG2 can be recorded on a computer-readable recording medium. The remote assistance program PROG2 can also be provided via a network.

[0121] The control device 250 performs communication with the management device 300 via the communication device 210 and with the vehicle 100 via the relay device 400. The control device 250 receives the assistance request REQ and the vehicle information VCL transmitted from the vehicle 100. The control device 250 prompts the remote assistant S of the vehicle information VCL by displaying the vehicle information VCL on the display device. The remote assistant S can recognize the state of the vehicle 100 and / or the surrounding situation based on the vehicle information VCL. The remote assistant S inputs the assistant instruction INS using the input device 230. The control device 250 transmits the input assistant instruction INS to the vehicle 100 via the relay device 400.

[0122] 5. Large-scale remote assistance system

[0123] As mentioned in sections 2-3, the remote assistance system 1 is particularly effective when it is used on a large scale. Hereinafter, a large-scale embodiment of the remote assistance system 1 will be described with reference to Figure 9 , in which the remote assistance system 1 is used by multiple assistant candidates Sc and multiple terminal candidates 200c belonging to a base (which can also be called a "remote assistance center") and is provided as a service. A person who receives this service provision is referred to as a user. In addition, the aforementioned assistance requester 10 means a user who transmits an assistance request REQ. In addition, multiple registered vehicles 100R that can be service objects are registered in the remote assistance system 1. A vehicle that transmits an assistance request REQ among the multiple registered vehicles 100R is a vehicle 100 that actually utilizes remote assistance.

[0124] 5-1. Management device selection

[0125] In the case where the remote assistance system 1 is used on a large scale, the remote assistance system 1 can also have multiple management device candidates 300c. The multiple management device candidates 300c have the same configuration and functions as the management device 300 described earlier. In this case, the vehicle 100 selects the management device 300 that actually manages remote assistance for the vehicle 100 from among the multiple management device candidates 300c. Hereinafter, this process will be referred to as "management device selection". The substantial subject of management device selection is the control device 150 included in the vehicle 100.

[0126] The vehicle 100 can access a "management device database MDB". The connection information (IP address, location information, etc.) of each of the multiple management device candidates 300c is registered in the management device database MDB. The management device database MDB can also be shared among multiple registered vehicles 100R.

[0127] At the start of remote assistance, the vehicle 100 refers to the management device database MDB, and performs management device selection. The vehicle 100 is able to communicate with each of a plurality of management device candidates 300c via the communication device 110. The vehicle 100 sequentially transmits an assistance request REQ to the plurality of management device candidates 300c in the case where remote assistance is required. For example, the vehicle 100 sequentially transmits the assistance request REQ from the management device candidate 300c located close to the vehicle 100 based on the position information of the plurality of management device candidates 300c. Each of the plurality of management device candidates 300c starts to work as a management device 300 for remote assistance for the vehicle 100 in the case where the assistance request REQ is received. Specifically, the management device 300 performs resource selection, and allocates a remote assistant S and a remote assistance terminal 200 to the vehicle 100. The assistance request REQ transmitted in the management device selection can also contain the identification information and the position information of the vehicle 100, the identification information of the assistance requester 10, the content of the required remote assistance, and the like. After the management device selection, the management device 300 performs resource selection.

[0128] 5-2. Resource selection

[0129] Each of the plurality of management device candidates 300c is provided with a resource database RDB that stores information on assistance resources. The assistant information SI stored in the resource database RDB contains information on each of a plurality of assistant candidates Sc. Likewise, the terminal information TI contains information on each of a plurality of terminal candidates 200c. The resource database RDB can also be shared among the plurality of management device candidates 300c. In the case where the management device 300 performs relay device selection, the resource database RDB contains relay device information RDI.

[0130] In the resource database RDB, information other than the assistant information SI, the terminal information TI, and the relay device information RDI can also be stored. For example, information on a plurality of registered vehicles 100R and users can also be registered in advance. By storing and managing various kinds of information in the resource database RDB, the management device 300 is able to flexibly perform resource selection.

[0131] 5-2-1. Specific example of resource selection

[0132] Hereinafter, several specific examples of resource selection will be described.

[0133] <Remote assistant selection>

[0134] The management device 300 can use the position information of each of the plurality of assistant candidates Sc included in the assistant information SI for resource selection. For example, the management device 300 transmits the allocation notification to the plurality of assistant candidates Sc in order from the one closest to the vehicle 100. In a case where a remote assistant S does not respond to the allocation notification within a certain period, the allocation notification can be transmitted to the next closest remote assistant S.

[0135] In the resource database RDB, the usage language of each of the plurality of assistant candidates Sc can also be registered. In this case, a remote assistant S who can cope with the usage language of the assistance requester 10 can be allocated. For example, in a case where an English-speaking tourist who is traveling in Japan uses the remote assistance system 1 as the assistance requester 10, the management device 300 preferentially allocates a remote assistant S who can cope with English.

[0136] In addition, in the resource database RDB, the remote assistance history of each of the plurality of assistant candidates Sc can also be registered. The remote assistance history includes information such as the total time, the number of times, and the region in which remote assistance has been performed. In this case, the management device 300 preferentially allocates a remote assistant S who has experience in remote assistance in the region where the vehicle 100 is located. In addition, the remote assistant S can be selected in such a manner that the workload based on the total time and / or the number of times of remote assistance is balanced after the workload is calculated.

[0137] The identification information of the user, the usage language, and the like described above are stored in the resource database RDB as the user information UI. In addition, the usage language, the remote assistance history, and the like of the plurality of assistant candidates Sc are included in the assistant information SI.

[0138] <Remote Assistance Terminal Selection>

[0139] The management device 300 can use the position information of each of the plurality of terminal candidates 200c for resource selection. For example, the management device 300 can preferentially allocate a terminal candidate 200c that is close to the vehicle 100 as the remote assistance terminal 200 with respect to the plurality of terminal candidates 200c.

[0140] In addition, instead of the management device 300 selecting the remote assistance terminal 200, the remote assistance terminal 200 can be selected by the remote assistant S accessing any of the plurality of terminal candidates 200c that are free.

[0141] <Relay Device Selection>

[0142] The management device 300 can select the relay device 400 from the plurality of relay device candidates 400c by relay device selection. Details of the relay device selection are as described above.

[0143] 5-3. Effects

[0144] The remote assistance system 1 exerts the following effects which are important for the large-scale of services.

[0145] The remote assistance system 1 can flexibly cope with the change of service scale. In the remote assistance system 1, the information of assistance resources is gathered in the resource database RDB. As the large-scale of services, the amount of required assistance resources increases, and the geographical location of resources also expands. The remote assistance system 1 can also manage the information of the increased assistance resources from the resource database RDB, and can flexibly cope with the expansion of geography by additionally providing the management device 300 and / or the assistance resource as needed.

[0146] Further, the remote assistance system 1 has high service continuity. The plurality of registered vehicles 100R can access the plurality of management device candidates 300c via the management device database MDB. Thereby, even if a part of the plurality of management device candidates 300c stops working, the service can be continued without problems by accessing the alternative device. In addition, the management device 300 selects the assistance resource via the resource database RDB. Thereby, even if a part of the assistance resources is not available, an alternative resource can be selected.

[0147] Further, the remote assistance system 1 can select the remote assistant S considering regionalism and / or language. The remote assistant S preferably fully understands the traffic regulations and customs of the area where the vehicle 100 is located. In addition, in the case where there is an assistance requester 10, the remote assistant S preferably can understand the language of the assistance requester 10. These points are important from the viewpoint of, for example, the safety of remote assistance and the sense of security of the assistance requester 10.

[0148] Furthermore, the effects brought by the division of responsibilities of the management device 300 and the relay device 400 described in Sections 2-3, and the effects of the selection of the relay device described in Section 3 are exerted even in the case where the remote assistance system 1 is used on a large scale.

Claims

1. A remote assistance system for remote assistance for a mobile body, comprising: a management device; and one or more relay devices different from the management device, the management device, receiving an assistance request transmitted from the mobile body, selecting an assistance resource required for the remote assistance, the one or more relay devices relaying data communication between the mobile body and a remote assistance terminal during the remote assistance.

2. The remote assistance system according to claim 1, the one or more relay devices at least temporarily recording at least a part of data received via the data communication in the remote assistance.

3. The remote assistance system according to claim 2, further comprising one or more storage devices different from the one or more relay devices, the one or more relay devices, further, transmitting the recorded data to the one or more storage devices, deleting the data within the one or more relay devices after the transmission of the data.

4. The remote assistance system according to claim 1, the management device selecting the one or more relay devices from a plurality of relay device candidates.

5. The remote assistance system according to claim 4, the management device, setting a priority for each of the plurality of relay device candidates, selecting the one or more relay devices from the plurality of relay device candidates based on the priority.

6. The remote assistance system according to claim 5, the priority for each of the plurality of relay device candidates being set based on a communication distance between the mobile body and the remote assistance terminal via the relay device candidate, the shorter the communication distance, the higher the priority.

7. The remote assistance system according to claim 5, the priority for each of the plurality of relay device candidates being set based on a network load generated in a network connecting the mobile body and the remote assistance terminal via the relay device candidate, the smaller the network load, the higher the priority.

8. The remote assistance system according to claim 5, the priority for each of the plurality of relay device candidates being set based on a processing load generated in the relay device candidate, the smaller the processing load, the higher the priority.

9. A management device for performing remote assistance for a mobile body, comprising: receiving an assistance request transmitted from the mobile body, selecting an assistance resource required for the remote assistance, selecting one or more relay devices different from the management device and relaying data communication between the mobile body and a remote assistance terminal from a plurality of relay device candidates.

10. The management device according to claim 9, setting a priority for each of the plurality of relay device candidates, selecting the one or more relay devices from the plurality of relay device candidates based on the priority.

11. The management device according to claim 10, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ the priority of each of the plurality of relay device candidates is set based on a communication distance between the mobile body and the remote assistance terminal via the relay device candidate, the shorter the communication distance, the higher the priority.

12. The management device according to claim 10, the priority of each of the plurality of relay device candidates is set based on a network load generated in a network connecting the mobile body and the remote assistance terminal via the relay device candidate, the smaller the network load, the higher the priority.

13. The management device according to claim 10, the priority of each of the plurality of relay device candidates is set based on a processing load generated in the relay device candidate, the smaller the processing load, the higher the priority.

Images