Mobile object and remote support system

Abstract

To provide a technique for appropriately selecting a device that receives a remote support request from a vehicle and allocates a necessary resource.SOLUTION: A mobile object is subject to a remote support. The mobile object includes a control device and a management device database in which information of a plurality of management devices is registered. Each of the plurality of management devices has a function of allocating a support resource necessary for the remote support. The control device is configured to transmit the support request to a first management device registered in the management device database, and to transmit the support request to a second management device registered in the management device database and different from the first management device when a response to the support request is not received from the first management device within a first period from the transmission of the support request.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a technology for remotely supporting a mobile object. [Background technology]

[0002] Patent Document 1 discloses a method for remotely controlling a vehicle. The method disclosed in Patent Document 1 performs remote control via a dispatcher that receives assistance requests from the vehicle and a control center that allocates resources required for remote control. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-205037 Summary of the Invention [Problem to be solved by the invention]

[0004] In remote support, the processing when there are multiple devices that receive support requests from vehicles and multiple devices that allocate the necessary resources is not fully considered. For example, Patent Document 1 does not consider the details of the processing for selecting each when there are multiple dispatchers and control centers.

[0005] One object of the present disclosure is to provide a technology for appropriately selecting a device that receives a support request from a vehicle and allocates the necessary resources. [Means for solving the problem]

[0006] The first aspect relates to the mobile object that is the target of remote support. The moving body is a control device; a management device database in which information on a plurality of management devices is registered; Equipped with each of the plurality of management devices has a function of allocating support resources necessary for remote support; The control device Sending a support request to a first management device registered in the management device database; If a response to the support request is not received from the first management device within a first period of time from the sending of the support request, the support request is sent to a second management device registered in the management device database and different from the first management device. It is configured as follows.

[0007] The second aspect relates to a remote support system for remotely supporting a mobile object. The remote support system one or more processors; Multiple management devices Equipped with Each of the plurality of management devices includes a database that stores information about support resources required for remote support; One or more processors may: Sending a support request to a first management device among the plurality of management devices; If a response to the support request is not received from the first management device within a first period of time from the transmission of the support request, the support request is transmitted to a second management device different from the first management device among the plurality of management devices. It is configured as follows. [Effects of the Invention]

[0008] The control device in the first aspect or the one or more processors in the second aspect transmits a support request to a second management device if a response is not received from the first management device within a first period of time from transmitting the support request, thereby enabling the remote support system to be maintained even if some of the multiple management devices are not functioning. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of remote support for a vehicle. [Figure 2] FIG. 10 is a schematic diagram illustrating an example of management device selection from a plurality of management devices. [Figure 3] FIG. 10 is a schematic diagram showing an example of resource allocation when multiple management devices exist. [Figure 4] 3 is a block diagram showing an example of the configuration of each of a vehicle and a plurality of management devices involved in management device selection; FIG. [Figure 5] FIG. 10 is a schematic diagram illustrating an example of an expanded scale of the remote support system. DETAILED DESCRIPTION OF THE INVENTION

[0010] In this embodiment, the description will be based on an example in which the "moving body" is a "vehicle." The technology of the present disclosure can be applied even if the "vehicle" described below is replaced (generalized) with "moving body." Moving bodies include drones, robots, ships, etc. in addition to vehicles.

[0011] 1. Remote support system 1 is a schematic diagram showing an example of remote support for a vehicle 100. The remote support is performed by a remote support system 1. The remote support system 1 includes the vehicle 100, a remote support terminal 200, and a management device. The vehicle 100, the remote support terminal 200, and the management device can communicate with each other via a communication network.

[0012] Vehicle 100 is a target for remote support. Vehicle 100 may be an autonomous vehicle. Here, autonomous driving is assumed to be one that does not necessarily require the driver to concentrate 100% on driving (so-called autonomous driving at level 3 or higher). Vehicle 100 may also be an autonomous vehicle at level 4 or higher that does not require a driver. Vehicle 100 is equipped with various sensors including a camera. For example, the camera photographs the situation around vehicle 100 and acquires an image showing the situation around vehicle 100. Vehicle 100 performs autonomous driving using various sensors.

[0013] Remote support refers to general intervention in the vehicle 100 performed via communication by a remote supporter S located away from the vehicle 100. Remote support is a concept that includes remote monitoring, remote assistance, and remote driving. Remote monitoring includes monitoring the environment around the vehicle 100, the vehicle status of the vehicle 100, and the status of the occupants of the vehicle 100, etc. Remote assistance refers to the remote supporter S providing instructions regarding the driving of the vehicle 100 upon a request from the vehicle 100. For example, if the vehicle 100, which is an autonomous vehicle, is unable to determine the appropriate timing for starting, changing lanes, turning right or left, etc., it sends an assistance request to the remote support system 1. The remote supporter S, upon receiving the assistance request, instructs the autonomous vehicle on the appropriate timing. Remote driving refers to the remote supporter S controlling the driving of the vehicle 100 based on operations input into the remote support terminal 200.

[0014] The remote support terminal 200 is a terminal device used by the remote supporter S when providing remote support to the vehicle 100. The remote support terminal 200 can also be called a remote support HMI (Human Machine Interface).

[0015] The management device manages remote support. The management of remote support includes assigning a remote supporter S to a vehicle 100 that requires remote support. The management device can communicate with the vehicle 100 and the remote support terminal 200 via a communication network. Typically, the management device is a management server on a cloud. The management server may be configured with multiple servers that perform distributed processing.

[0016] Typically, situations where remote support by the remote supporter S is necessary are situations where autonomous driving is difficult. For example, remote support may be necessary when the camera of the vehicle 100 cannot recognize the color of the traffic light at an intersection or when the vehicle 100 cannot determine the timing to change lanes.

[0017] When the vehicle 100 determines that remote support is necessary, it requests remote support. Specifically, the vehicle 100 transmits a support request REQ to the management device. The vehicle 100 that issued the support request REQ, i.e., the vehicle 100 that requires remote support, will be referred to as the "target vehicle 100T" hereinafter.

[0018] The vehicle 100 according to this embodiment is not limited to an autonomous vehicle. The remote support system 1 is also applicable to cases where a driver or passenger who is (manually) driving the vehicle 100 needs remote support. In this specification, the expression "the vehicle 100 requests remote support" also includes cases where a driver or passenger on board the vehicle 100 requests remote support. A driver or passenger on board the vehicle 100 who requests remote support will be referred to as a "support requester 10" hereinafter.

[0019] In response to a support request REQ from the target vehicle 100T, the management device allocates the support resources required for remote support to the remote support of the target vehicle 100T. Support resources refer to the equipment and personnel required for remote support, and include at least the remote supporter S and the remote support terminal 200. The process of allocating support resources to the remote support of the target vehicle 100T is hereinafter referred to as "resource allocation." Furthermore, the remote supporter S and the remote support terminal 200 allocated to the remote support of the target vehicle 100T through resource allocation are referred to as the "allocated supporter SA" and the "allocated support terminal 200A," respectively.

[0020] The management device transmits an allocation notification including a support request REQ and information about the target vehicle 100T to the allocation support terminal 200A. The allocation support terminal 200A presents the allocation notification to the allocation supporter SA. The allocation supporter SA recognizes that it has been assigned to the target vehicle 100T and begins remote support for the target vehicle 100T.

[0021] During remote support, the target vehicle 100T and the allocation support terminal 200A communicate with each other. The target vehicle 100T transmits vehicle information VCL to the allocation support terminal 200A. The vehicle information VCL includes the state of the vehicle 100, images of the surroundings of the vehicle 100 obtained by a camera, the results of vehicle processing, etc. The allocation support terminal 200A presents the vehicle information VCL received from the target vehicle 100T to the allocation supporter SA. The allocation supporter SA checks the vehicle information VCL, recognizes the surrounding conditions of the target vehicle 100T, and provides remote support for the target vehicle 100T. The supporter instruction INS is an instruction to the target vehicle 100T input by the allocation supporter SA.

[0022] In this way, remote support by the remote supporter S for the vehicle 100 is realized. The present disclosure particularly focuses on a case where the remote support system 1 includes a plurality of management devices (hereinafter referred to as "a plurality of management devices 300"). Detailed aspects will be described below.

[0023] 2. Remote support system including multiple management devices When the remote support system 1 includes multiple management devices 300, the target vehicle 100T executes a process of selecting one of the multiple management devices 300 to connect to the target vehicle 100T. This process is hereinafter referred to as "management device selection." Furthermore, one of the multiple management devices 300 selected through management device selection is specifically referred to as the "selected management device 300A." Below, management device selection and resource allocation when the remote support system 1 includes multiple management devices 300 will be described.

[0024] 2-1. Management device selection FIG. 2 is a schematic diagram showing an example of management device selection from multiple management devices 300. The target vehicle 100T has a "management device database MDB." The management device database MDB registers connection information (IP address, location, etc.) for each of the multiple management devices 300 that are targets for management device selection. Here, the management device database MDB includes information on the first to third management devices 300-1 to 300-3. When remote support is required, the target vehicle 100T transmits a support request REQ to the first management device 300-1. If the first management device 300-1 properly receives the support request REQ, it transmits a response RES to the target vehicle 100T notifying the same and begins operating as the selected management device 300A. On the other hand, if the target vehicle 100T does not receive a response RES from the first management device 300-1 within a first period from the transmission of the support request REQ, it transmits the support request REQ to a second management device 300-2, which is different from the first management device 300-1. For example, if the first control device 300-1 does not function properly due to a breakdown or maintenance, the response RES is not sent or received, and the support request REQ is sent to the second control device 300-2.

[0025] The target vehicle 100T may determine the target to which the support request REQ is to be sent based on the priority set for each of the multiple management devices 300 registered in the management device database MDB.

[0026] The priority may be linked to the distance between the target vehicle 100T and each of the multiple management devices 300. In this case, the target vehicle 100T acquires its own position using a position sensor. For example, a Global Navigation Satellite System (GNSS) is an example of the position sensor. Among the multiple management devices 300, the shorter the distance between the target vehicle 100T and the management device 300, the higher the priority is set. In other words, the target vehicle 100T transmits a support request REQ to the multiple management devices 300 in descending order of distance from the target vehicle 100T. In FIG. 2, the first management device 300-1 is located closest to the target vehicle 100T, and the second management device 300-2 is located next closest. Therefore, the target vehicle 100T first transmits a support request REQ to the first management device 300-1. If the target vehicle 100T does not receive a response RES from the first management device 300-1 within a first period, the target vehicle 100T transmits a support request REQ to the second management device 300-2. When the second control device 300-2 receives the support request REQ and the target vehicle 100T receives its response RES, the second control device 300-2 begins to operate as the selected control device 300A. In this case, the support request REQ is not sent to the third control device 300-3. If the target vehicle 100T does not receive the response RES from the second control device 300-2 within the first period, the target vehicle 100T sends the support request REQ to the third control device 300-3.

[0027] In this way, management device selection refers to a series of processes for determining the selected management device 300A by the target vehicle 100T sequentially sending a support request REQ to multiple management devices 300. The selection management device 300A executes resource allocation and allocates an allocation supporter SA and an allocation support terminal 200A to the target vehicle 100T.

[0028] <Effects> By having the target vehicle 100T select the selected management device 300A from among the multiple management devices 300, the remote support system 1 can be maintained even if some of the multiple management devices 300 are not functioning. In other words, by incorporating management device selection, the redundancy and service continuity of the remote support system 1 can be improved.

[0029] Furthermore, when the support request REQ is sent preferentially to targets that are close to the target vehicle 100T, it is less likely that communication will be returned to targets that are far away. Therefore, the remote support system 1 contributes to reducing the waste of communication resources associated with long-distance communication.

[0030] 2-2. Resource Allocation 3 is a schematic diagram showing an example of resource allocation when multiple management devices 300 are present. Each of the multiple management devices 300 has a resource database RDB that stores information about support resources. The information about support resources includes supporter information SI, which is information about each remote supporter S, and terminal information TI, which is information about each remote support terminal 200. Examples of the supporter information SI include availability, experience in remote support, license information, etc. Furthermore, the terminal information TI includes information about availability, the number of displays provided, the performance of input / output devices, etc.

[0031] Furthermore, the resource database RDB manages each remote supporter S in association with a remote support terminal 200 accessible by each remote supporter S. In this case, the correspondence between the remote supporter S and the remote support terminal 200 does not need to be one-to-one. For example, if one remote supporter S possesses multiple remote support terminals 200, multiple remote support terminals 200 are associated with one remote supporter S. Furthermore, in the case of a facility (remote support center) that has many remote supporters S and remote support terminals 200, all of the remote supporters S belonging to the remote support center are associated with all of the remote support terminals 200.

[0032] The resource database RDB may be shared among multiple management devices 300. In the case of Fig. 3, the first resource database RDB-1 is shared by the first management device 300-1 and the second management device 300-2, while the second resource database RDB-2 is used only by the third management device 300-3.

[0033] In this way, when the remote support system 1 has multiple management devices 300, resource allocation refers to the process in which the management device 300 allocates specific support resources (including allocation supporter SA and allocation support terminal 200A) from the resource database RDB to remote support.

[0034] <Effects> When each of the multiple management devices 300 has a resource database RDB, the selection management device 300A can allocate alternative resources even when some of the remote supporters S or remote support terminals 200 are unavailable. In other words, it can be said that resource allocation further improves the redundancy and service continuity of the remote support system 1.

[0035] 3. Configuration example for selecting management device FIG. 4 is a block diagram showing an example of the configuration of each of the vehicle 100 and the plurality of management devices 300 involved in management device selection.

[0036] 3-1.Vehicles The vehicle 100 includes a communication device 110 , a sensor group 120 , a driving device 130 , and a control device 150 .

[0037] The communication device 110 communicates with the outside of the vehicle 100. For example, the communication device 110 communicates with a remote support terminal 200 and a plurality of management devices 300.

[0038] The sensor group 120 includes a recognition sensor, a vehicle state sensor, a position sensor, etc. The recognition sensor recognizes (detects) the situation around the vehicle 100. Examples of the recognition sensor include a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, etc. The vehicle state sensor detects the state of the vehicle 100. The vehicle state sensor includes a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, etc. The position sensor detects the position and orientation of the vehicle 100. For example, the position sensor includes a GNSS (Global Navigation Satellite System).

[0039] The traveling device 130 includes a steering device, a drive device, and a braking device. The steering device steers the wheels. For example, the steering device includes an electric power steering (EPS) device. The drive device is a power source that generates driving force. Examples of the drive device include an engine, an electric motor, and an in-wheel motor. The braking device generates braking force.

[0040] The control device 150 is a computer that controls the vehicle 100. The control device 150 includes one or more processors 160 (hereinafter simply referred to as processors 160) and one or more storage devices 170 (hereinafter simply referred to as storage devices 170). The processor 160 can also be called a processing circuit. The processor 160 executes various processes. For example, the processor 160 includes a CPU. The storage device 170 stores various information required for processing by the processor 160. Examples of the storage device 170 include a volatile memory, a non-volatile memory, an HDD, an SSD, etc. The control device 150 may include one or more ECUs (Electronic Control Units).

[0041] The vehicle control program PROG1 is a computer program executed by the processor 160. The functions of the control device 150 are realized by the processor 160 executing the vehicle control program PROG1. The vehicle control program PROG1 is stored in the storage device 170. Alternatively, the vehicle control program PROG1 may be recorded on a computer-readable recording medium.

[0042] The management device database MDB includes connection information for each of a plurality of management devices 300 that can be targets for management device selection.

[0043] The control device 150 uses the sensor group 120 to acquire driving environment information ENV indicating the driving environment of the vehicle 100. The driving environment information ENV is stored in the storage device 170. The driving environment information ENV includes surrounding situation information indicating the recognition results of the recognition sensors. For example, the surrounding situation information includes images captured by a camera. The driving environment information ENV also includes vehicle state information indicating the vehicle state detected by the vehicle state sensor.

[0044] Furthermore, the driving environment information ENV includes vehicle position information indicating the position and direction of the vehicle 100. The vehicle position information is obtained by a position sensor. Highly accurate vehicle position information may be obtained by a self-position estimation process (localization) using map information and surrounding situation information (object information).

[0045] The control device 150 performs vehicle driving control to control the driving of the vehicle 100. The vehicle driving control includes steering control, drive control, and braking control. The control device 150 performs vehicle driving control by controlling the driving device 130 (steering device, drive device, and braking device). The control device 150 may perform automatic driving control based on the driving environment information ENV.

[0046] A case where remote support is provided to the vehicle 100 will be described. The control device 150 refers to the management device database MDB to select a management device. The control device 150 can communicate with each of the multiple management devices 300 via the communication device 110. The overview of management device selection is as described in Section 2-1 above. For example, during autonomous driving control, the control device 150 determines whether autonomous driving control is difficult based on the driving environment information ENV. When encountering a situation where autonomous driving control is difficult, the control device 150 first sends a support request REQ to the first control device 300-1. If the control device 150 does not receive a response RES from the first control device 300-1 within a first period, it sends the support request REQ to the second control device 300-2.

[0047] The control device 150 may also transmit the vehicle information VCL together with the support request REQ. The vehicle information VCL is information necessary for remote support by the remote supporter S, and includes at least a part of the driving environment information ENV. For example, the vehicle information VCL includes surrounding situation information. The vehicle information VCL may further include vehicle state information, vehicle position information, a driving plan, etc.

[0048] 3-2.Management device Each of the multiple management devices 300 includes a communication device 310 and a control device 350. The communication device 310 communicates with the vehicle 100 and the remote support terminal 200. The control device 350 controls each of the multiple management devices 300. The control device 350 includes one or more processors 360 (hereinafter simply referred to as processors 360) and one or more storage devices 370 (hereinafter simply referred to as storage devices 370). The processor 360 can also be called a processing circuit. The processor 360 executes various processes. For example, the processor 360 includes a CPU (Central Processing Unit). The storage device 370 stores various information required for processing by the processor 360. Examples of the storage device 370 include volatile memory, non-volatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0049] The management program PROG3 is a computer program executed by the processor 360. The functions of the control device 350 are realized by the processor 360 executing the management program PROG3. The management program PROG3 is stored in the storage device 370. Alternatively, the management program PROG3 may be recorded on a computer-readable recording medium. The management program PROG3 may be provided via a network.

[0050] Furthermore, the storage device 370 stores a resource database RDB. The resource database RDB stores and manages information about support resources required for remote support. The resource database RDB includes at least supporter information SI and terminal information TI.

[0051] The control device 350 communicates with the vehicle 100 and the remote support terminal 200 via the communication device 310. The control device 350 receives a support request REQ and vehicle information VCL transmitted from the target vehicle 100T. The support request REQ and vehicle information VCL are stored in the storage device 370. The control device 350 refers to the resource database RDB and performs resource allocation to allocate specific support resources to the target vehicle 100T. The overview of resource allocation is as described above in Section 2-2.

[0052] The control device 350 transmits an allocation notification including a support request REQ and information on the target vehicle 100T to the allocation support terminal 200A. In response to the allocation notification, the allocation supporter SA provides remote support to the target vehicle 100T.

[0053] During the remote support, the control device 350 may receive the vehicle information VCL from the target vehicle 100T. The vehicle information VCL is stored in the storage device 370. The control device 350 may transmit the received vehicle information VCL to the allocation support terminal 200A.

[0054] The control device 350 receives the supporter instruction INS transmitted from the allocation support terminal 200A. The supporter instruction INS is stored in the storage device 370. The control device 350 transmits the received supporter instruction INS to the target vehicle 100T.

[0055] 4. Large-scale remote support system FIG. 5 is a schematic diagram illustrating an example of an expanded scale of the remote support system 1. It is assumed that the remote support system 1 here is managed and operated by a base (which may also be called a remote support center) to which many remote supporters S and remote support terminals 200 belong, and is provided as a service. The remote support system 1 includes a management device database MDB and a resource database RDB. The vehicle 100 can access the management device database MDB, and the management device database MDB may be shared between the vehicles 100.

[0056] When remote support begins, the target vehicle 100T first selects a management device. The support request REQ sent when selecting a management device indicates that remote support is required. The support request REQ may also include the vehicle ID and location information of the vehicle 100, identification information of the support requester 10 (user ID, described below), the details of the remote support required, etc. Next, the selection management device 300A performs resource allocation. The overview of management device selection and resource allocation has already been described.

[0057] The resource database RDB may store various information other than the supporter information SI and terminal information TI described above. For example, information on vehicles 100 and users that may be the target of the service (e.g., the ID of the vehicle 100 and the ID of the user) may be registered in advance. A user is a person who receives the provision of the service. Furthermore, a support requester 10 refers to a user who sends a support request REQ.

[0058] In addition to the remote supporter S and the remote support terminal 200, a relay device 400 may be included as a support resource. The relay device 400 is a device that relays communication between the target vehicle 100T and the allocation support terminal 200A during remote support. The relay device 400 is provided separately from the management device 300. In principle, the management device 300 can also function as the relay device 400.

[0059] Relay device information RDI, which is information about the relay device 400, is also stored in the resource database RDB. The relay device 400 is also subject to resource allocation by the selection management device 300A. In Fig. 5, the relay device 400 allocated for remote support is referred to as an allocated relay device 400A. [Explanation of symbols]

[0060] 1: Remote support system, 10: Support requester, 100: Vehicle, 100T: Target vehicle, 150: Control device, 200: Remote support terminal, 200A: Allocation support terminal, 300: Management device, 300A: Selection management device, 350: Control device, 400: Relay device, 400A: Allocation relay device, INS: Supporter instruction, MDB: Management device database, RDB: Resource database, REQ: Support request, RES: Response, S: Remote supporter, SA: Allocation supporter

Claims

1. A mobile object that is a target of remote support, a control device; a management device database in which information on a plurality of management devices is registered; Equipped with each of the plurality of management devices has a function of allocating support resources necessary for the remote support; The control device Sending a support request to a first management device registered in the management device database; If a response to the support request is not received from the first management device within a first period of time from the transmission of the support request, the support request is transmitted to a second management device registered in the management device database and different from the first management device. It was configured as Mobile object.

2. The moving body according to claim 1, the control device sets a priority for each of the plurality of management devices when transmitting the support request; The first management device and the second management device are selected according to the priority. Mobile object.

3. The moving body according to claim 2, the priority is linked to a distance between the mobile object and each of the plurality of management devices; The shorter the distance, the higher the priority. Mobile object.

4. A remote support system for remote support of a moving object, comprising: one or more processors; Multiple management devices Equipped with each of the plurality of management devices includes a database that stores information about support resources required for the remote support; the one or more processors: sending a support request to a first management device among the plurality of management devices; If a response to the support request is not received from the first management device within a first period from the transmission of the support request, the support request is transmitted to a second management device different from the first management device among the plurality of management devices. It was configured as Remote support system.

5. 5. The remote support system according to claim 4, The database is shared among the plurality of management devices. It was configured as Remote support system.

Images