Remote operation delegation system, remote operation delegation method, and storage medium

Abstract

The present application relates to a remote operation delegation system, a remote operation delegation method, and a storage medium. The remote operation delegation method includes: in a case where an autonomous vehicle is difficult to continue autonomous driving or in a case where it is predicted that the autonomous vehicle will be difficult to continue autonomous driving, a computer delegating remote operation of the autonomous vehicle to a remote operator; in an autonomous driving area in which the autonomous driving can be continued by accepting remote support, the remote support of at least a part of the judgment for the autonomous driving by the remote operator is delegated; and outside the autonomous driving area, remote driving of at least one of steering operation and acceleration / deceleration operation of the autonomous vehicle by the remote operator is delegated.

Description

TECHNICAL FIELD

[0001] The present application relates to a remote operation entrustment system, a remote operation entrustment method, and a non-transitory storage medium. BACKGROUND

[0002] A technology related to remote operation of an automated driving vehicle is disclosed in Japanese Patent Application Publication No. 2018-077649. According to the technology, in a case where automated driving becomes difficult, communication is performed between the vehicle and a remote operation management device, and the vehicle is remotely driven by a remote operator.

[0003] However, in a case where automated driving becomes difficult, a switch to remote driving is not necessarily required. Sometimes, automated driving can be continued by eliminating the cause of the difficulty. In addition, the number of remote operators who can cope with remote driving is limited. Therefore, if remote driving is requested every time automated driving becomes difficult, the remote operators will be given a large burden. SUMMARY

[0004] The present disclosure provides a technology that enables continuation of operation of an automated driving vehicle in a case where automated driving of the automated driving vehicle becomes difficult, and reduces the burden on remote operators.

[0005] A first aspect of the present disclosure is a remote operation entrustment system including a remote operation entrustment device. The remote operation entrustment device includes a memory storing a program, and a processor coupled to the memory. The processor is configured to entrust remote operation of an automated driving vehicle to a remote operator in a case where the automated driving vehicle is difficult to continue automated driving, or in a case where the automated driving vehicle is predicted to be difficult to continue automated driving. The remote operation includes remote support by the remote operator of at least a part of judgment for automated driving, and remote driving by the remote operator of at least one of steering operation and acceleration / deceleration operation of the automated driving vehicle. The processor is configured to entrust the remote support in an automated driving area in which automated driving can be continued by accepting the remote support, and to entrust the remote driving outside the automated driving area, at the time of execution of the program.

[0006] In the first aspect, the processor can be configured to entrust the remote support without entrusting the remote driving in the automated driving area and in a remote driving area in which remote driving can be performed. That is, the processor can be configured to give priority to the remote support over the remote driving in an area in which both the remote support and the remote driving can be performed.

[0007] In the first aspect, the processor can be configured to determine whether it is difficult for the autonomous vehicle to continue the autonomous driving to the front specified time or the front specified distance based on at least the generated path to the front specified time or the front specified distance.

[0008] In the first aspect, the autonomous driving area can have a region that is set in advance.

[0009] In the first aspect, the autonomous driving area can have a region that changes over time.

[0010] In the first aspect, the processor can be configured to perform the inside-outside determination of the autonomous driving area based on external information and internal information acquired by the autonomous vehicle.

[0011] In the first aspect, the processor can be configured to entrust remote driving outside the autonomous driving area and inside a remote driving area in which remote driving is possible, and to stop or change a track of the autonomous vehicle outside the autonomous driving area and outside the remote driving area.

[0012] In the first aspect, the remote driving area can have a region that is set in advance.

[0013] In the first aspect, the remote driving area can have a region that changes over time.

[0014] In the first aspect, the processor can be configured to perform the inside-outside determination of the remote driving area based on external information and internal information acquired by the autonomous vehicle.

[0015] In the first aspect, the remote operation entrusting system can include a server configured to be connected to a remote operation entrusting device through a communication network. The remote operation entrusting device can be configured to entrust remote support inside an autonomous driving area in which autonomous driving can be continued by accepting the remote support and to entrust remote driving outside the autonomous driving area by cooperating with the server.

[0016] In the first aspect, the remote operation entrusting device can also be configured to transmit first information for inside-outside determination of an automatic driving area possessed by the automatic driving vehicle to the server in a case where the automatic driving vehicle is difficult to continue automatic driving, or in a case where it is predicted that the automatic driving vehicle will be difficult to continue automatic driving. The server can also be configured to perform inside-outside determination of the automatic driving area based on the first information for inside-outside determination of the automatic driving area transmitted from the remote operation entrusting device and second information that affects automatic driving acquired by the server, and to decide whether to entrust remote support or remote driving.

[0017] A second aspect of the present disclosure is a remote operation entrusting method. The remote operation entrusting method includes: in a case where an automatic driving vehicle is difficult to continue automatic driving, or in a case where it is predicted that the automatic driving vehicle will be difficult to continue automatic driving, a computer entrusting remote operation of the automatic driving vehicle to a remote operator; inside an automatic driving area in which automatic driving can be continued by accepting remote support of at least a part of judgment for the automatic driving by the remote operator, entrusting remote support; and outside the automatic driving area, entrusting remote driving of at least one of steering operation and acceleration-deceleration operation of the automatic driving vehicle by the remote operator. The remote operation has the remote support and the remote driving.

[0018] A third aspect of the present disclosure provides a non-transitory storage medium storing a program that is executable by one or more processors and causes the one or more processors to perform functions. The functions include: in a case where an automatic driving vehicle is difficult to continue automatic driving, or in a case where it is predicted that the automatic driving vehicle will be difficult to continue automatic driving, entrusting remote operation of the automatic driving vehicle to a remote operator; inside an automatic driving area in which automatic driving can be continued by accepting remote support of at least a part of judgment for the automatic driving by the remote operator, entrusting remote support; and outside the automatic driving area, entrusting remote driving of at least one of steering operation and acceleration-deceleration operation of the automatic driving vehicle by the remote operator. The remote operation has the remote support and the remote driving.

[0019] According to the first aspect, the second aspect, and the third aspect of the present disclosure, in a case where it is difficult for the automated driving vehicle to continue automated driving, or in a case where it is predicted that it will be difficult for the automated driving vehicle to continue automated driving, remote support is delegated to a remote operator in an automated driving area in which it is possible to continue automated driving by accepting the remote support. According to the remote support, at least a part of the judgment for automated driving is performed by the remote operator, and thus it is possible to continue the operation of the automated driving vehicle that is realized by automated driving. On the other hand, outside the automated driving area, remote driving is delegated to the remote operator. According to the remote driving, at least one of the steering operation and the acceleration and deceleration operation of the automated driving vehicle is performed by the remote operator, and thus it is possible to continue the operation of the automated driving vehicle even in a situation where it is difficult to continue automated driving. In this way, in a case where automated driving of the automated driving vehicle becomes difficult, remote support and remote driving are appropriately separated according to the situation, and thus it is possible to continue the operation of the automated driving vehicle, and it is possible to reduce the burden on the remote operator. BRIEF DESCRIPTION OF DRAWINGS

[0020] Features, advantages, and technical and industrial significance of exemplary embodiments of the present application will be described below with reference to the accompanying drawings, wherein the same reference numerals are denoted the same elements throughout the several views, and wherein:

[0021] Figure 1 is a diagram for schematically showing the configuration of a remote operation system.

[0022] Figure 2 is a block diagram showing one example of the configuration of an automated driving vehicle.

[0023] Figure 3 is a block diagram showing one example of the configuration of a remote operation center.

[0024] Figure 4 is a conceptual diagram for explaining the inside / outside determination of an automated driving area and the inside / outside determination of a remote driving area.

[0025] Figure 5 is a conceptual diagram for explaining the outline of the remote operation delegation method of the first embodiment of the present disclosure.

[0026] Figure 6 is a conceptual diagram for explaining the outline of the remote operation delegation method of the first embodiment of the present disclosure.

[0027] Figure 7 is a block diagram showing the configuration of a remote operation system of the first embodiment of the present disclosure.

[0028] Figure 8 is a conceptual diagram for explaining the outline of the remote operation delegation method of the second embodiment of the present disclosure.

[0029] Figure 9 is a block diagram showing the configuration of a remote operation system of a second embodiment of the present disclosure.

[0030] Figure 10 is a conceptual diagram for explaining an outline of a remote operation delegation method of a third embodiment of the present disclosure.

[0031] Figure 11 is a conceptual diagram for explaining an outline of a remote operation delegation method of a third embodiment of the present disclosure.

[0032] Figure 12 is a conceptual diagram for explaining an outline of a remote operation delegation method of a third embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the embodiment shown below, in the case where a numerical value, a number, an amount, a range, and the like of each element are mentioned, the technical idea of the present disclosure is not limited to the mentioned numerical value, unless otherwise specifically mentioned or it is obvious from the principle. Further, as for the configuration and the like described in the embodiment shown below, it is not necessarily essential in the technical idea of the present disclosure, unless otherwise specifically mentioned or it is obvious from the principle.

[0034] 1. Outline configuration of remote operation system

[0035] Figure 1 is a diagram that schematically shows the configuration of a remote operation system common to all embodiments described later. The remote operation system 100 is a system that remotely operates an autonomous vehicle 20 from a remote operation center 30. As the autonomous driving level of the autonomous vehicle 20, for example, there are assumed to be Level 3, Level 4, or Level 5. Hereinafter, the autonomous vehicle 20 will be simply referred to as the vehicle 20.

[0036] The remote operation in the present disclosure includes remote support and remote driving. The remote support and the remote driving are performed when the vehicle 20 is difficult to continue autonomous driving, or when it is predicted that the vehicle 20 will be difficult to continue autonomous driving. Here, as the situation in which it is difficult to continue autonomous driving, for example, the following situations can be given.

[0037] 1) A situation in which the vehicle 20 is trapped in a situation where the vehicle 20 does not know whether it can enter a crossroads when the vehicle 20 and other vehicles arrive at the crossroads in an all-way stop manner

[0038] 2) The vehicle 20 cannot determine whether a vehicle that is stopping on a shoulder is a vehicle that is parking on a road or a vehicle that is at the end of a line of vehicles waiting to enter a facility facing a road or the like and is in a situation of being stranded

[0039] 3) In a situation in which the vehicle 20 starts from a bus stop, merges at a merge road, or separates / merges at a diverge road, changes lanes, or the like, the vehicle 20 cannot understand the "atmosphere in which other vehicles around are yielding" and is in a situation of being stranded

[0040] In the above situation, the vehicle 20 can continue to stop in place, or can not be able to reach the destination. A process performed in order to avoid such a situation is remote support or remote driving, by the remote operator 35, 37. Hereinafter, the remote operator 35 who performs remote support will be referred to as a remote supporter 35, and the remote operator 37 who performs remote driving will be referred to as a remote driver 37.

[0041] In remote support, the remote supporter 35 performs at least a part of the judgment for automatic driving by the vehicle 20. Basic calculations related to cognition, judgment, and operation required for driving are performed in the vehicle 20. The remote supporter 35 judges an action that the vehicle 20 should take based on information transmitted from the vehicle 20, and thereby performs support for the vehicle 20. Among the support contents of remote support performed by the remote supporter 35 for the vehicle 20, there are included travel of the vehicle 20 and stop of the vehicle 20. Furthermore, among the support contents of remote support, there can also be included deviation avoidance with respect to an obstacle ahead, overtaking of a preceding vehicle, emergency retreat, and the like.

[0042] However, "at least a part of the judgment" performed by the remote supporter 35 is not limited to making at least one judgment with respect to a plurality of judgment elements. For example, the judgment result is considered to be obtained as a result of going through a plurality of steps and the like.

[0043] • A step in which the remote supporter 35 infers / understands the situation

[0044] • A step in which the remote supporter 35 selects one from among a plurality of options

[0045] • A step in which the remote supporter 35 evaluates a plurality of options that are effective

[0046] • A step in which the remote supporter 35 selects one within the evaluation result that is a criterion or more

[0047] Alternatively, other processes in which the judgment result is obtained based on a step of retrieving statistical information from a database of past cases or the like, which is not included in the above steps, can also be considered. Therefore, the phrase "at least a part of the judgment" can also include a contribution by the remote supporter 35 to at least one step for causing the vehicle 20 to obtain a desired judgment result.

[0048] In remote driving, the remote driver 37 performs driving of the vehicle 20, specifically, at least a part of steering operation or acceleration / deceleration operation. In remote driving, the remote driver 37 takes the burden of cognition, judgment, and operation required for driving. The remote driver 37 drives the vehicle 20 from a remote place in the same manner as driving performed at the driver's seat of the vehicle 20. However, in remote driving, the remote driver 37 does not necessarily need to perform all of the cognition, judgment, and operation. At least a part of the cognition, judgment, and operation can be assisted by the function of the vehicle 20.

[0049] The server 32, the remote support terminal 34, and the remote driving terminal 36 are provided in the remote operation center 30. The vehicle 20 is connected to the server 32 via the communication network 10 including 4G (fourth generation mobile communication technology), 5G (fifth generation mobile communication technology). The number of vehicles 20 capable of communicating with the server 32 is one or more, preferably a plurality of vehicles.

[0050] The remote support terminal 34 is an operation terminal for remote support operated by the remote supporter 35. The remote driving terminal 36 is an operation terminal for remote driving operated by the remote driver 37. The remote support terminal 34 is provided in at least one or more, preferably a plurality. Similarly, the remote driving terminal 36 is provided in at least one or more, preferably a plurality. The remote operation center 30 is prepared with the number of remote supporters 35 corresponding to the number of remote support terminals 34. In addition, the remote driver 37 is prepared in the number corresponding to the number of remote driving terminals 36. However, the remote operator 35 and the remote operator 37 can be different persons or the same person. Note that the remote support terminal 34 and the remote driving terminal 36 can be different terminals or the same terminal capable of switching between remote support function and remote driving function.

[0051] Each remote support terminal 34 and each remote driving terminal 36 is connected to the server 32 via a communication network including LAN (Local Area Network), the Internet. Note that the remote operation center 30 does not necessarily need to be an actually existing facility. Here, the system formed by connecting the remote support terminal 34 and the remote driving terminal 36 to the server 32 via the communication network is referred to as the remote operation center 30. Therefore, it can be that the server 32 is provided on the cloud, and the remote support terminal 34 and the remote driving terminal 36 are provided in satellite offices (Satellite Office) in various places, or in the remote operator's own home. However, the remote support terminal 34 and the remote driving terminal 36 can be connected to different servers, respectively. In addition, the remote support terminal 34 and the remote driving terminal 36 can be provided in different places.

[0052] Figure 2 is a block diagram showing one example of a configuration of a vehicle 20. The vehicle 20 is provided with an in-vehicle computer 21. The in-vehicle computer 21 is a collection of a plurality of ECUs (Electronic Control Units) mounted on the vehicle 20. In addition, the vehicle 20 is provided with an external sensor 22, an internal sensor 23, an actuator 24, and a communication device 25. They are connected to the in-vehicle computer 21 using a vehicle-mounted network such as a CAN (Controller Area Network).

[0053] The in-vehicle computer 21 is provided with one or a plurality of processors 21a (hereinafter, simply referred to as processors 21a) and one or a plurality of memories 21b (hereinafter, simply referred to as memories 21b) coupled with the processors 21a. In the memories 21b, one or a plurality of programs 21c (hereinafter, simply referred to as programs 21c) executable by the processors 21a and various kinds of information associated therewith are stored.

[0054] Various processes performed by the processors 21a are realized by the execution of the programs 21c by the processors 21a. The programs 21c include, for example, a program for realizing automatic driving, a program for realizing remote support, and a program for realizing remote driving. In addition, the programs 21c include a remote operation delegation program that causes the in-vehicle computer 21 to function as a remote operation delegation device described later. The memories 21b include a main storage device and an auxiliary storage device. The programs 21c can be stored in the main storage device or in a computer-readable recording medium as the auxiliary storage device. In addition, a map database that manages map information used for automatic driving can also be stored in the auxiliary storage device.

[0055] The external sensor 22 includes a camera that captures the surroundings of the vehicle 20, particularly the front. The camera can be a monocular camera or a stereo camera. A plurality of cameras can be provided, and can capture the side and the rear in addition to the front of the vehicle 20. In addition, the camera can be shared between automatic driving and remote operation, or a camera for automatic driving and a camera for remote operation can be provided separately.

[0056] The external sensor 22 includes an identification sensor other than a camera. The identification sensor is a sensor for identifying the situation around the vehicle 20. As the identification sensor other than the camera, a LiDAR (Laser Imaging Detection and Ranging) and a millimeter wave radar are exemplified. Further, the external sensor 22 includes a position sensor that detects the position and orientation of the vehicle 20. As the position sensor, a GPS (Global Positioning System) sensor is exemplified. The information obtained by the external sensor 22 is sent to the in-vehicle computer 21. Further, the external sensor 22 can also include a microphone that collects the sound around the vehicle 20.

[0057] The internal sensor 23 includes a state sensor that acquires information related to the motion of the vehicle 20. As the state sensor, for example, a wheel speed sensor, an acceleration sensor, an angular velocity sensor, and a rudder angle sensor are exemplified. The acceleration sensor and the angular velocity sensor can also be referred to as an IMU (Inertial Measurement Unit). The information obtained by the internal sensor 23 is sent to the in-vehicle computer 21. Hereinafter, the information obtained by the internal sensor 23 is referred to as internal information, and the information obtained by the external sensor 22 is referred to as external information.

[0058] The actuator 24 includes a steering device that steers the vehicle 20, a drive device that drives the vehicle 20, and a brake device that brakes the vehicle 20. The steering device includes, for example, a power steering system, a steer-by-wire system, and a rear wheel steering system. The drive device includes, for example, an engine, a BEV (battery electric vehicle) system, and a hybrid system. The brake device includes, for example, a hydraulic brake and an electric power regenerative brake. The actuator 24 acts by a control signal sent from the in-vehicle computer 21.

[0059] The communication device 25 is a device that controls wireless communication with the outside of the vehicle 20. The communication device 25 communicates with the server 32 via the communication network 10. The information processed by the in-vehicle computer 21 is sent to the server 32 using the communication device 25. The information processed by the server 32 is read into the in-vehicle computer 21 using the communication device 25. Further, in a case where inter-vehicle communication with other vehicles, road-to-vehicle communication with infrastructure is required for automatic driving, communication with these external devices is also performed by the communication device 25.

[0060] Figure 3is a block diagram showing one example of a configuration of the remote operation center 30. The remote operation center 30 is provided with a server 32. The server 32 is one computer, or a collection of a plurality of computers connected through a communication network. Further, the remote operation center 30 is provided with a remote support terminal 34, a remote driving terminal 36, and a communication device 38. They are connected to the server 32 using a communication network. As described above, it is also possible that a plurality of remote support terminals 34 and a plurality of remote driving terminals 36 are connected to the server 32.

[0061] The server 32 is provided with one or a plurality of processors 32a (hereinafter, simply referred to as processor 32a) and one or a plurality of memories 32b (hereinafter, simply referred to as memory 32b) coupled with the processor 32a. In the memory 32b, one or a plurality of programs 32c (hereinafter, simply referred to as program 32c) executable by the processor 32a and various kinds of information associated therewith are stored.

[0062] Various processes performed by the processor 32a are realized by the processor 32a executing the program 32c. The program 32c includes, for example, a program for realizing remote support and a program for realizing remote driving. The memory 32b includes a main storage device and an auxiliary storage device. The program 32c can be stored in the main storage device or in a computer-readable recording medium as the auxiliary storage device. Further, a map database that manages map information used for automatic driving can also be stored in the auxiliary storage device. The map database can be stored in at least one of the server 32 and the in-vehicle computer 21.

[0063] The remote support terminal 34 is provided with an information output section 34a. The information output section 34a is a device that outputs information required for remote support of the vehicle 20 to the remote supporter 35. The information output section 34a includes a display that outputs an image. In the display, for example, an image of the front of the vehicle 20 captured by a camera of the vehicle 20 is displayed. The display can have a plurality of display screens, and can display an image of the side and / or the rear of the vehicle 20. Further, the information output section 34a can include a speaker that outputs sound. For example, it is also possible that a sound or a voice for communicating the start or the end of remote support is emitted from the speaker. Further, in the case where the vehicle 20 is provided with a microphone, it is also possible to communicate the situation around the vehicle 20 collected by the microphone to the remote supporter 35 from the speaker.

[0064] The remote support terminal 34 is provided with an operation input section 34b. The operation input section 34b is a device that inputs an operation of the remote supporter 35 for remote support. As a specific example of the input device, a button, a lever, and a touch panel can be exemplified. For example, support of the progress / stop of the vehicle 20 or support of movement to the lateral direction can be performed in accordance with the direction in which the lever is pushed down. The movement to the lateral direction includes, for example, deviation avoidance against an obstacle in front, lane change, and overtaking of a preceding vehicle.

[0065] The remote driving terminal 36 is provided with an information output section 36a. The information output section 36a is a device that outputs information necessary for remote driving of the vehicle 20 to the remote driver 37. The information output section 36a includes a display that outputs an image. In the display, for example, an image of the front of the vehicle 20 captured by a camera of the vehicle 20 is displayed. As a method of display thereof, for example, the same scenery as that when the front is observed from the driver's seat of the vehicle 20 can be displayed on the display. The display can have a plurality of display screens, and can display an image of the side and / or the rear of the vehicle 20. Further, the information output section 36a can also include a speaker that outputs sound. For example, a sound or a voice for communicating the start or the end of remote driving can be emitted from the speaker. Further, in a case where the vehicle 20 is provided with a microphone, a situation around the vehicle 20 captured by the microphone can be communicated to the remote driver 37 from the speaker.

[0066] The remote driving terminal 36 is provided with an operation input section 36b. The operation input section 36b is a device that inputs an operation of the remote driver 37 for remote driving. In order to simulate an operation necessary for actual driving of the vehicle 20, the operation input section 36b is provided with a steering wheel for steering operation, an accelerator pedal for acceleration operation, and a brake pedal for deceleration operation. Further, if the vehicle 20 is provided with a transmission, the operation input section 36b can also be provided with a lever or a switch of the transmission. In addition thereto, the operation input section 36b is also provided with an operation lever for operating a direction indicator of the vehicle 20, an operation lever for causing a wiper to act, and the like, which are devices for inputting an operation necessary for safe driving.

[0067] The communication device 38 is a device that controls communication with the outside of the remote operation center 30. The communication device 38 communicates with one or a plurality of vehicles 20 via the communication network 10. Information processed by the server 32 is transmitted to the vehicle 20 using the communication device 38. Information processed by the vehicle 20 is read into the server 32 using the communication device 38.

[0068] 2. In-out determination of the automatic driving area and in-out determination of the remote driving area

[0069] The automatic driving by the vehicle 20 is realized by all conditions of road conditions, geographical conditions, environmental conditions, hardware conditions, and other various conditions being met. Hereinafter, a region in which all conditions for realizing the automatic driving are met is referred to as an automatic driving region. When the automatic driving is performed, it is necessary to determine whether the vehicle 20 is located in the automatic driving region, that is, it is necessary to perform in-out determination of the automatic driving region.

[0070] The automatic driving region in the present disclosure is designed as the same framework as an ODD (Operational Design Domain) or a higher level framework than the ODD, which means a driving environment condition that becomes a premise for the operation of the automatic driving system. For example, the framework of a general ODD is defined to represent only the driving environment condition, and ODD-out means that "the automatic driving system cannot normally operate due to a reason outside the vehicle". On the other hand, in the framework of the automatic driving region in the present disclosure, the automatic driving region-out can also include "the automatic driving system cannot normally operate due to a reason inside the vehicle". In addition, the driving environment condition that defines the ODD in general is a parameter that can be represented on a map. Therefore, it can be considered that the ODD represents the in-out relationship with the host vehicle when defined in a positional relationship. On the other hand, the automatic driving region in the present disclosure can also include the in-out relationship with the host vehicle when defined in a time axis.

[0071] The aforementioned remote support and remote driving are remote operations performed when the vehicle 20 is difficult to continue the automatic driving or when it is predicted that the vehicle 20 will be difficult to continue the automatic driving. Among them, the remote support is an operation by the remote supporter 35 to continue the automatic driving for a part of the judgment for the automatic driving. Therefore, the vehicle 20 being located in the automatic driving region becomes a premise for the remote support.

[0072] In a case where the vehicle 20 is located outside the automatic driving region, the automatic driving cannot be performed, and the remote support cannot be performed. In such a case, the method used to operate the vehicle 20 is remote driving. Among them, the remote driving is realized by all conditions of road conditions, geographical conditions, environmental conditions, hardware conditions, and other various conditions being met. Hereinafter, a region in which all conditions for realizing the remote driving are met is referred to as a remote driving region. When the remote driving is performed, it is necessary to determine whether the vehicle 20 is located in the remote driving region, that is, it is necessary to perform in-out determination of the remote driving region. The remote driving region in the present disclosure is designed as the same framework as the ODD or a higher level framework than the ODD, like the automatic driving region.

[0073] Hereinafter, the remote driving region will be described with reference to Figure 4The inside-outside determination of the automatic driving area and the remote driving area will be described. Here, for the convenience of the description using the drawings, the automatic driving area ADD and the remote driving area RDD are represented on a map together with the route RT on which the vehicle 20 advances.

[0074] According to Figure 4 In the example shown, at the current position, the vehicle 20 is located inside the automatic driving area ADD. As the vehicle 20 advances along the route RT, it is predicted that the vehicle 20 will enter the remote driving area RDD while remaining in the state of having entered the automatic driving area ADD at the point P1. It is predicted that the vehicle 20 will exit the automatic driving area ADD but remain in the remote driving area RDD at the point P2. It is predicted that the vehicle 20 will exit the remote driving area RDD but re-enter the automatic driving area ADD at the point P3.

[0075] Thus, in the inside-outside determination of the automatic driving area ADD and the remote driving area RDD, not only the inside-outside relationship of the current vehicle 20 with the automatic driving area ADD and the remote driving area RDD is determined, but also the inside-outside relationship of the future vehicle 20 with the automatic driving area ADD and the remote driving area RDD is determined. Note that, in the determination of the future inside-outside relationship, the inside-outside relationship of the vehicle 20 with the automatic driving area ADD and the remote driving area RDD is determined when the position relationship is defined, and the inside-outside relationship of the vehicle 20 with the automatic driving area ADD and the remote driving area RDD is determined when the time axis is defined. That is, it is determined whether the vehicle 20 is located inside or outside the automatic driving area ADD at a future point or time, and it is determined whether the vehicle 20 is located inside or outside the remote driving area RDD at a future point or time.

[0076] Hereinafter, examples of the condition of the automatic driving area ADD and the remote driving area RDD and the criterion of the inside-outside determination will be described.

[0077] <Condition Example 1> Road Condition

[0078] Criterion Example 1-1

[0079] The upper limit of the vehicle speed of the vehicle 20 must cover the limit speed of the road on which the vehicle 20 travels. For example, if the remote driving corresponds to an upper limit of 30 km / h, the service of the remote driving cannot be provided on an expressway. In this case, it can be determined that the expressway is outside the remote driving area RDD.

[0080] Criterion Example 1-2

[0081] In order to move to a destination requiring a lane change, a lane change function is required. For example, in order to merge between vehicles in a congested environment, non-verbal communication of drivers to each other is important. However, such communication is difficult in automated driving, and thus can be determined to be outside the automated driving area ADD.

[0082] Reference Example 1-3

[0083] In the case of automated driving using a road surface mark as a clue, for example, a riverbank land not paved can be determined to be outside the automated driving area ADD.

[0084] <Condition Example 2> Geographical Condition

[0085] Reference Example 2-1

[0086] In order to realize remote driving, communication with the server 32 is required. Therefore, a mountainous area where radio waves do not reach can be determined to be outside the remote driving area RDD.

[0087] Reference Example 2-2

[0088] In a land where a map is not made, or a land where a map deviates greatly due to construction, a service of automated driving in which the vehicle 20 autonomously travels using map information as a clue cannot be provided. That is, a land where map information cannot be used can be determined to be outside the automated driving area ADD.

[0089] <Condition Example 3> Environmental Condition

[0090] Reference Example 3-1

[0091] Although automated driving is performed based on ranging information obtained by a LiDAR, light used by the LiDAR can be diffused in the case of rain or fog. Therefore, an environment where it is raining, or an environment where it is foggy can be determined to be outside the automated driving area ADD.

[0092] Reference Example 3-2

[0093] In the case where the vehicle 20 is equipped with a low-sensitivity camera, an external image cannot be captured inside a tunnel or at night. If such a camera image is used in remote driving, it can be determined that inside a tunnel or at night is outside the remote driving area RDD.

[0094] <Condition Example 4> Hardware State

[0095] Reference Example 4-1

[0096] If the hardware for automatic driving only performs the desired action or returns the desired output, it can be determined to be within the automatic driving area ADD. For example, in the case of LiDAR being the hardware for automatic driving only, and in the case of performing the desired action or returning the desired output, it can be determined to be within the automatic driving area ADD. Further, if the hardware for remote driving only performs the desired action or returns the desired output, it can be determined to be within the remote driving area RDD. Note that the hardware here includes, for example, a computing device and a sensor.

[0097] Reference Example 4-2

[0098] It can be considered to perform prediction of sensor action at a future place / time based on map information, weather information, and the like, and thereby to estimate the automatic driving area ADD and the remote driving area RDD from now on. For example, rain can adversely affect the use of LiDAR, so if the place where a shower occurs is known based on weather information, it can be determined that the time of arrival at that place is outside the automatic driving area ADD.

[0099] Reference Example 4-3

[0100] In the case where the camera used in remote driving is weak in terms of backlight, if the place / time where the signal cannot be recognized due to backlight can be predicted based on the orientation of the vehicle 20 and the direction of the sun, it can be determined that the place / time is outside the remote driving area RDD.

[0101] Reference Example 4-4

[0102] In the case of performing automatic driving using a sensor mounted at a low position that can be affected by splashed mud, it can be determined to be outside the automatic driving area ADD on a rainy day. Further, in the case of performing automatic driving using a sensor mounted at a roof position of the vehicle 20, in the case of traveling on land where deciduous trees grow, it can be determined to be outside the automatic driving area ADD.

[0103] Reference Example 4-5

[0104] If it is a GPS antenna, it can be predicted from the number of satellites based on the place / time that the place where GPS is not favorable. If automatic driving is performed using GPS, it can be determined that the place predicted to be unfavorable for GPS is outside the automatic driving area ADD.

[0105] <Condition Example 5> Software State

[0106] Reference Example 5-1

[0107] Software utilized in automated driving or remote driving is mostly independent and indispensable. Further, even if the software acts, it can freeze. Symptoms thereof are various, and there are various symptoms such as a symptom that proper output cannot be performed due to a large amount of calculation, a symptom that a calculation speed decreases due to thermal runaway, or a symptom that it cannot be escaped from an inappropriate error due to a bug. Therefore, a health condition check of the software is performed by periodically performing communication with the software to confirm the presence or absence of an expected output or the like. For example, if a path planning module can calculate a path to be advanced in the future, it can be determined that it is within the automated driving area ADD.

[0108] Reference Example 5-2

[0109] In automated driving, self-position estimation is performed. In the self-position estimation, an accuracy, that is, a reliability of the estimation result can be calculated based on a variance / distribution or the like. If the reliability is equal to or higher than a threshold value, it can be determined that it is within the automated driving area ADD.

[0110] <Condition Example 6> Other Conditions

[0111] Reference Example 6-1

[0112] In a case where a unique rule is set for each municipality, a function that does not conform to the rule cannot be utilized. For example, in a case where utilization of remote driving is prohibited in a certain municipality, the inside of the municipality becomes outside the remote driving area RDD.

[0113] As in the above condition examples, the automated driving area ADD includes a region that is set in advance, and also includes a region that changes over time. Further, the inside-outside determination of the automated driving area ADD is performed based on external information and internal information acquired by the external sensor 22 and the internal sensor 23. Similarly, the remote driving area RDD includes a region that is set in advance, and also includes a region that changes over time. Further, the inside-outside determination of the remote driving area RDD is performed based on external information and internal information acquired by the external sensor 22 and the internal sensor 23.

[0114] 3. Remote operation delegation method of first embodiment

[0115] Figure 5 And Figure 6 are conceptual diagrams for describing an outline of the remote operation delegation method of the first embodiment of the present disclosure. Hereinafter, the remote operation delegation method of the first embodiment will be described using these diagrams.

[0116] In Figure 5In the example shown, at the current time point, the vehicle 20 is located within the automatic driving area ADD and is automatically driven along the path RT generated to the prescribed time. If the vehicle 20 moves along the path RT, the vehicle 20 will enter an area that is within the automatic driving area ADD and within the remote driving area RDD. In this area, remote driving is also possible, but operation by automatic driving is prioritized. For the remote driver 37, a skill as high as that in the case of actually driving the vehicle 20 is required, and thus lightening the burden on the remote driver 37 is one of the reasons for prioritizing automatic driving.

[0117] Here, assume that at time T1, the vehicle 20 is unable to continue automatic driving. Alternatively, assume that it is predicted that the vehicle 20 will be unable to continue automatic driving. At time T1, the vehicle 20 is located within the automatic driving area ADD, and thus can continue automatic driving by receiving remote support. As one example, assume a case in which the vehicle 20 is to make a right turn at an intersection at which a right turn is indicated without a signal. In this case, the judgment of making the right turn is made by a person instead of the vehicle 20, and thus the vehicle 20 can make the right turn at the intersection and continue automatic driving.

[0118] In the first embodiment, the vehicle 20 performs the in-out determination of the automatic driving area ADD and the in-out determination of the remote driving area RDD. At time T1, the vehicle 20 transmits a request for remote support RQS to the remote supporter 35. The remote supporter 35 that has accepted the request for remote support RQS operates the remote support terminal 34 based on information sent from the vehicle 20 such as a camera image. An operation signal RMS for remotely supporting the vehicle 20 is transmitted from the remote support terminal 34 to the vehicle 20. Remote support is performed by the operation signal RMS, and thus the vehicle 20 can continue automatic driving again.

[0119] If the vehicle 20 moves along the path RT, at time T2, the vehicle 20 will exit to outside the automatic driving area ADD. The vehicle 20 exits to outside the automatic driving area ADD, and thus the vehicle 20 is unable to continue automatic driving. In this case, although the vehicle 20 exits to outside the automatic driving area ADD, it is still located within the remote driving area RDD. Thus, by switching from automatic driving to remote driving, operation of the vehicle 20 can be continued. At time T2, the vehicle 20 transmits a request for remote driving RQD to the remote driver 37. The remote driver 37 that has accepted the request for remote driving RQD operates the remote driving terminal 36 based on information sent from the vehicle 20 such as a camera image. An operation signal RMD for remotely driving the vehicle 20 is transmitted from the remote driving terminal 36 to the vehicle 20. Remote driving by the operation signal RMD is continued until the vehicle 20 is located within the remote driving area RDD and enters the automatic driving area ADD again.

[0120] If the vehicle 20 moves along the route RT, at time T3, the vehicle 20 will again enter the automatic driving area ADD. The vehicle 20 enters the automatic driving area ADD, whereby the vehicle 20 can again start automatic driving. In this case, the remote driving is stopped, and switching to automatic driving by the vehicle 20 is performed. Thus, in the area where automatic driving is possible, automatic driving is prioritized over remote driving, whereby the burden on the remote driver 37 can be reduced.

[0121] In Figure 6 In the example shown, if the vehicle 20 that started remote driving at time T2 moves along the route RT, at time T4, the vehicle 20 will exit the remote driving area RDD. The vehicle 20 exits the remote driving area RDD, whereby remote driving by the remote driver 37 cannot be continued. However, at time T4, the vehicle 20 is not located within the automatic driving area ADD, and thus cannot be switched to automatic driving. In this case, in order to ensure safety, the vehicle 20 decelerates and stops in place. Alternatively, the vehicle 20 can be changed in course so as to stop at a safe location such as a shoulder.

[0122] Note that, in the stage of deciding the route RT, a route in which the automatic driving area ADD or the remote driving area RDD is continuous until the destination, i.e., a route in which automatic driving or remote driving continues until the destination, can also be selected. If there are multiple candidates for the route RT, a route in which the automatic driving area ADD continues for a longer time can also be selected. Further, the time at which switching between the automatic driving area ADD and the remote driving area RDD is performed (e.g., time T2) can be calculated, and thus a route in which the time at which the remote supporter 35 and the remote driver 37 can respond coincides with the time at which switching is performed can also be selected. For example, assume that the time at which the remote supporter 35 can respond (end of work time, etc.) is until time T5, and the time at which the remote driver 37 can respond (start of work time, etc.) is after time T5. In this case, a route in which the vehicle 20 is switched from the automatic driving area ADD to the remote driving area RDD at time T5, or a route in which time T5 is included in both the automatic driving area ADD and the remote driving area RDD can be selected from among the candidates for the route RT.

[0123] 4. Configuration of the remote operation system of the first embodiment

[0124] The remote operation delegation method described above can be implemented by the remote operation system of the first embodiment having the configuration shown in Figure 7 Figure 7 ​In the diagram, the functions of the vehicle-mounted computer 21 and the server 32 are represented by boxes. Hereinafter, the remote operating system of the first embodiment will be described focusing on the functions of the vehicle-mounted computer 21 and the server 32. However, descriptions of components and functions already described will be omitted or simplified. It should be noted that... Figure 7 Examples of external sensors 22 include LiDAR 22a, camera 22b, millimeter-wave sensor 22c, and GPS 22d.

[0125] The onboard computer 21 includes an autonomous driving ECU 210, a remote support ECU 211, a remote driving ECU 212, a remote operation authorization determination unit 213, and a remote function selection unit 214. The remote operation authorization determination unit 213 and the remote function selection unit 214 can be separate ECUs or functions of a single ECU. Signals including external and internal information are input to the autonomous driving ECU 210 and the remote driving ECU 212 from LiDAR 22a, camera 22b, millimeter-wave sensor 22c, GPS 22d, and internal sensors 23. However, not all signals need to be input to both ECUs 210 and 212. For example, the input of the signal from LiDAR 22a to the remote driving ECU 212 can be omitted.

[0126] The autonomous driving ECU 210 includes a vehicle position recognition unit 210a, a surrounding condition recognition unit 210b, a driving plan unit 210c, and an autonomous driving area investigation unit 210d. These functions of the autonomous driving ECU 210 are implemented when the processor executes the program stored in the memory of the autonomous driving ECU 210.

[0127] The vehicle position recognition unit 210a identifies the position of the vehicle 20 on the map based on the position information of the vehicle 20 received by the GPS 22d, the motion-related information of the vehicle 20 detected by the internal sensor 23, and the map information obtained from the map database. Furthermore, the vehicle position recognition unit 210a can also estimate the position of the vehicle 20 based on the relative position of features detected by the LiDAR 22a, the camera 22b, or the millimeter-wave sensor 22c relative to the vehicle 20, the motion-related information of the vehicle 20 detected by the internal sensor 23, and the position of the detected features on the map.

[0128] The surrounding situation recognition unit 210b recognizes objects in the surroundings of the vehicle 20 using a pattern matching, deep learning, or the like, with respect to information received from the LiDAR 22a, the camera 22b, or the millimeter wave sensor 22c, and determines the existence position and the category thereof. Among the object recognized by the surrounding situation recognition unit 210b, for example, include mobile objects such as vehicles, motorcycles, bicycles, pedestrians, and the like, and stationary objects. The surrounding situation recognition unit 210b outputs the objects whose position and category are determined as targets.

[0129] The travel plan unit 210c makes a travel plan of the vehicle 20, for example, based on the route RT recorded in the map database and the target information obtained by the surrounding situation recognition unit 210b. The travel plan is made so that the vehicle 20 appropriately travels on the route RT in compliance with the criteria of safety, law observance, travel efficiency, and the like. The travel plan unit 210c generates a target trajectory based on the travel plan made. The target trajectory includes a set of target positions of the vehicle 20 in the coordinate system fixed to the vehicle 20 and a target speed at each target point. The autonomous driving ECU 210 calculates an actuator control amount for causing the vehicle 20 to follow the target trajectory generated by the travel plan unit 210c.

[0130] The autonomous driving area investigation unit 210d performs the inside-outside determination of the autonomous driving area ADD in accordance with the prescribed determination criteria in each of the criteria examples in the aforementioned condition example. The determination result of the inside-outside determination of the autonomous driving area ADD is input to the remote operation delegation determination unit 213 in the subsequent stage.

[0131] The remote support ECU 211 has a remote support necessity determination unit 211a. The remote support necessity determination unit 211a is realized as a function of the remote support ECU 211 when a program stored in the memory of the remote support ECU 211 is executed by the processor. The remote support ECU 211 communicates with the autonomous driving ECU 210 and acquires information necessary for determination of the necessity of remote support from the autonomous driving ECU 210.

[0132] The remote support necessity determination unit 211a determines whether or not the vehicle 20 is difficult to continue autonomous driving, or whether or not it is predicted that the vehicle 20 will be difficult to continue autonomous driving, based on the information from the autonomous driving ECU. Then, the remote support necessity determination unit 211a determines whether or not remote support should be requested, based on the determination result of the difficulty of the continuation of autonomous driving and the determination result of the inside-outside determination of the autonomous driving area ADD by the autonomous driving ECU. The determination result obtained by the remote support necessity determination unit 211a is input to the remote operation delegation determination unit 213 in the subsequent stage.

[0133] The remote driving ECU 212 has a vehicle position recognition unit 212a and a remote driving area investigation unit 212b. They are realized as functions of the remote driving ECU 212 when a program stored in a memory of the remote driving ECU 212 is executed by a processor.

[0134] The vehicle position recognition unit 212a has the same function as the vehicle position recognition unit 210a of the automatic driving ECU 210. That is, the vehicle position recognition unit 212a recognizes the position of the vehicle 20 on the map. Note that, in the example shown in FIG. 2, the recognition of the position of the vehicle 20 is performed in the automatic driving ECU 210 and the remote driving ECU 212 respectively, but it can be performed by another ECU independent of them, and the automatic driving ECU 210 and the remote driving ECU 212 acquire the recognition result thereof. Figure 7

[0135] The remote driving area investigation unit 212b performs the inside / outside determination of the remote driving area RDD in accordance with the prescribed determination criteria like each of the criteria examples in the aforementioned condition examples. The determination result of the inside / outside determination of the remote driving area RDD is input to the later-stage remote operation delegation determination unit 213.

[0136] The remote operation delegation determination unit 213 determines whether to delegate remote operation on the basis of the determination result of the inside / outside determination of the automatic driving area ADD, the determination result of the necessity of remote support, and the determination result of the inside / outside determination of the remote driving area RDD. Further, the remote operation delegation determination unit 213 determines which of the remote support and the remote driving is delegated in the case of delegating remote operation. The determination result obtained by the remote operation delegation determination unit 213 is input to the later-stage remote function selection unit 214 and the communication device 25.

[0137] The remote function selection unit 214 sends a control signal to the actuator 24 in accordance with the determination result obtained by the remote operation delegation determination unit 213. Specifically, in the case where remote operation is not performed and automatic driving is continued, the actuator control amount calculated by the automatic driving ECU 210 is sent to the actuator 24 as a control signal. In the case where remote operation is selected, the actuator control amount calculated by the automatic driving ECU 210 and the remote support signal (for example, a Go signal and a No-Go signal) sent from the remote operation center 30 are sent to the actuator 24 as control signals. In the case where remote driving is selected, the remote driving signal (for example, a signal including an actuator control amount for remote driving) sent from the remote operation center 30 is sent to the actuator 24 as a control signal.

[0138] ​The determination result obtained by the remote operation commissioning determination section 213 input to the communication device 25 is transmitted from the communication device 25 to the communication device 38 of the remote operation center 30. However, the determination result is transmitted not only in the case where the remote operation is determined to be one of the remote support and the remote driving. By the determination result being transmitted, the commissioning of the remote support or the commissioning of the remote driving from the vehicle-mounted computer 21 to the remote operation center 30 is completed. As described above, the vehicle-mounted computer 21 of the first embodiment functions as a remote operation commissioning device that commissions the remote operation of the vehicle 20 to the remote supporter 35 or the remote driver 37.

[0139] The determination result obtained by the remote operation commissioning determination section 213 received by the communication device 38 is input to the server 32. The server 32 is provided with an operator allocation section 321. The operator allocation section 321 is realized as a function of the server 32 when a program stored in a memory of the server 32 is executed by a processor.

[0140] The operator allocation section 321 allocates the commissioning of the remote operation from the vehicle 20 to an appropriate remote operator on the basis of the determination result obtained by the remote operation commissioning determination section 213. In the case where the commissioning of the remote operation is the commissioning of the remote support, the commissioning is transmitted to the remote support terminal 34. Further, the operator allocation section 321 transmits information required for the remote support acquired from the vehicle 20 to the remote support terminal 34. A remote support signal input by the remote supporter 35 is transmitted from the remote support terminal 34 to the operator allocation section 321.

[0141] In the case where the commissioning of the remote operation is the commissioning of the remote driving, the commissioning is transmitted to the remote driving terminal 36. Further, the operator allocation section 321 transmits information required for the remote driving acquired from the vehicle 20 to the remote driving terminal 36. A remote driving signal input by the remote driver 37 is transmitted from the remote driving terminal 36 to the operator allocation section 321.

[0142] The signal transmitted from the remote support terminal 34 or the remote driving terminal 36 to the operator allocation section 321 is transmitted from the operator allocation section 321 to the vehicle 20 via the communication device 38. The remote support signal or the remote driving signal transmitted from the remote operation center 30 to the vehicle 20 is transmitted from the communication device 25 to the remote function selection section 214 of the vehicle-mounted computer 21.

[0143] 5. Remote operation commissioning method of the second embodiment

[0144] Next, the use of the remote operation commissioning method of the second embodiment will be described. Figure 8The remote operation entrusting method of the second embodiment of the present disclosure will be described. In the remote operation entrusting method of the first embodiment, in a case where the vehicle 20 is difficult to continue automatic driving, or in a case where it is predicted that the vehicle 20 will be difficult to continue automatic driving, the on-board computer 21 of the vehicle 20 decides which one of remote support and remote driving is to be entrusted. In the remote operation entrusting method of the second embodiment, the server 32 performs the decision.

[0145] In Figure 8 In the example shown, at time T1, in a case where the vehicle 20 is difficult to continue automatic driving, the information INF required for the decision of the remote operation method is transmitted from the vehicle 20 to the server 32. The information INF transmitted to the server 32 includes the result of the one-time determination of the inside-outside determination of the automatic driving area ADD and the result of the one-time determination of the inside-outside determination of the remote driving area RDD, which are obtained by the on-board computer 21.

[0146] The server 32 determines whether or not to entrust remote operation based on the information transmitted from the vehicle 20. Further, the server 32 decides which one of remote support and remote driving is to be entrusted in a case where remote operation is to be entrusted. The server 32 can acquire information that the vehicle 20 does not hold, particularly, information that affects automatic driving, information that affects remote driving. The server 32 performs the inside-outside determination of the automatic driving area ADD and the inside-outside determination of the remote driving area RDD based on the information acquired from the vehicle 20 and the information acquired by the server 32 alone, thereby deciding whether to entrust remote support or remote driving.

[0147] At time T1, the server 32 transmits the entrustment RQS of remote support to the remote supporter 35. The remote supporter 35 who accepted the entrustment RQS of remote support operates the remote support terminal 34 based on the information sent from the vehicle 20 such as the camera image. The operation signal RMS for remotely supporting the vehicle 20 is transmitted from the remote support terminal 34 to the vehicle 20. The vehicle 20 is remotely supported by the operation signal RMS, whereby the vehicle 20 can continue automatic driving again.

[0148] If the vehicle 20 moves along the route RT, at time T2, the vehicle 20 will run out of the automatic driving area ADD. The vehicle 20 runs out of the automatic driving area ADD, whereby the vehicle 20 is difficult to continue automatic driving. In this case, the vehicle 20 transmits the information INF required for the decision of the remote operation method to the server 32 again. The server 32 performs the inside-outside determination of the automatic driving area ADD and the inside-outside determination of the remote driving area RDD based on the information acquired from the vehicle 20 and the information acquired by the server 32 alone, as at time T1, thereby deciding whether to entrust remote support or remote driving.

[0149] At time T2, the server 32 transmits a remote driving commission RQD to the remote driver 37. The remote driver 37 who accepted the remote driving commission RQD operates the remote driving terminal 36 based on information sent from the vehicle 20 such as a camera image. An operation signal RMD for remote driving of the vehicle 20 is transmitted from the remote driving terminal 36 to the vehicle 20. The remote driving by the operation signal RMD is continued until the vehicle 20 is located within the remote driving area RDD and enters the automatic driving area ADD again.

[0150] 6. Configuration of remote operation system of second embodiment

[0151] The remote operation commissioning method described above can be implemented by the remote operation system of the second embodiment having the configuration shown in FIG. 6. In the remote operation system of the second embodiment, the remote operation commissioning determination unit is moved from the on-vehicle computer 21 to the server 32. Therefore, the determination results of the in-out determination of the automatic driving area ADD, the determination results of the necessity of remote support, and the determination results of the in-out determination of the remote driving area RDD, which are input to the remote operation commissioning determination unit in the on-vehicle computer 21, are transmitted to the server 32 of the remote operation center 30 via the communication device 25. Figure 9 Figure 9 In the remote operation system of the second embodiment, the server 32 has an operator allocation unit 321, a remote operation commissioning determination unit 322, a traffic information database 323, and an operator information database 324. The operator allocation unit 321 and the remote operation commissioning determination unit 322 are implemented as functions of the server 32 when a program stored in a memory of the server 32 is executed by a processor. The traffic information database 323 and the operator information database 324 are stored in a secondary storage device of the server 32.

[0152] The remote operation system of the first embodiment and the remote operation system of the second embodiment differ mainly in that the function of the remote operation commissioning determination unit is moved from the on-vehicle computer 21 to the server 32. Therefore, the determination results of the in-out determination of the automatic driving area ADD, the determination results of the necessity of remote support, and the determination results of the in-out determination of the remote driving area RDD, which are input to the remote operation commissioning determination unit in the on-vehicle computer 21, are transmitted to the server 32 of the remote operation center 30 via the communication device 25.

[0153] The determination results transmitted from the on-vehicle computer 21 to the server 32 are used as once determination results, and final determination is made by the server 32. In the remote operation system of the second embodiment, the server 32 has an operator allocation unit 321, a remote operation commissioning determination unit 322, a traffic information database 323, and an operator information database 324. The operator allocation unit 321 and the remote operation commissioning determination unit 322 are implemented as functions of the server 32 when a program stored in a memory of the server 32 is executed by a processor. The traffic information database 323 and the operator information database 324 are stored in a secondary storage device of the server 32.

[0154] ​The remote operation request determination unit 322 comprehensively determines whether to request remote operation on the basis of each determination result of the determination transmitted from the in-vehicle computer 21 and information held only by the server 32. The information held only by the server 32 includes traffic information registered in the traffic information database 323 and operator information registered in the operator information database 324. The remote operation request determination unit 322 comprehensively determines which of remote support and remote driving to request in a case where remote operation is requested, on the basis of information acquired from the in-vehicle computer 21 and information held only by the server 32. The determination result obtained by the remote operation request determination unit 322 is input to the operator assignment unit 321 of the subsequent stage.

[0155] Hereinafter, an example of determining which of remote support and remote driving to request by using information not held by the vehicle 20 will be described.

[0156] Determination Example 1

[0157] It is possible to determine whether to cope with by remote support or not by remote support by using detailed accident information. The detailed accident information is, for example, information on the degree of blocking of a road, only a specific lane is blocked, accident handling has been completed, and the like. Although it is possible to observe a shoulder parking vehicle from the vehicle 20, it is difficult to observe "a lot of metal pieces are scattered on the road" from the vehicle 20. In order to acquire such information, it is necessary to effectively use police information and the like.

[0158] Determination Example 2

[0159] It is possible to determine which of remote support and remote driving to request by using information on a communication state. For example, in an environment in which people frequently come and go and communication can be unstable, it is difficult to use remote driving.

[0160] Determination Example 3

[0161] It is also possible to determine which of remote support and remote driving to request in accordance with the free condition of a remote operator. For example, if the remote supporter 35 is already in a tight schedule and the remote driver 37 is free, remote driving is adopted even in a case where it is possible to cope with by remote support.

[0162] Determination Example 4

[0163] For automatic driving, it is desirable that the prediction of the behavior of other vehicles is simple. That is, for the vehicle 20 that performs automatic driving, it is more desirable that there are vehicles around which can be predicted by applying a model to the behavior of the vehicle, or vehicles whose behavior can be acquired in advance by inter-vehicle communication, compared to vehicles that take unexpected behavior. Therefore, if the proportion of vehicles whose behavior is predicted simply among vehicles around is equal to or higher than a threshold value, it is also possible to preferentially adopt remote support.

[0164] As described above, in the second embodiment, the onboard computer 21 and the server 32 cooperate to determine whether the autonomous driving area (ADD) or the remote driving area (RDD) is being delegated, thereby determining whether to delegate remote support or remote driving. In other words, the onboard computer 21 and server 32 in the second embodiment function as a remote operation delegation system that delegates the remote operation of the vehicle 20 to a remote supporter 35 or a remote driver 37.

[0165] 7. Remote Operation Delegation Method of the Third Implementation

[0166] Next, use Figure 10 to Figure 12 The remote operation delegation method according to the third embodiment of this disclosure will be described. In the third embodiment, when the vehicle 20 is unable to continue autonomous driving, or when it is predicted that the vehicle 20 will be unable to continue autonomous driving, the determination of the interior and exterior of the autonomous driving area and the determination of the interior and exterior of the remote driving area are made based on the target trajectory.

[0167] Figure 10 The text describes a scenario where an autonomous vehicle 20 follows a preceding vehicle 40 to a stop, and then resumes its journey. At the point where the preceding vehicle 40 stops, it is unclear whether it will start moving again or remain parked as before. In this situation, it is difficult to continue autonomous driving, and remote operation needs to be delegated to a remote operator to continue the operation of vehicle 20.

[0168] In the third embodiment, the trajectory TJRA of vehicle 20 when remote support is selected and the trajectory TJRDK of vehicle 20 when remote driving is selected are calculated respectively. The trajectory TJRA is calculated based on the maximum steering speed, steering angle and minimum vehicle speed when remote support is performed. Similarly, the trajectory TJRDK is calculated based on the maximum steering speed, steering angle and minimum vehicle speed when remote driving is performed.

[0169] Next, the calculated trajectories TJRA and TJRDK are determined to be in contact with the preceding vehicle 40. If trajectory TJRA is in contact with the preceding vehicle 40, it can be determined that vehicle 20 is outside the autonomous driving area. If trajectory TJRDK is in contact with the preceding vehicle 40, it can be determined that vehicle 20 is outside the remote driving area. Figure 10 In the example shown, trajectory TJRA is in contact with the preceding vehicle 40, but trajectory TJRDK is not. Therefore, remote driving is selected as the remote operation to be delegated. If we assume that neither trajectory TJRA nor TJRDK is in contact with the preceding vehicle 40, then the delegation of remote support takes priority based on the difference in workload between remote supporter 35 and remote driver 37.

[0170] existFigure 11 The situation in which the vehicle 20 in the autonomous driving merges into the own lane from the merging lane is depicted in FIG. 27. In order to merge into the own lane without contacting the other vehicle 42 traveling in the own lane, it is necessary to recognize the speed of the other vehicle 42 to determine the timing of the merging. In the situation in which the other vehicle 42 is traveling in the own lane, it is difficult to perform the action of merging into the own lane from the merging lane by the autonomous driving, and therefore the remote operator is entrusted with the remote operation.

[0171] In the third embodiment, the trajectory TJRA of the vehicle 20 in the case in which the remote support is selected and the trajectory TJRDK of the vehicle in the case in which the remote driving is selected are respectively calculated. The trajectory TJRA is calculated in accordance with the maximum speed and the maximum acceleration at the time of the remote support. Likewise, the trajectory TJRDK is calculated in accordance with the maximum speed and the maximum acceleration at the time of the remote driving. In Figure 11 In FIG. 26, the circles on each trajectory TJRA, TJRDK indicate the position of the vehicle 20 at each time. From the difference in the interval of the circles, it is known that the acceleration at the time of the merging is higher in the trajectory TJRDK than in the trajectory TJRA.

[0172] Next, the TTC (Time To Collision) between each calculated trajectory TJRA, TJRDK and the other vehicle 42 is calculated. In the case in which the TTC between the trajectory TJRA and the other vehicle 42 is below a prescribed threshold value, it is determined that the vehicle 20 is outside the autonomous driving region. In the case in which the TTC between the trajectory TJRDK and the other vehicle 42 is below the threshold value, it is determined that the vehicle 20 is outside the remote driving region. In the case in which the TTC between both trajectories TJRA, TJRDK and the other vehicle 42 is greater than the threshold value, the remote operation of the trajectory that provides the greater TTC is selected. In Figure 11 In the example shown in FIG. 27, the remote driving is selected as the remote operation that should be entrusted.

[0173] In Figure 12 The situation in which the vehicle 20 in the autonomous driving decelerates and stops in front of the obstacle 44 is depicted in FIG. 28. In the case in which it is difficult to stop the vehicle 20 by the autonomous driving, the remote operator is entrusted with the remote operation.

[0174] In the third embodiment, the trajectory TJRA of the vehicle 20 in the case in which the remote support is selected and the trajectory TJRDK of the vehicle in the case in which the remote driving is selected are respectively calculated. The trajectory TJRA is calculated in accordance with the maximum deceleration at the time of the remote support. Likewise, the trajectory TJRDK is calculated in accordance with the maximum deceleration at the time of the remote driving. In Figure 12In the middle, the circles on each trajectory TJRA, TJRDK indicate the position of the vehicle 20 at each time. From the difference in the interval of the circles, it is known that the deceleration at the time of stopping is higher in the trajectory TJRDK than in the trajectory TJRA.

[0175] Next, the TTC is calculated between the calculated trajectories TJRA, TJRDK and the obstacle 44. In the case where the TTC between the trajectory TJRA and the obstacle 44 is below a prescribed threshold, it is determined that the vehicle 20 is outside the automatic driving region. In the case where the TTC between the trajectory TJRDK and the obstacle 44 is below the threshold, it is determined that the vehicle 20 is outside the remote driving region. In the case where the TTC between both trajectories TJRA, TJRDK and the obstacle 44 is greater than the threshold, the remote operation of the trajectory that provides the greater TTC is selected. In the case where the TTC between both trajectories TJRA, TJRDK and the obstacle 44 is below the threshold, the remote operation of the trajectory that provides the smaller TTC is selected. Figure 12 In the example shown, remote driving is selected as the remote operation that should be entrusted.

[0176] 8. Other Embodiments

[0177] The server 32 can also be configured as a remote operation entrusting device. That is, the server 32 can be made to have all of the functions of the vehicle-mounted computer 21 of the first embodiment as a remote operation entrusting device. In this case, the server 32 monitors each vehicle 20. Then, in the case where a vehicle 20 for which it is difficult to continue automatic driving is sensed or a vehicle 20 for which it is predicted that it will be difficult to continue automatic driving, the server 32 entrusts the remote operation of the vehicle 20 to a remote operator.

Claims

1. A remote operation delegation system, characterized in that, Including remote operation consignment devices, The remote operation delegation device includes: Memory, which stores programs; and The processor, coupled to the memory, The processor is configured to delegate remote operation of the autonomous vehicle to a remote operator when the autonomous vehicle is unable to continue autonomous driving or when it is predicted that the autonomous vehicle will be unable to continue autonomous driving. The remote operation has the following characteristics: Remote support, comprising at least a portion of the decisions made by the remote operator for the autonomous driving; and Remote driving, wherein the remote operator performs at least one of the steering and acceleration / deceleration operations of the autonomous vehicle. The processor is configured to, during the execution of the program, Delegating remote support within the autonomous driving area where the autonomous driving can continue by accepting the remote support; and Delegating remote driving outside the autonomous driving area, The processor is configured to delegate remote support without delegating remote driving within the autonomous driving area and within a remote driving area capable of remote driving based on a different benchmark than the autonomous driving area.

2. The remote operation delegation system according to claim 1, characterized in that, The processor is configured to determine whether it is difficult for the autonomous vehicle to continue autonomous driving to the predetermined time or the predetermined distance ahead, based on generating a path that leads to at least the predetermined time or distance ahead.

3. The remote operation delegation system according to claim 1 or 2, characterized in that, The autonomous driving area has a pre-defined region.

4. The remote operation delegation system according to claim 1 or 2, characterized in that, The autonomous driving area has regions that change over time.

5. The remote operation delegation system according to claim 1 or 2, characterized in that, The processor is configured to perform an internal / external determination of the autonomous driving area based on external and internal information acquired by the autonomous vehicle.

6. The remote operation delegation system according to claim 1 or 2, characterized in that, The processor is configured to: Delegating remote driving outside the autonomous driving area and within a remote driving area where remote driving is possible; and Outside the autonomous driving area and outside the remote driving area, the autonomous vehicle is stopped or its trajectory is changed.

7. The remote operation delegation system according to claim 6, characterized in that, The remote driving area has a pre-defined zone.

8. The remote operation delegation system according to claim 6, characterized in that, The remote driving area has a region that changes over time.

9. The remote operation delegation system according to claim 6, characterized in that, The processor is configured to perform an internal / external determination of the remote driving area based on external and internal information acquired by the autonomous vehicle.

10. The remote operation delegation system according to claim 1, characterized in that, Also includes: The server is configured to connect to the remote operation delegation device via a communication network. The remote operation delegation device is mounted on the autonomous vehicle. The remote operation delegation device is configured as follows: In cooperation with the server, remote support is delegated within autonomous driving areas where the autonomous driving can continue by accepting the remote support; and The remote driving is delegated outside the autonomous driving area through collaboration with the server.

11. The remote operation delegation system according to claim 10, characterized in that, The remote operation delegation device is configured to: when the autonomous vehicle is unable to continue autonomous driving, or when it is predicted that the autonomous vehicle will be unable to continue autonomous driving, send first information of the autonomous vehicle used for determining whether the autonomous driving area is inside or outside the autonomous driving area to the server. The server is configured as follows: The determination of whether an autonomous driving area is inside or outside is performed based on the first information sent from the remote operation authorization device for determining whether the autonomous driving area is inside or outside, and the second information obtained from the server that affects the autonomous driving. as well as The decision is whether to delegate the remote support or the remote driving.

12. A method for remote operation delegation, characterized in that, include: When an autonomous vehicle is unable to continue autonomous driving, or when it is predicted that an autonomous vehicle will be unable to continue autonomous driving, the computer delegates remote operation of the autonomous vehicle to a remote operator. Within the autonomous driving area where the autonomous driving can continue by receiving remote support, the remote support is delegated to the remote operator for at least a portion of the decisions made for the autonomous driving. as well as Outside the autonomous driving zone, remote driving of the autonomous vehicle is delegated to the remote operator, including at least one of the steering and acceleration / deceleration operations. The remote operation includes the remote support and the remote driving. Within the autonomous driving area and in a remote driving area where remote driving is possible based on a different benchmark than the autonomous driving area, remote support is delegated instead of remote driving.

13. A non-transitory storage medium storing a program executable by one or more processors and causing the one or more processors to perform the following functions, characterized in that the functions include: In cases where the autonomous vehicle is unable to continue autonomous driving, or when it is predicted that the autonomous vehicle will be unable to continue autonomous driving, the remote operation of the autonomous vehicle is delegated to a remote operator. Within the autonomous driving area where the autonomous driving can continue by receiving remote support, the remote support is delegated to the remote operator for at least a portion of the decisions made for the autonomous driving. as well as Outside the autonomous driving zone, remote driving of the autonomous vehicle is delegated to the remote operator, including at least one of the steering and acceleration / deceleration operations. The remote operation includes the remote support and the remote driving. Within the autonomous driving area and in a remote driving area where remote driving is possible based on a different benchmark than the autonomous driving area, remote support is delegated instead of remote driving.

Images