Remote function selection device

Abstract

The present disclosure relates to a remote function selection device. In an autonomous vehicle that performs both autonomous driving and remote driving and is equipped with multiple remote functions, the remote function selection device selects the remote function by determining whether the autonomous driving cannot be performed at a predetermined time. If the autonomous driving is determined to be impossible, the device determines which remote function is executable at the predetermined time. The device predicts the behavior of objects around the autonomous vehicle based on detection results from external sensors detecting the objects, calculates a prediction confidence level for the predicted behavior of the objects, and selects the remote function. If the device determines that multiple remote functions are executable, the device selects the remote function based on the calculated prediction confidence level.

Description

Technical Field

[0001] The present disclosure relates to a remote function selection device. Background Art

[0002] There are autonomous vehicles that can perform both autonomous driving and remote driving using a remote function controlled by a remote operator. For example, when autonomous driving is unavailable, such autonomous vehicles use the remote function controlled by a remote operator to perform remote driving. Such autonomous vehicles are described, for example, in Japanese Patent No. 6663506.

[0003] In the aforementioned autonomous driving vehicle, multiple remote functions may be executed. In such a case, it is required to appropriately select which of the multiple remote functions to execute. Summary of the Invention

[0004] The first aspect of the present disclosure is a remote function selection device. The remote function selection device is configured to select a remote function to be executed in an autonomous vehicle, wherein the autonomous vehicle is configured to perform autonomous driving and remote driving (driving based on remote instructions from a remote operator) and is equipped with multiple remote functions for performing remote driving. The remote function selection device includes: an autonomous driving determination unit configured to determine whether autonomous driving is not possible at a predetermined time; a remote function determination unit configured to determine a remote function that can be executed at the predetermined time if the autonomous driving determination unit determines that autonomous driving is not possible; a confidence calculation unit configured to predict the behavior of an object based on detection results of an external sensor that detects an object around the autonomous vehicle and to calculate a behavior prediction confidence level for the predicted object behavior; and a function selection unit configured to select a remote function to be executed. If the remote function determination unit determines that the multiple remote functions are executable, the function selection unit is configured to select a remote function to be executed from the multiple remote functions based on the behavior prediction confidence level calculated by the confidence calculation unit.

[0005] The remote function selection device is configured to calculate the behavior prediction confidence level of objects surrounding the autonomous vehicle. If the remote function determination unit determines that multiple remote functions are executable, the remote function selection device is configured to select a remote function to be executed based on the object's behavior prediction confidence level. According to the first aspect, even when multiple remote functions are executable, the remote function selection device can use the behavior prediction confidence level to appropriately select a remote function to be executed.

[0006] In the first embodiment, the function selection unit may be configured to select a remote function that requires a longer remote operator engagement time when the confidence level of the predicted behavior is low, compared to when the confidence level of the predicted behavior is high. Here, when the confidence level of the predicted behavior of an object is low, the object may sometimes perform actions that are unpredictable in the autonomous vehicle. In the presence of such an object, the remote operator's intervention in the driving operation of the autonomous vehicle can flexibly respond to changes in the object's behavior. Therefore, when the confidence level of the predicted behavior of an object is low, the remote function selection device selects a remote function that requires a longer remote operator engagement time. With this configuration, the remote function selection device can select a more appropriate remote function based on the confidence level of the predicted behavior of the object.

[0007] In the first aspect, the remote function selection device may include a distance calculation unit configured to calculate the relative distance between the target object and the autonomous vehicle. The function selection unit may also be configured to select a remote function to be executed from among the multiple remote functions determined by the remote function determination unit based on the action prediction confidence level and the relative distance. With this configuration, the remote function selection device can select a more appropriate remote function by considering the relative distance between the target object and the autonomous vehicle.

[0008] In the first embodiment, the function selection unit may be configured to select a remote function that requires a shorter remote operator time when the relative distance is long, compared to when the relative distance is short. Here, when the relative distance between the autonomous vehicle and the detected target object is long, there is sufficient time for the autonomous vehicle to approach the detected target object. In such a situation, active driving of the autonomous vehicle by the remote operator may not be necessary. Therefore, when the relative distance between the autonomous vehicle and the target object is long, the remote function selection device selects a remote function that requires a shorter remote operator time. This prevents excessive selection of remote functions that require a longer remote operator time. According to the above configuration, a more appropriate remote function can be selected based on the relative distance between the autonomous vehicle and the target object.

[0009] In the first aspect, the plurality of remote functions may include remote support and remote driving. According to the above configuration, a more appropriate remote function can be selected from the remote functions including remote support and remote driving.

[0010] According to the first aspect of the present disclosure, even when a plurality of remote functions can be executed, a remote function to be executed can be appropriately selected. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, wherein like reference numerals denote like elements, and wherein:

[0012] Figure 1 It is a diagram for explaining a remote driving system according to one embodiment.

[0013] Figure 2 This is a block diagram showing an example of the configuration of an autonomous driving vehicle.

[0014] Figure 3 This is a block diagram showing an example of the functional configuration of the remote function selection unit.

[0015] Figure 4A This diagram illustrates the relationship between an autonomous vehicle and the interior and exterior of its operating range.

[0016] Figure 4B This diagram illustrates the relationship between an autonomous vehicle and the interior and exterior of its operating range.

[0017] Figure 5A This diagram is used to illustrate the state of an autonomous driving vehicle from within the autonomous driving operating range to outside the autonomous driving operating range.

[0018] Figure 5B This diagram is used to illustrate a situation where an autonomous driving vehicle that has reached outside the autonomous driving operating range exists within the remote function operating range.

[0019] Figure 6A A diagram illustrating remote support for an autonomous vehicle.

[0020] Figure 6B A diagram illustrating remote driving of an autonomous vehicle.

[0021] Figure 7 This is a flowchart showing the flow of a remote function selection process performed by the remote function selection unit.

[0022] Figure 8 This is a flowchart showing details of a remote function selection process performed by the function selection unit of the remote function selection unit. DETAILED DESCRIPTION

[0023] Hereinafter, exemplary embodiments will be described with reference to the accompanying drawings. It should be noted that in each figure, the same or corresponding elements are denoted by the same reference numerals and redundant descriptions are omitted.

[0024] Figure 1It is a diagram for explaining a long-distance travel system 1 according to one embodiment. Figure 1 The autonomous vehicle 2 shown is capable of both autonomous driving and remote driving (driving based on remote instructions). The remote driving system 1 is a system that remotely drives the autonomous vehicle 2 based on remote instructions from a remote operator R, in response to a remote request from the autonomous vehicle 2. It should be noted that autonomous driving refers to driving based on the autonomous vehicle 2 making autonomous decisions, not based on remote instructions from the remote operator R. The autonomous vehicle 2 makes a remote request, for example, when it cannot autonomously make the decision to operate autonomously.

[0025] A remote operator R is a person who issues remote instructions for the automated vehicle 2 to execute remote driving. The number of remote operators R is not limited and can be one or more. The number of automated vehicles 2 that can communicate with the remote driving system 1 is also not particularly limited. Multiple remote operators R can alternately issue remote instructions to a single automated vehicle 2, or a single remote operator R can issue remote instructions to two or more automated vehicles 2.

[0026] In this embodiment, the autonomous vehicle 2 is equipped with multiple remote functions. The autonomous vehicle 2 uses any of these remote functions to remotely drive based on remote instructions from a remote operator R. A remote function is a function for remotely driving the autonomous vehicle 2 based on remote instructions from the remote operator R. In this embodiment, examples of the multiple remote functions include remote support and remote driving.

[0027] Remote support refers to the following technology: the output of sensors mounted on the autonomous driving vehicle 2 (such as camera images, lidar point groups, etc.) is sent to a remote location, and a remote operator R at the remote location replaces the judgment made by the autonomous driving vehicle 2.

[0028] (Application scenario example 1)

[0029] As an example of a remote support application scenario, consider a scenario where autonomous vehicle 2 needs to avoid parked vehicles or fallen objects in its lane by traveling outside the lane. As an example of remote support in this scenario, for example, a remote operator R checks the forward camera image transmitted from autonomous vehicle 2. Then, remote operator R grants autonomous vehicle 2 permission to travel outside the lane. Based on this instruction, autonomous vehicle 2 autonomously generates a trajectory, enabling it to avoid parked vehicles, etc.

[0030] (Remote Support Example 2)

[0031] An example of a remote support scenario involves a police officer using hand signals to guide traffic at an intersection. In this scenario, for example, a remote operator R checks the forward camera image transmitted from an autonomous vehicle 2. The remote operator R then recognizes the police officer's hand signal and grants permission for the autonomous vehicle 2 to start. Based on this instruction, the autonomous vehicle 2 autonomously generates a route, enabling it to navigate the intersection.

[0032] (Remote Support Example 3)

[0033] As an example of a remote support scenario, consider a scenario where autonomous vehicle 2 receives notification via the network that the road it plans to travel on is blocked before approaching a blocked location. As an example of remote support in this scenario, remote operator R, for example, uses the vehicle's location information and road blockage information transmitted from autonomous vehicle 2 to develop a route around the blockade and instructs autonomous vehicle 2 to change its route. Upon receiving the new route, autonomous vehicle 2 autonomously generates a new route based on the received route, allowing it to continue driving.

[0034] (Remote Support Example 4)

[0035] As an example of a remote support scenario, consider a scenario where autonomous vehicle 2 detects a sign that differs from the map information, such as a temporary sign erected due to road construction. As an example of remote support in this scenario, for example, remote operator R checks the forward camera image transmitted from autonomous vehicle 2. Remote operator R then identifies the sign and provides the identified sign information to autonomous vehicle 2. Based on this sign information, autonomous vehicle 2 autonomously generates a route, enabling it to pass through the location.

[0036] (Remote Support Example 5)

[0037] As an example of a remote support scenario, consider a scenario where an emergency vehicle is approaching autonomous vehicle 2. As an example of remote support in this scenario, for example, remote operator R instructs autonomous vehicle 2 on a retreat location based on forward camera images transmitted from autonomous vehicle 2 and information indicating the relative positional relationships with surrounding objects identified by autonomous vehicle 2. Based on this instruction, autonomous vehicle 2 autonomously generates a route to the retreat location, thereby enabling it to travel toward the retreat destination.

[0038] (Remote Support Example 6)

[0039] As an example of a remote support scenario, consider the case of an autonomous vehicle 2 entering an intersection without traffic lights. In this scenario, for example, a remote operator R checks the forward camera image transmitted from autonomous vehicle 2 and information indicating the relative positional relationships of autonomous vehicle 2 with surrounding objects. The remote operator R then grants autonomous vehicle 2 permission to enter the intersection. Based on this instruction, autonomous vehicle 2 autonomously generates a route, allowing it to enter the intersection.

[0040] Remote driving doesn't necessarily require automated driving. Instead, it involves transmitting sensor information (primarily camera images) from a vehicle to a remote location. A remote operator (R) at that location performs all cognitive, judgment, and operational functions using input devices (steering wheel, accelerator pedal, brake pedal, gear shift, turn signals, etc.).

[0041] In the remote driving system 1, for example, a remote operator R is requested to input a remote instruction (remote support or remote driving instruction) in response to a remote request from an autonomous vehicle 2. The remote operator R inputs the remote instruction to an operator interface 3. The remote driving server 4 transmits the remote instruction to the autonomous vehicle 2 via a network N. The autonomous vehicle 2 executes the remote function according to the remote instruction and drives accordingly.

[0042] An example of the configuration of the autonomous driving vehicle 2 will be described. Figure 2 : is a block diagram showing an example of the structure of the autonomous driving vehicle 2. Figure 2 As shown, as an example, the autonomous driving vehicle 2 includes an autonomous driving ECU 20. The autonomous driving ECU 20 is an electronic control unit that includes a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory). In the autonomous driving ECU 20, various functions are implemented by, for example, the CPU executing programs stored in the ROM or RAM. The autonomous driving ECU 20 may also be composed of multiple electronic units.

[0043] The automatic driving ECU 20 is connected to the external sensor 11 , the internal sensor 12 , the map database 13 , the communication unit 14 , and the actuator 15 .

[0044] The external sensor 11 is a vehicle-mounted sensor that detects the external environment of the autonomous vehicle 2. The external sensor 11 includes at least a camera. The camera is a photographing device that photographs the external environment of the autonomous vehicle 2. The camera is, for example, located on the back side of the windshield of the autonomous vehicle 2 to photograph the front of the vehicle. The camera sends photographing information related to the external environment of the autonomous vehicle 2 to the autonomous driving ECU 20. The camera can be either a monocular camera or a stereo camera. A plurality of cameras can also be provided to photograph the left and right sides and the rear in addition to the front of the autonomous vehicle 2. The autonomous vehicle 2 can also be equipped with an external camera facing the remote operator R. The external camera facing the remote operator R photographs at least the front of the autonomous vehicle 2. The external camera facing the remote operator R can also be composed of a plurality of cameras that photograph the surroundings including the sides and the rear of the autonomous vehicle 2.

[0045] The external sensor 11 may also include a radar sensor. A radar sensor is a detection device that uses radio waves (such as millimeter waves) or light to detect objects around the autonomous driving vehicle 2. Radar sensors include, for example, millimeter wave radars or laser radars (LIDAR: Light Detection and Ranging). The radar sensor sends radio waves or light to the vicinity of the autonomous driving vehicle 2, and receives radio waves or light reflected by objects, thereby detecting the objects. The radar sensor sends the detected object information to the autonomous driving ECU 20. Objects include not only fixed objects such as guardrails and buildings, but also moving objects such as pedestrians, bicycles, and other vehicles. In addition, the external sensor 11 may also include a sound detection sensor that detects sounds outside the autonomous driving vehicle 2.

[0046] In addition, as described in detail later, when determining whether the autonomous driving operating range and the remote function operating range are inside or outside based on the radio wave state, the external sensor 11 may include a sensor such as a portable radio wave antenna.

[0047] Here, in a vehicle equipped with an autonomous driving system that performs autonomous driving and a remote system that performs a remote function, it is possible to consider a case where each of the external sensors 11 is shared between the autonomous driving system and the remote system, or a case where each of the external sensors 11 is not shared. As an example of a case where sensors are shared, for example, a camera that recognizes the color of traffic lights in the autonomous driving system and a camera that captures images to be transmitted to a remote operator R during remote driving in the remote function can be shared.

[0048] In addition, an example of a case where sensors are not shared is described. As such a case, for example, it can be considered that an autonomous driving system uses radar to identify the vehicle speed. On the other hand, it is difficult to imagine that the remote operator R visually observes the speed numerical information observed from the radar when performing remote driving in a remote function. For example, in a collision prevention system (such as a pre-collision safety system) in an autonomous driving system, although it is conceivable that the system uses radar, it is difficult to imagine that the remote operator R directly uses the radar. In addition, as another example, it can be considered that a lidar information is used in order to accurately measure the distance in an autonomous driving system. On the other hand, it is difficult to imagine that in remote driving, the remote operator R uses lidar information in priority to the image captured by the camera.

[0049] Internal sensors 12 are onboard sensors that detect the status of the autonomous vehicle 2. They include a GPS (Global Positioning System) sensor, an IMU (Inertial Measurement Unit), and a vehicle speed sensor. The GPS sensor measures the position of the autonomous vehicle 2 (e.g., the latitude and longitude of the autonomous vehicle 2) by receiving signals from three or more GPS satellites. The GPS sensor transmits the measured position information of the autonomous vehicle 2 to the autonomous vehicle ECU 20.

[0050] The IMU includes an acceleration sensor and a yaw angular velocity sensor, etc. The acceleration sensor is a detector that detects the acceleration of the autonomous driving vehicle 2. The acceleration sensor includes, for example, a front and rear acceleration sensor that detects the acceleration in the front and rear directions of the autonomous driving vehicle 2. The acceleration sensor may also include a lateral acceleration sensor that detects the lateral acceleration of the autonomous driving vehicle 2. The acceleration sensor, for example, sends the acceleration information of the autonomous driving vehicle 2 to the autonomous driving ECU 20. The yaw angular velocity sensor is a detector that detects the yaw angular velocity (rotational angular velocity) of the center of gravity of the autonomous driving vehicle 2 around the vertical axis. For example, a gyroscope sensor can be used as the yaw angular velocity sensor. The yaw angular velocity sensor sends the detected yaw angular velocity information of the autonomous driving vehicle 2 to the autonomous driving ECU 20.

[0051] The vehicle speed sensor detects the speed of the autonomous vehicle 2. A wheel speed sensor can be used as the vehicle speed sensor. This wheel speed sensor is provided on the wheels of the autonomous vehicle 2 or on a drive shaft that rotates integrally with the wheels, and detects the rotational speed of each wheel. The vehicle speed sensor transmits the detected vehicle speed information (wheel speed information) to the autonomous driving ECU 20.

[0052] The autonomous driving ECU 20 can calculate the motion state (position, speed, acceleration, azimuth, roll angle, pitch angle, yaw angle, rotation speed, etc.) of the autonomous driving vehicle 2 based on the detection results of the internal sensor 12 .

[0053] It should be noted that, although the sensors are classified into the external sensor 11 and the internal sensor 12 here, there is no difference in the functions of the external sensor 11 and the internal sensor 12 as components that observe objects and output data to the automatic driving ECU 20 .

[0054] The map database 13 is a database that records map information. The map database 13 is formed in a recording device such as an HDD (Hard Disk Drive) installed in the autonomous driving vehicle 2. The map information includes road location information, road shape information (such as curvature information), intersection and fork location information, etc. The map information may also include traffic control information such as legal speed associated with the location information. The map information may also include object landmark information used to obtain the location information of the autonomous driving vehicle 2. Road signs, road surface markings, traffic lights, utility poles, etc. can be used as landmarks. The map database 13 can also be configured on a server that can communicate with the autonomous driving vehicle 2.

[0055] The communication unit 14 is a communication device that controls wireless communication with the outside of the autonomous driving vehicle 2. The communication unit 14 transmits and receives various information to and from the remote driving server 4 via the network N.

[0056] The actuator 15 is a device for controlling the autonomous driving vehicle 2. The actuator 15 includes at least a drive actuator, a brake actuator, and a steering actuator. The drive actuator controls the amount of air supplied to the engine (throttle opening) according to the control signal from the autonomous driving ECU 20, thereby controlling the driving force of the autonomous driving vehicle 2. It should be noted that when the autonomous driving vehicle 2 is a hybrid vehicle, in addition to the amount of air supplied to the engine, a control signal from the autonomous driving ECU 20 is input to the motor serving as the power source to control the driving force. When the autonomous driving vehicle 2 is an electric vehicle, a control signal from the autonomous driving ECU 20 is input to the motor serving as the power source to control the driving force. The motor serving as the power source in these cases constitutes the actuator 15.

[0057] The brake actuator controls the braking system based on control signals from the autonomous driving ECU 20, thereby controlling the braking force applied to the wheels of the autonomous driving vehicle 2. For example, a hydraulic brake system can be used as the braking system. The steering actuator controls the driving of the assist motor in the electric power steering system, which controls the steering torque, based on control signals from the autonomous driving ECU 20. Thus, the steering actuator controls the steering torque of the autonomous driving vehicle 2.

[0058] Next, an example of the functional configuration of the autonomous driving ECU 20 will be described. The autonomous driving ECU 20 includes a vehicle position acquisition unit 21, an external environment recognition unit 22, a driving state recognition unit 23, a remote function selection unit (remote function selection device) 24, a route generation unit 25, and a driving control unit 26.

[0059] The vehicle position acquisition unit 21 acquires the vehicle position information of the autonomous vehicle 2 based on the position information from the GPS sensor and the map information in the map database 13. Alternatively, the vehicle position acquisition unit 21 may utilize the object information included in the map information in the map database 13 and the detection results of the external sensor 11 to acquire the vehicle position information of the autonomous vehicle 2 using SLAM (Simultaneous Localization and Mapping) technology. Alternatively, the vehicle position acquisition unit 21 may identify the lateral position of the autonomous vehicle 2 relative to the lane (the position of the autonomous vehicle 2 in the lane width direction) based on the relationship between the lane dividing lines and the position of the autonomous vehicle 2, and include this lateral position in the vehicle position information. Alternatively, the vehicle position acquisition unit 21 may acquire the vehicle position information of the autonomous vehicle 2 using a predetermined method.

[0060] The external environment recognition unit 22 recognizes the external environment of the autonomous vehicle 2 based on the detection results of the external sensor 11. The external environment includes the relative positions of the surrounding objects relative to the autonomous vehicle 2. The external environment information may also include the relative speed and movement direction of the surrounding objects relative to the autonomous vehicle 2. The external environment information may also include the types of objects such as other vehicles, pedestrians, bicycles, etc. The type of object can be identified by methods such as pattern matching. The external environment information may also include the results of the identification of the dividing line around the autonomous vehicle 2 (white line identification). The external environment information may also include the identification results of the lighting state of the traffic light. For example, the external environment recognition unit 22 can identify the lighting state of the traffic light in front of the autonomous vehicle 2 (whether it is a lighting state that allows passing or a lighting state that prohibits passing, etc.) based on the image of the camera of the external sensor 11.

[0061] The driving state recognition unit 23 recognizes the driving state of the autonomous vehicle 2 based on the detection results of the internal sensor 12. The driving state includes the speed, acceleration, and yaw rate of the autonomous vehicle 2. Specifically, the driving state recognition unit 23 recognizes the speed of the autonomous vehicle 2 based on the acceleration information from the speed sensor. The driving state recognition unit 23 recognizes the acceleration of the autonomous vehicle 2 based on the speed information from the acceleration sensor. The driving state recognition unit 23 recognizes the heading of the autonomous vehicle 2 based on the yaw rate information from the yaw rate sensor.

[0062] The remote function selection unit 24 determines whether a remote request should be made to the remote operator R. Here, the remote function selection unit 24 makes a remote request when the autonomously driven vehicle 2 is unable to autonomously drive itself. Furthermore, if multiple executable (operable) remote functions exist at the time of the remote request, the remote function selection unit 24 selects the remote function to be executed and makes the remote request to the remote operator R.

[0063] In more detail, Figure 3 As shown, the remote function selection unit 24 includes an automatic driving investigation unit 31 , a remote function investigation unit 32 , an action determination unit 33 , a confidence calculation unit 34 , a distance calculation unit 35 , a function selection unit 36 ​​, and a remote driving request unit 37 .

[0064] The autonomous driving investigation unit 31 investigates the operating status (whether it is in operation or not) of the system used by the autonomous driving vehicle 2 at a predetermined time point. Here, the autonomous driving investigation unit 31 investigates the operating status at the current and future time points, which are the predetermined time points. Specifically, the autonomous driving investigation unit 31 determines whether the vehicle is within or outside the autonomous driving operating range based on input from the external sensors 11 and the internal sensors 12. It should be noted that if the vehicle is within the autonomous driving operating range, autonomous driving can be performed; if it is outside the autonomous driving operating range, autonomous driving cannot be performed.

[0065] Here, the so-called inside and outside of the autonomous driving working range includes: the inside and outside relationship between the autonomous driving vehicle 2 and the autonomous driving working range when defined by the position relationship; and the inside and outside relationship between the autonomous driving vehicle 2 and the autonomous driving working range when defined by the time axis.

[0066] First, the method of expressing the internal and external relationship between the autonomous driving vehicle 2 and the autonomous driving operating range when the position relationship is defined is described. For example, the internal and external relationship in this case is as follows: Figure 4AAs shown, the autonomous driving operating range A1 and the autonomous driving vehicle 2 are configured in space, and it can be shown at the current position of the autonomous driving vehicle 2 whether the autonomous driving vehicle 2 is within the autonomous driving operating range A1 or outside the autonomous driving operating range A1. Figure 4A In the example shown, at the current position, the autonomous driving vehicle 2 is outside the autonomous driving operation range A1.

[0067] In addition, there is a method in which the position of the autonomous driving vehicle 2 at a future time is used to indicate whether the autonomous driving vehicle 2 is within or outside the autonomous driving operating range A1 at a future time based on information from a navigation system or the like. Figure 4A In the example shown, the location P that the autonomous driving vehicle 2 will arrive at at a certain point in the future falls within the autonomous driving operation range A1.

[0068] In addition, for example, the internal and external relationship Figure 4B As shown, it is possible to indicate whether the vehicle is within or outside the automatic driving operation range A2 on the linear or strip-shaped route L based on the route L of the automatic driving system. Figure 4B In the example shown, at the current position, the autonomous driving vehicle 2 is outside the autonomous driving operation range A2. In addition, the area from point P1 to point P2 on the route L where the autonomous driving vehicle 2 will travel in the future is within the autonomous driving operation range A2. Figure 4A and Figure 4B The examples described here differ only in the way they are expressed, not in the techniques themselves.

[0069] Next, the method of expressing the internal and external relationship between the autonomous driving vehicle 2 and the autonomous driving operation range when the time axis is defined will be described. Figure 4A In the example shown, it can be represented by the time of arrival at location P. Figure 4B In the example shown, the time taken to travel from point P1 to point P2 can be expressed. These times can be estimated based on a navigation system, a route plan of an autonomous driving system, or congestion information.

[0070] The autonomous driving investigation unit 31 outputs autonomous driving operation information indicating whether the vehicle is inside or outside the autonomous driving operation range to the action determination unit 33. This autonomous driving operation information includes either or both of the aforementioned information indicating the relationship between the autonomous driving vehicle 2 and the autonomous driving operation range, defined by position, and the information indicating the relationship between the autonomous driving vehicle 2 and the autonomous driving operation range, defined by time.

[0071] Based on this autonomous driving operation information, it can be determined whether the current time point (current location) is within or outside the autonomous driving operation range. Furthermore, based on the autonomous driving operation information, it can be determined whether a future time point or location is within or outside the autonomous driving operation range.

[0072] It should be noted that the autonomous driving system in this embodiment may be any system. Furthermore, its autonomous driving level is irrelevant. If external devices such as databases, vehicle-to-vehicle communication, and other infrastructure equipment are required to implement the autonomous driving system, the autonomous driving investigation unit 31 may also receive various information inputs from these devices.

[0073] Furthermore, the autonomous driving investigation unit 31 may also utilize a database for determining whether the autonomous driving operating range is inside or outside. For example, if the inside and outside of the autonomous driving service provision area are defined by location, the determination of whether the autonomous driving operating range is inside or outside can be performed based on map data, GPS sensor data, and the like. Alternatively, if the autonomous driving operating range is based on radio wave reception status, the autonomous driving investigation unit 31 may receive real-time information from the radio wave reception status to determine whether the autonomous driving operating range is inside or outside. It should be noted that the autonomous driving investigation unit 31 may also determine whether the autonomous driving operating range is inside or outside based not on a single piece of information, but rather based on multiple pieces of information (e.g., a combination of a map database and radio wave reception status).

[0074] The remote function investigation unit 32 investigates the operating status (whether it is in an operating state or a non-operating state) of the remote function used for remote driving of the autonomous driving vehicle 2 at a specified time point. Here, the remote function investigation unit 32 investigates the operating status at the current and future time points as the specified time points. Specifically, the remote function investigation unit 32 determines whether it is within the operating range of the remote function or outside the operating range of the remote function based on the input from the external sensor 11 and the internal sensor 12. It should be noted that if it is within the operating range of the remote function, remote driving using the remote function can be performed, and if it is outside the operating range of the remote function, remote driving using the remote function cannot be performed.

[0075] Here, similar to the autonomous driving operating range, the terms "inside" and "outside" of the remote function operating range include: the positional relationship between the autonomous driving vehicle 2 and the remote function operating range; and the temporal relationship between the autonomous driving vehicle 2 and the remote function operating range. The remote function investigation unit 32 can determine whether the remote function operating range is inside or outside the remote function operating range using the same method as the autonomous driving investigation unit 31.

[0076] In particular, the remote function investigation unit 32 determines whether each of the multiple remote functions is within or outside the remote function's operating range. Specifically, in this embodiment, the remote function investigation unit 32 determines, for example, whether the remote function is within or outside the operating range for remote driving enabled by remote support, a remote function; and whether the remote function is within or outside the operating range for remote driving enabled by remote driving, a remote function.

[0077] The remote function investigation unit 32 outputs remote function operation information indicating whether the vehicle is within or outside the remote function operation range to the action determination unit 33. This remote function operation information includes either or both information indicating the internal / external relationship between the autonomous driving vehicle 2 and the remote function operation range, as defined by position, and information indicating the internal / external relationship between the autonomous driving vehicle 2 and the remote function operation range, as defined by time. Furthermore, as described above, this remote function operation information includes information indicating whether each of the multiple remote functions is within or outside the remote function operation range.

[0078] The operation determination unit 33 determines whether remote driving using the remote function is possible when autonomous driving is not possible based on the automatic driving operation information and the remote function operation information. Here, the operation determination unit 33 determines whether remote driving using each remote function is possible.

[0079] Specifically, the action determination unit 33 can calculate whether the current time point is outside the autonomous driving operating range, as well as the position and / or time at which the vehicle will be outside the autonomous driving operating range in the future, based on the autonomous driving operation information from the autonomous driving investigation unit 31. Furthermore, the action determination unit 33 can calculate whether the current time point is outside the remote function operating range, as well as the position and / or time at which the vehicle will be outside the remote function operating range in the future, based on the remote function operation information from the remote function investigation unit 32. This allows the action determination unit 33 to determine whether the vehicle is inside or outside the remote function operating range for each remote function at the position and / or time at which the vehicle will be outside the autonomous driving operating range.

[0080] For example, Figure 5A As shown, the action determination unit 33 can calculate the position and / or time when the autonomous driving vehicle 2 moves from within the autonomous driving operation range A3 to outside the autonomous driving operation range A3 based on the autonomous driving operation information from the autonomous driving investigation unit 31. Figure 5A FIG shows a state where the autonomous driving vehicle 2 moves from within the autonomous driving operating range A3 to outside the autonomous driving operating range A3 at location P3. Figure 5BAs shown, the action determination unit 33 can determine whether the autonomous driving vehicle 2 is within the remote function operation range when it arrives at the location P3 based on the remote function operation information from the remote function investigation unit 32. Here, the action determination unit 33 determines whether it is within the remote function operation range for each remote function. Figure 5B , when the autonomous driving vehicle 2 arrives at the point P3 , it is shown that the autonomous driving vehicle 2 is in a state within a remote function operating range B3 where one of the multiple remote functions can be executed.

[0081] In this way, the action determination unit 33 can determine whether the vehicle is inside or outside the autonomous driving operating range at a future time. Therefore, the action determination unit 33 can determine in advance how long or how many times the autonomous driving vehicle 2 will be outside the autonomous driving operating range in the future trajectory. If the vehicle is outside the autonomous driving operating range, the action determination unit 33 determines that remote driving using a remote function (remote support or remote driving) is necessary.

[0082] Here, the action determination unit 33 functions as an automatic driving determination unit that determines whether automatic driving is not possible at a predetermined time point (currently or at a future time point). Furthermore, if the action determination unit 33 determines that automatic driving is not possible, it functions as a remote function determination unit that determines whether a remote function is executable at a predetermined time point (currently or at a future time point).

[0083] The confidence calculation unit 34 predicts the behavior of objects (e.g., other vehicles, pedestrians, etc.) based on the detection results of the external sensors 11 that detect objects around the autonomous vehicle 2. Hereinafter, the objects around the autonomous vehicle 2 detected by the external sensors 11 are referred to as peripheral objects. Furthermore, the confidence calculation unit 34 calculates the behavior prediction confidence level for the predicted behavior of the peripheral objects. This behavior prediction confidence level represents the confidence level in the prediction of the predicted behavior of the peripheral objects. The confidence calculation unit 34 can calculate the behavior prediction confidence level for the peripheral objects using various methods.

[0084] For example, the confidence calculation unit 34 can apply a probabilistic model to calculate the confidence level of the predicted behavior of surrounding objects. In this case, for example, the confidence calculation unit 34 inputs the motion information (position, speed, direction, etc.) of the surrounding objects detected by the external sensor 11 into a vehicle motion prediction model constructed based on general traffic data. The confidence calculation unit 34 then calculates all possible future behaviors of the surrounding objects and the probability of each behavior occurring. The confidence calculation unit 34 can use the maximum of the calculated probabilities as the predicted behavior confidence level.

[0085] Alternatively, the confidence calculation unit 34 may construct a probability model based on observational data of representative abnormal behaviors. The confidence calculation unit 34 can then calculate the inverse of the likelihood (closeness to the model) of the behavior of the surrounding objects relative to the probability model and use the calculated value as the behavior prediction confidence. In other words, the confidence calculation unit 34 determines that behavior prediction is difficult if the behavior of the surrounding objects appears to be abnormal.

[0086] Alternatively, the confidence calculation unit 34 may use qualitative information as the action prediction confidence. For example, the confidence calculation unit 34 may calculate the action prediction confidence based on whether the drivers of other vehicles around the autonomous vehicle 2 are driving under the influence. It is believed that if the drivers of other vehicles are driving under the influence, the other vehicles may take unexpected actions. Therefore, the confidence calculation unit 34 lowers the action prediction confidence if the drivers of other vehicles are driving under the influence.

[0087] For example, the confidence calculation unit 34 obtains a camera image of the driver of another vehicle from the camera of the external sensor 11. The confidence calculation unit 34 may also determine whether the driver of the other vehicle is driving under the influence by performing image processing on the obtained camera image of the driver of the other vehicle. Alternatively, other vehicles may be equipped with a driver monitoring device to monitor the driver's condition. In this case, the confidence calculation unit 34 obtains a camera image from a camera installed in the driver monitoring device of the other vehicle, which is capturing the driver's condition. The confidence calculation unit 34 may then determine whether the driver of the other vehicle is driving under the influence based on the camera image obtained from the other vehicle. Alternatively, the confidence calculation unit 34 may obtain information on whether the driver of the other vehicle is driving under the influence from a driver monitoring device installed in the other vehicle. Furthermore, other vehicles may be equipped with an alcohol checker. In this case, the confidence calculation unit 34 may also obtain the test results of the alcohol checker from the other vehicle to determine whether the driver of the other vehicle is driving under the influence.

[0088] In addition, the confidence calculation unit 34 can determine that an emergency vehicle such as an ambulance is an object whose behavior prediction is difficult. In this case, the confidence calculation unit 34 can lower the confidence of the behavior prediction of the emergency vehicle. In addition, the confidence calculation unit 34 can also lower the confidence of the behavior prediction of the objects around the autonomous driving vehicle 2 when the emergency vehicle is approaching. For example, the confidence calculation unit 34 can determine whether an emergency vehicle is approaching based on the detection results of the emergency vehicle's sirens obtained by a sound detection sensor (audio sensor). Alternatively, the confidence calculation unit 34 can also determine whether an emergency vehicle is approaching based on the notification of the approach of the emergency vehicle notified by the road-to-vehicle communication.

[0089] The distance calculation unit 35 calculates the relative distance between the surrounding object and the autonomous driving vehicle 2. Here, the distance calculation unit 35 calculates the relative distance between the surrounding object whose behavior prediction confidence is calculated by the confidence calculation unit 34 and the autonomous driving vehicle 2.

[0090] The distance calculation unit 35 may, for example, use a value directly measured by the laser radar of the external sensor 11 as the relative distance. Alternatively, the distance calculation unit 35 may calculate the relative distance through image processing based on a camera image captured by the camera of the external sensor 11. Furthermore, the distance calculation unit 35 may use a value directly measured by the radar of the external sensor 11 as the relative distance. Furthermore, the distance calculation unit 35 may obtain GPS information of other vehicles through inter-vehicle communication and calculate the relative distance based on the position of the other vehicles based on the obtained GPS information and the position of the autonomous driving vehicle 2.

[0091] The distance calculation unit 35 may also calculate the relative distance by taking the direction of travel into consideration. For example, when the surrounding object is moving in a direction away from the autonomous driving vehicle 2, the distance calculation unit 35 may determine that the relative distance is long, and adjust the relative distance. In addition, in addition to using the spatial distance, the distance calculation unit 35 may also use the time until approaching the surrounding object or TTC (Time to Collision) as the relative distance. The distance calculation unit 35 may also use a probabilistic indicator such as the possibility of collision with the surrounding object as the relative distance. In addition, the distance calculation unit 35 may also combine these multiple distance benchmarks as the relative distance.

[0092] The function selection unit 36 ​​selects a remote function to be executed from among the multiple remote functions determined by the action determination unit 33 to be executable. Here, the function selection unit 36 ​​selects the remote function to be executed based on the action prediction confidence calculated by the confidence calculation unit 34 and the relative distance calculated by the distance calculation unit 35. It should be noted that if only one remote function is determined by the action determination unit 33 to be executable, the function selection unit 36 ​​selects the remote function determined by the action determination unit 33 to be executable.

[0093] In remote support, a remote function, remote operators R do not operate the steering wheel, accelerator, or brakes. Therefore, compared to remote driving, remote support has less stringent requirements for communication latency and capacity, and does not require specialized operating devices (such as steering wheel controls). Furthermore, remote support reduces the time required by remote operators R, resulting in lower costs.

[0094] Specifically, the remote operator R who performs remote support only performs a replacement judgment. Therefore, the information for remote support (such as camera images, etc.) sent and received between the autonomous driving vehicle 2 and the remote driving server 4 can be stopped immediately after the remote operator R makes a judgment. For example, Figure 6A As shown, remote support may be provided for the automated vehicle 2 to enter an intersection. In this case, the transmission and reception of information for remote support begins at point S immediately before the intersection. Furthermore, after the remote support operator determines that the vehicle has entered the intersection, the transmission and reception of information ceases at point E.

[0095] On the other hand, the remote operator R who performs remote driving performs all the recognition, judgment and operation. Therefore, the information for remote driving (such as the images captured by the camera, etc.) sent and received between the autonomous driving vehicle 2 and the remote driving server 4 needs to be sent and received until the remote driving of the autonomous driving vehicle 2 is completed. For example, Figure 6B As shown, remote driving may be performed to allow the autonomous vehicle 2 to pass through an intersection. In this case, the transmission and reception of information for remote driving begins at point S immediately before entering the intersection. The remote driving operator then begins remote driving of the autonomous vehicle 2, and the transmission and reception of information ceases at point E, after the autonomous vehicle 2 has passed the intersection.

[0096] Thus, between the occupied time of the remote support operator and the occupied time of the remote driving operator, the occupied time of the remote driving operator who generally performs remote driving is longer.

[0097] Even in scenarios like those in Examples 1-6 above, which serve as examples of remote support, remote support can sometimes be difficult to resolve, depending on the behavior of other vehicles and pedestrians around the autonomous vehicle 2. This is particularly true when there are other vehicles nearby, such as those with drunk drivers, or when there are other vehicles or pedestrians whose behavior is difficult to predict. In such situations, remote support can be a suitable option because the remote operator R has a longer availability and offers the flexibility of long-term human judgment and operation.

[0098] Therefore, when there are surrounding objects that make behavior prediction difficult, the function selection unit 36 ​​selects the remote driving function in which the remote operator R is occupied for a longer period of time, between remote support and remote driving. Here, the function selection unit 36 ​​uses the behavior prediction confidence calculated by the confidence calculation unit 34 as an indicator of whether behavior prediction is difficult. That is, when the behavior prediction confidence is low, the function selection unit 36 ​​selects the remote function in which the remote operator R is occupied for a longer period of time than when the behavior prediction confidence is high.

[0099] Furthermore, when the relative distance to the surrounding objects calculated by the distance calculation unit 35 is long, the function selection unit 36 ​​selects a remote function that reduces the time occupied by the remote operator R compared to when the relative distance is short. In this way, the function selection unit 36 ​​selects remote functions based on the relative distance to the surrounding objects in addition to the action prediction confidence level.

[0100] When autonomous driving of the autonomous vehicle 2 becomes unavailable (outside the autonomous driving operating range), the remote driving request unit 37 remotely requests the remote driving server 4 (remote operator R) to execute the remote function selected by the function selection unit 36. The remote driving request unit 37 transmits various information necessary for executing the requested remote function, such as identification information of the autonomous vehicle 2, vehicle position information, and external environment information, to the remote driving server 4 along with the remote request.

[0101] return Figure 2 The route generation unit 25 generates a route for use in autonomous driving of the autonomous vehicle 2. The route generation unit 25 generates the route for autonomous driving based on a preset driving route, map information, position information of the autonomous vehicle 2, the external environment of the autonomous vehicle 2, and the driving state of the autonomous vehicle 2.

[0102] The driving route refers to the route that the autonomous vehicle 2 travels during autonomous driving. The route generation unit 25 calculates the autonomous driving route based on, for example, the destination, map information, and the position information of the autonomous vehicle 2. The driving route can also be set by the navigation system. The destination can be set by the occupant of the autonomous vehicle 2 or automatically provided by the autonomous driving ECU 20 or the navigation system.

[0103] The route includes the path along which the vehicle will travel in autonomous driving mode and the vehicle speed profile during autonomous driving. The path is the predetermined trajectory that the autonomous vehicle will follow along the route. For example, the path can be data (steering angle profile) showing changes in the steering angle of the autonomously driven vehicle 2 corresponding to positions along the route. Positions along the route, for example, are set longitudinal positions at predetermined intervals (e.g., 1 meter) in the direction of travel along the route. The steering angle profile is data that associates a target steering angle with each set longitudinal position.

[0104] The route generation unit 25 generates a path for the autonomous vehicle 2 to travel based on, for example, the travel route, map information, the external environment of the autonomous vehicle 2, and the travel state of the autonomous vehicle 2. For example, the route generation unit 25 generates a path so that the autonomous vehicle 2 passes through the center (the center in the lane width direction) of a lane included in the travel route.

[0105] The vehicle speed curve is, for example, data that associates a target vehicle speed with each set longitudinal position. It should be noted that the set longitudinal position can also be set based on the travel time of the autonomous driving vehicle 2 rather than the distance. The set longitudinal position can also be set as the vehicle's arrival position in one second or two seconds.

[0106] The route generation unit 25 generates a speed profile based on, for example, traffic control information such as legal speeds included in the route and map information. A pre-set speed for a location or section on the map may be used in place of the legal speed. The route generation unit 25 generates an autonomous driving route based on the route and speed profile. It should be noted that the route generation method used by the route generation unit 25 is not limited to the one described above; methods related to autonomous driving may be employed. The same applies to the content of the route.

[0107] When a request for remote support in the remote function is made to the remote driving server 4 through the remote function selection unit 24, the route generation unit 25 pre-generates a route corresponding to the requested remote support. The options for the remote support content are predetermined according to the status of the autonomous driving vehicle 2. For example, the options for the remote support content when turning right at an intersection include a proposal to start turning right (moving) and a proposal to wait. The options for the remote support content when turning right at an intersection may include a proposal to give up turning right and go straight, and may also include a proposal for emergency retreat. It should be noted that the route generation unit 25 does not necessarily need to generate the route in advance, and may also generate the route corresponding to the remote support content after receiving the remote support content.

[0108] The travel control unit 26 executes autonomous driving of the autonomous vehicle 2. The travel control unit 26 executes autonomous driving of the autonomous vehicle 2 based on, for example, the external environment of the autonomous vehicle 2, the driving state of the autonomous vehicle 2, and the route generated by the route generation unit 25. The travel control unit 26 executes autonomous driving of the autonomous vehicle 2 by sending control signals to the actuator 15.

[0109] When a remote request is made to the remote travel server 4 via the remote function selection unit 24, the travel control unit 26 waits for receipt of a remote instruction from the remote travel server 4. Upon receiving the remote instruction, the travel control unit 26 controls the travel of the autonomous vehicle 2 in such a manner as to execute the remote support or remote driving indicated in the remote instruction. For example, upon receiving a remote instruction for remote support, the travel control unit 26 performs autonomous driving of the autonomous vehicle 2 based on the remote support instruction. Furthermore, upon receiving a remote instruction for remote driving, for example, the travel control unit 26 transmits a control signal to the actuator 15 to cause the autonomous vehicle 2 to travel in accordance with the remote driving instruction.

[0110] Next, use Figure 7 The flowchart shown in FIG. 1 illustrates the flow of the remote function selection process performed in the remote function selection unit 24. Figure 7 As shown in the following example. Figure 7 As shown, the operation determination unit 33 of the remote function selection unit 24 determines whether the automatic driving is currently in a state where it cannot be continued based on the automatic driving operation information from the automatic driving investigation unit 31 (S101). If the automatic driving is currently in a state where it cannot be continued (S101: Yes), the operation determination unit 33 determines a remote function that can be executed at the current time based on the remote function operation information from the remote function investigation unit 32 (S102).

[0111] On the other hand, if the automatic driving can be continued at the current time point (S101: No), the action determination unit 33 determines whether the automatic driving will become uncontinuable at a future time point based on the automatic driving operation information from the automatic driving investigation unit 31 (S103). It should be noted that the future time point can be set to a time point from the current time point to a time point after a predetermined time. If the automatic driving cannot be continued at a future time point (S103: Yes), the action determination unit 33 determines the remote function that can be executed at the future time point when the automatic driving cannot be continued based on the remote function operation information from the remote function investigation unit 32 (S104). On the other hand, if the automatic driving can be continued at a future time point (S103: No), the automatic driving ECU 20 continues the automatic driving of the automatic driving vehicle 2.

[0112] After determining the remote functions that can be executed (after processing S102 or S104), the function selection unit 36 ​​determines whether the remote function determined to be executable is one or more (S105). Here, the function selection unit 36 ​​determines whether the remote functions determined to be executable are one or more. It should be noted that if there are no remote functions determined to be executable, there are no remote functions that can be executed when autonomous driving cannot be performed. In this case, when the autonomous driving ECU 20 cannot continue autonomous driving, it can take various measures such as stopping the autonomous driving vehicle 2 or reporting to the passengers of the autonomous driving vehicle 2.

[0113] If only one remote function is determined to be executable (S105: Yes), the function selection unit 36 ​​selects the remote function determined to be executable as the remote function to be executed (S106). On the other hand, if more than one remote function is determined to be executable (S105: No), that is, if there are multiple remote functions, the function selection unit 36 ​​selects a remote function to be executed from among the multiple remote functions based on the behavior prediction confidence calculated by the confidence calculation unit 34 and the relative distance calculated by the distance calculation unit 35 (S107). After selecting the remote function to be executed (after the processing of S106 or S107), the remote driving request unit 37 makes a remote request to the remote driving server 4 to execute the selected remote function at a time (at the current time or a future time) when autonomous driving cannot be executed (S108).

[0114] Next, use Figure 8 Flowchart of Figure 7 The details of the remote function selection process performed in S107 are described. Figure 8The example shown is an example of a representative embodiment. Here, a predetermined confidence threshold is used to divide the case where the action prediction confidence is high and the case where the action prediction confidence is low. That is, the case where the action prediction confidence is above the predetermined confidence threshold is set as the case where the action prediction confidence is high, and the case where the action prediction confidence is less than the predetermined confidence threshold is set as the case where the action prediction confidence is low. Similarly, a predetermined relative distance threshold is used to divide the case where the relative distance to the surrounding objects is long and the case where the relative distance to the surrounding objects is short. That is, the case where the relative distance is above the predetermined relative distance threshold is set as the case where the relative distance is long, and the case where the relative distance is less than the predetermined relative distance threshold is set as the case where the relative distance is short.

[0115] like Figure 8 As shown, the function selection unit 36 ​​determines whether the action prediction confidence calculated by the confidence calculation unit 34 is greater than a preset confidence threshold (S201). If the action prediction confidence is greater than the confidence threshold (S201: Yes), the function selection unit 36 ​​selects a remote function (in this embodiment, remote support) with a shorter occupancy time of the remote operator R from among the executable remote functions (S202).

[0116] On the other hand, if the action prediction confidence level is not equal to or greater than the confidence level threshold (S201: No), the function selection unit 36 ​​determines whether the relative distance to the surrounding objects calculated by the distance calculation unit 35 is equal to or greater than a predetermined relative distance threshold (S203). If the relative distance to the surrounding objects is equal to or greater than the predetermined relative distance threshold (S203: Yes), the function selection unit 36 ​​performs the process of S202 described above. If the relative distance to the surrounding objects is not equal to or greater than the predetermined relative distance threshold (S203: No), the function selection unit 36 ​​selects the remote function (in this embodiment, remote driving) that has been occupied by the remote operator R for a long time from among the executable remote functions (S204).

[0117] It should be noted that the preset confidence threshold and relative distance threshold can be fixed or variable. If these thresholds are variable, the distance calculator 35 can increase or decrease them based on the relative vehicle speed (relative velocity). For example, if the relative vehicle speed is high, the distance calculator 35 can increase the confidence threshold and / or relative distance threshold because the time required to achieve close proximity is short.

[0118] In addition to using thresholds to select remote functions, the function selection unit 36 ​​may also use, for example, a decision map that associates action prediction confidence and relative distance with the remote function to be selected. Thus, the function selection unit 36 ​​may also select remote functions based on the relative relationship between action prediction confidence and relative distance. Furthermore, the function selection unit 36 ​​may use probabilistic statistical methods to select the remote function to be selected, or may use a determinator constructed using methods such as machine learning to select the remote function to be selected.

[0119] As described above, the confidence calculation unit 34 calculates the predicted behavior confidence levels of the surrounding objects surrounding the autonomously driven vehicle 2. If the function selection unit 36 ​​determines that multiple remote functions are executable when autonomous driving is disabled, it selects the remote function to be executed based on the predicted behavior confidence levels of the surrounding objects. In this way, even when multiple remote functions are executable, the remote function selection unit 24 can use the predicted behavior confidence levels to appropriately select the remote function to be executed.

[0120] When the behavior prediction confidence level is low, the function selection unit 36 ​​selects a remote function that requires the remote operator R to be active longer than when the behavior prediction confidence level is high. When the behavior prediction confidence level of surrounding objects is low, the surrounding objects may behave in ways that are unpredictable by the autonomous vehicle 2. In the presence of such surrounding objects, the intervention of the remote operator R in the driving operation of the autonomous vehicle 2 allows for flexible response to changes in the behavior of the surrounding objects, thereby minimizing obstruction to traffic flow. Therefore, when the behavior prediction confidence level of a surrounding object is low, the remote function selection unit 24 selects a remote function that requires the remote operator R to be active longer. This allows the remote function selection unit 24 to select an appropriate remote function even when there are nearby vehicles whose behavior is difficult to predict, such as those driving under the influence. In this way, the remote function selection unit 24 can select a more appropriate remote function based on the behavior prediction confidence level of the surrounding objects.

[0121] The remote function selection unit 24 includes a distance calculation unit 35 that calculates the relative distance between the surrounding objects, for which the action prediction confidence is calculated by the confidence calculation unit 34, and the autonomous vehicle 2. The function selection unit 36 ​​selects a remote function to be executed based on the calculated action prediction confidence and the relative distance. In this case, the remote function selection unit 24 can select a more appropriate remote function by taking into account the relative distance between the surrounding objects and the autonomous vehicle 2.

[0122] When the relative distance between the autonomous vehicle 2 and the surrounding objects is long, the function selection unit 36 ​​selects a remote function that requires less time for the remote operator R to operate, compared to when the relative distance is short. When the relative distance between the autonomous vehicle 2 and the surrounding objects is long, there's more time for the autonomous vehicle 2 to approach the surrounding objects. In such situations, active driving of the autonomous vehicle 2 by the remote operator R may not be necessary. Therefore, when the relative distance between the autonomous vehicle 2 and the surrounding objects is long, the remote function selection unit 24 selects a remote function that requires less time for the remote operator R to operate. This prevents excessive selection of remote functions (remote driving in this embodiment) that require more time for the remote operator R to operate, thereby reducing the cost of engaging the remote operator R. In this way, the remote function selection unit 24 can select more appropriate remote functions based on the relative distance between the autonomous vehicle 2 and the surrounding objects.

[0123] The remote functions include remote support and remote driving. Thus, the remote function selection unit 24 can select a more appropriate remote function from the remote functions including remote support and remote driving.

Claims

1. A remote function selection device configured to select a remote function to be executed in an autonomous vehicle, wherein: The autonomous driving vehicle is configured to perform autonomous driving and remote driving based on remote instructions from a remote operator, and is equipped with a plurality of remote functions for performing the remote driving. The remote function selection device is characterized by including: an automatic driving determination unit configured to determine whether the automatic driving cannot be performed at a predetermined time point; a remote function determination unit configured to determine the remote function that can be executed at the predetermined time point when the autonomous driving determination unit determines that the autonomous driving cannot be executed; a confidence calculation unit configured to predict the behavior of an object based on a detection result of an external sensor configured to detect the object around the autonomous driving vehicle, and configured to calculate a behavior prediction confidence of the predicted behavior of the object; and a function selection unit configured to select the remote function to be executed, The function selection unit is configured to select the remote function to be executed from among the multiple remote functions based on the action prediction confidence calculated by the confidence calculation unit when the remote function determination unit determines that the multiple remote functions can be executed.

2. The remote function selection device according to claim 1, characterized in that: The function selection unit is configured to select the remote function for which the remote operator's occupation time is longer when the action prediction confidence is low than when the action prediction confidence is high.

3. The remote function selection device according to claim 1 or 2, characterized in that: Also includes: a distance calculation unit configured to calculate the relative distance between the object and the autonomous driving vehicle, The function selection unit is configured to select the remote function to be executed from among the plurality of remote functions determined by the remote function determination unit based on the action prediction confidence level and the relative distance.

4. The remote function selection device according to claim 3, characterized in that: The function selection unit is configured to select the remote function for which the remote operator's occupation time is shorter when the relative distance is long than when the relative distance is short.

5. The remote function selection device according to claim 1 or 2, characterized in that: The plurality of remote functions include remote support and remote driving.

Images