Notification Control Device

Abstract

To provide a vehicle control device and a vehicle control method that can reduce the possibility of contact with another vehicle in a road section after passing through a gate.SOLUTION: A processor with which an automatic driving ECU is provided determines whether or not a host vehicle has entered a gate region, which is a section in which a gate is present, on the basis of map data and the like. A processor 31 displays an eyes-on request image on the basis of the gate region having been entered by automatic driving control. The eyes-on request image is for asking a driver to monitor the surrounding traffic situation. However, the processor 31 may omit the display of the eyes-on request image if there is not another vehicle present around the host vehicle.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a technology for controlling notifications to a driver when passing through a toll road gate while autonomously driving. [Background technology]

[0002] Patent Document 1 discloses a vehicle control device that automatically passes through a gate of a toll road. The vehicle control device can change the gate through which the vehicle is to pass depending on whether or not a card for paying a toll is inserted in the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6692935 Summary of the Invention [Problem to be solved by the invention]

[0004] Around gates, vehicle tracks are likely to cross, which can increase the risk of collision compared to when driving on a straight road.

[0005] The present disclosure has been made based on the above considerations or points of view, and one of its objectives is to provide a notification control device and a notification control method that can reduce the possibility of contact with other vehicles when driving autonomously near a gate. [Means for solving the problem]

[0006] The notification control device disclosed herein is a notification control device used in a vehicle configured to perform autonomous driving control, and performs the following operations: acquires data indicating whether the vehicle is traveling under autonomous driving control; acquires information about gate points, which are points on a toll road where multiple gates are provided; determines whether the vehicle has entered a gate area defined based on the gate points; and, based on the vehicle's entry into the gate area under autonomous driving control, notifies the driver to check the surrounding traffic conditions.

[0007] In addition, the notification control method disclosed herein is a notification control method executed by a processor provided in a vehicle configured to perform autonomous driving control, and includes obtaining data indicating whether the vehicle is traveling under autonomous driving control, obtaining information about a gate point which is a point on a toll road where multiple gates are provided, determining whether the vehicle has entered a gate area defined based on the gate point, and notifying the driver to check the surrounding traffic conditions based on the vehicle's entry into the gate area under autonomous driving control.

[0008] According to the above device / method, not only the automated driving system but also the driver can check the surrounding traffic conditions near the gate. Therefore, even if there is another vehicle approaching the vehicle excessively, the possibility that the driver will operate the vehicle appropriately can be increased. As a result, the risk of contact with the other vehicle can be reduced. Note that the reference characters in parentheses in the claims indicate a correspondence relationship with the specific means described in the embodiment described later as one aspect, and do not limit the technical scope of the present disclosure. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an autonomous driving system. [Diagram 2] FIG. 2 is a functional block diagram of an autonomous driving ECU. [Diagram 3] 11 is a diagram for explaining a process for setting a target gate. FIG. [Figure 4] 10 is a flowchart showing the operation of the automatic driving ECU when passing through a gate. [Diagram 5] FIG. 13 is a diagram showing an example of an icon image displayed as an eyes-on request. [Figure 6] FIG. 13 is a diagram showing an example of a gate guide image. [Figure 7] FIG. 13 is a diagram showing another example of a gate guide image. [Figure 8] 10 is a flowchart for explaining an example of operation of a notification control unit depending on the presence or absence of nearby vehicles. [Figure 9] 10 is a flowchart illustrating another example of the operation of the notification control unit depending on the presence or absence of nearby vehicles. [Figure 10] 10 is a flowchart for explaining an example of the operation of a notification control unit according to a payment method of a target gate. [Figure 11] 10 is a flowchart for explaining an example of operation of a notification control unit according to an operation mode of an autonomous driving ECU when entering a gate area. [Figure 12] 10 is a flowchart for explaining an example of the operation of a notification control unit depending on whether or not a lane change is to be made in a gate area. [Figure 13] 11 is a diagram for explaining a case where the display content of a gate guide image is changed depending on the operation mode of an autonomous driving ECU when entering a gate area. FIG. [Figure 14] FIG. 13 is a diagram showing an example of a gate passing icon. [Figure 15] 13 is a diagram for explaining a case where the display content of a gate guide image is changed depending on whether or not a preceding vehicle is being followed when entering a gate area. FIG. [Figure 16] FIG. 13 is a diagram showing an example of a trackless gate image. [Figure 17] FIG. 11 is a diagram illustrating an example of control of an operation mode. [Figure 18] FIG. 11 is a diagram illustrating another example of control of the operation mode. [Figure 19] FIG. 11 is a diagram illustrating another example of control of the operation mode. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] <Introduction> Hereinafter, one embodiment of the present disclosure will be described with reference to the drawings. The present disclosure is not limited to the following embodiment, and can be implemented with various modifications within the scope of the gist of the present disclosure. The various supplements and modifications described below can be implemented in appropriate combinations within the scope of no technical contradiction. Members having the same function are given the same reference numerals, and their description may be omitted. In addition, when only a part of the configuration is mentioned, the previous description can be applied to the other parts.

[0011] FIG. 1 is a diagram showing an example of a schematic configuration of an automatic driving system Sys according to the present disclosure. Hereinafter, a vehicle equipped with the automatic driving system Sys is also referred to as a host vehicle. In addition, the term "host vehicle lane" in the present disclosure refers to a lane in which the host vehicle is traveling among a plurality of lanes provided on a road. The host vehicle lane can also be called an ego lane. An adjacent lane is a lane adjacent to the host vehicle lane.

[0012] In this disclosure, a preceding vehicle refers to a vehicle that is in front of the host vehicle, travels in the same lane as the host vehicle, and is closest to the host vehicle. A following vehicle refers to another vehicle traveling behind the host vehicle in the host vehicle lane. A preceding vehicle is not limited to a vehicle traveling in front of the host vehicle in the host vehicle lane, but also includes another vehicle traveling in front of the host vehicle in one or more adjacent lanes. Similarly, a following vehicle includes not only a following vehicle, but also a vehicle traveling diagonally behind the host vehicle.

[0013] The driver in this disclosure refers to a person who sits in the driver's seat, that is, a driver's seat occupant, regardless of whether or not the person is actually driving. For example, the driver in this disclosure may refer to a person who should receive the authority and responsibility of driving operation from the automatic driving system Sys when automatic driving ends. The description of the driver in this disclosure can be replaced with the driver's seat occupant. The vehicle may be a remote-operated vehicle that is remotely operated by an operator outside the vehicle. The person who takes over the driving operation from the automatic driving system Sys may be an operator outside the vehicle. The operator here refers to a person who has the authority to control the vehicle by remote operation from outside the vehicle. The operator is also included in the concept of the driver.

[0014] The automated driving system Sys provides a so-called automated driving function that allows the vehicle to autonomously drive along a predetermined route. There can be multiple levels of automation of driving operations (hereinafter referred to as automation levels), as defined by the Society of Automotive Engineers (SAE International). Automation levels can be classified into six levels, for example, from level 0 to 5, as follows.

[0015] Level 0 is a level equivalent to fully manual driving where the system does not control anything. Level 1 is a level where the system supports either steering or acceleration / deceleration. Level 1 includes cases where only Adaptive Cruise Control (ACC) is executed. Level 2 refers to a level where the system performs both speed adjustment by accelerator and brake operation and left / right control (i.e. steering) by steering wheel operation. At level 2, the system essentially drives the vehicle autonomously, although the driver is required to monitor the surroundings (so-called eyes-on). In this disclosure, control equivalent to level 2 is also referred to as automated driving control with a requirement to monitor the surroundings, level 2 automated driving control, or semi-automated driving control.

[0016] Level 3 refers to a level where the system performs all driving tasks within the Operational Design Domain (ODD), but in an emergency, the system transfers operational authority to the driver. ODD specifies the conditions under which automated driving can be performed. Level 4 is a level where the system performs all driving tasks, except under specific circumstances such as on designated roads that cannot be handled and in extreme environments. Level 5 is a level where the system performs all driving tasks in all environments.

[0017] Automation levels 3 to 5 are automation levels at which the driver does not need to monitor the surroundings, in other words, levels that correspond to autonomous driving. Therefore, in this disclosure, vehicle control equivalent to level 3 or higher is also referred to as autonomous driving control without the obligation to monitor the surroundings.

[0018] The automated driving system Sys described below can be modified as appropriate to suit the local laws and customs, the characteristics of the vehicle / equipment, etc. in which it is used. Unless otherwise specified, the term "system" below refers to the automated driving system Sys.

[0019] <Overall configuration of the Autonomous Driving System Sys> The autonomous driving system Sys includes various components as shown in Fig. 1 as an example. That is, the autonomous driving system Sys includes a surroundings monitoring sensor 11, a vehicle state sensor 12, a locator 13, a map storage unit 14, a wireless communication device 15, an occupant state sensor 16, a body ECU 17, an exterior display device 18, and a driving actuator 19. The autonomous driving system Sys also includes an in-vehicle HMI 20 and an autonomous driving ECU 30. Note that ECU is an abbreviation for Electronic Control Unit, which means an electronic control device. HMI is an abbreviation for Human Machine Interface.

[0020] The autonomous driving ECU 30 is connected to each of the above devices / sensors such as the periphery monitoring sensor 11 via an in-vehicle network IvN so that they can communicate with each other. The in-vehicle network IvN is a communication network built inside the vehicle. Various standards such as Controller Area Network (hereinafter, CAN: registered trademark) and Ethernet (registered trademark) can be adopted as standards for the in-vehicle network IvN. In addition, some of the devices / sensors may be directly connected to the autonomous driving ECU 30 by dedicated signal lines. The connection form between the devices can be changed as appropriate.

[0021] The perimeter monitoring sensor 11 is a sensor that detects an object present within a detection range. The perimeter monitoring sensor 11 can be understood as an autonomous sensor that senses the surrounding environment of the vehicle. The perimeter monitoring sensor can be called an object detection sensor. The autonomous driving system Sys can be equipped with a plurality of perimeter monitoring sensors 11. The autonomous driving system Sys is equipped with, for example, a camera 111 and a millimeter wave radar 112 as the perimeter monitoring sensors 11.

[0022] The camera 111 is a so-called forward camera that is arranged to capture an image of the front of the vehicle at a predetermined angle of view. The camera 111 is arranged on the upper end of the windshield on the interior side of the vehicle, on the front grill, on the roof top, etc. The camera 111 may include a camera ECU in addition to a camera main body that generates an image frame. The camera main body includes at least an image sensor and a lens. The camera ECU includes a processor and a memory. The processor is a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), etc. The camera ECU is an ECU that detects a predetermined detection target by performing recognition processing on the image frame. The camera ECU detects and identifies an object registered as a detection target using, for example, a classifier to which deep learning is applied. The camera ECU also calculates the relative position coordinates of the detection object with respect to the vehicle from position information (for example, pixel coordinates) of the detection object in the image frame.

[0023] Objects detected by the camera 111 include, for example, moving objects such as pedestrians and other vehicles. Objects detected by the camera 111 also include features such as road edges, road markings, and structures installed along the road. Road markings include lane markings that indicate lane boundaries, pedestrian crossings, stop lines, guidance strips, safety zones, and regulatory arrows. Structures installed along the road include road signs, guardrails, curbs, utility poles, traffic lights, and the like. The camera 111 can also detect the lighting status of lighting devices such as hazard lights and direction indicators (so-called blinkers) of the vehicle ahead.

[0024] The automatic driving system Sys may include multiple cameras 111. For example, the automatic driving system Sys may include, as the camera 111, a side camera that captures images of the sides of the vehicle and a rear camera that captures images of the rear of the vehicle, in addition to a front camera. The function of detecting a detection target object by analyzing camera images may be provided in another ECU, such as the automatic driving ECU 30. The functional arrangement in the automatic driving system Sys can be changed as appropriate. The camera 111 outputs data related to the detected object to the in-vehicle network IvN. The data flowing through the in-vehicle network IvN is referred to by the automatic driving ECU 30 as appropriate.

[0025] The millimeter wave radar 112 is a device that detects the relative position and relative speed of an object with respect to the vehicle by transmitting a search wave such as a millimeter wave or a quasi-millimeter wave in a predetermined direction and analyzing the received data of the reflected wave that is reflected by the object. The autonomous driving system Sys may include a plurality of millimeter wave radars 112. The plurality of millimeter wave radars 112 include a forward millimeter wave radar and a rear millimeter wave radar. The forward millimeter wave radar is a millimeter wave radar 112 that transmits a search wave toward the front of the vehicle, and is installed, for example, on the front grill or the front bumper. The rear millimeter wave radar is a millimeter wave radar 112 that transmits a search wave toward the rear of the vehicle, and is installed, for example, on the rear bumper. Each millimeter wave radar 112 generates data indicating the relative position and relative speed of the detected object and outputs the data as a detection result to the autonomous driving ECU 30 or the like. The objects detected by the millimeter wave radar 112 may include other vehicles, pedestrians, manholes (iron plates), three-dimensional structures as landmarks, and the like.

[0026] The perimeter monitoring sensor 11 may be a camera 111 and a millimeter wave radar 112, as well as a LiDAR, a sonar, etc. LiDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging. LiDAR is a device that generates three-dimensional point cloud data indicating the position of a reflection point for each detection direction by irradiating laser light. LiDAR is also called a laser radar. The autonomous driving system Sys may also be equipped with multiple LiDARs and sonars. The combination of the perimeter monitoring sensors 11 equipped in the autonomous driving system Sys can be changed as appropriate. The detection results of each perimeter monitoring sensor 11 are input to the autonomous driving ECU 30.

[0027] The vehicle state sensor 12 is a sensor that detects information related to the state of the host vehicle. The vehicle state sensor 12 includes a vehicle speed sensor, a steering angle sensor, an acceleration sensor, a yaw rate sensor, an accelerator pedal sensor, and the like. The vehicle speed sensor is a sensor that detects the traveling speed of the host vehicle. The steering angle sensor is a sensor that detects the steering angle. The acceleration sensor is a sensor that detects the acceleration acting in the forward / rearward direction of the host vehicle, the lateral acceleration acting in the left / right direction, and the like. The yaw rate sensor is a sensor that detects the angular velocity of the host vehicle. The accelerator pedal sensor is a sensor that detects the depression amount / depression force of the accelerator pedal. The brake pedal sensor is a sensor that detects the depression amount / depression force of the brake pedal. The vehicle state sensor 12 outputs data indicating the current value (i.e., the detection result) of the physical state quantity to be detected to the in-vehicle network IvN. The type of sensor used by the autonomous driving system Sys as the vehicle state sensor 12 may be designed appropriately.

[0028] The locator 13 is a device that calculates and outputs the position coordinates of the host vehicle using navigation signals transmitted from positioning satellites that constitute the Global Navigation Satellite System (GNSS). The locator 13 includes a GNSS receiver, an inertial sensor, and the like. The locator 13 sequentially calculates the host vehicle's position, traveling direction, and the like by combining the navigation signals received by the GNSS receiver, the measurement results of the inertial sensor, and the vehicle speed information flowing through the in-vehicle network IvN, and the like. In this disclosure, data indicating the position coordinates of the host vehicle calculated by the locator 13 is referred to as host vehicle position data. The locator 13 outputs the host vehicle position data to the autonomous driving ECU 30.

[0029] The map storage unit 14 is a storage device in which map data is stored. The map data held by the map storage unit 14 may be so-called HD (High Definition) map data. The map data stored in the map storage unit 14 includes the three-dimensional shape of the road, the installation positions of road markings such as lane markings, the installation positions of traffic signs, and the like, with the accuracy required for autonomous driving and the like. The map data includes gate point data for each gate point. A gate point is a point on a toll road where a gate for toll collection is installed. The expression "gate point / gate" in this disclosure can be read as "toll gate."

[0030] The gate point data is data that indicates the structure of the gate point, etc. At one gate point, multiple gates may be installed side by side in the road width direction. The gate point data includes data related to the representative position coordinate, the number of gates installed, the detailed location of each gate, and the settlement method for each gate. The number of gates installed can be rephrased as the number of lanes. Each gate provides one lane (passage). The representative position coordinate is a position coordinate that roughly indicates the position of the gate point. The representative position coordinate may be, for example, the position coordinate of the gate (hereinafter referred to as the representative gate) that is located in the middle, the right end, or the left end of multiple gates lined up horizontally.

[0031] In the present disclosure, a road section within a predetermined distance before and after a gate point represented by the representative position coordinates is also referred to as a gate area. The gate area may be a section that exists before and after a gate and does not have lane markings (hereinafter, a laneless section). The gate area may be a section whose road width is expanded relative to a connecting road. The gate area can be divided into a pre-gate area and a post-gate area. The pre-gate area refers to the area of ​​the gate area that is located on the entrance side of the gate (in other words, forward). The post-gate area refers to the area of ​​the gate area that is located on the exit side of the gate (in other words, rearward). If a branch point exists behind the gate, the processor 31 may regard the area up to the branch point as the post-gate area.

[0032] The data of the detailed gate position may be coordinate data such as latitude and longitude. The detailed gate position may be expressed by a number, with the rightmost or leftmost gate being number 1. The data of the settlement method indicates the settlement (payment) method of the road toll. The settlement method can be classified into a manual settlement method and an automatic settlement method. In the manual settlement method, the driver pays the toll by handing cash or a credit card to the gate staff or by inserting it into a settlement machine installed at the gate. In the automatic settlement method, a wireless communication device installed in the vehicle (a so-called on-board device) and a wireless communication equipment installed at the gate (a so-called roadside device) communicate wirelessly with each other, and a settlement is made according to the vehicle type and the passage section. In Japan, the manual settlement method may be called "general" and the automatic settlement method may be called "ETC (registered trademark)". ETC is an abbreviation for Electronic Toll Collection.

[0033] The map data stored in the map storage unit 14 may be updated by data received by the wireless communication device 15 from a map server or the like. The map storage unit 14 may be a storage device for temporarily storing the map data received by the wireless communication device 15 from the map server until the validity period of the data expires. The map data stored in the map storage unit 14 may be navigation map data, which is map data for navigation, as long as it includes gate point data.

[0034] The wireless communication device 15 is a device for the vehicle to perform wireless communication with an external device. The external device may include a server, a traffic information center, a roadside device, and some or all of other vehicles. The wireless communication device 15 is configured to be capable of performing cellular communication. Cellular communication refers to wireless communication conforming to LTE (Long Term Evolution), 4G, 5G, etc. The wireless communication device 15 may be configured to perform cellular V2X (PC5 / SideLink / Uu).

[0035] Further, the wireless communication device 15 is configured to be capable of performing short-range communication. In the present disclosure, short-range communication refers to wireless communication in which the communication distance is limited to within several hundred meters. The method of short-range communication used may be DSRC (Dedicated Short Range Communications) compatible with IEEE802.11p, Wi-Fi (registered trademark), or Bluetooth (registered trademark) Low Energy. The short-range communication method may be the above-mentioned cellular V2X. The wireless communication device 15 may be configured to be capable of performing data communication related to toll payment with a roadside device installed at a gate when passing through the gate. For example, the wireless communication device 15 may be an on-board device compatible with ETC2.0.

[0036] The wireless communication device 15 may receive information about the gate point from an external device. For example, the wireless communication device 15 may receive location information of the gate point, information about passable gates, and closed gates from a server or a center. The wireless communication device 15 may receive vehicle information from surrounding vehicles through vehicle-to-vehicle communication. The vehicle information may include speed, current position, turn signal operation status, acceleration, and movement trajectory. The surrounding vehicles here refer to vehicles that are present within a range where vehicle-to-vehicle communication is possible.

[0037] The occupant status sensor 16 is a sensor that detects the state of the driver. The occupant status sensor 16 may be, for example, a driver status monitor (hereinafter, DSM). The DSM is a sensor that detects the direction of the driver's face, the direction of the line of sight, the degree of opening of the eyelids, and the like based on an image of the driver's face. The DSM as the occupant status sensor 16 is arranged on the instrument panel, the upper end of the windshield, and the like, with the optical axis directed toward the headrest of the driver's seat, so that the driver's face can be photographed. The DSM as the occupant status sensor 16 transmits driver status data indicating the direction of the driver's face, the direction of the line of sight, the degree of opening of the eyelids, and the like to the autonomous driving ECU 30. The occupant status sensor 16 may be a pulse sensor, a thermal camera, and the like.

[0038] The body ECU 17 is an ECU that comprehensively controls the body-related in-vehicle devices mounted on the vehicle. The body-related in-vehicle devices include lighting devices, a horn, a door lock motor, etc. The lighting devices include headlights, hazard lights, turn signals, backlights, welcome lights, etc. The body-related in-vehicle devices may also include an exterior display device 18.

[0039] The exterior display device 18 is a projector that projects an image on the rear window. The exterior display device 18 can display an image for communicating with the driver of another vehicle based on an input signal from the automatic driving ECU 30. For example, the exterior display device 18 displays an image showing the direction of travel of the vehicle itself, or an image requesting a transfer of the right of way (in other words, permission to cut in) to a rear vehicle traveling in an adjacent lane. The exterior display device 18 is provided, for example, on the ceiling of the vehicle interior (for example, near the upper end of the window frame) in a position where the irradiated light hits the rear window.

[0040] The exterior display device 18 may be one that projects onto a side window or onto the road surface around the vehicle. The exterior display device 18 may be provided on a side mirror so as to project an image onto the road surface near the vehicle. The headlights or backlights may be configured to operate as the exterior display device 18. The exterior display device 18 may be a liquid crystal display or the like arranged with the display surface facing the side or rear of the vehicle.

[0041] The in-vehicle HMI 20 is a group of interfaces for exchanging information between the occupant and the automatic driving system Sys. The in-vehicle HMI 20 includes a display 21 and a speaker 22 as notification devices for notifying the driver of information. The in-vehicle HMI 20 also includes an input device 23 as an input interface for receiving operations from the occupant.

[0042] The autonomous driving system Sys includes one or more of a head-up display (HUD), a meter display, and a center display as the display 21. The HUD is a device that projects image light onto a predetermined area of ​​the windshield to display a virtual image that can be perceived by the driver. The meter display is a display arranged in an area of ​​the instrument panel located in front of the driver's seat. The center display is a display provided in the center of the instrument panel in the vehicle width direction. The meter display and the center display can be realized using a liquid crystal display or an organic EL display. The display 21 displays an image corresponding to a signal input from the autonomous driving ECU 30. The speaker 22 is a device that outputs a sound corresponding to a signal input from the autonomous driving ECU 30. The expression "sound" in this disclosure includes notification sound, voice, music, and the like.

[0043] The autonomous driving system Sys may also include a vibrator, an ambient light, and the like, as notification devices other than those described above. The ambient light is a lighting device that is realized by a number of LEDs (light emitting diodes) and is capable of adjusting the light emission color and light emission intensity. The ambient light is provided on the instrument panel, the steering wheel, the A-pillar, and the like. The A-pillar is a pillar that exists next to the windshield. The A-pillar may also be called the front pillar.

[0044] The input device 23 is a device for receiving an instruction operation from the driver to the automatic driving system Sys. As the input device 23, a steering switch provided on the spoke part of the steering wheel, an operation lever provided on the steering column, a touch panel stacked on the center display, etc. can be adopted. The automatic driving system Sys may be equipped with a plurality of types of devices as the input device 23.

[0045] The input device 23 outputs an operation signal, which is an electric signal corresponding to the driver's operation, to the automatic driving ECU 30. The operation signal includes information indicating the contents of the driver's operation. The automatic driving system Sys receives an instruction for changing the operation mode via the input device 23. The instruction for changing the operation mode includes an instruction for starting and ending automatic driving. The automatic driving system Sys may be configured to be able to acquire various instructions from the driver by voice recognition. A device for voice input such as a microphone may also be included in the input device 23. Note that, for example, an HCU (HMI Control Unit) may be interposed between the in-vehicle HMI 20 and the automatic driving ECU 30. The HCU is a device that comprehensively controls the information output (in other words, notifications) to the driver.

[0046] The autonomous driving ECU 30 is an ECU that controls the traveling actuator 19 based on the detection results of the surrounding monitoring sensor 11 and performs some or all of the driving operation on behalf of the driver. The autonomous driving ECU 30 is also called an automatic driving device. The traveling actuator 19 includes, for example, a brake actuator, an electronic throttle, and a steering actuator. The steering actuator includes an EPS (Electric Power Steering) motor. Note that other ECUs, such as a steering ECU that performs steering control, a power unit control ECU that performs acceleration / deceleration control, and a brake ECU, may be interposed between the autonomous driving ECU 30 and the traveling actuator 19.

[0047] The autonomous driving ECU 30 is realized by using a computer including a processor 31, a memory 32, a storage 33, a communication interface 34, and a bus connecting these. The memory 32 is a rewritable volatile storage medium. The memory 32 is, for example, a RAM (Random Access Memory). The storage 33 is, for example, a rewritable non-volatile memory such as a flash memory. The storage 33 stores a vehicle control program which is a program executed by the processor 31. The vehicle control program also includes a notification control program for controlling notifications to the driver regarding passing through a gate. Execution of the notification control program by the processor 31 corresponds to execution of a notification control method.

[0048] The autonomous driving ECU 30 has a plurality of operation modes with different automation levels. Each operation mode has a different range of driving tasks that the driver is responsible for, in other words, a different range of driving tasks in which the system intervenes. The operation mode can be rephrased as a driving mode. Here, as an example, the autonomous driving ECU 30 is configured to be able to switch between a plurality of operation modes including at least a fully manual mode, a level 2 mode, and a level 3 mode.

[0049] The fully manual mode is an operation mode in which the driver performs all driving tasks. The fully manual mode corresponds to a mode in which the autonomous driving ECU 30 does not perform substantial vehicle control. The fully manual mode may be a mode in which the autonomous driving ECU 30 stops operating (so-called a stop mode). In the fully manual mode, the autonomous driving ECU 30 may continue to perform a recognition process of the driving environment in the background (in other words, potentially) as a preparatory process for transitioning to the level 2 or level 3 mode.

[0050] The level 2 mode is an operation mode in which an automatic driving control with a duty to monitor the surroundings is performed, in other words, a vehicle control equivalent to an automation level 2 is performed. The level 2 mode can be called a semi-automated driving mode or an eyes-on automatic driving mode. The level 2 mode may be subdivided into a hands-on level 2 mode and a hands-off level 2 mode. In the automatic driving ECU 30 of this embodiment, the hands-on level 2 mode is a mode in which the driver needs to hold the steering wheel. The hands-off level 2 mode is an operation mode in which the driver does not need to hold the steering wheel, in other words, an operation mode in which hands-off is permitted. Note that hands-on in this disclosure refers to holding the steering wheel. Hands-off refers to the act of taking your hands off the steering wheel. Eyes-on refers to monitoring the outside area (mainly the front) related to the moving direction of the vehicle. Eyes-off refers to the act of taking your eyes off the outside area related to the moving direction of the vehicle.

[0051] The level 3 mode is an operation mode that executes autonomous driving control without the obligation to monitor the surroundings, that is, vehicle control equivalent to automation level 3. The autonomous driving ECU 30 may be capable of executing autonomous driving control equivalent to level 4 or higher. The level 3 mode may be called an autonomous driving mode or an eyes-off autonomous driving mode. The autonomous driving ECU 30 may include multiple processors 31. The processor that executes autonomous driving control at level 3 or higher may be provided separately from the processor that executes vehicle control at level 2 or lower.

[0052] While in the autonomous driving mode, the autonomous driving ECU 30 automatically steers, accelerates, decelerates (in other words, brakes), and the like of the vehicle so that the vehicle travels along a planned travel route toward a destination set by the driver. Even if a destination is not set, the autonomous driving ECU 30 may select a route to continue traveling / traveling within an area that satisfies the ODD, and continue autonomous traveling.

[0053] ODD may include, for example, (a) the road is an expressway or a motor vehicle-only road with a median strip and guardrails, (b) the amount of rainfall is equal to or less than a predetermined threshold, and (c) the vehicle is in a congested state. A motor vehicle-only road here is a road where pedestrians and bicycles are prohibited from entering, and includes, for example, toll roads such as expressways. A congested state refers to, for example, a state in which the driving speed is equal to or less than a congestion judgment value (for example, about 30 km / h) and there are other vehicles within a predetermined distance (for example, 20 m) in front of and behind the vehicle. In addition, (d) all / a predetermined number or more of the surrounding monitoring sensors 11 are operating normally, and (e) there are no parked vehicles on the road, etc., can also be included in ODD. The conditions for judging whether autonomous driving is possible / impossible, in other words, the detailed conditions for defining ODD, can be changed as appropriate.

[0054] In addition, the autonomous driving ECU 30 performs control for driving the vehicle substantially autonomously even while in the level 2 mode. That is, it recognizes the driving environment, plans the driving trajectory, and reflects / feeds back to the control. Reflection to the control includes speed adjustment by acceleration and deceleration, steering control, etc. In the following, the term "autonomous driving" can be replaced with semi-autonomous driving equivalent to level 2, unless otherwise noted.

[0055] In addition, the autonomous driving ECU 30 allows the driver to perform a second task in the autonomous driving mode. The second task allowed in level 3 autonomous driving may be limited to a task that allows the driver to immediately return to driving, such as reading or operating a smartphone. The autonomous driving mode is terminated due to the driver's steering / pedal operation (so-called override), as well as due to system limitations, exiting the ODD, etc.

[0056] <About the Autonomous Driving ECU Configuration> 2 as functional units realized by executing the autonomous driving program. That is, the autonomous driving ECU 30 has an information acquisition unit F1, an environment recognition unit F2, a mode control unit F3, a planning unit F4, a vehicle control unit F5, and a notification control unit F6.

[0057] The information acquisition unit F1 is configured to acquire various information for implementing vehicle control such as autonomous driving and driving assistance. The information acquisition unit F1 acquires sensing data (i.e., detection results) from various surrounding monitoring sensors 11 including a camera 111. The sensing data includes data on objects present around the vehicle, such as moving bodies, features, and obstacles. Data on each detected object may include the position, moving speed, and type or size of the detected object.

[0058] The sensing data on features may include data on the detection results of lane markings and road edges. The data on lane markings may include not only position data but also line type data. The line type may be expressed as a continuous line (solid line) or a dashed line. The sensing data may also include data indicating the recognition status of lane markings, such as whether or not the lane markings can be recognized, and the recognition status of the road edges, such as whether or not the road edges can be recognized.

[0059] The information acquisition unit F1 also acquires data indicating the state of the vehicle, such as the vehicle's running speed, acceleration, yaw rate, and external illuminance, from the vehicle state sensor 12. Furthermore, the information acquisition unit F1 acquires vehicle position data from the locator 13. The information acquisition unit F1 acquires surrounding map information by referring to the map storage unit 14.

[0060] The information acquisition unit F1 acquires data transmitted from an external device in cooperation with the wireless communication device 15. For example, the information acquisition unit F1 may acquire vehicle information transmitted from a vehicle ahead via vehicle-to-vehicle communication. In addition, the information acquisition unit F1 acquires dynamic map data for a road section that the vehicle is scheduled to pass through within a predetermined time in cooperation with the wireless communication device 15. The dynamic map data here includes traffic congestion information, merging vehicle information, and the like.

[0061] The information acquisition unit F1 also acquires the driver's operation on the automatic driving system Sys based on a signal from the input device 23. For example, the information acquisition unit F1 acquires an instruction signal related to the start and end of automatic driving from the input device 23. The information acquisition unit F1 also acquires data related to the operation state of the automatic driving system Sys from various devices / software modules. For example, the information acquisition unit F1 also acquires data such as the operation state (on / off) of the ACC function and whether or not a preceding vehicle is recognized. The information acquisition unit F1 also manages the operation states of various components, such as whether or not the periphery monitoring sensor 11 is operating normally. The information acquisition unit F1 acquires driver state data indicating the eye opening degree and line of sight from the occupant state sensor 16.

[0062] Various data sequentially acquired by the information acquisition unit F1 is stored in a temporary storage medium such as the memory 32, and is used by the environment recognition unit F2, the mode control unit F3, and the like. The various pieces of information may be classified by type and stored in the memory 32. The various pieces of information may also be sorted and stored, for example, with the most recent data at the top. Data that has been acquired for a certain period of time may be discarded. In this disclosure, "acquisition" also includes generation / detection / determination by the autonomous driving ECU 30 itself through calculation based on data input from other devices / sensors. This is because the functional arrangement within the system can be changed as appropriate.

[0063] The environment recognition unit F2 recognizes the driving environment of the vehicle based on various data acquired by the information acquisition unit F1. The environment recognition unit F2 may recognize the driving environment of the vehicle by a sensor fusion process that integrates the detection results of multiple surrounding monitoring sensors 11, such as the camera 111 and the millimeter wave radar 112, with a predetermined weight.

[0064] The driving environment includes the curvature of the road, the number of lanes, the vehicle lane number, the weather, the road surface condition, the traffic volume, the remaining distance to the gate point, and the like. The vehicle lane number is a number indicating the position of the vehicle lane on the road, and is determined based on the left road edge. The vehicle lane number directly or indirectly indicates the number of lanes existing to the left of the vehicle lane. Of course, the vehicle lane number may be expressed based on the right road edge. The vehicle lane number may be specified using the distance from the road edge to the vehicle, the number of lane markings detected on the left and right, and part or all of the map data. The vehicle lane number may be specified from the map data and the vehicle position data. The vehicle lane number may be specified by the camera 111 or the locator 13. The weather and road surface condition can be specified by combining the recognition result of the camera 111 and the weather information acquired by the information acquisition unit F1. The road structure may be specified using the recognition result of the camera 111, map data, or trajectory information of a forward vehicle.

[0065] The environment recognition unit F2 acquires information on the structure of a road existing within a predetermined distance ahead of the vehicle based on at least one of an output signal of the perimeter monitoring sensor 11, a received signal from an external device, and map data. The road structure includes the location of a gate point, the location of a branch road, the number of lane markings, the road width, and the like. The environment recognition unit F2 acquires the remaining distance to the gate point as detailed information on the gate point. The remaining distance to the gate point may be acquired based on map data, or may be determined based on data of a guide sign detected by the camera 111. The environment recognition unit F2 may determine the remaining distance to the gate point based on behavior data or sensing data received from a vehicle ahead. The environment recognition unit F2 may acquire the number of gates and the payment method for each gate from the map data or the travel trajectory of the vehicle ahead. The environment recognition unit F2 may regard a gate that requires stopping to pass as a gate of a manual payment method, and a gate that the vehicle ahead passes through without stopping as a gate of an automatic payment method. The functional unit that acquires information related to the gate point in the environment recognition unit F2 corresponds to the gate recognition unit F21.

[0066] The driving environment includes the position, type, and moving speed of objects around the vehicle. The environment recognition unit F2 recognizes the position and behavior of surrounding vehicles based on various data acquired by the information acquisition unit F1. A software / hardware module in charge of processing to recognize surrounding vehicles corresponds to the surrounding vehicle recognition unit F22. The environment recognition unit F2 as the surrounding vehicle recognition unit F22 can calculate a collision risk for each other vehicle detected. The collision risk may be, for example, TTC (Time-To-Collision) or collision margin (MTC: Margin-To-Collision). For example, the environment recognition unit F2 calculates the TTC for each other vehicle. TTC and MTC are parameters that mean that the smaller the value, the greater the collision risk. In addition, the environment recognition unit F2 acquires outside vehicle environment information related to ODD and driver state data.

[0067] The mode control unit F3 controls the operation mode of the autonomous driving ECU 30 based on various information acquired by the information acquisition unit F1. The switching of the operation mode is executed based on an operation signal input from the input device 23. For example, when the driving environment satisfies the ODD and an instruction signal to start autonomous driving is input from the input device 23, the mode control unit F3 switches the operation mode from the fully manual mode or the level 2 mode to the autonomous driving mode. In addition, when the mode control unit F3 predicts that the driving environment recognized by the environment recognition unit F2 will no longer satisfy the ODD during the autonomous driving mode, it may decide to transition to the fully manual mode and notify the planning unit F4 of that effect.

[0068] In addition, when an override operation by the driver is detected during the autonomous driving mode or level 2 mode, the mode control unit F3 switches to the fully manual mode. An override operation refers to an operation by a passenger on a driving operation member such as a steering wheel, a brake pedal, or an accelerator pedal. When the autonomous driving ECU 30 detects that an override operation has been performed by the driver, it promptly transfers driving authority to the driver and notifies the driver by voice output or the like that the driving has been switched to manual driving. Note that the operation mode to which the vehicle transitions when the autonomous driving mode ends may be the level 2 mode.

[0069] The planning unit F4 is a configuration for planning the control contents to be executed as level 2 or higher autonomous driving. The planning unit F4 can be enabled when the operation mode is level 3 or level 2 mode. While in level 3 or level 2 mode, the planning unit F4 generates a driving plan for autonomous driving based on the recognition result of the driving environment by the environment recognition unit F2. The driving plan can also be called a control plan. The planning unit F4 corresponds to a configuration for creating a driving plan for the vehicle. The driving plan includes a driving position for each time, a target speed, a steering angle, etc. That is, the driving plan can include schedule information of acceleration / deceleration for speed adjustment on the calculated route and schedule information of steering amount.

[0070] For example, the planning unit F4 performs a route search process as a medium- to long-term driving plan, and determines a planned driving route from the vehicle's position to the destination. If the destination is not set, the planning unit F4 may select a route on which autonomous driving can be continued as the planned driving route. The planned driving route includes data on roads to be traveled within a predetermined time (for example, 10 minutes) from now.

[0071] The planning unit F4 generates a driving plan for changing lanes, a driving plan for driving in the center of the lane, a driving plan for following the preceding vehicle, and a driving plan for avoiding obstacles as a short-term control plan for driving according to the mid- to long-term driving plan. For example, the planning unit F4 generates a driving plan for driving in the center of the recognized lane of the vehicle as a short-term control plan, or generates a driving plan for a route that follows the behavior or driving trajectory of the recognized preceding vehicle as a driving plan. The control plan created by the planning unit F4 is input to the vehicle control unit F5.

[0072] The planning unit F4 performs gate passing planning processing as a configuration related to passing through a gate point. The gate passing planning processing includes setting a target gate, generating a travel trajectory to the target gate, and generating a trajectory after passing through the gate. The target gate is a gate through which the vehicle passes among multiple gates provided at the gate point. A method for setting the target gate will be described separately later.

[0073] In addition to control plans directly related to vehicle driving, the planning unit F4 also formulates plans related to notification processing to the occupants using notification devices such as the display 21. For example, the planning unit F4 plans the timing of issuing notifications / requests to the driver, such as behavior notification, mode change notification, eyes-on request, hands-on request, and TOR (Take Over Request) notification. The behavior notification is a process of notifying the driver of planned vehicle behavior such as lane change, overtaking, deceleration, etc. The mode change notification is a process of notifying the driver that the operation mode will be changed or that the operation mode will be changed.

[0074] The eyes-on request is a process that requests the driver to monitor the surroundings just in case in level 3 mode. The hands-on request is a process that requests the driver to lightly grip the steering wheel in level 3 mode or hands-off level 2 mode. The TOR notice is a process that notifies the driver that the possibility of TOR is increasing. TOR requests the driver to take over driving operations, in other words, ends automated driving.

[0075] Various notifications including advance notices and requests include displaying an icon image according to the content on the display 21. Various notifications may be accompanied by some or all of the following, depending on the importance and urgency: output of a notification sound, output of a voice message, blinking of the ambient light, and vibration of the vibrator. The planning unit F4 creates notification plan data indicating the content of the notification and the timing of the notification, and transmits it to the notification control unit F6.

[0076] The vehicle control unit F5 generates control commands based on the control plan formulated by the planning unit F4, and sequentially outputs the control commands to the traveling actuator 19. The vehicle control unit F5 also controls the illumination states of turn signals, headlights, hazard lights, and the like, in accordance with the driving plan and the external environment, based on the plan of the planning unit F4 and the external environment.

[0077] The vehicle control unit F5 includes an ACC system F51 as a subsystem for executing the control to follow the preceding vehicle. The ACC system F51 executes the control to follow the preceding vehicle based on the plan created by the planning unit F4. That is, when the ACC system F51 can recognize the preceding vehicle, it controls the vehicle speed so that the distance / time between the preceding vehicle is constant within the range of the set vehicle speed. When the ACC system F51 does not recognize the preceding vehicle or when the speed of the preceding vehicle exceeds the set vehicle speed, it adjusts the speed so as to maintain the set vehicle speed. The ACC system F51 provides the notification control unit F6 with data indicating the recognition state of the preceding vehicle and the implementation state of the control to follow the preceding vehicle. The ACC system F51 can also be called a control unit to follow the preceding vehicle.

[0078] The software / hardware modules including the mode control unit F3, the planning unit F4, and the vehicle control unit F5 correspond to the autonomous driving unit Fn. Of course, the information acquisition unit F1 and the environment recognition unit F2 can also be included in the autonomous driving unit Fn.

[0079] The notification control unit F6 is a subsystem for making notifications / suggestions to the driver using notification devices such as the display 21 and the speaker 22. Various notifications / suggestions can be realized by displaying an image on the display 21 or outputting a voice message from the speaker 22. The notification control unit F6 executes various notifications based on the plan of the planning unit F4.

[0080] The notification control unit F6 also acquires data on the relative position of the vehicle to the gate point as the recognition result of the environment recognition unit F2. For example, the notification control unit F6 acquires whether the vehicle has entered the gate area, the remaining distance to the target gate, whether the vehicle has passed the target gate, whether the vehicle has left the gate area, and the like. The notification control unit F6 also acquires the positions of surrounding vehicles, the operating state of the preceding vehicle following control, the recognition state of the preceding vehicle, the current operating mode, and the like. The processor 31 as the notification control unit F6 performs gate approach response processing when passing through a gate. The gate approach response processing will be described separately later.

[0081] The notification control unit F6 of this embodiment is configured to selectively adopt two notification modes, a conspicuous mode and a discreet mode, as a notification mode for an eyes-on request or the like. The discreet mode refers to a mode in which stimuli such as light and sound are weakened compared to the conspicuous mode. The discreet mode refers to a notification mode that aims to not annoy the occupants. A notification in a discreet mode refers to a notification mode in which an image display is the main mode, no vibration is applied to the driver, and the output volume of the notification sound is set to a predetermined value or less. Setting the output volume to a predetermined value or less includes not outputting any sound. The discreet mode can also be rephrased as an inconspicuous mode.

[0082] A conspicuous notification refers to a notification in a manner intended to ensure that the driver is clearly aware of the notification content. A conspicuous notification may involve output of a voice message / sound effect at a volume equal to or greater than a predetermined value. A conspicuous notification may involve application of vibration to the driver. A conspicuous notification corresponds to outputting a stimulus with a necessary and sufficient strength to attract the driver's interest.

[0083] <Setting the target gate> Here, a method for setting a target gate will be explained with reference to Fig. 3. The road shown in Fig. 3 has four gates at one gate point, and is structured so that it branches into a first road Rt1 and a second road Rt2 behind the gate point.

[0084] When multiple gates are installed at a gate point as shown in FIG. 3, the processor 31 as the planning unit F4 sets a gate corresponding to a post-gate road among the multiple gates as a target gate. The post-gate road is a road along which the vehicle is scheduled to travel after passing the gate point. A gate corresponding to a post-gate road refers to a gate located in front of the post-gate road, in other words, a gate from which the vehicle can enter the post-gate road by traveling straight after passing the gate. A gate corresponding to a post-gate road can be interpreted as a gate that continues to the post-gate road. From the opposite perspective, a road corresponding to a certain gate can be interpreted as a road located in front of the gate, a road closest to the gate, or a road that continues along the road edge closest to the gate.

[0085] 3, the first gate Gt1 and the second gate Gt2 are gates corresponding to the first road Rt1, while the third gate Gt3 and the fourth gate Gt4 are gates corresponding to the second road Rt2.

[0086] When there are multiple gates corresponding to the road after the gate, the processor 31 may set the gate closest to the extension of the current vehicle lane as the target gate. For example, when the second road Rt2 is the road after the gate for the vehicle and the current vehicle lane is the first lane, the processor 31 sets the third gate Gt3 as the target gate. This is because the third gate Gt3 is closer to the vehicle lane than the fourth gate Gt4. Note that "Hv" in FIG. 3 is a symbol indicating the vehicle. The processor 31 sets the target gate so that the amount of lateral movement after passing through the gate is as small as possible.

[0087] Of course, if the third gate Gt3 satisfies a specific non-use condition, the processor 31 may set the fourth gate Gt4 as the target gate instead of the third gate Gt3. The non-use condition may be satisfied, for example, when the gate is closed, when the payment method is a manual payment method, when the third gate Gt3 is more crowded than the fourth gate Gt4, etc. The target gate selection algorithm may be changed as appropriate.

[0088] Processor 31 may select a target gate from among gates for which the automatic settlement method is available. Also, if the vehicle is unable to carry out the automatic settlement process, processor 31 may select a target gate from among gates for which the manual settlement method is available. A case in which the automatic settlement process is unable to be carried out refers to a case in which a card for automatic settlement is not inserted in a specified onboard device, etc. If there is only one gate through which the vehicle can pass in terms of the settlement method, etc., processor 31 may set that gate as the target gate.

[0089] If there is only one gate available due to closure or the like, the processor 31 may also set that gate as the target gate. In addition, if a destination is not set, the processor 31 may set a gate on an extension of the vehicle's lane as the target gate. In addition, if a destination is not set, the processor 31 may set a road on which the level 3 mode can be maintained as a post-gate road and then set that as the target gate. Note that if there is no gate that can be passed through while maintaining autonomous driving, the notification control unit F6 may execute TOR.

[0090] Once the target gate has been set, the processor 31 creates a pre-gate trajectory and a post-gate trajectory based on the position of the target gate, the current vehicle position, and the position of the planned driving lane on the post-gate road. The pre-gate trajectory is the trajectory until entering the target gate. The post-gate trajectory is the trajectory from leaving the target gate to entering the post-gate road. The pre-gate trajectory may include a course change (e.g., a lane change) toward the target gate. The dashed line indicated by "Tr1" in FIG. 3 conceptually indicates the pre-gate trajectory, and the dashed line indicated by "Tr2" conceptually indicates the post-gate trajectory.

[0091] In a situation where the vehicle is located in front of the gate (entrance side), the gate makes it difficult to detect objects that are located behind the gate. If the target gate is set so that the amount of lateral movement after passing through the gate is larger than the amount of lateral movement before passing through the gate, the difficulty of control after passing through the gate increases. This is because there is a possibility that an obstacle that was not detected before passing through the gate may be present. In other words, there is a smaller possibility of overlooking a surrounding vehicle before passing through the gate than after passing through the gate. As described above, by setting the target gate so that the amount of lateral movement before passing through the gate is equal to or greater than the amount of lateral movement after passing through the gate, safety can be improved. This configuration corresponds to a configuration in which the target gate is set so that the amount of lateral movement after passing through the gate is as small as possible.

[0092] The trajectory data for the target gate and the vicinity of the gate set by the processor 31 as the planner F4 as described above can be referenced not only by the vehicle controller F5 but also by the notification controller F6.

[0093] <Gate approach response processing> Here, the gate approach response process performed by the notification control unit F6 will be described with reference to the flowchart shown in Fig. 4. The flowchart shown in Fig. 4 may be periodically executed while in the level 3 mode. The flowchart shown in Fig. 4 includes steps S101 to S110, as an example.

[0094] Step S101 is a step in which the notification control unit F6 acquires data indicating the relative position of the vehicle with respect to the gate point. The processing of step S101 is also periodically executed after step S103. Note that step S101 may include a step of acquiring the current operating mode of the autonomous driving ECU 30 and a step of acquiring data indicating the planned traveling trajectory created by the planning unit F4. Step S101 can be interpreted as a step in which the notification control unit F6 acquires data necessary for implementing various notifications.

[0095] Step S102 is a step of judging whether or not the vehicle has entered a gate area. As described above, the gate area may be a road section within a certain distance from the gate point. The distance regarded as the gate area may be 100 m, 250 m, 400 m, or the like. In a configuration in which a section within a certain distance from the gate point is defined as the gate area, step S102 can be interpreted as a step of judging whether or not the remaining distance to the gate point is equal to or less than a predetermined value. As described above, the output of the periphery monitoring sensor 11, data received from an external device, map data, or the like can be used as a judging material for the remaining distance to the gate point. In addition, the gate area may be a laneless section as another aspect. In a configuration in which a laneless section near the gate is regarded as the gate area, the notification control unit F6 can judge that the vehicle has entered the gate area based on the fact that the lane dividing line is no longer detected by the camera 111. In addition, the gate area may be an area in which the road width is expanded near the gate point. Whether or not the vehicle has entered the gate area may be judged using the vehicle's position on the map. The determination of entry into the gate region may be performed in a variety of ways.

[0096] If the host vehicle has entered the gate area (S102 YES), processor 31 executes the sequence from step S103 onward. On the other hand, if the host vehicle has not entered the gate area (S102 NO), this flow ends. When this flow ends, this flow may be executed again if a predetermined pause time has elapsed since the end point. The pause time may be set to, for example, 500 milliseconds, 1 second, 2 seconds, or the like.

[0097] Step S103 is a step of executing an eyes-on request. Step S103 corresponds to a step in which the system requests the driver to check the surrounding situation based on entering the gate area. Implementing an eyes-on request makes it easier for the driver to take over driving operations. The eyes-on request includes displaying an eyes-on icon Im1 illustrated in FIG. 5 at a predetermined position on the display 21. The eyes-on icon Im1 is an icon image / pictogram that simulates the driver looking ahead. The eyes-on request may be accompanied by output of a predetermined notification sound or turning on an ambient light.

[0098] The notification control unit F6 of this embodiment implements the eyes-on request in a discreet manner in the area in front of the gate. The discreet eyes-on request may be, for example, to display the eyes-on icon Im1 in green or yellow. The discreet eyes-on request may be to display the eyes-on icon Im1 in a predetermined size in a corner of the display 21. The display of the eyes-on icon Im1 requesting eyes-on may be continued until the front of the target gate is reached.

[0099] After executing step S103, the notification control unit F6 displays a gate guide image Im2 on the display 21 in step S104. The gate guide image Im2 is an image showing the road structure near the gate. The gate guide image can also be interpreted as a map image near the gate. The gate guide image may include an image showing the trajectory of the host vehicle near the gate.

[0100] For example, the gate guide image Im2 includes, as image elements, a vehicle image E1, a gate front trajectory image E21, a gate rear trajectory image E22, and a gate image E3, as shown in FIG. 6. The vehicle image E1 is an image element that represents the position of the vehicle. The gate front trajectory image E21 is an image element that represents the trajectory from the current position to the target gate. The gate rear trajectory image E22 is an image element that represents the trajectory of the vehicle after passing through the target gate. The gate image E3 is an image that represents the position of the target gate. The gate image E3 may include not only an image of the target gate, but also images of other gates. The gate image E3 may be an image of multiple gates installed at the gate location. In the gate image E3, the target gate may be displayed in a manner different from the other gates. For example, gates other than the target gate may be grayed out. The target gate may be given an effect or decoration such as blinking. This configuration makes it easier for the driver to recognize the position of the target gate.

[0101] The gate pre-trajectory image E21 and the gate post-trajectory image E22 are represented by arrows or the like. The gate pre-trajectory image E21 and the gate post-trajectory image E22 may be belts / belts / lines representing the trajectory. The gate pre-trajectory image E21 and the gate post-trajectory image E22 may be connected. The gate pre-trajectory image E21 and the gate post-trajectory image E22 are also collectively referred to as the trajectory image E2. If the trajectory image E2 overlaps the gate image E3, the visibility of the target gate may be deteriorated. Therefore, it is preferable that the trajectory image E2 is interrupted in front of the gate so as not to overlap with the gate image E3. As in the gate guide image Im2 illustrated in FIG. 6, by displaying the trajectory image E2 divided into the gate pre-trajectory image E21 and the gate post-trajectory image E22, the driver can easily confirm the target gate.

[0102] Furthermore, if the gate guide image Im2 includes both the pre-gate trajectory image E21 and the post-gate trajectory image E22, the driver can easily recognize the overall behavior of the vehicle passing through the gate. The gate area is an area that is more difficult to control than a straight road. By visualizing the behavior plan of the autonomous driving ECU 30 in the gate area, the driver can determine the validity and feasibility of the plan. If the driver determines that the plan of the autonomous driving ECU 30 is reckless, the driver can override it without waiting for a request from the system.

[0103] In addition, the post-gate trajectory image E22 is an optional element in the gate guide image Im2. The gate guide image Im2 may be an image showing the behavior before entering the gate, as shown in FIG. 7. However, according to a configuration in which the road shape of the road after the gate or the post-gate trajectory is additionally displayed as exemplified in FIG. 6, the driver can be notified of the movement of the host vehicle after passing through the gate. The direction / object that the driver should pay attention to may differ depending on the trajectory after passing through the gate, for example, whether the vehicle proceeds straight, diagonally right, or diagonally left. According to a configuration in which the gate guide image Im2 including the post-gate trajectory image E22 is displayed, the driver can easily pay attention to the appropriate direction.

[0104] The gate guide image Im2 does not have to be an image of the road viewed from above (a so-called bird's-eye view image). The gate guide image Im2 may be a stereoscopic image from the driver's viewpoint. The gate guide image Im2 may also be an image of the target gate viewed from a virtual viewpoint arranged above the vehicle. Furthermore, the gate guide image Im2 may be displayed on the head-up display so as to overlap with the actual view ahead of the vehicle. The notification control unit F6 may display the gate pre-gate trajectory image E21 and an image showing the target gate as the gate guide image Im2, superimposed on the view ahead. The display of the gate guide image Im2 may continue until the gate is passed or the gate area is exited.

[0105] Step S105 is a step for determining whether or not the vehicle has passed through the gate. Whether or not the vehicle has passed through the gate may be determined based on the vehicle's position data on a map, the recognition result by the periphery monitoring sensor 11, and the communication status with an external device. If the vehicle has passed through the gate (S105 YES), the notification control unit F6 changes the display mode of the eyes-on icon Im1 to a more conspicuous mode in step S106. For example, the notification control unit F6 changes the color of the eyes-on icon Im1 to an accent color such as red or orange. The accentuation of the eyes-on icon Im1 may be any of blinking, increasing the display size, and changing the display position.

[0106] Such step S106 corresponds to a step of re-notifying the eyes-on request. Step S106 can also be interpreted as a step of making the eyes-on request more noticeable than before passing through the gate. After passing through the gate, the trajectories of vehicles are more likely to cross each other than before passing through the gate. As a result, the possibility of other vehicles approaching the vehicle excessively increases. By re-outputting the eyes-on request after passing through the gate, the possibility that the driver will monitor the surroundings can be increased. As a result, the possibility of contact with other vehicles in the post-gate area can be further reduced.

[0107] The eye-on request in a conspicuous manner may be accompanied by output of a notification sound, display of a text message requesting eye-on, or generation of vibration. The display of the eye-on icon Im1 may be continued until the gate exit. When the notification control unit F6 executes / starts the eye-on request in a conspicuous manner, it executes step S107.

[0108] Step S107 is a step for determining whether the vehicle has left the gate area. This determination can be made based on various data, similar to step S102. If the vehicle has left the gate area (S107 YES), the notification control unit F6 ends the eyes-on request (S108). Ending the eyes-on request corresponds to, for example, canceling the display of the eyes-on icon Im1.

[0109] In the above, the embodiment in which the eyes-on request is executed even while driving through the area in front of the gate has been described, but the present invention is not limited to this. The eyes-on request in the area in front of the gate may be omitted. Conversely, the notification control unit F6 may execute the eyes-on request in the area in front of the gate, but may omit the eyes-on request after passing through the gate. The notification control unit F6 may end the eyes-on request at the timing of passing through the gate or entering the gate.

[0110] In addition, it is also an optional control to change the strength of the eyes-on request before and after passing through the gate. The notification control unit F6 may execute the eyes-on request in the same manner after passing through the gate as before passing through the gate.

[0111] As long as the autonomous driving ECU 30 is in level 3 mode (i.e., autonomous driving mode), the driver's eyes on is an optional element and is not required. Even during autonomous driving, the notification control unit F6 requests the driver to keep his / her eyes on as an optional element just in case, which is expected to further improve safety. In addition, even in level 3 mode, by making an eyes-on request in response to approaching a gate, the handover of driving operations can be made smoother even if it becomes necessary to switch from level 3 mode to hands-on level 2 mode.

[0112] <Supplementary information on the implementation conditions of EyesOn requests> While the vehicle is autonomously driving through the gate area, the notification control unit F6 may change whether to make an eyes-on request depending on whether there are surrounding vehicles as shown in FIG. 8. Step S201 shown in FIG. 8 is a step for determining whether there are surrounding vehicles. The surrounding vehicles here are other vehicles that are present within a predetermined distance (e.g., 100 m) from the vehicle. The surrounding vehicles may also be limited to other vehicles whose TTC calculated by the environment recognition unit F2 or the like is less than a predetermined value (e.g., 5 seconds). The presence or absence of surrounding vehicles can be determined based on the recognition results of the environment recognition unit F2, and therefore the detection results of the surrounding monitoring sensor 11 or data received from an external device.

[0113] If there is a nearby vehicle (S201 YES), the notification control unit F6 executes an eyes-on request in step S202. On the other hand, if there is no nearby vehicle (S201 NO), the notification control unit F6 decides to omit / postpone the eyes-on request (S203). Note that the flowchart shown in FIG. 8 may be executed periodically while traveling in the gate area in level 3 mode until an eyes-on request is executed.

[0114] The notification control unit F6 may change the notification mode (strength) of the eyes-on request depending on whether or not there are surrounding vehicles as shown in Fig. 9. For example, when there are surrounding vehicles (S211 YES), the notification control unit F6 issues the eyes-on request in a conspicuous manner (S212). On the other hand, when there are no surrounding vehicles (S211 NO), the notification control unit F6 issues the eyes-on request in a discreet manner (S213).

[0115] According to the control examples shown in Figs. 8 and 9, it is possible to improve safety while reducing the risk of causing annoyance to the driver.

[0116] The determination process of step S201 may be replaced by a determination process of whether or not there is another vehicle whose collision risk is equal to or greater than a predetermined value. The notification control unit F6 may be configured to perform an eyes-on request when there is another vehicle whose collision risk is equal to or greater than a predetermined value while traveling in the gate area, and to omit the eyes-on request when there is no other vehicle whose collision risk is equal to or greater than the predetermined value. The determination process of step S201 may be replaced by a determination process of whether or not there is a preceding vehicle. The determination process of step S201 may be replaced by a determination process of whether or not there is another vehicle diagonally ahead of the vehicle, in other words, whether or not there is another vehicle that may cut in front of the vehicle.

[0117] The determination process of step S211 may also be replaced with a determination process of whether or not there is another vehicle with a collision risk equal to or greater than a predetermined value, whether or not there is a preceding vehicle, or whether or not there is another vehicle diagonally ahead of the vehicle. The eyes-on request condition, which is a condition for implementing the eyes-on request, may include the above sub-conditions in addition to being in a gate area. The above sub-conditions refer to the presence of a surrounding vehicle, the presence of another vehicle with a collision risk equal to or greater than a predetermined value, the presence of a preceding vehicle, the presence of another vehicle that may cut in front of the vehicle, etc.

[0118] Similarly, the strong request condition, which is a condition for making the eyes-on request in a conspicuous manner, may include any of the above sub-conditions in addition to traveling in a gate area. Conversely, the notification control unit F6 may be configured to make the eyes-on request in a more modest manner than usual when the modest request condition, which is a condition for making the eyes-on request in a modest manner, is satisfied. The modest request condition may be any of the following: no surrounding vehicles, no other vehicles with a collision risk equal to or greater than a predetermined value, no preceding vehicles, and no other vehicles that may cut in front of the host vehicle.

[0119] Also, the notification control unit F6 may change whether or not to make an eyes-on request depending on the settlement method of the target gate as shown in FIG. 10. Step S301 shown in FIG. 10 is a step for determining whether or not the target gate supports the manual settlement method. The settlement method supported by the target gate can be identified based on map data, the behavior of the vehicle ahead, or data received from the roadside device. The notification control unit F6 makes an eyes-on request when the target gate supports the manual settlement method (S302). On the other hand, when the target gate does not support the manual settlement method, in other words, when the target gate is a gate that can implement only the automatic settlement method, the notification control unit F6 omits the eyes-on request (S303).

[0120] At gates that support manual settlement, there is a possibility that the preceding vehicle may suddenly decelerate / stop before the gate. In order to deal with such a possibility, if the vehicle is planning to pass through a gate that supports manual settlement, an eyes-on request is made, making it easier for the driver to perform evasive maneuvers such as braking at their own discretion according to the behavior of the preceding vehicle.

[0121] Of course, the contents of steps S302 to S303 can be replaced with steps S212 to S213. That is, the notification control unit F6 may be configured to execute the eyes-on request in a more conservative manner when the target gate does not support the manual payment method, compared to when the target gate supports the manual payment method.

[0122] The notification control unit F6 may switch whether to execute the eyes-on request depending on whether the operation mode when traveling through the gate area is a hands-off mode. The hands-off mode is a mode in which the driver's hands are allowed to be off among the operation modes in which the vehicle travels substantially automatically. The hands-off mode includes a level 3 mode and a hands-off level 2 mode.

[0123] Fig. 11 shows an example of the operation of the notification control unit F6 based on this technical idea. That is, in step S401, the notification control unit F6 determines whether or not the current operation mode (hereinafter, current mode) is a hands-off enabled mode while traveling through a gate area. This step S401 may be executed periodically while traveling through the gate area. Step S401 may be executed only when a predetermined event occurs, such as when entering a gate area, when passing through a gate, or when a surrounding vehicle is detected.

[0124] The notification control unit F6 executes an eyes-on request (S402) when the current mode is a hands-off possible mode while traveling in a gate area (S401 YES). On the other hand, the notification control unit F6 omits the eyes-on request when the current mode is not a hands-off possible mode while traveling in a gate area (S401 NO). An operating mode that is not a hands-off possible mode can be called a hands-off prohibited mode or a hands-on required mode. The hands-on level 2 mode corresponds to the hands-off prohibited mode.

[0125] Since the hands-off prohibition mode is an operation mode that is based on the premise that the driver has his / her eyes on, when the current mode is the hands-off prohibition mode, the eyes-on request corresponds to a notification that is unnecessary for the driver. When the current mode is the hands-off prohibition mode, the risk of causing annoyance to the driver can be reduced by omitting the eyes-on request.

[0126] In principle, the driver should have their eyes on even in hands-off level 2 mode. However, the driver's level of involvement in driving operations is lower in hands-off level 2 mode than in hands-on level 2 mode. Therefore, the driver's attention may be less focused in hands-off level 2 mode than in hands-on level 2 mode. In hands-off level 2 mode, the driver's attention can be encouraged by requesting eyes on again.

[0127] Note that steps S402 to S403 can be replaced with steps S212 to S213. That is, the notification control unit F6 may be configured to perform an eyes-on request in a less restrictive manner when the current mode is a hands-off prohibited mode, compared to a hands-off permitted mode.

[0128] The notification control unit F6 may switch whether to make an eyes-on request depending on whether a course change is planned after passing through the gate as shown in Fig. 12. If a course change is planned after passing through the gate (S501 YES), the notification control unit F6 makes an eyes-on request (S502). On the other hand, if a plan to change course after passing through the gate has not been made (S501 NO), the notification control unit F6 omits the eyes-on request.

[0129] Here, lane change refers to lateral movement. Lane change includes lane change and driving diagonally to the road direction in a laneless section. Lane change after passing through a gate includes lane change in the area after the gate.

[0130] As described above, the area after the gate is more likely to have vehicle trajectories cross each other than the area before the gate, so the need for perimeter monitoring is increased. On the other hand, if there are no plans to change course after passing through the gate, the need for perimeter monitoring may be reduced. With the above configuration, it is possible to increase safety while reducing the risk of causing inconvenience to the driver.

[0131] Note that steps S502 to S503 can also be replaced with steps S212 to S213. That is, the notification control unit F6 may be configured to perform an eyes-on request in a more conservative manner when a course change after passing through the gate is not planned, compared to when a course change after passing through the gate is planned. Also, the notification control unit F6 may switch whether to perform an eyes-on request depending on whether a course change is planned in the area in front of the gate. Step S501 may be a step of determining whether a course change is planned in the area in front of the gate.

[0132] The notification control unit F6 may switch whether to make an eyes-on request depending on whether there is a branch road behind the gate. The notification control unit F6 may implement an eyes-on request when there is a branch road behind the gate, but may omit implementing the eyes-on request when there is no branch road behind the gate. A branch road behind the gate means that there is a branch point within a specified distance (e.g., 100 m) behind the gate. When there is no branch road near the gate, the tracks of vehicles are less likely to cross compared to when there is a branch road near the gate. The above configuration can reduce unnecessary eyes-on requests.

[0133] The notification control unit F6 may switch whether to make an eyes-on request depending on whether the road width narrows behind the gate. The notification control unit F6 may make an eyes-on request when the road width narrows behind the gate, but may omit making an eyes-on request when the road width does not narrow behind the gate. The case where the road width narrows behind the gate corresponds to a case where the number of lanes of the road behind the gate is fewer than the number of gates. When the number of lanes decreases after passing through the gate, merging / cutting in is likely to occur, and the behavior of other vehicles may become more complicated. According to the above configuration, it is possible to reduce unnecessary eyes-on requests while reducing the risk of abnormal approach / contact with other vehicles immediately after passing through the gate. The road structure behind the gate, such as the presence or absence of a branch road and the phenomenon of road width, can be identified based on map data and trajectory data of the forward vehicle.

[0134] <Display control of gate guide images> The notification control unit F6 may change the display mode of the gate guide image Im2 depending on whether the operation mode when entering the gate area is a hands-off mode or not. For example, as shown in Fig. 13, when the operation mode when entering the gate area is a hands-off mode (S601 YES), the notification control unit F6 displays the gate guide image Im2 in which the target gate portion is emphasized (S602). On the other hand, when the operation mode when entering the gate area is not a hands-off mode (S601 NO), the notification control unit F6 displays the normal gate guide image Im2.

[0135] The gate guide image Im2 in which the target gate is emphasized is, for example, an image in which the target gate is blinked or surrounded by a frame. The normal gate guide image Im2 is a gate guide image Im2 that is not decorated or processed in the hands-off mode.

[0136] In the hands-off mode, steering is left to the system, so the driver may be less interested in the gate the vehicle is about to pass through than in the operation mode that requires hands-on driving. In the hands-off mode, even if the system sets a gate different from the gate that the driver thinks is optimal as the target gate, the driver is unlikely to notice this. As a result, in the hands-off mode, there is a possibility that an override may occur just before the gate. To address such issues, the above configuration makes it easier for the driver to recognize the target gate position even in the hands-off mode.

[0137] When the hands-off mode is adopted when entering the gate area, the notification control unit F6 may display a gate passage icon Im3 on the display 21 instead of or together with the gate guide image Im2. Fig. 14 shows an example of the gate passage icon Im3.

[0138] The gate passing icon Im3 includes, for example, a vehicle image E4, a lane marking image E5, and a gate image E6. The vehicle image E4 is an image element representing the vehicle. The lane marking image E5 is an image element representing a lane marking. The gate image E6 is an image element representing a gate. The gate image E6 is an optional element and may be omitted.

[0139] When the camera 111 or the like can recognize the demarcation line or the end of the gate in front of the gate, the notification control unit F6 displays a gate passing icon Im3 in which the demarcation line image E5 is displayed in green, white, or the like on the display 21. According to this configuration, the driver can know that the vehicle can pass through the gate or that the system can normally recognize the passage position within the gate based on the display state of the gate passing icon Im3.

[0140] On the other hand, when the camera 111 or the like is unable to recognize the dividing line or the end of the gate in front of the gate, the notification control unit F6 may display on the display 21 a gate passing icon Im3 in which the dividing line image E5 is represented in gray or a broken line. When the camera 111 or the like is unable to recognize the dividing line or the end of the gate in front of the gate, the notification control unit F6 may display a gate passing icon Im3 with a question mark added near the dividing line image E5. According to these configurations, the driver can know that the system has not yet recognized the passage position in the gate based on the display state of the gate passing icon Im3. In this way, the gate passing icon Im3 is an image indicating whether or not the dividing line associated with the gate can be recognized. The gate passing icon Im3 corresponds to a passing image.

[0141] The notification control unit F6 may be configured to highlight / indicate the lane marking image E5 based on the fact that the vehicle is located in front of the target gate. The notification control unit F6 may be configured to gray out the lane marking image E5 when the vehicle is not located in front of the target gate.

[0142] The notification control unit F6 may also change the display mode of the gate guide image Im2 depending on whether or not the vehicle is following a preceding vehicle when entering the gate area. For example, as shown in Fig. 15, when the vehicle is following a preceding vehicle when entering the gate area (S701 YES), the notification control unit F6 displays a trackless gate guide image Im2a (S702). On the other hand, when the vehicle is not following a preceding vehicle when entering the gate area (S701 NO), the notification control unit F6 displays a normal gate guide image Im2 (S703).

[0143] The trackless gate guide image Im2a is a gate guide image that does not include the track image E2, as shown in Fig. 16. The normal gate guide image Im2 is an image that includes image elements that indicate the track of the host vehicle, as shown in Fig. 6 and Fig. 7. The normal gate guide image Im2 can be rephrased as a tracked gate guide image or a track guide image.

[0144] The state of following a preceding vehicle here means a state in which the function of the control for following the preceding vehicle is enabled and the preceding vehicle is actually recognized. When following a preceding vehicle, there is less need to notify the driver of the trajectory of the vehicle than when not following a preceding vehicle. By omitting the presentation of information that is less useful to the driver, it is possible to reduce the possibility of causing annoyance to the driver. Conversely, when not following a preceding vehicle, it is possible to give the driver a sense of security by displaying the planned travel trajectory of the vehicle.

[0145] The above description of "entering the gate area" may be interpreted as the time when the remaining distance to the gate point becomes less than a predetermined value. Also, the time when the vehicle is traveling in the area in front of the gate, such as the time when the vehicle has traveled a predetermined distance after entering the gate area, may be included in the time when the vehicle is traveling in the area in front of the gate.

[0146] <Using maps to determine entry into gate areas> Map data may include node data and link data. Node data is data about characteristic points (nodes) on multiple roads. For example, points where roads intersect, merge or branch, points where lanes increase or decrease, gate points, etc. are set as nodes. Link data is data about road sections (links) connecting nodes. Link data includes data about a link ID, which is a unique number that identifies a link, a link length that indicates the length of the link, a link direction, shape information of the link, node coordinates or node numbers of the start and end of the link, and road attributes. Node data includes data such as a node ID, which is a unique number for each node, the position coordinates, name, type of the node, and the link ID of the link connecting to the node.

[0147] Such node data may include intra-node map data that indicates the road shapes of an area related to the node. The intra-node map data corresponds to partial map data within a certain range based on the node.

[0148] When the map data stored in the map storage unit 14 includes intra-node map data as described above, the processor 31 may perform various processes from step S103 onward based on the host vehicle's entry into the range indicated by the intra-node map data associated with the gate point. In other words, the host vehicle's entry into a gate area also includes the host vehicle's entry into the range indicated by the intra-node map data associated with the gate point. The gate area may be the range indicated by the intra-node map data associated with the gate point. In addition, when data indicating a range is assigned to each node, the range may correspond to the gate area.

[0149] <Example of operation mode control> As shown in FIG. 17, the processor 31 may continue to maintain the level 3 mode even when passing through a gate. In that case, the processor 31 may request the driver to put his / her eyes on depending on the situation while maintaining the level 3 mode. The processor 31 may also perform a TOR notice depending on the traffic conditions such as whether there are surrounding vehicles and the complexity of the road structure. According to this configuration, the driver acts as a partner / assistant for the system in driving operations, which may improve safety. In addition, by requesting the driver to put his / her eyes on in advance, there is also an advantage that the driver can smoothly take over the driving operation even when it is actually necessary to switch driving from the system to the driver. The processor 31 may switch from the level 3 mode to the hands-on level 2 mode or the hands-off level 2 mode depending on the traffic conditions such as whether there are surrounding vehicles while traveling in the gate area.

[0150] Moreover, the processor 31 may be configured to temporarily switch from the level 3 mode to the level 2 mode at a predetermined timing after entering the gate area, as shown in FIG. 18. The processor 31 may automatically change the operation mode according to the position of the vehicle relative to the gate point. FIG. 18 illustrates a pattern in which the processor 31 maintains the level 3 mode when it enters the pre-gate area, while switching to the hands-on level 2 mode in a section where the remaining distance to the gate point is less than a predetermined value (e.g., 100 m). FIG. 17 illustrates a pattern in which the processor 31 switches to the hands-off level 2 mode when it passes through the gate, and then switches to the level 3 mode when it leaves the gate area. The normal area refers to an area that is neither a pre-gate area nor a post-gate area.

[0151] The timing of switching to the hands-on level 2 mode in the area in front of the gate may be a predetermined number of seconds (e.g., 5 seconds) after an eyes-on request is made. In the area just before the gate, lateral movement (lane changes) between vehicles toward the target gate may also be active. By switching the area in front of the gate to the hands-on level 2 mode, the driver can change lanes toward the target gate at his / her own responsibility. In addition, by switching the area in front of the gate to the hands-on level 2 mode, it becomes easier for the driver to select an arbitrary gate as the target gate.

[0152] In addition, when it is programmed to transition to the hands-on level 2 mode in the pre-gate area, it is preferable that the notification control unit F6 executes an eyes-on request in a stronger manner based on the fact that the vehicle has entered the gate area. This configuration can more strongly lead the driver's state to a state suitable for the hands-on level 2 mode. In addition, when transitioning from the level 3 mode to the hands-on level 2 mode, by requesting eyes-on before hands-on, rather than simultaneously implementing hands-on and eyes-on, it is expected that the burden on the driver involved in transitioning between operation modes can be reduced.

[0153] Of course, the control example of the operation mode for passing through the gate is not limited to this. As shown in Fig. 19, the processor 31 may switch to the hands-off level 2 mode when the remaining distance to the gate point becomes less than a predetermined value, and may switch to the hands-on level 2 mode when passing through the gate. After passing through the gate, the paths of vehicles are likely to cross due to branching roads and narrowing of the road width. By setting the operation mode in the post-gate area to one that requires the driver's hands on, it may be possible to flexibly deal with a forceful cut-in or abnormal approach by another vehicle.

[0154] <Changes in notification content> The above-mentioned eyes-on request may be replaced with a TOR notice. That is, the notification control unit F6 may execute a TOR notice based on the fact that the vehicle has entered the gate area. The TOR notice also has the effect of essentially encouraging the driver to check the surrounding conditions. In other words, the TOR notice can be understood as a kind of eyes-on request.

[0155] <Other Modifications> The notification control unit F6 may display a surveillance target image on the display 21 based on entry into the gate area. The surveillance target image is an image showing a surveillance target, which is another vehicle with a collision risk equal to or greater than a predetermined value. The surveillance target image may be an image showing the direction in which the surveillance target exists. The surveillance target image may include information about the characteristics of the surveillance target, such as color, size, and vehicle type. The surveillance target image may be, for example, an image in which the surveillance target is shown in a color different from that of the other vehicles in an overhead image showing another vehicle present around the vehicle.

[0156] By displaying the image of the warning target, the driver can recognize the vehicle that the system is paying attention to near the gate. By issuing an eyes-on request in conjunction with the image of the warning target, it is possible to align the system's judgment with the driver's sense. As a result, it is expected that the driver will be more likely to trust the system. The notification control unit F6 may output a message requesting the driver to check whether there are any dangerous vehicles other than the warning target (i.e., whether they have overlooked anything).

[0157] The above technical idea is to prepare for emergencies by requesting the driver to put their eyes on in a situation where eyes on is not actually required. Such an idea can also be applied to the hands-off level 2 mode, which is one step lower in automation level. In a situation where an operation mode that does not actually require hands on is applied, the driver may be requested to perform hands on or to stand by for hands on discreetly. The notification control unit F6 may output a hands-on request based on entering a gate area in the hands-off level 2 mode. In the hands-off level 2 mode, the notification control unit F6 may replace the above eyes-on request with a hands-on request and perform the various controls described above.

[0158] <System configuration> The notification control unit F6 may be disposed outside the autonomous driving ECU 30. For example, the notification control unit F6 may be provided in another computer such as an HCU.

[0159] <Vehicles to which this disclosure is applicable> The above-described embodiment is applicable to various vehicles that run on roads. The present disclosure can be mounted on various vehicles that can run on roads, such as two-wheeled vehicles, three-wheeled vehicles, and the like, in addition to four-wheeled vehicles. A motorized bicycle can also be included in the category of two-wheeled vehicles. The vehicle itself may be an electric vehicle or an engine vehicle. The electric vehicle can include not only electric vehicles, but also plug-in hybrid vehicles, hybrid vehicles, and fuel cell vehicles. A vehicle to which the system / apparatus / method, etc. of the present disclosure is applied may be a privately owned car or a service car. A service car refers to a vehicle provided for, for example, a car sharing service or a vehicle rental service. Service cars include taxis, route buses, and shared buses.

[0160] <Additional remarks (1)> This specification discloses the following technical ideas and combinations thereof. In addition, a vehicle control method and a computer program corresponding to the technical ideas listed below are also included in the scope of the present disclosure.

[0161] [Technical thought 1] A notification control device used in a vehicle configured to be able to perform automatic driving control, Obtaining data indicating whether the vehicle is traveling under the automatic driving control; Obtaining information about a gate point, which is a point where a plurality of gates are provided on a toll road; determining whether the vehicle has entered a gate area defined based on the gate point; A notification control device that notifies the driver to check the surrounding traffic conditions based on the vehicle entering the gate area under the automatic driving control.

[0162] [Technical thought 2] The notification control device according to technical idea 1 continues the notification until the gate is passed and terminates the notification upon passing the gate.

[0163] [Technical thought 3] The notification control device according to technical idea 1 or 2, which continues the notification until the gate area is exited.

[0164] [Technical thought 4] Acquire data on other vehicles present around the vehicle; A notification control device described in any one of technical ideas 1 to 3, which stops the notification when the other vehicle is not present in the vicinity of the vehicle.

[0165] [Technical thought 5] Acquire data on other vehicles present around the vehicle; A notification control device described in any one of technical ideas 1 to 3, which weakens the intensity of the notification when there is no other vehicle in the vicinity of the vehicle compared to when there is another vehicle in the vicinity of the vehicle.

[0166] [Technical Thought 6] acquiring data indicating a payment method for a target gate which is a gate through which the vehicle passes; A notification control device described in any one of technical ideas 1 to 5, which executes the notification when the target gate is a gate where manual settlement is possible, and omits the notification when the target gate is a gate where manual settlement is not possible.

[0167] [Technical Thought 7] A notification control device according to any one of technical ideas 1 to 6, which is used in a vehicle configured to selectively implement a hands-off prohibited mode in which the driver is obliged to hold the steering wheel and a hands-off enabled mode in which the driver is not obliged to hold the steering wheel, A notification control device that changes the manner of the notification depending on whether the vehicle is traveling through the gate area in the hands-off prohibited mode or the hands-off permitted mode.

[0168] [Technical Thought 8] The notification control device described in Technical Idea 7 strengthens the strength of the notification when driving through the gate area in the hands-off enabled mode compared to when driving through the gate area in the hands-off prohibited mode.

[0169] [Technical Thought 9] When traveling through the gate area in the hands-off mode, a gate recognition state notification image is displayed, which indicates that the lane markings associated with the gate can be recognized. The notification control device according to technical idea 7 or 8, in which the gate recognition state notification image is not displayed when driving through the gate area in the hands-off prohibited mode.

[0170] [Technical Thought 10] acquiring data indicating whether or not to perform a lane change after passing through the gate; When a course change is planned to be implemented after passing through the gate, the notification is implemented, while A notification control device described in any one of technical ideas 1 to 9, which omits the notification when it is not planned to change course after passing through the gate, or weakens the intensity of the notification compared to when it is planned to change course after passing through the gate.

[0171] [Technical Thought 11] Obtaining data indicative of whether or not the vehicle is following a preceding vehicle; A notification control device described in any one of technical ideas 1 to 10, which implements the following: when the vehicle is not following the preceding vehicle in the gate area, an image of the gate area is displayed on a display in which an image element indicating the vehicle's trajectory is superimposed on an image of the gate area, and when the vehicle is not following the preceding vehicle in the gate area, an image of the gate area not including an image element indicating the vehicle's trajectory is displayed on a display.

[0172] [Technical Thought 12] A notification control device according to any one of technical ideas 1 to 11, which displays a trajectory image showing the trajectory of the vehicle until it passes through the gate on a display, A notification control device that does not display the orbit image in a position overlapping with the gate.

[0173] [Technical Thought 13] A notification control device according to technical idea 3, which changes the manner of the notification before passing through the gate point and after passing through the gate point.

[0174] The notification control device may obtain data indicating whether or not the vehicle is traveling under automatic driving control from the automatic driving unit (Fn), which is a component for implementing automatic driving control. The notification control device may obtain information about a gate point, which is a point where multiple gates are provided on a toll road, from an output signal of a perimeter monitoring sensor, a wireless signal received from an external device, or map data. The determination of whether or not the vehicle has entered a gate area defined based on the gate point may also be performed based on the output signal of a perimeter monitoring sensor, a wireless signal received from an external device, or map data.

[0175] <Additional remarks (2)> The various flowcharts shown in the present disclosure are all examples, and the number of steps constituting the flowcharts and the order of execution of the processes can be changed as appropriate. The various process flows shown in the present disclosure may be performed in parallel with, in combination with, or partially replaced by other processes. Note that the expression corresponding to the case where the remaining distance to the gate point in the present disclosure is less than a predetermined value may be replaced with the case where the vehicle enters the gate area. For example, step S102 may be a step of determining whether the vehicle has entered the gate area.

[0176] The device, system, and method thereof described in the present disclosure may be realized by a dedicated computer that configures a processor programmed to execute one or more functions embodied by a computer program. The device and method described in the present disclosure may be realized by a dedicated hardware logic circuit. The device and method described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. For example, some or all of the functions of the processor 31 may be realized as hardware. A form in which a certain function is realized as hardware includes a form in which it is realized using one or more ICs, etc. As the processor (computation core), a CPU, an MPU, a GPU, a DFP (Data Flow Processor), etc. can be adopted. Some or all of the functions of the processor 31 may be realized using any of a system-on-chip (SoC), an integrated circuit (IC), and a field-programmable gate array (FPGA). The computer program may be stored in a computer-readable non-transitive tangible storage medium as instructions executed by a computer. A hard-disk drive (HDD), a solid-state drive (SSD), a flash memory, or the like may be used as the program storage medium. The scope of the present disclosure also includes forms such as a program for causing a computer to function as the processor 31, and a non-transitive tangible storage medium such as a semiconductor memory on which the program is stored. [Explanation of symbols]

[0177] 11 Surrounding monitoring sensor, 14 Map memory unit, 15 Wireless communication device, 18 External display device, 21 Display, 22 Speaker, 30 Autonomous driving ECU, 31 Processor, 32 Memory, F1 Information acquisition unit, F2 Environment recognition unit, F21 Gate recognition unit, F22 Surrounding vehicle recognition unit, F3 Mode control unit, F4 Planning unit, F5 Vehicle control unit, F51 ACC system, F6 Notification control unit, Fn Autonomous driving unit, Sys Autonomous driving system

Claims

1. A notification control device used in a vehicle configured to be able to perform automatic driving control, Obtaining data indicating whether the vehicle is traveling under the automatic driving control; Obtaining information about a gate point, which is a point where a plurality of gates are provided on a toll road; determining whether the vehicle has entered a gate area defined based on the gate point; Acquiring data on other vehicles present around the vehicle; When the other vehicle is present around the vehicle, a notification is given to the driver to prompt the driver to check the surrounding traffic conditions based on the fact that the vehicle has entered the gate area under the automatic driving control; A notification control device configured to stop the notification when the other vehicle is not present in the vicinity of the vehicle.

2. A notification control device used in a vehicle configured to be able to perform automatic driving control, Obtaining data indicating whether the vehicle is traveling under the automatic driving control; Obtaining information about a gate point, which is a point where a plurality of gates are provided on a toll road; determining whether the vehicle has entered a gate area defined based on the gate point; Acquiring data on other vehicles present around the vehicle; Based on the fact that the vehicle has entered the gate area under the automatic driving control, a notification is given to the driver to prompt the driver to check a surrounding traffic condition, moreover, A notification control device configured to weaken the intensity of the notification when the other vehicle is not present in the vicinity of the vehicle compared to when the other vehicle is present in the vicinity of the vehicle.

3. A notification control device used in a vehicle configured to be able to perform automatic driving control, Obtaining data indicating whether the vehicle is traveling under the automatic driving control; Obtaining information about a gate point, which is a point where a plurality of gates are provided on a toll road; determining whether the vehicle has entered a gate area defined based on the gate point; acquiring data indicating a payment method for a target gate which is a gate through which the vehicle passes; When the target gate is a gate for which manual settlement is possible, a notification is given to the driver to urge the driver to check the surrounding traffic conditions based on the fact that the vehicle has entered the gate area under the automatic driving control; A notification control device configured to omit the notification when the target gate is a gate for which manual settlement is not possible.

4. A notification control device for use in a vehicle configured to selectively implement a hands-off prohibited mode in which automatic driving control is performed with the driver being obligated to hold the steering wheel, and a hands-off enabled mode in which automatic driving control is performed without the driver being obligated to hold the steering wheel, Obtaining data indicating whether the vehicle is traveling under the automatic driving control; Obtaining information about a gate point, which is a point where a plurality of gates are provided on a toll road; determining whether the vehicle has entered a gate area defined based on the gate point; Based on the fact that the vehicle has entered the gate area under the automatic driving control, a notification is given to the driver to prompt the driver to check the surrounding traffic conditions; A notification control device configured to change the manner of the notification when driving through the gate area in the hands-off prohibited mode and when driving through the gate area in the hands-off permitted mode.

5. The notification control device according to claim 4 , wherein when the vehicle is traveling through the gated area in the hands-off enabled mode, the intensity of the notification is increased compared to when the vehicle is traveling through the gated area in the hands-off disabled mode.

6. When traveling through the gate area in the hands-off mode, a gate recognition state notification image is displayed, which indicates that the lane markings associated with the gate can be recognized. The notification control device according to claim 4 , wherein the gate recognition state notification image is not displayed when the vehicle travels through the gate area in the hands-off prohibition mode.

7. A notification control device used in a vehicle configured to be able to perform automatic driving control, Obtaining data indicating whether the vehicle is traveling under the automatic driving control; Obtaining information about a gate point, which is a point where a plurality of gates are provided on a toll road; determining whether the vehicle has entered a gate area defined based on the gate point; Based on the fact that the vehicle has entered the gate area under the automatic driving control, a notification is given to the driver to prompt the driver to check a surrounding traffic condition, moreover, acquiring data indicating whether a lane change is planned after passing through the gate; When a course change is planned to be implemented after passing through the gate, the notification is implemented, while The notification control device is configured to omit the notification when a lane change is not planned to be made after passing through the gate, or to weaken the intensity of the notification compared to when a lane change is planned to be made after passing through the gate.

8. A notification control device used in a vehicle configured to be able to perform automatic driving control, Obtaining data indicating whether the vehicle is traveling under the automatic driving control; Obtaining information about a gate point, which is a point where a plurality of gates are provided on a toll road; determining whether the vehicle has entered a gate area defined based on the gate point; Based on the fact that the vehicle has entered the gate area under the automatic driving control, a notification is given to the driver to prompt the driver to check the surrounding traffic conditions. Obtaining data indicative of whether or not the vehicle is following a preceding vehicle; When the vehicle is not following the preceding vehicle in the gate area, an image of the gate area is displayed on a display in which an image element showing a trajectory of the vehicle is superimposed; and when the vehicle is not following the preceding vehicle in the gate area, displaying an image of the gate area on a display that does not include image elements showing the vehicle's trajectory.

9. The notification control device according to claim 1 , wherein the notification is continued until the gate is passed and is terminated based on the fact that the gate is passed.

10. The notification control device according to claim 1 , wherein the notification is continued until the gated area is exited.

11. 2. The notification control device according to claim 1, further comprising: a display that displays a trajectory image showing a trajectory of the vehicle until the vehicle passes through the gate; A notification control device that does not display the orbit image in a position overlapping with the gate.

12. The notification control device according to claim 10, wherein a mode of the notification is changed between before passing through the gate point and after passing through the gate point.

Images