Driving Support Device, Driving Support Method, and Storage Medium

Abstract

Provided are a driving support device, a driving support method, and a storage medium that can perform more appropriate driving support. The driving support device according to the embodiment includes: an identification unit that identifies the surrounding conditions of the vehicle; a driving control unit that controls at least the steering of the vehicle based on the surrounding conditions identified by the identification unit; a reception unit that receives an operation of an occupant of the vehicle; and an output control unit that causes an output unit to output the driving state of the vehicle. The driving control unit performs a lane change of the vehicle when an instruction to change lanes of the vehicle is received by the reception unit and the execution conditions for the lane change are satisfied. When the lane change by the driving control unit has not started even after a first specified time has elapsed since the instruction was received by the reception unit, the output control unit causes the output unit to output information related to the lane change.

Description

Technical Field

[0001] The present invention relates to a driving support device, a driving support method, and a storage medium. Background Art

[0002] In recent years, research related to autonomous driving that automatically controls the driving of a vehicle has been continuously progressing. In connection with this, the following technology is known: when an operation of a user to move a host vehicle object in an image displayed on a display unit from a first lane to a second lane is received, a command to change the driving lane of the host vehicle from the first lane to the second lane is output to an autonomous driving control unit that controls autonomous driving (for example, International Publication No. 2017 / 022197). Summary of the Invention

[0003] However, in a situation where the lane change cannot be started due to the influence of the surrounding conditions or the like after the occupant has indicated a lane change, the occupant sometimes forgets that a lane change has been indicated. Therefore, when the lane change is executed after a certain period of time has elapsed since the lane change was indicated, there is a case where the lane change is executed in a state unexpected by the occupant and appropriate driving support cannot be performed.

[0004] The solution of the present invention was completed in consideration of such a situation, and one of its purposes is to provide a driving support device, a driving support method, and a storage medium that can perform more appropriate driving support.

[0005] The driving support device, driving support method, and storage medium of the present invention adopt the following configuration.

[0006] (1): One aspect of the present invention relates to a driving support device, wherein the driving support device includes: an identification unit that identifies the surrounding conditions of the vehicle; a driving control unit that controls at least the steering of the vehicle based on the surrounding conditions identified by the identification unit; a reception unit that receives an operation of an occupant of the vehicle; and an output control unit that causes an output unit to output the driving state of the vehicle. The driving control unit executes a lane change of the vehicle when an instruction to change the lane of the vehicle is received by the reception unit and the execution conditions for the lane change are satisfied. When the lane change by the driving control unit has not started even after a first specified time has elapsed since the instruction was received by the reception unit, the output control unit causes the output unit to output information related to the lane change.

[0007] (2): Based on the solution of (1) above, the information related to the lane change includes information related to the cancellation of the lane change.

[0008] (3): Based on the solution in (1) above, when the output unit outputs information related to the lane change and the instruction to cancel the lane change is not received by the receiving unit even after a second specified time, the output control unit ends the output of the information, and the driving control unit continues to execute the lane change.

[0009] (4): Based on the solution in (3) above, the second specified time is a time shorter than the first specified time.

[0010] (5): Based on the solution in (1) above, the output unit includes a display device, and the output control unit causes the display device to display an image representing information related to the lane change.

[0011] (6): Based on the solution in (3) above, when the output control unit ends the output of the information and the lane change by the driving control unit has not started even after a third specified time, the output unit outputs information related to the lane change.

[0012] (7): Based on the solution in (6) above, the third specified time is the same as the first specified time, or a time that changes according to the number of times or time of outputting information related to the lane change.

[0013] (8): One aspect of the present invention relates to a driving support method, in which the driving support method causes a computer to perform the following processes: identifying the surrounding conditions of the vehicle; the driving control unit controlling at least the steering of the vehicle based on the identified surrounding conditions; receiving the operation of the vehicle occupant; causing the output unit to output the driving state of the vehicle; when receiving an instruction from the occupant to change the lane of the vehicle and satisfying the execution conditions for the lane change, executing the lane change of the vehicle; when the lane change by the driving control unit has not started even after a first specified time from receiving the instruction, causing the output unit to output information related to the lane change.

[0014] (9): One aspect of the present invention relates to a storage medium storing a program, in which the program causes a computer to perform the following processes: identifying the surrounding conditions of the vehicle; the driving control unit controlling at least the steering of the vehicle based on the identified surrounding conditions; receiving the operation of the vehicle occupant; causing the output unit to output the driving state of the vehicle; when receiving an instruction from the occupant to change the lane of the vehicle and satisfying the execution conditions for the lane change, executing the lane change of the vehicle; when the lane change by the driving control unit has not started even after a first specified time from receiving the instruction, causing the output unit to output information related to the lane change.

[0015] According to the above solutions (1) to (9), more appropriate driving support can be performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural diagram of a vehicle system using the driving support device of the embodiment.

[0017] Figure 2 It is a functional structural diagram of the first control unit and the second control unit.

[0018] Figure 3 It is a diagram showing an example of the relationship between the driving mode, the control state of the vehicle, and the tasks.

[0019] Figure 4 It is a diagram for explaining the lane change determination unit, the permission determination unit, and the HMI control unit.

[0020] Figure 5 It is a diagram showing an example of an image for proposing a lane change.

[0021] Figure 6 It is a diagram showing an example of an image showing information related to a lane change.

[0022] Figure 7 It is a diagram for explaining the case where the permission determination unit determines that the lane change has not been canceled.

[0023] Figure 8 It is a flowchart showing an example of the process executed by the automatic driving control device of the embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] Hereinafter, embodiments of the driving support device, the driving support method, and the storage medium of the present invention will be described with reference to the drawings. Hereinafter, as an example, an embodiment in which the driving support device is applied to an autonomous vehicle will be described. Autonomous driving means, for example, automatically controlling one or both of the steering and acceleration / deceleration of the vehicle to perform driving control. In the driving control of the vehicle, various driving support controls such as ACC (Adaptive Cruise Control), ALC (Auto Lane Changing), LKAS (Lane Keeping Assistance System), and TJP (Traffic Jam Pilot) can be included, for example. The autonomous vehicle can also sometimes be controlled by the manual driving of an occupant (driver).

[0025] [OVERALL STRUCTURE]

[0026] Figure 1FIG. 1 is a structural diagram of a vehicle system 1 that utilizes the driving support device according to the embodiment. The vehicle equipped with the vehicle system 1 is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using the generated electric power from a generator connected to the internal combustion engine, or the discharge electric power of a secondary battery or a fuel cell.

[0027] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, an operation unit 45, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitoring camera 70, a driving operation member 80, an autonomous driving control device 100, a driving force output device 200, a braking device 210, and a steering device 220. These devices and equipment are interconnected by a multi-channel communication line such as a CAN (Controller Area Network) communication line, a serial communication line, or a wireless communication network. Figure 1 The structure shown is only an example, and a part of the structure may be omitted, or other structures may be added. The HMI 30 is an example of an "output unit". The operation unit 45 is an example of a "receiving unit". The combination of the operation unit 45 and the autonomous driving control device 100 is an example of a "driving support device".

[0028] The camera 10 is, for example, a digital camera that uses a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is installed at an arbitrary position of the vehicle (hereinafter referred to as vehicle M) equipped with the vehicle system 1. When shooting the front, the camera 10 is installed at the upper part of the front windshield, the back of the interior rearview mirror, etc. The camera 10 periodically repeats shooting the periphery of the vehicle M. The camera 10 may also be a stereo camera.

[0029] The radar device 12 emits radio waves such as millimeter waves to the periphery of the vehicle M, and detects the radio waves (reflected waves) reflected by an object to detect at least the position (distance and azimuth) of the object. The radar device 12 is installed at an arbitrary position of the vehicle M. The radar device 12 may also detect the position and speed of an object by the FM-CW (Frequency Modulated Continuous Wave) method.

[0030] The LIDAR 14 irradiates light (or an electromagnetic wave with a wavelength close to light) to the periphery of the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to an object based on the time from light emission to light reception. The irradiated light is, for example, pulsed laser light. The LIDAR 14 is installed at any part of the vehicle M.

[0031] The object recognition device 16 performs sensor fusion processing on the detection results detected by a part or all of the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition result to the autonomous driving control device 100. The object recognition device 16 may directly output the detection results of the camera 10, the radar device 12, and the LIDAR 14 to the autonomous driving control device 100. The object recognition device 16 may also be omitted from the vehicle system 1.

[0032] The communication device 20 communicates with other vehicles existing in the periphery of the vehicle M or communicates with various server devices via a wireless base station, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc.

[0033] The HMI 30 outputs various information to the occupants of the vehicle M under the control of the HMI control unit 170. The HMI 30 may also function as a reception unit that receives input operations performed by the occupants. The HMI 30 includes, for example, a display device 32 and a speaker 34. The HMI 30 may also include a microphone, a buzzer, buttons, indicator lights, etc.

[0034] The display device 32 is, for example, an LCD (Liquid Crystal Display), an organic EL (ElectroLuminescence) display device, etc. The display device 32 is, for example, provided near the front of the driver's seat (the seat closest to the steering wheel) on the instrument panel, at a position where an occupant (especially, the driver) can visually confirm through the gap of the steering wheel or over the steering wheel. In the display device 32, information required for driving during manual driving or autonomous driving of the vehicle M (hereinafter referred to as driving support information) is displayed as an image. The driving support information includes, for example, the speed of the vehicle M, the engine speed, the fuel level, the radiator water temperature, the driving distance, the state of the shift lever, the lane (lane marking), other vehicles, etc. recognized by the object recognition device 16, the autonomous driving control device 100, etc., the lane in which the vehicle M should travel, the future target trajectory, etc. information. The driving support information may include information indicating the switching of the driving mode described later, the driving state under driving support, information related to lane changes under driving support (information related to the proposal of lane changes and the cancellation of lane changes), etc.

[0035] In addition to being provided at the above-described positions, the display device 32 may also be provided near the center of the instrument panel. In this case, in addition to displaying driving support information, the display device 32 also displays, for example, an image indicating the navigation result of the navigation device 50. On the display device 32, it is also possible to display TV programs, entries stored in a DVD, or movies and other entries downloaded from an external device via the communication device 20.

[0036] The display device 32 may, for example, also include a HUD (Head Up Display). The HUD projects an image onto a part of the windshield in front of the driver's seat, for example, so that the occupant sitting in the driver's seat can visually confirm a virtual image. In the HUD, for example, driving support information and the like are displayed. The display device 32 may also be configured as a touch panel having a function of receiving an operation input from an occupant.

[0037] At least one speaker 34 is provided inside the vehicle. The speaker 34 outputs sound, warning sounds, etc. under the control of the HMI control unit 170, for example.

[0038] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the vehicle M, an acceleration sensor that detects acceleration, a yaw rate sensor that detects the angular velocity about the vertical axis, an azimuth sensor that detects the orientation of the vehicle M, and the like. The vehicle sensor 40 may include a position sensor that acquires the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. The position sensor may also be a sensor that uses the GNSS (Global Navigation Satellite System) receiver 51 of the navigation device 50 to acquire position information.

[0039] The operation unit 45 receives operations from the occupants of the vehicle M. The operation unit 45 includes, for example, a mode changeover switch that receives the changeover of the driving mode described later, a permission switch that permits a proposal for a lane change under the control of the system side (the autonomous driving control device 100 side), a release switch that cancels the execution of the permitted lane change, and the like. The permission switch and the release switch may also be constituted by one switch. In this case, for example, by turning the switch on and off, the operations of permission and release can be performed. In the operation unit 45, for example, a mechanical switch is installed on the steering wheel or the instrument panel. The operation unit 45 may also be integrally formed with the HMI 30. For example, the above-described switches of the operation unit 45 are displayed on the display device 32 as GUI (Graphical User Interface) switches.

[0040] The navigation device 50 includes, for example, a GNSS receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores the first map information 54 in a storage device such as an HDD (Hard Disk Drive) or a flash memory. The GNSS receiver 51 determines the position of the vehicle M based on the signals received from GNSS satellites. The position of the vehicle M can also be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, a speaker, a touch panel, buttons, etc. Part or all of the navigation HMI 52 can also be shared with the aforementioned HMI 30. The route determination unit 53 determines, for example, a route (hereinafter referred to as the on-map route) from the position of the vehicle M determined by the GNSS receiver 51 (or an arbitrarily input position) to the destination input by the occupant using the navigation HMI 52 with reference to the first map information 54. The first map information 54 is information that represents the road shape, for example, by showing road segments and nodes connected by the road segments. The first map information 54 can also include information such as the curvature of the road and POI (Point Of Interest) information. The on-map route is output to the MPU 60. The navigation device 50 can also perform route guidance using the navigation HMI 52 based on the on-map route. The navigation device 50 can be implemented, for example, by the functions of a terminal device such as a smartphone or a tablet terminal held by the occupant. The navigation device 50 can also send the current position and the destination to a navigation server via the communication device 20 and obtain a route equivalent to the on-map route from the navigation server.

[0041] The MPU 60 includes, for example, a recommended lane determination unit 61 and stores the second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 divides the on-map route provided from the navigation device 50 into a plurality of blocks (for example, divided every 100 [m] in the vehicle traveling direction) and determines the recommended lane for each block with reference to the second map information 62. For example, when the currently traveling lane or the road that will be traveled in the near future has multiple lanes, the recommended lane determination unit 61 makes a decision on which lane to travel in, counting from the left. When there is a branch point on the on-map route, the recommended lane determination unit 61 determines the recommended lane so that the vehicle M can travel on a reasonable route for traveling to the branch destination.

[0042] The second map information 62 is map information with higher precision than the first map information 54. The second map information 62 includes, for example, information on the center of a lane or information on the boundary of a lane. The second map information 62 may include road information (road type), the number of lanes of a road, presence or absence of branching and merging, legal speed (speed limit, maximum speed, minimum speed), traffic restriction information, address information (address, postal code), facility information, telephone number information, etc. The second map information 62 can be updated at any time by communicating with other devices through the communication device 20.

[0043] The driver monitoring camera 70 is, for example, a digital camera that uses a solid-state imaging device such as a CCD or a CMOS. The driver monitoring camera 70 is installed, for example, at a position and in an orientation that enables it to capture the head of an occupant (driver) sitting in the driver's seat of the vehicle M from the front (the orientation for capturing the face). For example, the driver monitoring camera 70 is installed above a display device provided at the center of the instrument panel of the vehicle M.

[0044] The driving operation member 80 includes, for example, not only the steering wheel 82 but also an accelerator pedal, a brake pedal, a shift lever, and other operation members. Sensors for detecting the operation amount or the presence or absence of an operation are installed on the driving operation member 80, and the detection results are output to a part or all of the autonomous driving control device 100, or the driving force output device 200, the braking device 210, and the steering device 220. The steering wheel 82 is an example of an "operation member that receives a steering operation performed by the driver". The operation member does not necessarily have to be ring-shaped and may be in the form of a non-circular steering wheel, a joystick, a button, etc. A steering wheel gripping sensor 84 is installed on the steering wheel 82. The steering wheel gripping sensor 84 is implemented by a capacitance sensor or the like and outputs a signal capable of detecting whether the driver is gripping the steering wheel 82 (which means contacting it in a state of applying force) to the autonomous driving control device 100.

[0045] The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, an HMI control unit 170, and a storage unit 180. The first control unit 120, the second control unit 160, and the HMI control unit 170 are respectively implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components can be implemented by hardware (including a circuitry unit) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or can be implemented by the cooperation of software and hardware. The program can be pre-stored in a storage device (a storage device having a non-transitory storage medium) such as an HDD or a flash memory of the automatic driving control device 100, or can be stored in a removable storage medium such as a DVD or a CD-ROM, and is installed in the HDD or flash memory of the automatic driving control device 100 by being mounted on a driving device through the storage medium (non-transitory storage medium). The combination of the action plan generation unit 140 and the second control unit 160 is an example of a "driving control unit". The HMI control unit 170 is an example of an "output control unit".

[0046] The storage unit 180 can also be implemented by the above-mentioned various storage devices, or an SSD (Solid State Drive), an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), or a RAM (Random Access Memory), etc. In the storage unit 180, for example, information required to execute the driving support in the present embodiment, various other information, programs, etc. are stored.

[0047] Figure 2It is a functional structure diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, an identification unit 130, an action plan generation unit 140, and a mode determination unit 150. The first control unit 120 implements functions based on AI (Artificial Intelligence) and functions based on a pre-given model in parallel, for example. For example, the function of "identifying an intersection" can be implemented by "parallelly executing the identification of an intersection based on deep learning, etc., and the identification based on pre-given conditions (signals, road markings, etc. capable of pattern matching), and comprehensively evaluating by scoring both". Thereby, the reliability of autonomous driving is ensured.

[0048] Based on the information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16, the identification unit 130 identifies the position (relative position), speed (relative speed), acceleration, and other states of the objects (such as other vehicles, other obstacles) around the vehicle M. The position of the object is, for example, identified as the position on the absolute coordinates with the representative point (center of gravity, center of drive shaft, etc.) of the vehicle M as the origin, and is used for control. The position of the object can be represented by the representative points such as the center of gravity and corners of the object, or can be represented by a region. The "state" of the object, in the case where the object is a moving body such as another vehicle, may also include the acceleration, jerk, or "action state" of the other vehicle (for example, whether it is changing lanes or about to change lanes).

[0049] The identification unit 130 identifies, for example, the lane (travel lane) on which the vehicle M is traveling.

[0050] For example, the identification unit 130 compares the pattern of the road dividing line obtained from the second map information 62 (for example, the arrangement of solid lines and dashed lines) with the pattern of the road dividing line around the vehicle M identified from the image captured by the camera 10, thereby identifying the travel lane. The identification unit 130 is not limited to identifying the road dividing line, and can also identify the travel lane by identifying the road dividing line, the travel road boundary (road boundary) including the road shoulder, curb, median strip, guardrail, etc. In this identification, the position of the vehicle M obtained from the navigation device 50 and the processing result processed by the INS can also be added. The identification unit 130 identifies the temporary stop line, obstacles, red lights, toll stations, road signs, and other road phenomena. The identification unit 130 identifies the adjacent lanes adjacent to the travel lane. The adjacent lanes are, for example, lanes that can travel in the same direction as the travel lane.

[0051] When the recognition unit 130 recognizes the driving lane, it recognizes the position and posture of the vehicle M relative to the driving lane. For example, the recognition unit 130 may also recognize the deviation of the reference point of the vehicle M from the center of the lane and the angle formed by the traveling direction of the vehicle M with respect to the line connecting the centers of the lanes as the relative position and posture of the vehicle M relative to the driving lane. Alternatively, the recognition unit 130 may recognize the position of the reference point of the vehicle M relative to any side end (road dividing line or road boundary) of the driving lane as the relative position of the vehicle M relative to the driving lane. Here, the reference point of the vehicle M may be the center of the vehicle M or the center of gravity of the vehicle M. The reference point may be an end (front end, rear end) of the vehicle M or the position where one of the multiple wheels of the vehicle M is located.

[0052] The action plan generation unit 140 generates a target trajectory for the vehicle M to automatically (independent of the driver's operation) travel in the future in a manner that it travels on the recommended lane determined by the recommended lane determination unit 61 in principle and can cope with the surrounding conditions of the vehicle M. The target trajectory includes, for example, a speed element. For example, the target trajectory is represented as a trajectory formed by arranging in sequence the points (trajectory points) that the vehicle M should reach. The trajectory points are the points that the vehicle M should reach at regular driving distances (for example, about several [m]) along the travel distance. Different from this, the target speed and target acceleration at regular sampling times (for example, about zero point several [sec]) are generated as part of the target trajectory. The trajectory points may also be the positions that the vehicle M should reach at regular sampling times. In this case, the information of the target speed and target acceleration is represented by the interval of the trajectory points. The action plan generation unit 140 may also generate a target trajectory such that the speed of the vehicle M becomes the set speed within the range where it can travel when the preset speed of the vehicle M is determined.

[0053] When generating the target trajectory, the action plan generation unit 140 may set events (functions) for autonomous driving. Among the events for autonomous driving, there are events such as constant-speed driving event, low-speed following driving event, lane change event, branch event, merging event, takeover event, etc. The action plan generation unit 140 generates a target trajectory corresponding to the activated event. The action plan generation unit 140 may also, when executing the driving control of the vehicle M, a specified event, etc., propose (recommend) the driving control and the execution of the event to the occupant according to the driving mode of the vehicle M described later, and generate a corresponding target trajectory when the proposal is accepted.

[0054] The action plan generation unit 140 includes, for example, a lane change determination unit 142 and an acceptance determination unit 144. Details of their respective functions will be described later.

[0055] The mode determination unit 150 determines, based on the conditions of the vehicle M and the like, the driving mode to be executed by the vehicle M as any one of a plurality of driving modes with different tasks assigned to the occupants (in other words, a plurality of modes with different degrees of automation). The driving control unit drives the vehicle M in the mode determined by the mode determination unit 150. The mode determination unit 150 includes, for example, a driver state determination unit 152 and a mode change processing unit 154.

[0056] Figure 3 It is a diagram showing an example of the relationship between the driving mode, the control state of the vehicle M, and the tasks.

[0057] In the driving mode of the vehicle M, for example, there are five modes from mode A to mode E. Among modes A to E, regarding the control state, that is, the degree of automation of the driving control of the vehicle M, mode A is the highest, and then it decreases in the order of mode B, mode C, mode D, and mode E is the lowest. On the contrary, regarding the tasks assigned to the occupants, mode A is the least intensive, and then it becomes more intensive in the order of mode B, mode C, and mode D, and mode E for manual driving becomes the most intensive. In modes B to E, the control state becomes non-automatic driving, so the automatic driving control device 100 functions before ending the control related to automatic driving and transferring to driving support or manual driving. Modes A and B are examples of the "first driving mode", and mode C is an example of the "second driving mode". Hereinafter, the details of each mode will be described using a driver as an example of the occupant.

[0058] In mode A, it becomes an automatic driving state, and neither the task of surrounding monitoring of the vehicle M nor the task of holding the steering wheel 82 (hereinafter referred to as "steering wheel holding") is assigned to the driver. Regarding whether the driver is performing surrounding monitoring, for example, it is determined based on the analysis result of the captured image of the driver monitoring camera 70. Regarding whether the steering wheel is being held, for example, it is determined based on the detection result of the steering wheel holding sensor 84. The surrounding monitoring includes at least the monitoring of the front of the vehicle M. The front refers to the space in the traveling direction of the vehicle M visually confirmed through the front windshield. However, even in mode A, it is required that the driver be in a body posture that can quickly transfer to manual driving according to the requirements of the system centered on the automatic driving control device 100. The automatic driving here means that either the steering or the speed of the vehicle M is controlled without depending on the driver's operation.

[0059] Mode A is, for example, a driving mode that can be executed when the vehicle M is traveling at a speed equal to or lower than a specified speed (e.g., about 50 [km / h]) on a motor vehicle-only road such as a highway and there are conditions satisfied such as a preceding vehicle to be followed. Sometimes it is called the "TJP mode". When the conditions for traveling in Mode A are no longer satisfied or when a mode switching operation is received by the operation unit 45, the mode determination unit 150 changes the driving mode of the vehicle M to another mode (e.g., Mode B).

[0060] During the execution of Mode A, the driver can perform secondary tasks. Secondary tasks are, for example, behaviors other than driving that are permitted for the driver during the autonomous driving of the vehicle M. Secondary tasks include, for example, watching TV, making a call on a mobile phone, sending and receiving emails, and eating.

[0061] In Mode B, it becomes a driving support state, and the driver is assigned the task of monitoring the surroundings of the vehicle M (hereinafter referred to as surrounding monitoring), but the task of holding the steering wheel 82 is not assigned. In Mode B, for example, ACC, ALC, LKAS, etc. are executed. ACC, ALC, LKAS can also be executed under the limitation of specified tasks in Modes C and D, for example. In Mode B, for example, without accepting a lane change instruction from the driver, the vehicle M's lane change (ALC) based on the path setting to the destination by the navigation device 50 is executed through the judgment on the vehicle system 1 side. A lane change means moving the vehicle M from the lane on which the vehicle M is traveling to an adjacent lane adjacent to the current lane.

[0062] In Mode C, it becomes a driving support state, and the driver is assigned the tasks of surrounding monitoring and holding the steering wheel 82. In Mode C, for example, when the vehicle system 1 side determines that a lane change of the vehicle M is required, a proposal is made to the driver via the HMI 30, and when permission for the lane change is received from the driver through the operation unit 45, the lane change is executed through driving control that controls at least the steering among the steering and acceleration / deceleration of the vehicle M.

[0063] In the case of Mode C, even when a lane change cannot be executed at the current time point, if it is determined that a lane change is advisable on the vehicle system 1 side in the future, a proposal is made to the driver to accept the permission (reservation) for the lane change. Subsequently, when the vehicle M meets the execution conditions for the lane change based on the surrounding conditions, etc., a lane change based on driving control is performed. The execution conditions for the lane change, for example, refer to the generation of a target trajectory by the action plan generation unit 140 for changing the lane from the driving lane of the vehicle M to the lane of the lane change destination. For example, when there are other vehicles in the driving lane of the vehicle M and the lane of the lane change destination, the action plan generation unit 140 sets a target position (target position) as the lane change destination based on the relative position and relative distance between the vehicle M and the other vehicles, and generates a target trajectory for moving to the target position without contacting the other vehicles.

[0064] When the target position cannot be set due to road conditions such as congestion or construction, the action plan generation unit 140 does not generate a target trajectory for performing a lane change, but generates a target trajectory for continuing to drive in the current driving lane. When the occupant has given permission for the lane change, the action plan generation unit 140 repeatedly searches for the target position, and once the target position can be set, it generates an action plan for performing the lane change.

[0065] The above proposal for a lane change through driving control (driving support) can also be made when the driving mode is Mode B or Mode A. The following mode control is performed: when permission is obtained for the proposal for the lane change, the driving mode is changed to Mode C, and when permission is not obtained, the current Mode B or Mode A is continued.

[0066] Mode D is a driving mode in which at least one of the steering and acceleration / deceleration of the vehicle M requires a certain degree of driving operation by the driver. In Mode D, when an instruction to change the lane of the vehicle M is received through the driver's operation of the turn signal control lever, driving control for executing the lane change in the indicated direction is performed. The lane change in Modes B to D can also be an example of a lane change event.

[0067] In Mode E, it becomes a state of manual driving in which both the steering and acceleration / deceleration of the vehicle M require driving operations by the driver. Naturally, in both Modes D and E, the driver is assigned the task of monitoring the front of the vehicle M. The driving subject in Modes C to E is the driver.

[0068] The mode determination unit 150 determines the driving mode executed by the vehicle M based on, for example, the driver's state. For example, the mode determination unit 150 obtains the execution state of the task, and when the task related to the determined driving mode has not been executed by the driver, it changes to a driving mode with a more severe task assigned to the driver. For example, when the driver is in a body posture that cannot transfer to manual driving according to the requirements from the system during the execution of mode A (for example, continuously looking around outside the allowable area, detecting signs of driving difficulties), the mode determination unit 150 executes control to urge the driver to transfer to manual driving in mode E using the HMI control unit 170 and the HMI 30. When the driver does not respond even after the HMI control unit 170 executes control to urge the transfer to manual driving for a specified time, or when it is presumed that the driver is not in a state of performing manual driving, the mode determination unit 150 performs control to stop the vehicle M at the target position by autonomous driving and stop (end) the autonomous driving after the stop. After stopping the autonomous driving, the vehicle M becomes in the state of mode D or E, and the vehicle M can be started by the driver's manual operation. Hereinafter, the same applies to "stopping the autonomous driving".

[0069] In mode B, when the driver does not monitor the front, the mode determination unit 150 uses the HMI 30 to urge the driver to perform peripheral monitoring. If the driver does not respond, it performs control to stop the vehicle M at the target position and stop the autonomous driving. In the case where the driver does not monitor the front or does not hold the steering wheel 82 in mode C, the mode determination unit 150 uses the HMI 30 to urge the driver to perform peripheral monitoring and / or hold the steering wheel 82. If the driver does not respond, it performs control to stop the vehicle M at the target position and stop the autonomous driving. In mode C and mode D, when the vehicle M does not execute a lane change in the section for reaching the specified location, it is also possible to perform control to stop the vehicle M at the target position and stop the autonomous driving.

[0070] The driver state determination unit 152 determines whether the driver is in a state suitable for driving. For example, the driver state determination unit 152 monitors the driver's state for the above-mentioned mode change and determines whether the driver's state is a state corresponding to the task. For example, the driver state determination unit 152 analyzes the image captured by the driver monitoring camera 70 to perform a posture estimation process and determines whether the driver is in a body posture that cannot transfer to manual driving according to the requirements from the system.

[0071] The driver state determination unit 152 analyzes the images captured by the driver monitoring camera 70 to perform a line-of-sight estimation process, and determines whether the driver is monitoring the surroundings (more specifically, the front) of the vehicle M. When it is determined that the state not corresponding to the task has continued for a specified time or more, the driver state determination unit 152 determines that the driver is in a state unfit for driving the task. When it is determined that the state is corresponding to the task, the driver state determination unit 152 determines that the driver is in a state suitable for driving the task. The driver state determination unit 152 may also determine whether the driver is in a state where driving substitution can be performed.

[0072] The mode determination unit 150 may, instead of (or in addition to) the above-described driver state, determine the driving mode of the vehicle M based on the driving state of the vehicle M, whether the driver permits the executed driving control, etc. In this case, the mode determination unit 150 determines the driving mode of the vehicle M, for example, based on the determination result determined by the permission determination unit 144. Details of the determination of the driving mode based on the above-described determination result will be described later.

[0073] The mode change processing unit 154 performs various processes for changing from the current driving mode to the mode determined by the mode determination unit 150. For example, the mode change processing unit 154 causes the HMI control unit 170 to control the HMI 30 or instructs the generation of a target trajectory for executing or stopping the autonomous driving in order to prompt the driver to perform a specified operation. When the operation unit 45 receives an operation for switching the driving mode of the vehicle M to a specified mode, the mode change processing unit 154 performs various processes for executing the mode. When the operation unit 45 receives permission for a lane change executed by the vehicle system 1 or permission for canceling a lane change, etc., the mode change processing unit 154 executes various processes for switching to a mode corresponding to the received content.

[0074] The second control unit 160 controls the driving force output device 200, the braking device 210, and the steering device 220 so that the vehicle M passes through the target trajectory generated by the action plan generation unit 140 at a predetermined time.

[0075] The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on a target trajectory (trajectory points) generated by the action plan generation unit 140 and stores the information in a memory (not shown). The speed control unit 164 controls the driving force output device 200 or the braking device 210 based on the speed element attached to the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 combines feedforward control corresponding to the curvature of the road ahead of the vehicle M with feedback control based on the deviation from the target trajectory and executes it.

[0076] The HMI control unit 170 notifies a prescribed information to an occupant (driver) of the vehicle M through the HMI 30. The prescribed information includes, for example, driving support information. For example, the HMI control unit 170 may generate an image including the above-mentioned prescribed information and display the generated image on the display device 32, or may generate a sound representing the prescribed information and output the generated sound from the speaker 34. The HMI control unit 170 may also output the information received by the HMI 30 and the operation unit 45 to the communication device 20, the navigation device 50, the first control unit 120, etc. Details of the functions of the HMI control unit 170 will be described later.

[0077] The driving force output device 200 outputs a driving force (torque) for the vehicle M to travel to the drive wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above structure according to the information input from the second control unit 160 or the information input from the driving operation member 80.

[0078] The braking device 210 includes, for example, a brake caliper, a hydraulic cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the hydraulic cylinder, and a brake ECU. The brake ECU controls the electric motor according to the information input from the second control unit 160 or the information input from the driving operation member 80 so that a braking torque corresponding to the braking operation is output to each wheel. The braking device 210 may include a mechanism that transmits the hydraulic pressure generated by the operation of the brake pedal included in the driving operation member 80 to the hydraulic cylinder via a master hydraulic cylinder as a backup. The braking device 210 is not limited to the structure described above, and may also be an electronically controlled hydraulic braking device that controls an actuator according to the information input from the second control unit 160 to transmit the hydraulic pressure of the master hydraulic cylinder to the hydraulic cylinder.

[0079] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor applies a force to a rack and pinion mechanism, for example, to change the orientation of the steering wheels. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the steering wheel 82 of the driving operation member 80 to change the orientation of the steering wheels.

[0080] [Lane change determination unit, permission determination unit, and HMI control unit]

[0081] Hereinafter, the details of the functions of the lane change determination unit 142, the permission determination unit 144, and the HMI control unit 170 will be described. Figure 4 FIG. is for explaining the lane change determination unit 142, the permission determination unit 144, and the HMI control unit 170. In Figure 4 this example, a vehicle M traveling at a speed VM on a lane L2 in two lanes L1 and L2 that can travel in the same direction (the X-axis direction in the figure) is shown. The lane L1 is demarcated by the demarcation lines LL and CL, and the lane L2 is demarcated by the demarcation lines CL and RL. The demarcation line CL is a lane line on which lane change can be made between the lanes L1 and L2. The lanes L1 and L2 are, for example, exclusive motor vehicle roads such as highways. In the following description, the time t11 is the earliest, and the times t12, t13, t14, t15, t16, and t17 become later in sequence. Hereinafter, the vehicle M travels in mode B at a time earlier than the time t11, but it may also be the case that the vehicle travels in mode A.

[0082] The lane change determination unit 142 determines whether the vehicle M will change lanes through driving control at the current time point or in the future based on the surrounding conditions recognized by the recognition unit 130. For example, the lane change determination unit 142 determines whether to change the lane of the vehicle M based on the travel lane of the vehicle M and the number of lanes of the road that can travel in the same direction including the travel lane. The number of lanes can be recognized, for example, from the captured image of the camera 10 or from the map information (for example, the second map information 62). For example, the lane change determination unit 142 compares the recommended lane determined by the recommended lane determination unit 61 in the multi-lanes that can travel in the same direction with the travel lane of the vehicle M, and determines to change the lane of the vehicle M when the recommended lane and the travel lane are not the same lane.

[0083] For example, when the vehicle M is traveling in the left lane of a two-lane road and the recommended lane is the right lane, the lane change determination unit 142 determines to change the lane of the vehicle M to the right lane. When the vehicle M is traveling in the right or left lane among three lanes facing the traveling direction and the recommended lane is the central lane, it is determined to change the lane of the vehicle M to the central lane. When a destination is set in advance by the navigation device 50 or the like, the lane change determination unit 142 determines to change the lane to a lane that can travel in the direction of the destination when the traveling lane of the vehicle M is not a lane that can travel in the direction of the destination. When the vehicle M is traveling on a road with four or more lanes, the lane change determination unit 142 also determines that a lane change to the recommended lane is required when the vehicle M is not traveling in the pre-set recommended lane.

[0084] The lane change determination unit 142 may also compare the future recommended lane with the traveling lane in the case where the road shape in which the vehicle M travels will change in the future (for example, when there is a branch, a merge, an increase or decrease in the number of lanes several kilometers ahead), and determine to change the lane when they are not the same lane.

[0085] In Figure 4 the example of, at the time t11 when the vehicle M reaches the point P11, the lane change determination unit 142 determines to change the lane from the lane L2 to the lane L1. In this case, the HMI control unit 170 generates an image that proposes (recommends) a lane change through driving control, and causes the display device 32 to output the generated image to notify the driver.

[0086] Figure 5 FIG. is an example of an image IM10 that proposes a lane change. In the image IM10, for example, a proposed content display area A11 is included. In the proposed content display area A11, for example, information related to the proposal of the lane change is displayed. In Figure 5 the example of, in the proposed content display area A11, literal information such as "It is recommended to change the lane from the current traveling lane to the right lane." and "If you approve the lane change, please perform the approval operation." is displayed. The NMI control unit 170 may, on the basis of (or instead of) the above literal information, display an image (including an animated image) that reminds the occupant of the lane change, or may light up an indicator lamp indicating the lane change included in the HMI 30. The HMI control unit 170 may, on the basis of (or instead of) the display of the image and the indicator, generate a sound representing the proposed content (for example, related to Figure 5The text information shown (equivalent sound), and causes the generated sound to be output from the speaker 34. It is also possible to output the sounds of an alarm, a buzzer, etc. from the speaker 34. Thus, the driver of the vehicle M can easily recognize that a lane change has been proposed from the vehicle system 1 side and the operation content in the case of approval.

[0087] The approval determination unit 144 determines whether it has received the driver's approval of the above-mentioned lane change proposal. For example, when the HMI control unit 170 displays the image IM10 on the display device 32 and the operation unit 45 receives a specified operation, the approval determination unit 144 determines that the lane change by driving control has been approved. The specified operation can be, for example, an operation of pressing the approval switch provided on the operation unit 45 once in the state where the image IM10 is displayed, or an operation of turning on the approval switch. The approval determination unit 144 can also determine that no approval has been received when the operation unit 45 does not receive the specified operation within a specified time. In the case where no approval is received, the lane change is not executed.

[0088] In Figure 4 the example, after the image IM10 is displayed at time t11, at time t12 when the vehicle M reaches the location P12, the approval determination unit 144 determines that it has received the driver's approval of the lane change proposal (lane change approval). In this case, the mode determination unit 150 changes the driving mode of the vehicle M from mode B to mode C. By changing to mode C, not only the task of peripheral monitoring but also the task of holding the steering wheel is assigned to the driver. And, based on the surrounding conditions recognized by the recognition unit 130, the action plan generation unit 140 searches for the target position of the lane L1 of the lane change destination, generates a target track for changing lanes to the target position at the time when the searched target position is found, and starts the lane change according to the generated target track.

[0089] When the action plan generation unit 140 cannot set a target position due to the presence of other vehicles traveling on lane L1, obstacles such as construction work areas, etc., it generates an action plan to continue traveling on lane L2, and causes the vehicle M to travel according to the generated action plan. When the state where lane change cannot be performed continues for a long time after lane change is permitted, the driver may forget that lane change has been permitted. Therefore, when lane change is performed after a certain period of time after lane change is instructed, it is possible that the driver is surprised due to lane change being performed in a state unexpected by the occupants, resulting in unnecessary steering operations and braking operations. Therefore, when lane change by the driving control unit has not started even after a first specified time has elapsed after lane change is permitted, the HMI control unit 170 causes the HMI 30 to output information related to lane change. The so-called "lane change not started" means, for example, that the execution conditions for lane change are not satisfied. The so-called "lane change not started" can be the situation where the entire vehicle M is on the lane L2 side, the lateral movement amount (the movement amount along the lateral direction of the road, the movement amount in the Figure 4 -Y axis direction in the

[0090] In the Figure 4 example, at time t13 (the time point when the vehicle M reaches location P13) when a first specified time (ΔT1) has elapsed since time t12, the HMI control unit 170 generates an image representing information related to lane change, and causes the generated image to be displayed on the display device 32. Figure 6 FIG. is a diagram showing an example of an image IM20 showing information related to lane change. The image IM20 includes, for example, a driving state display area A21. In the driving state display area A21, as information related to lane change, for example, text information such as "The state where lane change cannot start continues." and "Lane change can be cancelled by the permission operation." is displayed.

[0091] By displaying Figure 6The displayed image IM20 enables the driver to grasp the situation where the vehicle M is in a state where it is about to perform a lane change through driving control but cannot start the lane change due to surrounding conditions or the like. Therefore, it is possible to make it difficult for the driver to forget (or make the driver recall) the fact that lane change through driving control has been permitted. Moreover, by displaying information related to the cancellation of the lane change, it is possible to suppress misoperations in the case of wanting to cancel the lane change. When the HMI control unit 170 notifies the driver of information related to the lane change, it notifies only by displaying the image IM20 and avoids making a voice-based notification. As a result, the situation where the driver gets bored with the notification can be reduced.

[0092] In a state where the image IM20 is displayed on the display device 32 by the HMI control unit 170, the permission determination unit 144 determines whether the execution of the lane change related to the permitted driving control has been cancelled (or whether the cancellation of the lane change has been permitted) based on whether a specified operation is received by the operation unit 45. The time when the image IM20 is displayed is the time for the driver to judge whether to continue the lane change. The specified operation can be, for example, an operation of pressing the cancellation switch provided on the operation unit 45 once in a state where the image IM20 is displayed, or an operation of pressing the permission switch once, or an operation of setting the permission switch to the off state.

[0093] In Figure 4 , for example, in a state where the image IM20 is displayed at time t13, at time t14 when the vehicle M reaches the point P14, the permission determination unit 144 determines that the lane change has been cancelled (permission for cancellation has been given). In this case, the driving control unit aborts the lane change to the lane L1. Then, the mode determination unit 150 changes the driving mode from mode C to mode B, and the driving control for the vehicle M to travel on the lane L2 is executed.

[0094] Figure 7 is a diagram for explaining the case where the permission determination unit 144 determines that the lane change has not been cancelled. Figure 7 The processing up to time t13 in Figure 7As shown, at time t15 (the time when the vehicle M reaches point P15) which is after the second specified time (ΔT2) has elapsed since time t13 when the image IM20 is displayed, the HMI control unit 170 ends the display of the image IM20. Thereby, by displaying the image IM20, it is possible to grasp that the vehicle M is to perform a lane change through driving control, and by ending the display of the image IM20 in a shorter time than the first specified time, the annoyance degree of the driver caused by the notification can be reduced. In the case where the driver does not perform an operation (release consent) for canceling the lane change, the lane change through driving control continues. Therefore, compared with the case where an operation for continuing the lane change is performed every time the image IM20 is displayed, the convenience of the driver can be improved.

[0095] The consent for the lane change also continues after time t13. Therefore, as long as the driving control unit satisfies the execution conditions for changing to lane L1, it executes the lane change to lane L1.

[0096] The HMI control unit 170 may also, after ending the display of the image IM20 at time t15, display the image IM20 on the display device 32 again at time t16 (the time when the vehicle M reaches point P16) which is after the third specified time (ΔT3) has elapsed. The third specified time is, for example, the same as the first specified time. The third specified time may also be changed according to the number of times the image IM20 is displayed or the display time (total display time). For example, it is predicted that the more the number of times and the time of the notification to the driver through the display of the image IM20 increase, the less likely the occupant is to forget that the lane change has been consented. Therefore, the more the number of times and the display time of the image IM20 increase, the longer the HMI control unit 170 sets the third specified time compared to the first specified time. The HMI control unit 170 may also perform control to cancel the next display once based on the number of times or the display time of the image IM20. It may also be that the longer the driving distance of the vehicle M in the state where the image IM20 is displayed, the longer the third specified time is adjusted. In this way, by adjusting the interval at which the image IM20 is displayed again, the annoyance degree caused by the display of the image IM20 can be reduced. By repeatedly displaying the image IM20, it is possible to prevent the driver from forgetting that the lane change has been consented, and thus more appropriate driving support can be achieved.

[0097] After the HMI control unit 170 displays the image IM20 at time t16, if no specified operation is received via the operation unit 45 at time t17 (the time when the vehicle M reaches the location P17) after the elapse of the fourth specified time ΔT4, the display of the image IM20 is ended. The fourth specified time is the same as the second specified time. The fourth specified time may also be shortened according to the number of times the image IM20 is displayed or the display time (total display time). However, the HMI control unit 170 adjusts the fourth specified time to prevent it from being less than a predetermined minimum display time. When a lane change has not started after time t17 and the cancellation of the lane change has not been permitted, the HMI control unit 170 repeatedly displays the image IM20 based on the third specified time and the fourth specified time.

[0098] [Processing Flow]

[0099] Figure 8 It is a flowchart showing an example of the processing flow executed by the automatic driving control device 100 of the embodiment. Hereinafter, the description will mainly focus on the processing related to the proposal of a lane change from the vehicle system 1 side, the approval of the proposal, and the cancellation of the lane change in the processing executed by the automatic driving control device 100. When the following processing is executed, the vehicle M is traveling in mode B. In the following processing, the state of the driver determined by the driver state determination unit 152 is a state that conforms to the tasks assigned to the driver in the driving mode executed by the vehicle M. Therefore, in Figure 8 the processing, no mode change is made based on the state of the driver.

[0100] In Figure 8 this example, the recognition unit 130 recognizes the surrounding situation of the vehicle M (step S100). Next, the lane change determination unit 142 determines whether to propose a lane change for the vehicle M based on the recognized surrounding situation of the vehicle M, etc. (step S102). When it is determined that a lane change is proposed, the HMI control unit 170 generates an image indicating the proposal of the lane change (for example, the image IM10) and causes the display device 32 to output the generated image (step S104).

[0101] Next, the consent determination unit 144 determines whether consent to the proposed lane change is received by the operation unit 45 (step S106). If it is determined that consent to the lane change is received, the mode determination unit 150 determines to switch the driving mode of the vehicle M from mode B to mode C (step S108). Thereby, the driving control unit performs the switching from mode B to mode C. Next, the action plan generation unit 140 determines whether the lane change to the lane change destination can be performed (step S110). If it is determined that the lane change can be performed, the driving control unit performs the lane change to the lane change destination (step S112). Next, the mode determination unit 150 determines to change the driving mode of the vehicle M from mode C to mode B (step S114). Thereby, the driving control unit performs the mode switching to mode B.

[0102] In the process of step S110, when the lane change cannot be performed after the consent to the lane change (in other words, when the lane change has not started), the action plan generation unit 140 determines whether the first specified time has elapsed since the consent to the lane change was received (step S116). If it is determined that the first specified time has not elapsed, the process returns to step S110. If it is determined that the first specified time has elapsed, the HMI control unit 170 generates an image (such as image IM20) including information related to the cancellation of the lane change, and causes the generated image to be displayed on the display device (step S118).

[0103] Next, the HMI control unit 170 determines whether consent to the cancellation of the lane change is received (step S120). If it is determined that the cancellation consent is received, the driving control unit performs the process of step S114 (the process of switching the driving mode from mode C to mode B) without performing the lane change. If it is determined that the cancellation consent is not received, the HMI control unit 170 determines whether the second specified time has elapsed since the image including the information related to the cancellation of the lane change was displayed on the display device 32 (step S122). If it is determined that the second specified time has not elapsed, the process returns to step S120. If it is determined that the second specified time has elapsed, the HMI control unit 170 ends the display of the image including the information related to the cancellation of the lane change (step S124), and returns to the process of step S110. Thereby, the process of this flowchart is executed. If it is determined in the process of step S102 that no lane change is proposed, or if it is determined in the process of step S106 that consent to the lane change is not received by the operation unit 45, the process of this flowchart ends.

[0104] <Variant Example>

[0105] In the above-described embodiment, control is performed when, after the images IM10 and IM20 are displayed, a prescribed operation is received via the operation unit 45. However, instead of this, a GUI (Graphical User Interface) switch for selecting whether to permit (or cancel) the display of the images IM10 and IM20 may be displayed, and a determination of permission may be made by receiving an operation of the displayed GUI switch by the driver. In this case, the display device 32 (touch panel) that has displayed an image including the GUI switch serves as an example of the "receiving unit".

[0106] According to the above-described embodiment, the driving support device includes: an identification unit 130 that identifies the surrounding conditions of the vehicle M; a driving control unit that controls at least the steering of the vehicle M based on the surrounding conditions identified by the identification unit 130; an operation unit (an example of the receiving unit) 45 that receives an operation of an occupant of the vehicle M; and an HMI control unit (an example of the output control unit) that causes the HMI 30 (an example of the output unit) to output the driving state of the vehicle M. The driving control unit performs a lane change of the vehicle M when an instruction to change the lane of the vehicle M is received by the operation unit 45 and the execution conditions for the lane change are satisfied. When the lane change by the driving control unit has not started even after a first prescribed time has elapsed since the instruction was received by the operation unit 45, the NMI control unit 170 causes the HMI 30 to output information related to the lane change, thereby enabling more appropriate driving support.

[0107] Specifically, according to the above-described embodiment, when a lane change cannot be performed after permission (reservation) of the lane change and a prescribed time has elapsed, information related to the lane change is notified, thereby suppressing the driver from forgetting about the permission of the lane change over time and avoiding the driver being surprised by the implementation of the lane change. According to the embodiment, the information related to the lane change notified to the occupant includes information related to whether it is necessary to cancel the lane change and the method for canceling the lane change. Therefore, it is not automatically canceled unless a cancellation operation is performed, thereby improving the convenience for the driver. According to the embodiment, the display time of the information related to the lane change is shortened, and the information related to the lane change is notified only by displaying an image on the display device 32, thereby reducing the annoyance of the driver with respect to the notification.

[0108] The above-described embodiment can be expressed in the following manner.

[0109] A driving support device, wherein,

[0110] The driving support device is configured to include:

[0111] A storage device storing a program; and

[0112] A hardware processor,

[0113] performs the following processing by executing a program stored in the storage device through the hardware processor:

[0114] Identify the surrounding conditions of the vehicle;

[0115] The driving control unit controls at least the steering in the vehicle based on the identified surrounding conditions;

[0116] Receive the operation of the occupant of the vehicle;

[0117] Cause the output unit to output the driving state of the vehicle;

[0118] When receiving an instruction from the occupant to change lanes of the vehicle and the execution conditions for lane change are met, perform the lane change of the vehicle;

[0119] When the lane change by the driving control unit has not started after a first specified time has elapsed since receiving the instruction, cause the output unit to output information related to the lane change.

[0120] As described above, the specific embodiments of the present invention have been described using the embodiments, but the present invention is in no way limited to such embodiments, and various modifications and substitutions can be made without departing from the gist of the present invention.

Claims

1. A driving support device, wherein: the driving support device includes: a recognition unit that recognizes the surrounding conditions of the vehicle; a driving control unit that controls at least the steering of the vehicle based on the surrounding conditions recognized by the recognition unit; a reception unit that receives the operations of the occupants of the vehicle; and an output control unit that causes an output unit to output the driving state of the vehicle, the driving control unit performs a lane change of the vehicle when an instruction to perform a lane change of the vehicle is received by the reception unit and the execution conditions for the lane change are satisfied, when the lane change by the driving control unit has not started after a first predetermined time has elapsed since the instruction was received by the reception unit, the output control unit causes the output unit to output information related to the lane change, when an instruction to cancel the lane change has not been received by the reception unit after a second predetermined time has elapsed since the information related to the lane change was output by the output unit, the output control unit ends the output of the information, the driving control unit continues the execution of the lane change.

2. The driving support device according to claim 1, wherein: the information related to the lane change includes information related to the cancellation of the lane change.

3. The driving support device according to claim 1, wherein: the second predetermined time is a time shorter than the first predetermined time.

4. The driving support device according to claim 1, wherein: the output unit includes a display device, the output control unit causes the display device to display an image representing information related to the lane change.

5. The driving support device according to claim 1, wherein: when the lane change by the driving control unit has not started after a third predetermined time has elapsed since the output control unit ended the output of the information, the output control unit causes the output unit to output information related to the lane change.

6. The driving support device according to claim 5, wherein: the third predetermined time is the same as the first predetermined time, or a time that changes according to the number of times or time of outputting information related to the lane change.

7. A driving support method, wherein: the driving support method causes a computer to perform the following processes: recognize the surrounding conditions of the vehicle; a driving control unit controls at least the steering of the vehicle based on the recognized surrounding conditions; receive the operations of the occupants of the vehicle; cause an output unit to output the driving state of the vehicle; perform a lane change of the vehicle when an instruction to perform a lane change of the vehicle is received and the execution conditions for the lane change are satisfied; cause the output unit to output information related to the lane change when the lane change by the driving control unit has not started after a first predetermined time has elapsed since the instruction was received; end the output of the information when an instruction to cancel the lane change has not been received after a second predetermined time has elapsed since the information related to the lane change was output; continue the execution of the lane change.

8. A storage medium stores a program, wherein, the program causes a computer to perform the following processes: recognize the surrounding conditions of the vehicle; a driving control unit controls at least the steering of the vehicle based on the recognized surrounding conditions; receive an operation of an occupant of the vehicle; cause an output unit to output a driving state of the vehicle; when receiving an instruction from the occupant to change lanes of the vehicle and the execution conditions for the lane change are satisfied, perform the lane change of the vehicle; when the lane change by the driving control unit has not started after a first specified time has elapsed since receiving the instruction, cause the output unit to output information related to the lane change; when an instruction to cancel the lane change has not been received after a second specified time has elapsed since the output unit output the information related to the lane change, end the output of the information; continue the execution of the lane change.

Images