Driving assistance device, driving assistance method, and program

Abstract

To provide a driver assistance device, a driver assistance method, and a program that can provide more appropriate driver assistance depending on the status of information acquisition regarding traffic signals. [Solution] The driving assistance device of the embodiment includes: an ambient information acquisition unit that acquires information about the surroundings of the vehicle; a traffic light information acquisition unit that acquires the location information and lighting status of traffic lights located around the vehicle based on the vehicle's location information; and a driving assistance unit that provides driving assistance for the vehicle based on the acquisition results from the ambient information acquisition unit and the traffic light information acquisition unit. The driving assistance unit suppresses the acceleration of the vehicle when, in the lane the vehicle is traveling in, the distance from the vehicle to the traffic light is less than a predetermined distance, and the traffic light information acquisition unit has not been able to acquire information on the color of the illuminated traffic light.

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device, a driving assistance method, and a program. [Background Art]

[0002] In recent years, initiatives to provide access to sustainable transportation systems that also take into account vulnerable people among traffic participants have become active. To achieve this goal, efforts are focused on research and development that further improves traffic safety and convenience through research and development related to driving assistance technology. In relation to this, in recent years, there have been known techniques in which, when a plurality of traffic signals are recognized, alarm output is controlled based on a combination of the lighting colors of the plurality of traffic signals, the vehicle is decelerated when the permitted traveling direction of an arrow signal is different from the planned traveling direction of the vehicle, or the vehicle is controlled in an autonomous mode based on the reliability of the traffic signal (see, for example, Patent Documents 1 to 3). [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2023-112545 [Patent Document 2] Japanese National Publication of International Patent Application No. 2015-518600 [Patent Document 3] Japanese Unexamined Patent Application Publication No. 2021-76899 [Summary of the Invention] [Problems to be Solved by the Invention]

[0004] By the way, in conventional driving assistance technology, there are several situations that can be assumed where the lighting color of a traffic signal is undefined, and there has been a problem that appropriate driving assistance such as speed control may not be provided for each of the assumed situations.

[0005] This application was made in consideration of these circumstances, and one of its objectives is to provide a driver assistance device, driver assistance method, and program that can provide more appropriate driver assistance depending on the status of information acquisition regarding traffic signals. Ultimately, this will contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The driver assistance device, driver assistance method, and program according to this invention employ the following configuration. (1) A driving assistance device according to one aspect of the present invention comprises: an ambient information acquisition unit that acquires information about the surroundings of the vehicle; a traffic light information acquisition unit that acquires the location information and lighting status of traffic lights located around the vehicle based on the vehicle's location information; and a driving assistance unit that provides driving assistance for the vehicle based on the acquisition results from the ambient information acquisition unit and the traffic light information acquisition unit, wherein the driving assistance unit suppresses the acceleration of the vehicle when, in the lane the vehicle is traveling in, the distance from the vehicle to the traffic light is less than a predetermined distance, and the traffic light information acquisition unit has not been able to acquire information on the color in which the traffic light is lit.

[0007] (2): In the embodiment of (1) above, the vehicle further comprises a driving operation detection unit for detecting driving operations performed by the driver of the vehicle, and the driving support unit suppresses the acceleration of the vehicle due to the driver's acceleration operation detected by the driving operation detection unit when the distance from the lane to the traffic light in the lane on which the vehicle is traveling is less than a predetermined distance and the traffic light information acquisition unit has not been able to acquire information on the color of the illuminated traffic light.

[0008] (3) In the embodiment of (2) above, the case in which the color of the illuminated traffic light has not been acquired includes the period from when the existence of the traffic light is acquired based on the acquisition results of the traffic light information acquisition unit until the information of the color of the illuminated traffic light is acquired.

[0009] (4): In the embodiment of (2) above, the driving support unit suppresses the acceleration of the vehicle when it is no longer possible to obtain information on the color of the illuminated traffic light after the traffic light information acquisition unit has determined that the illuminated color of the traffic light is blue.

[0010] (5) In the embodiment of (2) above, the driving support unit slows down its own vehicle when it is no longer possible to obtain information on the color of the illuminated traffic light after the traffic light information acquisition unit has determined that the illuminated color of the traffic light is red or yellow.

[0011] (6): In the embodiment of (5) above, the driving support unit shall, when the traffic signal is lit in yellow or red according to the traffic signal information acquisition unit, and the driving operation detection unit has not detected any deceleration operation by the driver or the degree of deceleration operation is below a threshold, perform at least one of the following: notify the driver and decelerate the vehicle.

[0012] (7) In the embodiment of (1) above, the driving support unit does not suppress the acceleration of the vehicle when the speed of the vehicle is below a predetermined speed.

[0013] (8) In the embodiment of (1) above, the driving support unit does not suppress the acceleration of the vehicle when the surrounding information acquisition unit acquires information about a following vehicle in the vicinity of the vehicle.

[0014] (9) In the embodiment of (1) above, the driver status detection unit further comprises a driver status detection unit for detecting the status of the driver of the vehicle, and the driver support unit does not suppress the acceleration of the vehicle when the driver status detection unit detects that the driver is monitoring the road ahead.

[0015] (10): In the embodiment of (1) above, the driving support unit shall not suppress the acceleration of the vehicle if the surrounding information acquisition unit determines that there is an adjacent lane adjacent to the lane in which the vehicle is traveling and that can proceed in the same direction as the lane, and there is another vehicle passing through the intersection where the traffic light is installed in the adjacent lane.

[0016] (11): In the embodiment of (1) above, the signal information acquisition unit acquires information regarding the color of the signal light that is illuminated based on an image captured by a camera mounted on the vehicle or information received by a communication device that communicates with the signal light.

[0017] (12): Another aspect of the present invention is a driving assistance method in which a computer acquires information about the surroundings of the vehicle, acquires information about the location and lighting status of traffic lights in the surroundings of the vehicle based on the location information of the vehicle, provides driving assistance for the vehicle based on the acquired information, and suppresses the acceleration of the vehicle when the distance from the vehicle to the traffic light is less than a predetermined distance in the lane in which the vehicle is traveling and information about the color of the illuminated traffic light has not been acquired.

[0018] (13): A program according to another aspect of the present invention causes a computer to acquire information about the surroundings of its own vehicle, to acquire information about the location and lighting status of traffic lights in the vicinity of the vehicle based on the vehicle's location information, to provide driving assistance for the vehicle based on the acquired information, and to suppress the acceleration of the vehicle when the distance from the vehicle to the traffic light in the lane the vehicle is traveling in is less than a predetermined distance and information about the color of the illuminated traffic light has not been acquired. [Effects of the Invention]

[0019] According to the embodiments described in (1) to (13) above, more appropriate driving assistance can be provided depending on the status of information acquisition regarding traffic signals. [Brief explanation of the drawing]

[0020] [Figure 1] It is a configuration diagram of a vehicle system 1 including the driving assistance device according to the embodiment. [Figure 2] It is a diagram showing an example of a scene where the driving assistance of the embodiment is executed. [Figure 3] It is a diagram showing an example of a screen when a following vehicle exists around the host vehicle M. [Figure 4] It is a diagram showing an example of a scene where the host vehicle M travels on a road shape having a plurality of lanes in which the host vehicle M can travel in the same direction. [Figure 5] It is a flowchart showing an example of processing executed by the driving assistance device 100. Mode for Carrying Out the Invention

[0021] Hereinafter, embodiments of a driving assistance device, a driving assistance method, and a program according to the present invention will be described with reference to the drawings. Hereinafter, an example in which the driving assistance device is applied to a vehicle will be described. Note that the driving assistance device may be applied to moving bodies other than vehicles, for example, moving bodies such as ships, aircraft, and standing riding vehicles having a power unit.

[0022] [Overall Configuration] Figure 1 is a configuration diagram of a vehicle system 1 including the driving assistance device according to the embodiment. A vehicle on which the vehicle system 1 is mounted (hereinafter referred to as the host vehicle M) is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle or a micro-mobility, and the drive source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or electric power discharged from a battery (storage battery) such as a secondary battery or a fuel cell.

[0023] Vehicle system 1 includes, for example, a camera 10, a radar device 12, a LiDAR (Light Detection and Ranging) 14, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an in-cabin camera 70, a driver control unit 80, a driver assistance device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. Note that the configuration shown in Figure 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added. Camera 10, radar device 12, and LiDAR 14 are examples of "detection devices DD". HMI 30 is an example of a "notification unit".

[0024] Camera 10 is a digital camera that uses a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle M on which the vehicle system 1 is installed. When imaging the front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, or on the front of the vehicle body. When imaging the rear, camera 10 is mounted on the top of the rear windshield or on the tailgate. When imaging the sides, camera 10 is mounted on the left and right door mirrors, etc. Camera 10 periodically and repeatedly images the area around the vehicle M. Camera 10 may also be a stereo camera.

[0025] The radar device 12 emits radio waves (radar) such as millimeter waves around the vehicle M and detects radio waves (reflected waves) reflected by surrounding objects to detect at least the position (distance and bearing) of an object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of an object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0026] The LIDAR 14 illuminates the area around the vehicle M with light and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time from emission to reception. The emitted light is, for example, pulsed laser light. The LIDAR 14 can be attached to any location on the vehicle M.

[0027] The communication device 20 communicates with other vehicles in the vicinity of its own vehicle M, terminal devices of users using its own vehicle M, or various server devices, for example, by utilizing networks such as cellular networks, Wi-Fi networks, Bluetooth®, DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), and the Internet.

[0028] The HMI30 outputs various information to the occupants of the vehicle M (including the driver) and accepts input operations from the occupants. The HMI30 includes, for example, a display unit and a speaker. The display unit is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display device. The display unit displays various images (including video) in the embodiment. The display unit may be configured integrally with the input unit as a touch panel. The speaker outputs predetermined sounds (for example, notification sounds or message sounds). The HMI30 may also include a microphone, buzzer, touch panel, switches, keys, etc. The switches may include switches for executing or terminating predetermined driving controls that can be executed by the driving control unit described later, and switches for approving (permitting) or rejecting driving control recommendations (suggestions) from the system (vehicle system 1). The switches may also include switches for operating the turn signals (turn signal switches), etc.

[0029] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, and a yaw rate sensor for detecting yaw rate (for example, the rotational angular velocity around the vertical axis passing through the center of gravity of the vehicle M). The vehicle sensor 40 may also include a steering angle sensor for detecting the steering angle of the vehicle M (which may be the angle of the steering wheels or the operating angle of the steering wheel), a steering angular velocity sensor for detecting the steering angular velocity, and a compass sensor for detecting the orientation of the vehicle M.

[0030] Furthermore, the vehicle sensor 40 may include a position sensor that detects the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that acquires position information using, for example, a GNSS (Global Navigation Satellite System) receiver of a navigation device 50. The vehicle sensor 40 may derive the speed of the vehicle M from the difference (i.e., distance) of position information at a predetermined time from the position sensor. The results detected by the vehicle sensor 40 are output to the driver assistance device 100.

[0031] The navigation device 50 includes, for example, a GNSS receiver, a navigation HMI, and a route determination unit. The navigation device 50 may store map information in a storage device such as an HDD (Hard Disk Drive) or flash memory, or it may acquire map information 182 stored in a storage unit 180, which will be described later. The GNSS receiver determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The navigation HMI includes a display device, speaker, touch panel, keys, etc. The GNSS receiver may be provided on the vehicle sensor 40. The navigation HMI may be partially or completely shared with the HMI 30 described above. The route determination unit determines, for example, a route (hereinafter referred to as a route on a map) from the position of the vehicle M determined by the GNSS receiver (or any input location) to a destination input by the occupant using the navigation HMI, by referring to, for example, map information 182, etc. Furthermore, the navigation device 50 provides route guidance using the navigation HMI based on the determined route on the map. The navigation device 50 may also transmit its current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0032] Here, map information 182 is information in which the shape of a road is represented by links indicating roads (an example of a travel route) and nodes connected by those links. Map information 182 may also include POI (Point of Interest) information, etc. Map information 182 also includes, for example, the number of lanes, the type and shape of road markings (hereinafter referred to as markings), information on the center of the lanes, or information on road boundaries. Map information 182 may also include information on whether the road boundary is a boundary (physical boundary) that includes structures that the vehicle cannot pass through (including crossing and contact). Physical boundaries include, for example, guardrails, curbs, median strips, fences, walls, etc. Map information 182 may also include road shape information, traffic regulation information, address information (address and postal code), information on the location and type of traffic signals (hereinafter referred to as traffic signals), location information of pedestrian crossings, facility information, parking information, telephone number information, etc. Road shape information includes, for example, information such as the curvature (or radius of curvature), width, road surface gradient, branching and merging points, and intersections (including T-junctions, etc.). Map information 182 may be updated as needed by the communication device 20 communicating with an external device.

[0033] The in-vehicle camera 70 is, for example, a digital camera using a solid-state image sensor such as a CCD or CMOS. The in-vehicle camera 70 is mounted at any location in the vehicle M in a position and orientation that allows it to capture the head of the driver seated in the driver's seat of the vehicle M from the front. For example, the in-vehicle camera 70 is mounted near (for example, above or below) the display device located in the center of the instrument panel of the vehicle M. The in-vehicle camera 70 may also capture images of the interior of the vehicle including the area of ​​the occupant (passenger) seated in the passenger seat of another vehicle. The in-vehicle camera 70 captures images of the interior of the vehicle repeatedly and periodically, for example.

[0034] The driver control unit 80 includes, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driver control unit 80 may also include a shift lever, a modified steering wheel, a joystick, or other controls. Each control of the driver control unit 80 is equipped with an operation detection unit that detects, for example, the amount of operation performed by the driver on the control unit or whether or not an operation has been performed. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel (for example, the amount of steering due to the driver's driving operation (steering input torque)), the rate of change of the steering torque, the amount of depression of the accelerator pedal and brake pedal, etc. The operation detection unit then outputs the detection results to the driver assistance device 100, or to one or both of the driving force output device 200, the brake device 210, and the steering device 220. The driver control unit 80 may also include a turn signal control unit (for example, a turn signal lever, a turn signal switch). When the turn signal control unit is operated, the turn lamp of the vehicle M corresponding to the operation is illuminated (flashes), and the operation details (including, for example, the detection result that the operation was performed by the driver) are output to the driver assistance device 100.

[0035] The driver assistance device 100 performs various controls to support the driver of the vehicle M and the ongoing driving control. The driver assistance device 100 includes, for example, an acquisition unit 120, a state detection unit 140, a driver assistance unit 160, and a storage unit 180. The acquisition unit 120, the state detection unit 140, and the driver assistance unit 160 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), or SOC (System On Chip), or by the cooperation of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the driver assistance device 100, or it may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the driver assistance device 100 when the storage medium (non-transient storage medium) is inserted into a drive device or card slot. The HMI control unit 166 is an example of a "notification control unit".

[0036] The storage unit 180 may be implemented using the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. The storage unit 180 may store, for example, map information 182, programs, and other various information. The storage unit 180 may also store various setting information used in the processing in this embodiment.

[0037] The acquisition unit 120 acquires various information from, for example, a detection device DD, a communication device 20, an HMI 30, a vehicle sensor 40, a navigation device 50, an in-vehicle camera 70, a driver control unit 80, a driving force output device 200, a brake device 210, a steering device 220, etc. The acquisition unit 120 also includes, for example, a surrounding information acquisition unit 122 and a traffic signal information acquisition unit 124.

[0038] The surrounding information acquisition unit 122 acquires information about the area around the vehicle M based on, for example, the detection results of the detection device DD (information input from the camera 10, radar device 12, and LIDAR 14). For example, the surrounding information acquisition unit 122 performs sensor fusion processing on some or all of the detection results from the camera 10, radar device 12, and LIDAR 14 to acquire the position (relative position), speed (relative speed), acceleration, shape, and other states of objects present in the area around the vehicle M (within a predetermined distance from the vehicle M). Objects acquired by the surrounding information acquisition unit 122 may include, for example, other vehicles, pedestrians, bicycles, and other traffic participants. Other vehicles include, for example, vehicles traveling in front of the vehicle M, vehicles traveling behind the vehicle M, adjacent vehicles, oncoming vehicles, stationary vehicles, and other surrounding vehicles present on the road.

[0039] The position of an object is recognized and used for control as a position on an absolute coordinate system with the origin being, for example, a representative point of the vehicle M (such as the center of gravity or the center of the drive axis). The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or by a represented region. The "state" of an object may include, for example, the acceleration or jerk of another vehicle, or the "action state" (for example, whether the other vehicle is changing lanes or is about to change lanes) if the object is another vehicle.

[0040] Furthermore, the surrounding information acquisition unit 122 may recognize, for example, intersections (including T-junctions, etc.), pedestrian crossings, stop lines, toll booths, other road features, road signs, and markings drawn on the road (for example, speed limits).

[0041] Furthermore, the surrounding information acquisition unit 122 recognizes the lane in which the vehicle M is traveling (driving lane) and other surrounding lanes (e.g., adjacent lanes and oncoming lanes) based on the detection results of the detection device DD or map information 182. For example, the surrounding information acquisition unit 122 recognizes lane markings from images captured by the camera 10 and recognizes the driving lane and adjacent lanes based on the positional relationship of the lane markings as seen from the recognized vehicle M. Alternatively, the surrounding information acquisition unit 122 refers to map information 182 based on the position information of the vehicle M obtained from the vehicle sensor 40, etc., to recognize the driving lane, adjacent lanes, lane markings, etc. of the vehicle M. Alternatively, the surrounding information acquisition unit 122 may recognize road boundaries instead of lane markings and recognize the driving lane and other lanes of the vehicle M based on the recognition results. The surrounding information acquisition unit 122 may also acquire the location and type of traffic lights, the location of intersections and pedestrian crossings, etc., that exist around the vehicle M from map information 182.

[0042] Furthermore, the surrounding information acquisition unit 122 may recognize the lateral position (position in the lane width direction) of the vehicle M relative to the driving lane and the attitude (orientation) of the vehicle M relative to the direction of extension of the driving lane, based on the positional relationship of the vehicle M with respect to the driving lane. For example, the surrounding information acquisition unit 122 may recognize the deviation of the vehicle M's reference point from the center of the lane, and the angle it makes with a line connecting the centers of the lanes in the direction of travel of the vehicle M, as the relative position and attitude of the vehicle M with respect to the driving lane. Alternatively, the surrounding information acquisition unit 122 may recognize the position of the vehicle M's reference point relative to any side edge (lane marking or road boundary) of the driving lane as the relative position (lateral position) of the vehicle M with respect to the driving lane. In addition, the surrounding information acquisition unit 122 may recognize the driving trajectory of other vehicles (e.g., preceding vehicles, following vehicles, adjacent vehicles, etc.) based on the detection results of the detection device DD, and may recognize a predicted future trajectory based on the driving trajectory.

[0043] The traffic light information acquisition unit 124 acquires the location information of traffic lights in the vicinity of the vehicle M, as well as the lighting status, including information on the color of the lights (illuminating parts) on the traffic lights, based on the location information of the vehicle M. For example, the traffic light information acquisition unit 124 performs known image analysis processing (for example, edge, shape, size, and color feature extraction, pattern matching, etc.) on the image captured by the camera 10 (hereinafter referred to as the camera image), and acquires the location of traffic lights in the direction of travel (forward) of the vehicle M based on the image analysis results. The traffic light information acquisition unit 124 may also acquire the location of traffic lights in the direction of travel of the vehicle M based on the output results of the radar device 12 or LIDAR 14. Furthermore, the traffic light information acquisition unit 124 may refer to map information 182 based on the location information of the vehicle M and acquire information on the location and type of traffic lights in the direction of travel of the vehicle M.

[0044] Furthermore, if a traffic light is present in the vicinity of the vehicle M, the traffic light information acquisition unit 124 acquires information on the color of the lamp (lighting part) on the traffic light that is lit (or flashing) (hereinafter sometimes referred to as "the color the traffic light is lit") based on the image analysis results (e.g., color features) of the camera image. Alternatively, the traffic light information acquisition unit 124 may communicate with traffic lights present in the vicinity of the vehicle M via the communication device 20 instead of (or in addition to) the camera image to acquire information on the location of the traffic light and the color the traffic light is lit. In other words, the traffic light information acquisition unit 124 can acquire information on the color the traffic light is lit based on the camera image captured by the camera 10 mounted on the vehicle M or information received by the communication device 20 that communicates with the traffic light.

[0045] The state detection unit 140 includes, for example, a driving operation detection unit 142 and a driver state detection unit 144. The driving operation detection unit 142 detects the driving operations of the vehicle M performed by the driver. For example, the driving operation detection unit 142 detects the amount of operation or presence or absence of operation of each control element (for example, the steering wheel, accelerator pedal, and brake pedal) performed by the driver using an operation detection unit provided on the driving control element 80. Alternatively, the driving operation detection unit 142 may detect the state of the driver's driving operations in correspondence with the magnitude or change (increase or decrease) of the output from the vehicle speed sensor and acceleration sensor included in the vehicle sensor 40. Alternatively, the driving operation detection unit 142 may detect the state of the driver's driving operations related to the steering of the vehicle M based on the magnitude or change in the output from the steering angle sensor and steering angular velocity sensor included in the vehicle sensor 40.

[0046] The driver state detection unit 144 performs known image analysis processing on images captured by, for example, the in-vehicle camera 70, and detects the state of the driver of the vehicle M based on the image analysis results. The driver state is, for example, a state based on the driver's posture or gaze. The driver state may also include, for example, whether or not the driver is monitoring the surroundings. Furthermore, the driver state may also include whether or not the driver is in a state where they can drive. For example, if it is determined from the image captured by the in-vehicle camera 70 that the driver is asleep, it is determined that the driver is not in a state where they can drive (is in an undriveable state).

[0047] The driver assistance unit 160 provides driving assistance for the vehicle M based on the acquisition results from the acquisition unit 120, the detection results from the state detection unit 140, and the driver's instructions from the HMI 30, etc. Driving assistance may include not only driving assistance based on the driver's manual operation of the vehicle M, but also driving assistance based on driving control (automatic driving) that controls at least one of the steering and speed of the vehicle M to drive the vehicle M without driver operation. Driving control may also include, for example, ACC (Adaptive Cruise Control System), LKAS (Lane Keeping Assistance System), ALC (Auto Lane Changing), etc. Driving control may also include control to stop the vehicle M in a safe position such as the shoulder of the road.

[0048] The driver assistance unit 160 includes, for example, a determination unit 162, a driving control unit 164, and an HMI control unit 166. The HMI control unit 166 is an example of a "notification control unit". The determination unit 162 performs various determinations based on the acquisition status of information such as the position and color of the traffic signals by the traffic signal information acquisition unit 124. The determination unit 162 also performs various determinations based on the driver's status detected by the status detection unit 140. Details of the functions of the determination unit 162 will be described later.

[0049] The driving control unit 164 controls the driving of the vehicle M based on the determination result from the determination unit 162. For example, the driving control unit 164 controls at least one of the steering and speed of the vehicle M based on the determination result. For example, when the vehicle M performs driving control, the driving control unit 164 generates a target trajectory that the vehicle M will travel in the future according to the surrounding conditions and the control content (e.g., the type of driving control to be performed), and controls the driving force output device 200, the brake device 210, and the steering device 220 so that the vehicle M passes through the generated target trajectory at the scheduled time. The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) that the vehicle M should reach. The trajectory points are points that the vehicle M should reach at predetermined travel distances (e.g., a few meters) along the road, and separately, target speed and target acceleration at predetermined sampling times (e.g., a few tenths of a second) are generated as part of the target trajectory. The trajectory points may be the positions that the vehicle M should reach at each predetermined sampling time interval. In this case, information on target velocity and target acceleration is represented by the intervals between trajectory points.

[0050] For example, when the driving control unit 164 performs lane keeping control such as LKAS, it generates a target trajectory so that the vehicle M travels in the center of the driving lane. Also, when the driving control unit 164 performs follow control such as ACC, which drives behind a preceding vehicle and maintains a predetermined distance from the preceding vehicle, it generates a target trajectory for the vehicle M to travel in the future along the driving trajectory of the preceding vehicle.

[0051] Furthermore, the driving control unit 164, while the driver is manually operating the vehicle, controls at least one of the steering and speed of the vehicle M based on the surrounding conditions and the driver's condition, for example, to prevent contact with obstacles or to enable the vehicle M to travel more safely, or to cause the HMI control unit 166 to notify the driver of warning information, etc. For example, the driving control unit 164 performs at least speed control according to the information acquired by the traffic signal information acquisition unit 124, such as the position and color of the illuminated traffic signals. Note that speed control may include not only acceleration control and deceleration control, but also control to suppress acceleration (not accelerating, or reducing the acceleration ratio in response to driving operations). Details of the functions of the driving control unit 164 will be described later.

[0052] The HMI control unit 166 notifies the occupants (including the driver) of predetermined information via the HMI 30 and receives information input via the HMI 30. The predetermined information includes, for example, information related to the driving of the vehicle M, such as information regarding the status of the vehicle M and information regarding driving control. Information regarding the status of the vehicle M includes, for example, the speed of the vehicle M, engine speed, and shift position. Information regarding driving control includes, for example, whether or not the driving support unit 160 is performing driving control (driving control), information regarding the status of the driving control, information regarding driving control recommendations (suggestions) from the system, and notification information to the driver (warnings, etc.). The predetermined information may also include information regarding the surrounding conditions recognized by the detection device DD. The predetermined information may also include information unrelated to the driving of the vehicle M, such as content (e.g., video) stored on a storage medium such as a television program or DVD. The predetermined information may also include, for example, the current location and destination of the vehicle M, and information regarding the remaining fuel level of the vehicle M. The HMI control unit 166 may output the information received by the HMI 30 to the communication device 20, navigation device 50, acquisition unit 120, status detection unit 140, driving support unit 160, etc.

[0053] Furthermore, the HMI control unit 166 may generate information such as acquisition results from the acquisition unit 120, detection results from the state detection unit 140, and determination results from the determination unit 162, and output the generated information to the HMI 30. The generated information may include images and sounds (including notification sounds, etc.). In addition, the HMI control unit 166 may transmit the various information to be output to the HMI 30 to terminal devices (for example, smartphones, tablet terminals, etc.) used by the occupants of the vehicle M via the communication device 20.

[0054] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to information input from the driving control unit 164 or information input from the accelerator pedal of the driver control unit 80.

[0055] The brake system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the driving control unit 164 or from the brake pedal of the driver control unit 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 210 may be equipped with a backup mechanism that transmits hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. The brake system 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the driving support unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0056] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the driving control unit 164 or from the steering wheel of the driver control unit 80.

[0057] [Driving Support Department] Next, the details of the driving support provided by the driving support unit 160 (mainly the determination unit 162 and the driving control unit 164) of the embodiment will be described. In the following example, the explanation will mainly focus on driving support that corresponds to the acquisition status of information regarding traffic signals. Furthermore, although the explanation will mainly focus on driving support related to speed control, driving support by steering control may be provided instead of (or in addition to) speed control.

[0058] Figure 2 shows an example of a scenario in which the driving assistance of the embodiment is performed. In the example in Figure 2, the lane L1 in which the vehicle M travels (driving lane), the lane L2 adjacent to lane L1 (opposing lane in Figure 2), and lanes L3 and L4 that intersect (cross at a level intersection) with lanes L1 and L2 (connected at intersection CR1) are shown. Lane L1 is demarcated by lane markings LL1 and CL1, and lane L2 is demarcated by lane markings CL1 and RL1. Lane L3 is demarcated by lane markings LL2 and CL2, and lane L4 is demarcated by lane markings CL2 and RL2.

[0059] In Figure 2, vehicle M is traveling at speed VM on lane L1 towards intersection CR1. Also in Figure 2, a stop line SL1 is located on the road surface of lane L1, prior to intersection CR1 from the perspective of vehicle M. Further from intersection CR1 from the perspective of vehicle M, there is a traffic light TL1 that displays information (signal) indicating whether vehicle M traveling on lane L1 is permitted to pass through intersection CR1. In the example in Figure 2, the position and speed of vehicle M at time T* are represented by M(T*) and VM(T*), respectively. That is, in Figure 2, the position and speed of vehicle M at time T1 are represented by M(T1) and VM(T1), the position and speed of vehicle M at time T2 are represented by M(T2) and VM(T2), and the position and speed of vehicle M at time T3 are represented by M(T3) and VM(T3). Furthermore, in the following explanation, time T1 is assumed to be the earliest, followed by T2 and then T3, in descending order of time.

[0060] For example, the vehicle M acquires information about its surroundings (for example, information about the direction of travel (forward) of the vehicle M) using the acquisition unit 120, and the driving support unit 160 performs driving support for the vehicle M based on the acquired information.

[0061] In the example in Figure 2, time T1 (when the vehicle M reaches point P1) is the time when the vehicle M acquires the information that intersection CR1 exists in the direction of travel of the vehicle M. Furthermore, the vehicle M travels toward intersection CR1, and at time T2 (when the vehicle M reaches point P2), it acquires the position of traffic light TL1 located near intersection CR1 for the vehicle M traveling in lane L1. Traffic light TL1 is a traffic light that outputs (lights up) information (signal) indicating whether or not the vehicle traveling in lane L1 can pass through intersection CR1. If the distance D1 from the current position of the vehicle M to the position of traffic light TL is less than a predetermined distance, the traffic light information acquisition unit 124 acquires information on the color that traffic light TL1 is lit in. The predetermined distance may be, for example, a fixed distance, or a variable distance set according to the speed VM of the vehicle M, the road shape, etc. In this embodiment, the timing of acquiring the position of traffic light TL1 and the timing of acquiring the color of traffic light TL1 may be simultaneous, or the position may be acquired after acquiring the color of traffic light TL1.

[0062] The driver assistance unit 160 then performs driver assistance for the vehicle M based on the acquisition status of information about the vehicle M's surroundings. Driver assistance includes driver assistance for the aforementioned LKAS and ACC, and driver assistance for the driver's operations of the vehicle M.

[0063] Here, when the signal information acquisition unit 124 acquires information on the color of the signal TL1 that is lit, the lit color may become undetermined (undetermined color state) due to several influences on the vehicle M. An undetermined color state includes cases where the light cannot be detected, where the light is lit but the type of color cannot be detected, and where the type of color is detected but the detected color is different from a predetermined (expected) color.

[0064] For example, if vehicle M is traveling at a distance from intersection CR1 (or traffic light TL1) (for example, the section from point P2 to point P3 as shown in Figure 2 (the section from time T2 to T3)), the distance from traffic light TL1 is far (more than a predetermined distance), and therefore the number of pixels of traffic light TL1 (or lamp) included in the camera image is small, which may result in an undefined color state (the above pattern will be referred to as "undefined color pattern 1").

[0065] Furthermore, the undefined color state may also occur when the vehicle M is closer to intersection CR1 (or traffic light TL1) than when it is in the undefined color pattern 1 state (for example, the section from point P3 shown in Figure 2 to intersection CR1 or stop line SL1 (the section after time T3)). For example, this can occur when the distance from traffic light TL1 becomes shorter (less than the predetermined distance), but the vehicle is affected by disturbances such as sunlight or its reflected light shining on the lamp or camera 10 of traffic light TL1, trees installed along the roadside obscuring the lamp of traffic light TL1, or large vehicles (trucks or buses) traveling ahead obscuring the lamp of traffic light TL1 from the vehicle M (the above pattern is referred to as "undefined color pattern 2").

[0066] Therefore, in this embodiment, when the color becomes indeterminate due to the color indeterminate patterns 1 and 2 described above, various controls are executed according to the status of information acquisition regarding the signal TL1 to prevent incorrect driving assistance.

[0067] For example, the determination unit 162 determines, based on the results obtained by the signal information acquisition unit 124, whether or not a signal TL1 exists in the lane path (the direction of travel of the vehicle M (forward)) in which the vehicle M is traveling. If it determines that a signal TL1 exists in the direction of travel of the vehicle M, the determination unit 162 determines whether or not the distance D1 between the vehicle M and the signal TL1 is within a predetermined distance.

[0068] If the determination unit 162 determines that the distance D1 from the vehicle M to the traffic light TL1 is less than a predetermined distance, it determines whether or not information about the color of the illuminated traffic light TL1 has been acquired based on the acquisition results from the traffic light information acquisition unit 124. If it is determined that the color information has not been acquired, the driving control unit 164 suppresses the acceleration of the vehicle M at least.

[0069] In this case, the driving control unit 164 suppresses the acceleration of its own vehicle M by, for example, not accepting acceleration operations from the driver via the driver control element 80 (more specifically, the accelerator pedal), or by not performing acceleration control in response to accepted acceleration operations. Alternatively, the driving control unit 164 may suppress the acceleration by making the acceleration ratio corresponding to the accepted acceleration operation (operation amount) smaller than under normal conditions. Furthermore, if the driving control unit 164 is executing driving control such as ACC and the pre-generated target trajectory includes an acceleration speed element, it may regenerate a target trajectory without acceleration to control the speed of its own vehicle M. In addition, for speed operations other than acceleration operations (deceleration operations and constant speed operations), the driving control unit 164 controls the speed of its own vehicle M according to the driver's operation instructions or the target trajectory.

[0070] Thus, if the color information of the illuminated traffic light TL1 cannot be obtained, the possibility of contact with an object near the traffic light TL1 (for example, at intersection CR1) can be reduced by suppressing the acceleration of the vehicle M. Furthermore, by suppressing acceleration instead of immediately performing deceleration control when color information cannot be obtained, unnecessary braking (deceleration) when the color is uncertain can be avoided, thereby reducing user discomfort and stabilizing the behavior of the vehicle M.

[0071] Furthermore, if the color of the illuminated traffic light TL1 has not been acquired (state), the period may include, for example, the time between the acquisition of information on the color of the illuminated traffic light TL1 based on the acquisition results of the traffic light information acquisition unit 124 and the acquisition of information on the color of the illuminated traffic light TL1. In other words, the driving control unit 164 suppresses the acceleration of the vehicle M until it acquires information on the illuminated color from the image analysis processing results of the camera image, or until it acquires information on the illuminated color from the information received from the traffic light TL1 via the communication device 20. Note that the phrase "after acquiring the presence of traffic light TL1" above may be rephrased as "after acquiring the position information of traffic light TL1". This makes it possible to suppress the execution of unnecessary control such as deceleration in situations where the vehicle M is traveling at a distance from intersection CR1 (traffic light TL1) and color information has not been acquired, such as in the color-undetermined pattern 1, thereby enabling more appropriate driving assistance (speed control).

[0072] Furthermore, if the determination unit 162 determines that it has acquired information on the color of the illuminated traffic light TL1, it determines the type of color acquired. For example, the determination unit 162 determines whether the illuminated color of the traffic light TL1 is blue or not. Here, blue is an example of a color that indicates permission for the vehicle M traveling in lane L1 to pass through intersection CR1, but it may also be another predetermined color such as green. If it is determined that the illuminated color is blue, the driving control unit 164 performs speed control (and / or steering control) such as acceleration or deceleration according to the driver's driving operation or a pre-generated target trajectory.

[0073] Furthermore, if, after it is determined that the color of the illuminated traffic light TL1 is blue, information about the color of the illuminated traffic light TL becomes unavailable due to, for example, the situation described in the color-indeterminate pattern 2 above, the driving control unit 164 may suppress the acceleration of the vehicle M. For example, even if it is determined that the color of the illuminated traffic light TL1 is blue at time T2 shown in Figure 2, if the color of the illuminated traffic light TL1 becomes indeterminate due to disturbances or other factors when approaching intersection CR1 at time T3 or later, there is a possibility that the color has changed from blue to another color (the signal has changed) during this indeterminate period. Therefore, in this embodiment, by suppressing the acceleration of the vehicle M in this situation, deceleration can be performed smoothly even if the color of the illuminated traffic light TL1 has changed, thereby stabilizing the behavior of the vehicle M. In addition, by performing only acceleration suppression control without immediately executing deceleration control based on the above information, unnecessary control during color indetermination due to disturbances can be suppressed. This enables more appropriate driving assistance.

[0074] Furthermore, if the determination unit 162 determines that the color of the illuminated traffic light TL1 is not blue, it determines whether the illuminated color is red or not. Here, red is just one example of a color that indicates that the vehicle M traveling in lane L1 is denied (not permitted to pass) through intersection CR1 (or should stop before the stop line SL1), and other predetermined colors may also be used. Alternatively, the determination unit 162 may determine whether the light is yellow or not instead of red. Yellow, like red, basically indicates that the vehicle is denied passage through intersection CR1.

[0075] If the illuminated color is determined to be red (or yellow), the determination unit 162 further determines whether the vehicle M is decelerating (or whether the vehicle M's speed VM is below a predetermined speed) due to the driver's operation or a pre-generated target trajectory. If it is determined that the vehicle M is not decelerating (or the vehicle M's speed VM is not below a predetermined speed), the driving control unit 164 executes deceleration control so that the vehicle M stops before the stop line SL1. Alternatively, the driving control unit 164 may, instead of (or in addition to) the above deceleration control, cause the HMI control unit 166 to output a warning or information prompting deceleration. The HMI control unit 166 generates images and sounds indicating warnings or information prompting deceleration, and outputs the generated images and sounds to the HMI 30 to inform the driver of the situation. If it is determined that the vehicle M is decelerating (or the vehicle M's speed VM is below a predetermined speed), the driving control unit 164 continues the current control.

[0076] Furthermore, if it is determined that the color of the illuminated traffic light TL1 is red (or yellow), but then it becomes impossible to obtain information about the color of the illuminated traffic light TL1, the driving control unit 164 may perform control to decelerate the vehicle M. For example, even if it is determined that the color of the illuminated traffic light TL1 is red (or yellow) at time T2 as shown in Figure 2, if the color of the illuminated traffic light TL1 becomes uncertain due to disturbances or other factors when approaching intersection CR1 at time T3 or later, there is a possibility that it has changed from red (or yellow) to another color during this uncertain period, but there is a high possibility that it will remain red, so by decelerating the vehicle M, safer driving assistance can be provided.

[0077] Furthermore, if the driving control unit 164 determines that the color of the illuminated traffic light TL1 is red (or yellow), and then becomes unable to obtain information on the color of the illuminated traffic light TL1, and the driving operation detection unit 142 has not detected any deceleration operation by the driver, or the degree of deceleration operation being performed is below a threshold, then the driving control unit 164 may perform at least one of the following: deceleration of the vehicle M, or notification to the driver by the HMI control unit 166. A degree of deceleration steering below a threshold includes, for example, the driver operating the brake pedal, but the speed VM of the vehicle M is below a predetermined speed, or the amount the brake pedal is pressed is below a predetermined amount. Notification to the driver by the HMI control unit 166 includes, for example, providing the driver with information prompting further deceleration, or notifying the driver that the traffic light TL1 ahead is red (the color of the illuminated traffic light TL1 is red).

[0078] Furthermore, if it is determined that the color of the illuminated traffic light TL1 is not blue and is not red (or yellow), the driving control unit 164 may perform control to suppress acceleration at least. By suppressing acceleration at least when it is unclear which color is illuminated, smooth deceleration control can be performed even if it is red or yellow, thereby further stabilizing the behavior of the vehicle M.

[0079] Furthermore, the HMI control unit 166 may vary the level of notification depending on the distance from the vehicle M to the traffic light TL1 (intersection CR1 or stop line SL1) and / or the speed VM of the vehicle M. For example, if the distance from the vehicle M to the traffic light TL1 is greater than or equal to a threshold, the HMI control unit 166 displays an image on the HMI 30 display unit indicating information to encourage deceleration, and if it is less than the threshold, it outputs a warning sound from the speaker along with the above image. In addition, deceleration control may be performed by increasing the deceleration force in response to the driver's deceleration operation (in other words, by increasing the deceleration even with a small amount of brake pedal depression) instead of (or in addition to) decelerating until the speed VM of the vehicle M reaches a predetermined speed. This allows for more appropriate driving assistance in response to the driver's driving operations (especially deceleration operations).

[0080] [Differentiation] In this embodiment, the driving control unit 164 may refrain from suppressing the acceleration of the vehicle M if the speed VM of the vehicle M is below a predetermined speed, even if the distance D1 from the vehicle M to the traffic light TL1 is less than a predetermined distance and information on the color of the illuminated traffic light TL1 cannot be obtained. For example, even if the color of the illuminated traffic light TL1 cannot be obtained from the camera image, the driver may be able to see the color of the traffic light TL1 and accelerate accordingly, or accelerate in accordance with the behavior of surrounding vehicles. Therefore, by not suppressing the acceleration of the vehicle M when the speed VM of the vehicle M is below a predetermined speed, unnecessary acceleration suppression can be suppressed, reducing user discomfort and enabling more appropriate driving assistance.

[0081] Furthermore, in this embodiment, the driving control unit 164 may not suppress the acceleration of the vehicle M when the surrounding information acquisition unit 122 acquires information about a vehicle following the vehicle M. Figure 3 is a diagram showing an example of a screen where a vehicle following the vehicle M is present. In the example in Figure 3, under the same road conditions as in Figure 2, a following vehicle m1 is traveling in lane L1 at a speed Vm1 behind the vehicle M.

[0082] In the example shown in Figure 3, the driving control unit 164 does not suppress the acceleration of the vehicle M if the distance D1 from the vehicle M to the traffic light TL1 is less than a predetermined distance (first predetermined distance), and the traffic light information acquisition unit 124 has not been able to acquire information on the color of the illuminated traffic light, but the surrounding information acquisition unit 122 indicates that a following vehicle m1 is present behind the vehicle M. The driving control unit 164 may also choose not to suppress acceleration if the distance (relative distance) D2 between the vehicle M and the following vehicle m1 is less than a predetermined distance (second predetermined distance), or if the time to collision (TTC) between the vehicle M and the following vehicle m1 is less than a threshold. The time to collision (TTC) can be derived, for example, by dividing the relative distance (distance D2) between the vehicle M and the following vehicle m1 by the relative speed (absolute value of "speed VM - speed Vm1"). This prevents contact with the following vehicle m1 by suppressing unnecessary acceleration.

[0083] Furthermore, in this embodiment, if the driving control unit 164 detects that the driver of the vehicle M, as detected by the driver state detection unit 144, is monitoring the road ahead, it may choose not to suppress the acceleration of the vehicle M even if the distance D1 from the vehicle M to the traffic light TL1 is less than a predetermined distance and information on the color of the illuminated traffic light TL1 has not been obtained. When the driver is monitoring the road ahead, there is a high probability that they are aware of the surrounding conditions (especially ahead). Therefore, by controlling the vehicle M not to suppress its acceleration in such situations, control can be performed in accordance with the driver's intentions, thereby reducing user discomfort and providing more appropriate driving assistance.

[0084] Furthermore, in this embodiment, even if the distance D1 from the vehicle M to the traffic light TL1 is less than a predetermined distance and the traffic light information acquisition unit 124 has not been able to acquire information on the color of the illuminated traffic light, the driving control unit 164 may control the vehicle M not to suppress its acceleration based on the driving conditions of adjacent vehicles acquired by the surrounding information acquisition unit 122.

[0085] Figure 4 shows an example of a scenario where a vehicle M is traveling on a road with multiple lanes that can travel in the same direction. In the example in Figure 4, lanes L5 and L6 intersect (cross at a level intersection) with lanes L3 and L4 (connected at intersection CR2). Lanes L5 and L6 are lanes that can travel in the same direction (X-axis direction in the figure). Lane L5 is demarcated by lane markings LL3 and CL3, and lane L6 is demarcated by lane markings CL3 and RL3.

[0086] In the example in Figure 4, vehicle M is traveling at speed VM in lane L6 towards intersection CR2. Therefore, lane L6 is the lane in which vehicle M is traveling, and lane L5 is the adjacent lane to lane L6. Assume that another vehicle m2 is traveling at speed Vm2 towards intersection CR2 in lane L5. Another vehicle m2 is traveling in the lane adjacent to vehicle M's lane and is an adjacent vehicle located within a predetermined distance from vehicle M. In the example in Figure 4, a traffic light TL2 is installed further away from intersection CR2 as seen from vehicle M, which displays information (signals) indicating whether or not it is permissible for vehicles traveling in lanes L5 and L6 to pass through intersection CR2. Note that traffic light TL2 is installed on the lane L5 side (adjacent lane side). Also, in lane L5, there is a stop line SL2 before intersection CR2 as seen from vehicle M, and in lane L6, there is a stop line SL3 before intersection CR2 as seen from vehicle M.

[0087] For example, in the scenario shown in Figure 4, the driving control unit 164 will not suppress the acceleration of the vehicle M if, even if the distance D1 from the vehicle M to the traffic light TL1 is less than a predetermined distance, and the traffic light information acquisition unit 124 has not been able to acquire information on the color of the illuminated traffic light TL2, the surrounding information acquisition unit 122 has acquired that there is an adjacent lane L5 that the vehicle M is traveling in the same direction as the lane it is traveling in, and that there is another vehicle (adjacent vehicle) m2 passing through intersection CR2 where the traffic light TL2 is installed in that adjacent lane. For example, the driving control unit 164 will determine that the other vehicle m2 is passing through intersection CR2 if the position of the other vehicle m2 is closer to intersection CR2 than the stop line SL2 of the adjacent lane L5, has passed through intersection CR2, or is traveling at a predetermined speed or higher and less than a predetermined distance from intersection CR2.

[0088] For example, even if the color of the illuminated traffic light TL2 is uncertain due to disturbances or other factors from the position of the vehicle M in lane L6, the angle to the traffic light TL2 (the direction of the traffic light as seen from the vehicle) from the position of another vehicle m2 traveling in the adjacent lane L5 is different from that of vehicle M. Therefore, there is a high probability that the color of the illuminated traffic light TL2 is visible from the position of the other vehicle m2 without being affected by disturbances or other factors (or that the driver of the other vehicle m2 can see the color of the illuminated traffic light TL2). For this reason, the driving control unit 164 can provide more appropriate driving assistance according to the surrounding conditions by not suppressing the acceleration of the vehicle M when the other vehicle m2 traveling in the adjacent lane L5 is about to travel through intersection CR2 at a predetermined speed or higher.

[0089] Furthermore, in this embodiment, it may be determined whether or not a traffic light is installed in a predetermined road condition, such as when there is a road shape like an intersection or when there is a pedestrian crossing, and if a traffic light is present in the predetermined road condition, various control functions related to the driving assistance described above may be executed. Also, in this embodiment, the content of the speed control of the vehicle M may be changed according to the road shape near the location of the traffic light. For example, the driving control unit 164 may perform control such as differentiating the degree of acceleration suppression and deceleration depending on whether the road shape near the traffic light is an intersection or when there is a pedestrian crossing on the road near the traffic light.

[0090] Furthermore, in this embodiment, the execution of the acceleration suppression and deceleration control described above may be controlled according to the time of day. For example, the situation in which the color of a traffic light illuminated becomes inconsistent due to the influence of sunlight, etc., is unlikely to occur at night, and the possibility of it becoming invisible is low. Therefore, based on the installation and orientation of the traffic light, a time period in which the illuminated color is expected to become inconsistent may be set, and the above control may be performed when passing near that traffic light during the set time period.

[0091] Furthermore, if the color of the illuminated traffic light is not one of the preset colors (blue, yellow, or red), the traffic light information acquisition unit 124 may infer the illuminated color based on which of the multiple lamps (illuminating parts) arranged on the traffic light is lit. Since the arrangement of the illuminated colors of the traffic light lamps is determined by preset criteria, the illuminated color can be inferred by acquiring the position of the lit lamp among the multiple lamps provided on the traffic light, and more appropriate driving assistance can be provided according to the inferred color.

[0092] Furthermore, when a traffic light lamp is flashing, it may have a different meaning than when the lamp is lit. For example, a flashing yellow light means to proceed while paying attention to other traffic, and a flashing red light means that you can stop at the stop line, check for safety, and then proceed. Therefore, the determination unit 162 may, for example, determine whether the color is flashing when the lit color is yellow or red. Also, the driving control unit 164 may perform different control when the light is flashing compared to when it is lit (for example, control that changes the degree of acceleration suppression or deceleration).

[0093] [Processing flow] Next, the processes performed by the driver assistance device 100 of the embodiment will be described. Figure 5 is a flowchart showing an example of the processes performed by the driver assistance device 100. In the following description, the processes performed by the driver assistance device 100 will mainly focus on the processes that control the operation of the vehicle M based on signal information. Furthermore, the processes shown below may be executed repeatedly at predetermined cycles or timings.

[0094] In the example in Figure 5, the surrounding information acquisition unit 122 acquires information about the surroundings of the vehicle M (step S100). Next, the traffic light information acquisition unit 124 acquires information about traffic lights located around the vehicle M (step S110). Next, the determination unit 162 determines whether or not it has acquired information about traffic lights located in the direction of travel of the vehicle M (the path of the lane the vehicle is traveling in) (step S120). If it determines that it has detected traffic light information, the determination unit 162 determines whether or not the distance from the vehicle M to the traffic light is less than a predetermined distance (step S130). If it determines that the distance is less than a predetermined distance, the determination unit 162 determines whether or not it has acquired the color of the illuminated traffic light (step S140).

[0095] If the determination unit 162 determines that it has been able to acquire the color of the illuminated light, it determines whether the color of the illuminated light is yellow or red (step S150). If it determines that the color of the illuminated light is not yellow or red, the determination unit 162 determines whether the color of the illuminated light is blue (step S160). If it determines that the color is not blue, the driving control unit 164 performs control to suppress the acceleration of the vehicle M because the color of the illuminated light has been acquired as a color other than a predetermined color (step S170). In step S170, instead of (or in addition to) the control to suppress the acceleration of the vehicle M, control to decelerate the vehicle M may be performed, and in that case, notification control may be performed to inform the driver of predetermined information via the HMI control unit 166. If it is determined that the color is blue, the processing of this flowchart ends.

[0096] Furthermore, in step S150, if it is determined that the color of the illuminated traffic light is yellow or red, the determination unit 162 determines whether or not the vehicle M is decelerating (step S180). If it is determined that the vehicle is not decelerating, the driving control unit 164 executes at least one of deceleration and notification control by the HMI control unit 166 (step S190), and the process of this flowchart ends. Also, if it is determined in step S190 that the vehicle is decelerating, the process of this flowchart ends.

[0097] Furthermore, if it is determined in step S140 that the illuminated color cannot be obtained, the driving control unit 164 suppresses the acceleration of its own vehicle M (step S200) and terminates the processing of this flowchart.

[0098] Furthermore, if it is determined in step S120 that information on traffic signals in the direction of travel has not been acquired, or if it is determined in step S130 that the distance to the traffic signal is not less than a predetermined distance, the process of this flowchart terminates.

[0099] According to the embodiments described above, the driver assistance device 100 includes a surrounding information acquisition unit 122 that acquires information about the surroundings of the vehicle M, a traffic light information acquisition unit 124 that acquires the location information and lighting status of traffic lights in the surroundings of the vehicle M based on the location information of the vehicle M, and a driver assistance unit 160 that provides driving assistance for the vehicle M based on the acquisition results of the surrounding information acquisition unit 122 and the traffic light information acquisition unit 124. The driver assistance unit 160 can provide more appropriate driving assistance according to the status of traffic light information acquisition by suppressing the acceleration of the vehicle M when the distance from the vehicle M to the traffic light is less than a predetermined distance in the lane in which the vehicle M is traveling, and when the traffic light information acquisition unit 124 has not been able to acquire information on the color of the illuminated traffic light.

[0100] For example, according to the embodiment, unnecessary deceleration control can be suppressed in situations where the recognition accuracy of the camera decreases due to the distance between the vehicle M and the traffic light, or due to the influence of external disturbances, or when the color of the illuminated traffic light becomes uncertain due to communication failures in the communication device 20. As a result, the instability of vehicle behavior due to sudden deceleration can be reduced, and more appropriate driving assistance can be provided depending on the situation. Furthermore, according to the embodiment, unnecessary braking can be suppressed when the color of the traffic light becomes uncertain, and inappropriate acceleration operations by the driver can also be suppressed.

[0101] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: Acquire information about the surroundings of your vehicle, Based on the vehicle's location information, the location information and lighting status of traffic lights in the vicinity of the vehicle are acquired. Based on the acquired information, the system provides driving assistance for the vehicle. In the lane in which the vehicle is traveling, if the distance from the vehicle to the traffic light is less than a predetermined distance and information on the color of the illuminated traffic light cannot be obtained, the acceleration of the vehicle is suppressed. Driving assistance system.

[0102] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0103] 1...Vehicle system, 10...Camera, 12...Radar device, 14...LIDAR, 20...Communication device, 30...HMI, 40...Vehicle sensor, 50...Navigation device, 70...In-cabin camera, 80...Driver control unit, 100...Driver support device, 120...Acquisition unit, 122...Surrounding information acquisition unit, 124...Traffic light information acquisition unit, 140...Status detection unit, 142...Drive operation detection unit, 144...Driver status detection unit, 160...Driver support unit, 162...Determination unit, 164...Driving control unit, 166...HMI control unit, 180...Storage unit, 200...Driving force output device, 210...Brake device, 220...Steering device, M...Own vehicle

Claims

1. A surrounding information acquisition unit that acquires information about the vehicle's surroundings, A signal information acquisition unit that acquires the location information and lighting status of signal lights in the vicinity of the vehicle based on the vehicle's location information, The system includes a driving support unit that provides driving support for the vehicle based on the results obtained from the surrounding information acquisition unit and the signal information acquisition unit, The aforementioned driving support unit suppresses the acceleration of the vehicle when, in the lane in which the vehicle is traveling, the distance from the vehicle to the traffic light is less than a predetermined distance, and the traffic light information acquisition unit has not been able to acquire information on the color of the illuminated traffic light. Driving assistance system.

2. The vehicle further includes a driving operation detection unit that detects driving operations performed by the driver of the vehicle, The driving support unit suppresses the acceleration of the vehicle caused by the driver's acceleration operation detected by the driving operation detection unit when the distance from the lane to the traffic light in the lane in which the vehicle is traveling is less than a predetermined distance, and the traffic light information acquisition unit has not been able to acquire information on the color of the illuminated traffic light. The driving support device according to claim 1.

3. The case in which the color of the illuminated traffic light cannot be obtained includes the period from when the existence of the traffic light is confirmed based on the results obtained by the traffic light information acquisition unit until the information on the color of the illuminated traffic light is obtained. The driving support device according to claim 2.

4. The aforementioned driving support unit suppresses the acceleration of the vehicle when, after the traffic light information acquisition unit has determined that the color of the illuminated traffic light is blue, it becomes impossible to acquire information about the color of the illuminated traffic light. The driving support device according to claim 2.

5. The aforementioned driving support unit, after the signal information acquisition unit has determined that the color of the signal is red or yellow, will decelerate the vehicle if it becomes impossible to acquire information about the color of the signal. The driving support device according to claim 2.

6. The aforementioned driving support unit, when the traffic signal information acquisition unit determines that the color of the illuminated traffic signal is yellow or red, and the driving operation detection unit determines that the driver has not performed a deceleration operation or that the degree of deceleration operation is below a threshold, shall perform at least one of the following: notify the driver and decelerate the vehicle. The driving support device according to claim 5.

7. The aforementioned driving support unit does not suppress the acceleration of the vehicle when the vehicle's speed is below a predetermined speed. The driving support device according to claim 1.

8. The aforementioned driving support unit, when the surrounding information acquisition unit acquires information about a following vehicle in the vicinity of the vehicle, does not suppress the acceleration of the vehicle. The driving support device according to claim 1.

9. The vehicle further includes a driver status detection unit that detects the status of the driver of the vehicle, The aforementioned driving support unit, when the driver state detection unit detects that the driver is monitoring the road ahead, does not suppress the acceleration of the vehicle. The driving support device according to claim 1.

10. The aforementioned driving support unit, when the surrounding information acquisition unit determines that there is an adjacent lane adjacent to the lane in which the vehicle is traveling and that can proceed in the same direction as the lane, and that there is another vehicle passing through an intersection where the traffic lights are installed in the adjacent lane, shall not suppress the acceleration of the vehicle. The driving support device according to claim 1.

11. The signal information acquisition unit acquires information regarding the color of the illuminated signal based on images captured by a camera mounted on the vehicle or information received by a communication device that communicates with the signal. The driving support device according to claim 1.

12. Computers Acquire information about the surroundings of your vehicle, Based on the vehicle's location information, the location information and lighting status of traffic lights in the vicinity of the vehicle are acquired. Based on the acquired information, the system provides driving assistance for the vehicle. In the lane in which the vehicle is traveling, if the distance from the vehicle to the traffic light is less than a predetermined distance and information on the color of the illuminated traffic light cannot be obtained, the acceleration of the vehicle is suppressed. Driving assistance methods.

13. On the computer, It acquires information about the surroundings of its own vehicle. Based on the vehicle's location information, the system acquires the location information and lighting status of traffic lights in the vicinity of the vehicle. Based on the acquired information, the system provides driving assistance for the vehicle. In the lane in which the vehicle is traveling, if the distance from the vehicle to the traffic light is less than a predetermined distance, and information on the color of the illuminated traffic light cannot be obtained, the acceleration of the vehicle is suppressed. program.

Images