Driving assistance system, program, driving assistance device

Abstract

Upon receipt of information indicating that an object having a time to collision (TTC hereinafter) less than a predetermined determination start value has arisen, a driving support ECU starts monitoring whether a driver has taken preliminary action, on the basis of an output signal from a motion sensor. If the driving support ECU is able to detect a specific preliminarily action of the driver during a determination period from when the TTC drops below the determination start value to when the TTC reaches a determination end value, the driving support ECU delays an output timing of a warning sound to be later than a standard timing. However, if the preliminarily action of the driver cannot be detected during the determination period, the driving support ECU sets the output timing of the warning sound to the standard timing or earlier than the standard timing.

Description

Driving assistance system, driving assistance method, program, and driving assistance device CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on Patent Application No. 2023-076990 filed in Japan on May 9, 2023, and the contents of the original application are incorporated by reference in their entirety.

[0002] The present disclosure relates to a technology for assisting a driver's driving operation.

[0003] Patent Document 1 discloses a technology for adjusting the timing of outputting the warning sound depending on the degree to which the driver is looking away in a driving assistance device that notifies the driver by sounding an alarm of the possibility of a collision between the vehicle and another object, such as a preceding vehicle. In the configuration disclosed in Patent Document 1, the degree to which the driver is looking away is evaluated based on the direction of the driver's line of sight detected using a camera.

[0004] Patent No. 4534789

[0005] Various configurations have been considered for controlling the output of an alarm sound using the driver's line of sight. In one assumed example, when the driver's line of sight is directed at an object, the volume of the alarm may be lowered or the timing of outputting the alarm sound may be delayed. This assumed example is expected to have the effect of reducing the risk of annoyance to the driver. The object here may be understood as an object that may come into contact with the vehicle.

[0006] However, as the developers of the present disclosure continued to study the above-mentioned assumed example, they noticed that there are cases where the driver is looking at an object but is unable to recognize the object (so-called cognitive inattention). If the driver is looking at an object but is unable to recognize the object, the timing of outputting the warning sound in the above-mentioned assumed example may be inappropriate.

[0007] The present disclosure has been made based on the above considerations or viewpoints, and one of its objectives is to provide a technology that enables assist control to be performed more appropriately.

[0008] One of the driving assistance systems disclosed herein includes an external sensor that detects the vehicle's surrounding environment, a motion sensor that detects the driver's movements, and a control unit that executes assistance control to prevent the vehicle from colliding with other objects. The control unit is configured to determine, based on the output signal of the external sensor, whether the current situation is a dangerous situation in which there is a possibility of collision with other objects, execute assistance control based on the determination that the situation is dangerous, determine, based on the output signal of the motion sensor, whether the driver has taken preparatory action to avoid a collision, and change the timing of execution of the assistance control depending on whether preparatory action has been taken within a predetermined determination period.

[0009] If the driver is aware of the danger, it is expected that they will take some kind of action to avoid it. Paradoxically, if such preparatory action is not observed, it is possible that the driver is not aware of the object even if they are looking at it.

[0010] According to the driving assistance system of the present disclosure, the execution timing of the assistance control is determined based on whether or not the driver has taken preparatory action, so that the assistance control can be executed appropriately even when the driver is looking at an object but is not aware of the object.

[0011] A driving assistance method included in the present disclosure is a driving assistance method for avoiding a collision of a vehicle with another object, and includes determining whether or not the current situation is a dangerous situation in which there is a possibility of collision with another object, based on an output signal from an external sensor that detects the vehicle's surrounding environment, implementing assistance control to avoid the vehicle from colliding with the other object based on the determination that the situation is dangerous, determining whether or not the driver has taken preparatory action to avoid the collision, based on an output signal from a motion sensor that detects the driver's movement, and changing the timing of executing the assistance control depending on whether or not the preparatory action has been taken within a predetermined determination period.

[0012] The program included in the present disclosure includes instructions to cause a computer to determine whether the current situation is a dangerous situation in which there is a possibility of collision with another object, based on an input signal from an external sensor that detects the vehicle's surrounding environment; to implement assistance control to prevent the vehicle from colliding with another object based on the determination that the situation is dangerous; to determine whether the driver has taken preparatory action to avoid a collision, based on an input signal from a motion sensor that detects the driver's movement; and to change the timing of execution of the assistance control depending on whether preparatory action has been taken within a predetermined determination period.

[0013] A first driving assistance device included in the present disclosure includes a communication unit for communicating with other devices, and a control unit that executes assistance control to prevent the vehicle from colliding with other objects based on data received by the communication unit, and the control unit is configured to acquire, via the communication unit, detection results from an external sensor that detects the vehicle's surrounding environment, determine whether or not a dangerous situation exists in which there is a possibility of collision with other objects based on the output signal of the external sensor, and execute assistance control based on the determination that a dangerous situation exists, acquire, via the communication unit, detection results from a motion sensor that detects the driver's movement, determine whether or not the driver has taken preparatory action to avoid a collision based on the detection result of the motion sensor, and change the timing of execution of the assistance control depending on whether or not preparatory action has been taken within a predetermined determination period.

[0014] The above-described driving assistance method, program, and driving assistance device are methods and programs corresponding to a driving assistance system, and have the same functions and effects as the driving assistance system.

[0015] A second driving assistance device included in the present disclosure includes a communication unit for communicating with other devices, and a control unit that executes behavior control, which is steering control or braking control, to avoid the vehicle colliding with another object based on data received by the communication unit. The control unit is configured to acquire, via the communication unit, detection results from an external sensor that detects the vehicle's surrounding environment, determine, based on the output signal of the external sensor, whether or not the vehicle is in a dangerous situation in which there is a possibility of collision with another object, and start behavior control based on the determination that the situation is dangerous; acquire, via the communication unit, detection results from a motion sensor that detects the driver's movement, determine, based on the detection result of the motion sensor, whether or not the driver has taken preparatory action to avoid a collision; and change the operation pattern of the behavior control depending on whether or not preparatory action has been taken within a predetermined determination period.

[0016] According to the second driving assistance device, the operation pattern of the behavior control is determined based on whether or not the driver has performed preparatory action. Therefore, even in a situation where the driver does not recognize an object, the behavior control can be executed in an appropriate manner. Note that the behavior control here may be considered as one form of assistance control.

[0017] Note that the symbols in parentheses in the claims indicate a correspondence with the specific means described in the embodiments described below as one aspect, and do not limit the technical scope of the present disclosure.

[0018] It is a diagram illustrating an example of the overall configuration of a driving assistance system. It is a diagram illustrating variations of a motion sensor. It is a functional block diagram of a driving assistance ECU. It is a flowchart showing a process for determining warning timing. It is a flowchart showing another example of a process for determining warning timing. It is a diagram showing an example of variations of warning thresholds.

[0019] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The present disclosure is not limited to the following embodiments, and various modifications can be made without departing from the spirit of the present disclosure. Various modifications may be implemented in appropriate combinations as long as no technical contradictions arise. The present disclosure also includes configurations that combine multiple modifications without explicit description. In the following description, components having the same function are given the same reference numerals, and specific descriptions thereof may be omitted. Furthermore, when only a portion of a configuration is mentioned, descriptions given elsewhere may apply to the other portions.

[0020] FIG. 1 is a diagram illustrating an example of a schematic configuration of a driving assistance system 100 according to the present disclosure. Hereinafter, the term "host vehicle" refers to a vehicle equipped with the driving assistance system 100. Furthermore, the term "host vehicle lane" in the present disclosure refers to the lane in which the host vehicle is traveling among multiple lanes of a road. The term "adjacent lane" refers to a lane adjacent to the host vehicle lane. The host vehicle lane may also be referred to as the "ego lane." The term "preceding vehicle" in the present disclosure may be interpreted as the vehicle traveling in the same lane as the host vehicle and being closest to the host vehicle among the vehicles present ahead of the host vehicle.

[0021] In this disclosure, a driver refers to a person sitting in the driver's seat, that is, a driver's seat occupant. The driver may be understood as a person who has the authority and responsibility to drive the vehicle. The vehicle may be a remotely operated vehicle that is remotely operated by an operator located outside the vehicle. The operator here refers to a person who has the authority to remotely control the vehicle from outside the vehicle. The operator is also included in the concept of a driver. The driver's seat may be a cockpit for the operator located outside the vehicle.

[0022] The driving assistance system 100 described below can be modified as appropriate to suit the local laws and customs, the characteristics of the vehicle / equipment, etc. Unless otherwise specified, the term "system" below refers to the driving assistance system 100.

[0023] <Overall Configuration of Driving Assistance System> The driving assistance system 100 includes various components shown in Fig. 1 as an example. That is, the driving assistance system 100 includes a periphery monitoring sensor 11, a vehicle state sensor 12, a motion sensor 13, a driver status monitor (DSM) 14, and a wireless communication device 15. The driving assistance system 100 also includes a display 21, a speaker 22, a driving actuator 23, and a driving assistance ECU 30. ECU is an abbreviation for Electronic Control Unit.

[0024] The driving assistance ECU 30 is connected to each of the above devices / sensors, such as the periphery monitoring sensor 11, via an in-vehicle network so that they can communicate with each other. The in-vehicle network is a communication network established within the vehicle. The in-vehicle network may be compliant with standards such as Controller Area Network (hereinafter referred to as CAN: registered trademark) or Ethernet (registered trademark). Some of the devices / sensors may be directly connected to the driving assistance ECU 30 via dedicated signal lines. The connection between the devices may be changed as appropriate.

[0025] The perimeter monitoring sensor 11 is a sensor that detects objects present within a detection range. The perimeter monitoring sensor 11 senses the surrounding environment of the vehicle. The perimeter monitoring sensor 11 may be referred to as an external sensor or an independent sensor. The driving assistance system 100 may include multiple perimeter monitoring sensors 11. The driving assistance system 100 may include a forward camera and a millimeter-wave radar as the perimeter monitoring sensors 11.

[0026] The front camera is a camera arranged on the vehicle to capture images of the area ahead of the vehicle at a predetermined angle of view. The front camera may be arranged on the upper edge of the windshield on the interior side of the vehicle, on the front grille, on the rooftop, etc. The front camera may include a camera main body and a camera ECU. The camera main body is a module including an image sensor and a lens. The camera main body sequentially generates image frames at a predetermined frame rate. The camera ECU includes a processor and memory. The processor is a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), etc. The camera ECU is an ECU that detects a predetermined detection target by performing recognition processing on the image frames. The camera ECU may be configured to detect and identify objects registered as detection targets using a classifier that applies deep learning. The camera ECU also calculates the relative position coordinates of the detected object with respect to the vehicle based on position information of the detected object in the image frame.

[0027] Objects detected by the forward camera include moving objects such as pedestrians and other vehicles. Objects detected by the forward camera may also include features such as road edges, road markings, and roadside structures. Road markings include lane markings that indicate lane boundaries, crosswalks, stop lines, navigation strips, safety zones, and roadway arrows. Roadside structures include road signs, guardrails, curbs, utility poles, and traffic lights.

[0028] In addition to the front camera, the driving assistance system 100 may also include a side camera that captures images of the sides of the vehicle and a rear camera that captures images of the rear of the vehicle. The function of detecting a detection target object by analyzing camera images may be provided in another ECU, such as the driving assistance ECU 30. The functional layout within the driving assistance system 100 can be changed as appropriate.

[0029] A millimeter-wave radar is a device that transmits probe waves in a predetermined direction and detects the relative position and relative speed of an object relative to the vehicle by analyzing the received data of the reflected waves that are returned after the transmitted waves are reflected by the object. The probe waves may be millimeter waves or quasi-millimeter waves. The driving assistance system 100 may be equipped with a forward millimeter-wave radar and a rearward millimeter-wave radar. The forward millimeter-wave radar is a millimeter-wave radar that transmits probe waves toward the front of the vehicle. The rearward millimeter-wave radar is a millimeter-wave radar that transmits probe waves toward the rear of the vehicle. Each millimeter-wave radar generates data indicating the relative position and relative speed of the detected object and outputs the data as a detection result to the driving assistance ECU 30, etc. Objects detected by the millimeter-wave radar may include the above-mentioned moving objects as well as three-dimensional structures serving as landmarks.

[0030] The perimeter monitoring sensor 11 may include LiDAR or sonar. LiDAR is an abbreviation for Light Detection and Ranging or Laser Imaging Detection and Ranging. LiDAR is a device that generates three-dimensional point cloud data indicating the positions of reflection points for each detection direction by emitting laser light. The combination of the perimeter monitoring sensors 11 included in the driving assistance system 100 may be changed as appropriate. Data indicating the detection results of each perimeter monitoring sensor 11 is input to the driving assistance ECU 30.

[0031] The vehicle state sensor 12 is a sensor that detects information related to the state of the host vehicle. The vehicle state sensor 12 may include a vehicle speed sensor, a steering angle sensor, an acceleration sensor, a yaw rate sensor, a shift position sensor, and the like. The vehicle speed sensor is a sensor that detects the traveling speed of the host vehicle. The steering angle sensor is a sensor that detects the steering angle. The acceleration sensor is a sensor that detects acceleration acting in the longitudinal direction of the host vehicle, lateral acceleration acting in the lateral direction, and the like. The yaw rate sensor is a sensor that detects the angular velocity of the host vehicle. The shift position sensor is a sensor that detects the shift position of the transmission. The vehicle state sensor 12 outputs data indicating the current value of the physical state quantity to be detected (i.e., the detection result) to the in-vehicle network. The data flowing through the in-vehicle network is referenced by the driving assistance ECU 30 as appropriate. The type of sensor used by the driving assistance system 100 as the vehicle state sensor 12 may be designed as appropriate.

[0032] The motion sensor 13 is a sensor that detects the movement of the driver. The motion sensor 13 may include a plurality of sensors. As shown in Fig. 2, the motion sensor 13 may include an accelerator pedal sensor (APS in the figure) 13A, a brake pedal sensor (BPS in the figure) 13B, a foot camera 13C, a surface pressure sensor 13D, a grip sensor 13E, and a room camera 13F.

[0033] The accelerator pedal sensor 13A is a sensor that detects the amount of depression of the accelerator pedal. The brake pedal sensor 13B is a sensor that detects the amount of depression of the brake pedal. The pedal depression amount means the amount to which the driver depresses the pedal. The term "pedal" may be read as either the accelerator pedal or the brake pedal. The term "pedal sensor" below may be read as either the accelerator pedal sensor 13A or the brake pedal sensor 13B.

[0034] The depression amount may be expressed as an angle or the like. In the following, the expression "accelerator pedal angle" refers to the depression amount of the accelerator pedal. Also, the expression "brake pedal angle" refers to the depression amount of the brake pedal. A state in which the depression amount / pedal angle is 0 corresponds to a state in which the driver is not depressing the pedal. The accelerator pedal sensor 13A and the brake pedal sensor 13B each output data indicative of the pedal angle (i.e., depression amount) to the driving assistance ECU 30.

[0035] The foot camera 13C is a camera for capturing images of the position / movement of the driver's feet near the pedals. The foot camera 13C may be installed above the brake pedal or accelerator pedal. The foot camera 13C may also be located to the right of the accelerator pedal or to the left of the brake pedal. The foot camera 13C may be attached at any position so as to capture an image of the driver's feet. The captured image signal of the foot camera 13C is output to the driving assistance ECU 30. The driving assistance system 100 may be equipped with a foot sonar / foot radar instead of / in addition to the foot camera 13C. The foot sonar / foot radar is a sonar / millimeter-wave radar for detecting the position of the feet. The foot sonar / foot radar may be located around the driver's feet, such as above or to the side of the pedals. Note that the position of the feet in the following description may be interpreted as the position of the part from the toes to the ankles / heels.

[0036] The accelerator pedal sensor 13A, the brake pedal sensor 13B, and the foot camera 13C are sensors for detecting the movement of the driver's feet. In this disclosure, sensors for detecting the movement of the driver's feet, such as these sensors, are also referred to as foot sensors 13X. A foot sonar / foot radar may also be included in the foot sensors 13X.

[0037] The foot camera 13C, foot sonar, and foot radar all correspond to foot position sensors that detect the position of the driver's feet. Hereinafter, the expression "foot position sensor" may be replaced with foot camera 13C, foot sonar, or foot radar. The surface pressure sensor 13D, which will be described next, is also a sensor that indirectly detects foot movement. Therefore, the surface pressure sensor 13D may also be considered a type of foot sensor 13X and foot position sensor. The pedal sensor may be referred to as a first sensor, and the foot position sensor may be referred to as a second sensor or sub-sensor.

[0038] The surface pressure sensor 13D is a sensor that detects pressure acting on the seating surface. The surface pressure sensor 13D may be a sheet-like module in which multiple pressure-sensitive points are arranged in a two-dimensional matrix, a so-called pressure sensor sheet. The surface pressure sensor 13D may be arranged over the entire seating surface. Alternatively, the surface pressure sensor 13D may be arranged only in the area where the back of the driver's knees or thighs come into contact. The surface pressure sensor 13D may be provided only in an area within 5 cm from the front end of the seating surface. The surface pressure sensor 13D may be configured to detect the distribution of pressure acting near the front end or over the entire seating surface. The surface pressure sensor 13D outputs data indicating the pressure distribution to the driving assistance ECU 30.

[0039] The driving assistance system 100 may include one or more load sensors instead of or in addition to the surface pressure sensor 13D. The load sensors may be distributed at the four corners and the center of the seating surface of the driver's seat. The multiple load sensors may also function as sensors that detect the distribution of pressure acting on the seating surface.

[0040] The grip sensor 13E is a sensor that detects the driver's grip on the steering wheel. The grip sensor 13E may be configured to detect not only whether the steering wheel is being gripped but also the grip force. The grip force functions as a parameter indicating whether the driver is firmly gripping the steering wheel or simply placing their hands on it. The grip force may also be referred to as grip pressure. The grip sensor 13E may be a capacitance-type touch sensor arranged on the outer circumferential surface of the steering wheel. The grip sensor 13E may also be a pressure sensor provided on the steering wheel. The pressure sensor detects the pressure with which the driver grips the steering wheel. The grip sensor 13E may be a piezoelectric ceramic sensor using piezoelectric ceramic. The grip sensor 13E outputs data indicating the driver's grip on the steering wheel to the driving assistance ECU 30. The grip sensor 13E corresponds to a hand sensor.

[0041] The room camera 13F is a camera installed so as to capture the face and upper body of the driver. The room camera 13F may be attached, for example, to the top surface of the instrument panel, the top edge of the windshield, or the A-pillar on the driver's seat side. The room camera 13F may be installed inside the vehicle in a position and orientation that allows it to capture the movements of the driver's upper body and head. The video signal from the room camera 13F is output to the driving assistance ECU 30.

[0042] The room camera 13F may be integrated with the DSM 14, which will be described next. The room camera 13F or the DSM 14 corresponds to a head sensor. The room camera 13F may be configured to detect the driver's hand movements in addition to the upper body / head. The room camera 13F may have a position, posture, and angle of view that allows it to capture the hand movements relative to the shift lever and steering wheel. Additionally, the motion sensor 13 may include a touch sensor provided at the shift position.

[0043] The DSM 14 is a device that sequentially detects the driver's state by analyzing the driver's facial image. The driver's state may include the driver's facial orientation (head angle) and eye opening. The DSM 14 also detects the driver's gaze direction by combining the driver's facial orientation vector and the eye orientation vector relative to the face. The DSM 14 includes a visible light / infrared camera installed in the vehicle cabin in a position that captures the driver's face. The DSM 14 may be disposed on the top surface of the steering column cover with its optical axis directed toward the headrest of the driver's seat. Data indicating the detection results (e.g., gaze direction) of the DSM 14 is transmitted to the driving assistance ECU 30. The DSM 14 corresponds to a gaze detector. The DSM 14 may also be used as a head sensor.

[0044] The wireless communication device 15 is a device that enables the vehicle to perform wireless communication with external devices. The external devices may include a server, a traffic information center, a roadside device, and some or all of other vehicles. The wireless communication device 15 may be configured to perform short-range communication. The short-range communication may include vehicle-to-vehicle communication, which is direct communication between vehicles, and road-to-vehicle communication, which is direct communication between a vehicle and a roadside device. The short-range communication may be wireless communication with a communication distance of several hundred meters. The short-range communication method (protocol) may be DSRC (Dedicated Short Range Communications) compatible with IEEE 802.11p, or cellular V2X (PC5 / SideLink / Uu). The wireless communication device 15 may receive vehicle information from nearby vehicles. The vehicle information may include speed, current location, turn signal operation status, acceleration, movement trajectory, etc. Here, nearby vehicles refer to vehicles within a range where vehicle-to-vehicle communication is possible. The wireless communication device 15 may also receive information about other vehicles from roadside devices. The information about the surrounding vehicles received by the wireless communication device 15 may be used by the surrounding monitoring sensor 11 or the driving assistance ECU 30 to detect other vehicles that are in the blind spot of the driver.

[0045] The display 21 is a device that displays an image according to a video signal input from the driving assistance ECU 30. The display 21 may be a head-up display (HUD), a meter display, or a center display. The HUD is a device that projects image light onto a predetermined area on the windshield to display a virtual image that can be perceived by the driver. The meter display is a display that is arranged in an area of ​​the instrument panel that is located in front of the driver's seat. The center display is a display that is provided in the center of the instrument panel in the vehicle width direction. The meter display and the center display may be liquid crystal displays or organic EL displays.

[0046] The speaker 22 is a device that outputs sound corresponding to a signal input from the driving assistance ECU 30. In this disclosure, the term "sound" may include notification sounds (alarm sounds), voice, music, etc. The display 21 and the speaker 22 are notification devices that notify the driver of information. The driving assistance system 100 may also include a vibration generator or ambient light as notification devices other than those described above. The vibration generator is a device that applies vibration stimulation to the driver's body, such as the hands, back, or chest. The vibration generator may be provided on the steering wheel or the driver's seat. The vibration generator may also be a device that vibrates a seat belt. The ambient light is a lighting device that is realized by multiple light emitting diodes (LEDs) and is capable of adjusting the light emission color and light emission intensity. The ambient light may be provided on the instrument panel or the steering wheel.

[0047] The traveling actuators 23 are actuators related to the traveling of the host vehicle, i.e., acceleration, deceleration, and steering. The traveling actuators 23 include a brake actuator and a steering actuator. The traveling actuators 23 may also include an electronic throttle or a traveling motor. The steering actuator may be an EPS (Electric Power Steering) motor. Other ECUs, such as a steering ECU, a hybrid control ECU, an engine ECU, a motor ECU, a brake ECU, etc., may be interposed between the driving assistance ECU 30 and the traveling actuators 23.

[0048] In addition to the above, various other in-vehicle devices may be directly or indirectly connected to the driving assistance ECU 30. The driving assistance ECU 30 is connected to a locator and a map storage unit via an in-vehicle network or using a dedicated cable to enable mutual communication. The locator is a device that calculates and outputs the position coordinates of the vehicle using navigation signals transmitted from positioning satellites that make up the Global Navigation Satellite System (GNSS). The map storage unit is a storage device that stores map data. The map data stored in the map storage unit includes the three-dimensional shape of roads, the locations of road markings such as lane markings, the locations of traffic signs, etc., with the accuracy required for autonomous driving, etc. The driving assistance ECU 30 may refer to map data within a range corresponding to the current location and use it to recognize the driving environment.

[0049] The driving assistance ECU 30 is a device that assists the driver in driving by presenting information to the driver or performing some driving operations on behalf of the driver based on signals input from the various in-vehicle devices described above. The driving assistance ECU 30 corresponds to a driving assistance device. The driving assistance ECU 30 may be an ECU that performs some driving operations on behalf of the driver by controlling the driving actuator 23 based on detection results from the perimeter monitoring sensor 11. The driving assistance ECU 30 may realize controls (functions) such as adaptive cruise control (ACC), pre-collision safety (PCS) control, automatic emergency braking (AEB) control, and lane keeping assist (LKA). The driving assistance ECU 30 may have a so-called automatic driving function that autonomously drives the vehicle along a predetermined route. The driving assistance ECU 30 may also be an automatic driving device.

[0050] The driving assistance ECU 30 may be a computer including a processor 31, a memory 32, a storage 33, a communication interface 34, and a bus connecting these components. The memory 32 is a rewritable volatile storage medium. The memory 32 may be a random access memory (RAM). The storage 33 is a rewritable non-volatile memory. The storage 33 may include multiple types of storage media, such as a read-only memory (ROM) and a flash memory. The storage 33 stores a driving assistance program, which is a program executed by the processor 31. The execution of the driving assistance program by the processor 31 corresponds to the execution of a driving assistance method. The communication interface 34 is a circuit module that enables the processor 31 to communicate with other on-board devices via an in-vehicle network. The communication interface 34 may include a PHY chip or the like that complies with the communication standard of the in-vehicle network. The communication interface 34 corresponds to a communication unit.

[0051] <Configuration of the Driving Assistance ECU> The driving assistance ECU 30 includes a plurality of functional units shown in Fig. 3 as functional units realized by executing a driving assistance program. That is, the driving assistance ECU 30 includes an information acquisition unit F1, an environment recognition unit F2, a preparatory action determination unit F3, and a support unit F4.

[0052] The information acquisition unit F1 is configured to acquire information (data) for implementing driving assistance from an in-vehicle device. The information acquisition unit F1 acquires sensing data (i.e., detection results) from various surrounding monitoring sensors 11, including a forward camera. The sensing data includes data on objects present around the vehicle, such as moving objects, features, and obstacles. The data on each detected object may include the position, moving speed, and type or size of the detected object.

[0053] The information acquisition unit F1 also acquires data indicating the state of the vehicle, such as the vehicle's traveling speed, acceleration, yaw rate, and shift position, from the vehicle state sensor 12. Furthermore, the information acquisition unit F1 may acquire the vehicle's position from a locator. The information acquisition unit F1 may also acquire surrounding map information by referring to a map storage unit.

[0054] The information acquisition unit F1 may acquire data transmitted from an external device in cooperation with the wireless communication device 15. The information acquisition unit F1 may acquire vehicle information transmitted from a preceding vehicle via vehicle-to-vehicle communication. The information acquisition unit F1 may also acquire position information of stopped vehicles, the status of traffic lights, and the positions and movement direction information of pedestrians and bicycles in cooperation with the wireless communication device 15.

[0055] The information acquisition unit F1 acquires data related to the driver's movements based on an input signal from the motion sensor 13. The information acquisition unit F1 may sequentially acquire data related to the accelerator pedal depression amount, brake pedal depression amount, foot position, pressure distribution acting on the seating surface, steering wheel grip state, upper body position, and head position. Note that the information acquisition unit F1 does not need to acquire data on all of the above items. The above items are merely examples. The information acquisition unit F1 may be configured to acquire data on only some of the above items.

[0056] The information acquisition unit F1 may have a function to identify the position of the driver's feet by analyzing the image captured by the foot camera 13C. The information acquisition unit F1 may also have a function to identify the position of the driver's upper body and head by analyzing the image captured by the room camera 13F. "Acquisition" in this disclosure also includes generation / detection / determination by the driving assistance ECU 30 itself through calculations based on data input from other devices / sensors. This is because the functional layout within the system can be changed as appropriate. Of course, the function to identify the position of the feet through image analysis may be provided by the foot camera 13C. The function to identify the position of the driver's upper body / head through image analysis may be provided by the room camera 13F.

[0057] Various data sequentially acquired by the information acquisition unit F1 is stored in a temporary storage medium such as the memory 32 and is used by the environment recognition unit F2, the preparatory action determination unit F3, and the support unit F4. The various information may be classified by type and stored in the memory 32. The various information may also be sorted and stored so that the most recent data is at the top. Data that has been acquired for a certain period of time may be discarded.

[0058] The environment recognition unit F2 recognizes the driving environment of the vehicle based on various data acquired by the information acquisition unit F1. The environment recognition unit F2 may recognize the driving environment of the vehicle by a sensor fusion process that integrates the detection results of multiple perimeter monitoring sensors 11, such as a front camera and a millimeter-wave radar, with a predetermined weight.

[0059] The driving environment includes the curvature of the road, the number of lanes, the speed limit, the weather, the road surface condition, the traffic volume, etc. The weather and road surface condition may be determined by combining the recognition result of the front camera with the weather information acquired by the information acquisition unit F1. The road structure and the speed limit may be determined using the recognition result of the front camera, map data, or trajectory information of the preceding vehicle.

[0060] The driving environment also includes the positions, types, and moving speeds of objects around the vehicle. The environment recognition unit F2 recognizes the positions and behaviors of preceding vehicles, oncoming vehicles, pedestrians, and bicycles based on the various data acquired by the information acquisition unit F1. Hereinafter, objects around the vehicle will also be referred to as "surrounding objects."

[0061] The environment recognition unit F2 calculates a collision risk for surrounding objects that exist in an area related to the traveling direction of the host vehicle. When the host vehicle is moving forward, the environment recognition unit F2 calculates a collision risk for surrounding objects that exist within a predetermined range ahead of the host vehicle. Here, "forward" may include diagonally forward. When the host vehicle is planning to change lanes or turn right or left, the environment recognition unit F2 may calculate a collision risk for surrounding objects that exist diagonally behind the host vehicle or near an intersection.

[0062] The collision risk may be TTC (Time-To-Collision) or MTC (Margin-To-Collision). TTC and MTC are parameters indicating that the smaller the value, the higher the collision risk. The collision risk may also be evaluated using TTC2nd, a distance condition evaluation index (KdB), THW (Time-Headway), RF (Risk Feeling), or the like. Below, the operation of each unit will be described using an example in which the environment recognition unit F2 uses TTC as the collision risk. That is, the environment recognition unit F2 calculates the TTC for each vehicle, pedestrian, and bicycle present ahead of the host vehicle. Hereinafter, a hazardous object may be interpreted as an object whose TTC is less than a predetermined value.

[0063] The preparatory action determination unit F3 is configured to determine whether the driver has taken preparatory action to avoid a collision based on the output signal of the motion sensor 13, in other words, the data related to the driver's behavior acquired by the information acquisition unit F1. Determining whether the driver has taken preparatory action corresponds to detecting preparatory action. In the present disclosure, the terms detection and determination may be interchangeable.

[0064] The preparatory action may be an action that leads to deceleration or steering. The preparatory action may be a reaction that the driver may take when the driver recognizes an object that the vehicle is likely to collide with (i.e., a dangerous object). The preparatory action may be interpreted as a movement that indicates that the driver has recognized the dangerous object. The preparatory action may be rephrased as a danger recognition reaction or an avoidance preparation behavior. The preparatory action may be broadly classified into (A) preparatory action of the feet, (B) preparatory action of the upper body / head, and (C) preparatory action of the hands.

[0065] The preparatory foot action may be (1) releasing the accelerator pedal, (2) lifting the right foot from the accelerator pedal and moving it closer to the brake pedal, or (3) placing the right foot on the brake pedal. The above preparatory foot action may be determined from time-series data of the output of the accelerator pedal sensor 13A and time-series data of the output of the brake pedal sensor 13B. The preparatory action determination unit F3 may obtain the amount and direction of movement of the foot position from images of the foot camera 13C and combine this with the output value of the pedal sensor to detect the above preparatory action. Time-series data for a certain parameter may be interpreted as data indicating observed values ​​for each time period, in other words, data indicating the time change of the parameter.

[0066] The preparatory action determination unit F3 may determine that the accelerator pedal has been released if the amount of decrease in the accelerator pedal angle within a certain period of time is equal to or greater than a predetermined value, or if the accelerator pedal angle is less than a predetermined value. Releasing the right foot from the accelerator pedal and moving it closer to the brake pedal may be an action of moving the right foot from the accelerator toward the brake pedal (i.e., to the left) by a predetermined distance or more. The preparatory action determination unit F3 may identify the movement / amount of movement of the right foot based on the image from the foot camera 13C.

[0067] The preparatory action determination unit F3 may determine that the right foot is applied to the brake pedal when the brake pedal angle is greater than 0 and less than a predetermined value (e.g., 5°). The preparatory action determination unit F3 may determine that the right foot is applied to the brake pedal when the driver's right foot is on the brake pedal in the image captured by the foot camera 13C. In addition, the action of pressing the left foot against the footrest may be registered as a preparatory action. The behavior of the left foot may correspond to an action to support the body for sudden braking.

[0068] The preparatory action determination unit F3 may determine that a preparatory action has been performed when the accelerator pedal angle or the brake pedal angle has changed in a predetermined pattern.Also, the preparatory action determination unit F3 may determine that a preparatory action has been performed when the right foot position, which is determined by analyzing the image from the foot camera 13C, has changed in a predetermined pattern.

[0069] Note that when the driver moves his / her right foot, the pressure distribution on the seating surface may vary. Furthermore, the pressure distribution pattern may differ between when the driver is pressing the accelerator pedal and when the driver has his / her foot near the brake pedal. Given this, the preparatory action determination unit F3 may determine that a preparatory action has been taken when the pressure distribution output by the surface pressure sensor 13D varies according to a predetermined pattern. The preparatory action determination unit F3 may determine that the driver has moved his / her foot toward the brake pedal, i.e., that a preparatory action has been taken, in response to a shift of the pressure point corresponding to the right thigh from right to left. Furthermore, the preparatory action determination unit F3 may determine that a preparatory action has been taken when the center of gravity of the pressure distribution has shifted leftward.

[0070] The preparatory behavior of the upper body / head may be (4) moving the upper body forward, (5) moving the face left and right, or (6) correcting posture. The act of moving the upper body forward corresponds to a behavior for checking parts that are difficult to see when there are many obstructions. This behavior is due to the intention to recognize a potential hazard, and therefore may be considered one of the preparatory behaviors. The movement of moving the upper body forward may be detected by analyzing the image from the room camera 13F or based on the output of the DSM 14. The movement of moving the upper body forward may also be detected based on time-series data of pressure distribution output by the surface pressure sensor 13D.

[0071] The act of moving one's head from side to side may correspond to an action to check whether it is possible to change lanes to avoid a dangerous object ahead. Therefore, the act of moving one's head from side to side may also be considered as a preparatory action. The preparatory action determination unit F3 may detect the act of moving one's head from side to side based on the image from the room camera 13F or the output of the DSM 14.

[0072] Correcting one's posture suggests that the driver may be preparing for contact with a hazardous object or for a sudden driving maneuver. Therefore, correcting one's posture may be considered a preparatory action. The preparatory action determination unit F3 may detect that the driver has corrected his / her posture by analyzing the image from the room camera 13F or based on the output of the DSM 14. When the driver corrects his / her driving posture, the head position moves upward. Therefore, the preparatory action determination unit F3 may determine that a preparatory action has been taken in response to the head position being raised. The preparatory action determination unit F3 may determine that a preparatory action has been taken when the position of the driver's upper body / head, as determined by image analysis, changes in a predetermined pattern.

[0073] The preparatory action of the hand may be (7) an action of tightening the grip on the steering wheel (grip firmly). The preparatory action determination unit F3 may detect the above preparatory action based on the output signal of the grip sensor 13E. Note that, if the driver was gripping the steering wheel with only one hand before the hazard was detected, the preparatory action determination unit F3 may determine that the preparatory action was performed in response to the driver gripping the steering wheel with both hands. The action of tightening the grip on the steering wheel may include an action of increasing the number of hands gripping the steering wheel from one to two.

[0074] A pattern model for detecting preparatory actions may be generated in advance by testing, machine learning, or the like, and stored in the storage 33. The driving environment may also be taken into consideration when determining whether preparatory actions have been performed. The preparatory action determination unit F3 may determine that preparatory actions have been performed in response to a decrease in the accelerator pedal angle when the current driving speed is equal to or less than the speed limit. The preparatory action determination unit F3 may also determine that preparatory actions have been performed in response to a decrease in the accelerator pedal angle when the vehicle is traveling uphill. That is, the preparatory action determination unit F3 may determine that preparatory actions have been performed in response to a decrease in the accelerator pedal angle by a predetermined amount or falling below a predetermined value in a driving environment in which the accelerator pedal angle is likely to be maintained or increased.

[0075] Alternatively, the preparatory action determination unit F3 may determine that a preparatory action has been taken when the shift position is changed from the drive position to the brake position. The brake position here refers to a shift position where engine braking or regenerative braking is applied. The drive position is a shift position for forward movement where engine braking or regenerative braking is relatively small. Switching the shift position from the drive position to the brake position is an operation for deceleration. Therefore, the above operation may be considered a preparatory action. Note that if the host vehicle is a manual transmission vehicle, the preparatory action determination unit F3 may determine that a preparatory action has been taken when the host vehicle is downshifted. The preparatory action determination unit F3 may also detect that the driver has placed his / her hand on the shift lever as a preparatory action. The driver's placing his / her hand on the shift lever may be detected from the image from the room camera 13F or from the output of a touch sensor provided on the shift lever.

[0076] The preparatory action determination unit F3 may be configured to determine whether the driver has performed the above-described preparatory action within a predetermined determination period. The determination period may be from the time an object appears whose TTC is less than a predetermined determination start value until the TTC for the object reaches a determination end value. The determination start value may be set to 4, 5, or 6 seconds, for example. The determination end value may be set to 2.5, 3, or 3.5 seconds, for example. The determination end value may be set to a value that is at least 1 second smaller than the determination start value. The determination period may be the period from the time the driving assistance ECU 30 recognizes a dangerous situation until the TTC reaches a standard threshold value, which will be described later. Note that the driving assistance ECU 30 may consider an object whose TTC is less than or equal to the determination start value to be a hazard. Of course, the threshold for setting a nearby object as a hazard may be set to a value greater than the determination start value.

[0077] The support unit F4 notifies the driver using notification devices such as the display 21 and the speaker 22. Various notifications / suggestions can be realized by displaying an image on the display 21 or outputting a voice message from the speaker 22.

[0078] The support unit F4 may be configured, as a basic operation, to output an alarm sound from the speaker 22 to notify the driver of the risk of collision when a hazardous object whose TTC has reached a predetermined standard threshold is present. The standard threshold may be set to 2.2 seconds, for example. The standard threshold may also be set to other values, such as 1.8 seconds, 2 seconds, 2.4 seconds, 2.6 seconds, or 3 seconds. For convenience, the timing at which the TTC reaches the standard threshold is also referred to as the standard timing.

[0079] In this embodiment, the support unit F4 is configured to output an audible warning when the TTC reaches a standard threshold if no preparatory behavior of the driver is detected during the determination period. On the other hand, if the preparatory behavior determination unit F3 detects that preparatory behavior has been performed during the determination period, the support unit F4 delays the output timing of the audible warning from the standard timing. If the preparatory behavior determination unit F3 detects that preparatory behavior has been performed during the determination period, the support unit F4 may output an audible warning when the TTC reaches a predetermined delayed threshold. The delayed threshold is a predetermined amount smaller than the standard threshold. The difference between the delayed threshold and the standard threshold may be set to 0.2 seconds, 0.4 seconds, 0.6 seconds, or the like. Thus, the support unit F4 includes, as a sub-function, a condition change unit F41 that changes the output timing of the audible warning depending on whether the driver has performed preparatory behavior during the determination period. The output timing of the audible warning will hereinafter also be referred to as the warning timing. In the present disclosure, a threshold value for the TTC that determines the output timing of the audible warning, such as a delayed threshold or a standard threshold, is also referred to as the warning threshold.

[0080] In addition, the support unit F4 automatically starts braking control (so-called automatic braking) when the TTC of the hazardous object becomes equal to or less than a predetermined braking threshold value. The braking threshold value may be set to a value of 1.4 seconds or less (1.6 seconds or less for large vehicles), such as 1.2 seconds.

[0081] After the start of the determination, if the TTC of the hazardous object becomes equal to or greater than the determination release value due to the driver's braking operation, etc., the support unit F4 may determine that the hazardous situation has been escaped. The determination release value may be the same as the determination start value, or may be set to be a predetermined amount larger than the determination start value.

[0082] <Example of Operation> The flowchart shown in Fig. 4 shows an example of a process for determining the timing of an alarm in the driving assistance ECU 30. The flowchart shown in Fig. 4 may be executed periodically while the host vehicle is traveling. The description of the processor 31 as the entity that executes the following steps may be replaced with the information acquisition unit F1, the environment recognition unit F2, the preparatory action determination unit F3, the assistance unit F4, or the assistance unit F4, depending on the context.

[0083] Step S101 is a step in which the processor 31 acquires data from various in-vehicle devices. In step S101, the processor 31 acquires the detection results of the perimeter monitoring sensor 11, the detection results of the motion sensor 13, the vehicle's traveling speed, and the like. Step S101 may be interpreted as a step in which received data stored in a buffer of the communication interface 34 is retrieved. After step S101, the processor 31 executes step S102. The process corresponding to step S101 may be periodically executed after step S102. In other words, step S101 may be executed as needed in parallel (independently) with the process after step S102.

[0084] Step S102 is a step in which the processor 31 determines whether the current situation of the host vehicle is a dangerous situation based on the detection result of the perimeter monitoring sensor 11 acquired in step S101. The processor 31 may determine that the current situation is a dangerous situation when an object whose TTC is less than a predetermined value (e.g., a determination start value) is present. The process in which the processor 31 calculates the TTC for each surrounding object based on the detection result of the perimeter monitoring sensor 11 may be included in step S101 or S102. Of course, the processor 31 may determine whether the current situation is a dangerous situation based on data other than the TTC.

[0085] Step S103 is a step for determining whether or not the driver has performed preparatory action. The determination of preparatory action may be performed sequentially during the determination period. As described above, whether or not preparatory action has been performed may be determined by various methods. Whether or not preparatory action has been performed may be determined based on a combination of time-series data for each item, such as the accelerator pedal angle, the brake pedal angle, the foot position, the head position, the steering wheel grip force, etc.

[0086] As a result of the preparatory behavior determination process in step S103, if it is determined that preparatory behavior has been performed (YES in step S104), the processor 31 executes step S105. On the other hand, if no preparatory behavior has been detected during the determination period (NO in step S104), the processor 31 executes step S106.

[0087] Step S105 is a step of setting the warning timing later than the standard timing. Step S105 may be interpreted as a step of setting the warning threshold to a later threshold. When step S105 is completed, this flow ends. Step S106 is a step of setting the warning timing to the standard timing. Step S106 may be interpreted as a step of setting the warning threshold to the standard threshold. When step S106 is completed, this flow ends. Thereafter, when the TTC actually reaches the warning threshold set in step S105 or S106, the support unit F4 outputs a warning sound from the speaker 22.

[0088] <Effects> According to the above configuration, when preparatory action is being taken in a dangerous situation, the timing of outputting the alarm sound is delayed compared to when preparatory action is not being taken. This reduces the risk of outputting the alarm sound in a situation where the driver is aware of a hazard and is about to take evasive action. As a result, the risk of causing annoyance to the driver is reduced.

[0089] Furthermore, according to the above configuration, if a driver is in a dangerous situation and has not taken any preparatory action, the warning sound is output at the standard timing. Even if the driver is looking at a dangerous object but is not aware that the situation is dangerous, the timing of outputting the warning sound is not set to be late. This makes it easier for the driver to recognize the dangerous object and take action to avoid it.

[0090] In one aspect, the above embodiment may be interpreted as a configuration in which, if a preparatory action is not taken during the determination period, the timing of executing the assist control (warning) is advanced compared to when a preparatory action is taken. By advancing the timing of executing the assist control when a preparatory action is not taken during the determination period, the time from when the driver recognizes a hazard to when the driver actually operates the vehicle can be extended. Furthermore, the above configuration may be interpreted as a configuration in which, if a preparatory action is taken during the determination period, the timing of executing the assist control is delayed compared to when a preparatory action is not taken during the determination period.

[0091] <Warning Timing When Preparatory Action Is Not Taken> In the above, when no preparatory action of the driver is detected during the determination period, the audible alarm is output at the timing when the TTC reaches the standard threshold (i.e., the standard timing). However, this is not limited to this. When no preparatory action of the driver is detected during the determination period, the assistance unit F4 may be configured to output the audible alarm at a timing earlier than the standard timing. To achieve this, the driving assistance ECU 30 may be configured to use an earlier threshold value that is greater than the standard threshold value as the warning threshold value. The earlier threshold value may be set to a value that is 0.2 seconds, 0.4 seconds, or 0.6 seconds greater than the standard threshold value.

[0092] In this case, the support unit F4 may adopt the standard threshold as the warning threshold when the driver has taken the first preparatory action but not the second preparatory action. Also, the support unit F4 may be configured to set the warning threshold to the delayed threshold when the driver has taken the second preparatory action.

[0093] The primary preparatory action here refers to a minor (elementary) preparatory action such as releasing the foot from the accelerator pedal or reducing the amount of depression of the accelerator pedal, etc. The primary preparatory action may be detected based on the output signal of the accelerator pedal sensor 13A, the foot camera 13C, or the surface pressure sensor 13D.

[0094] The secondary preparatory action may be, for example, moving the right foot near the brake pedal, placing the right foot on the brake pedal, or lightly stepping on the brake pedal. These actions may be detected based on the output signals of the brake pedal sensor 13B, the foot camera 13C, or the surface pressure sensor 13D. The secondary preparatory action may also be setting the shift position to the brake position or shifting down. These actions may be detected based on the output of the shift position sensor.

[0095] The secondary preparatory action may be to tighten the grip on the steering wheel or to correct the posture, which may be detected based on the output signals of the grip sensor 13E, the room camera 13F, or the surface pressure sensor 13D.

[0096] When the determination period has ended, the processor 31 serving as the support unit F4 may determine the warning timing according to the procedure of steps S111 to S115 shown in FIG. 5. First, in step S111, the support unit F4 determines whether or not the driver has performed a primary preparatory action during the determination period. If the driver has not performed a primary preparatory action, that is, if the driver's primary preparatory action has not been detected, the processor 31 executes step S112. Step S112 is a step of setting the warning timing earlier than the standard timing. Step S112 may be a step of setting the warning threshold to an earlier threshold. When step S112 is completed, this flow ends.

[0097] Furthermore, if the driver's primary preparatory behavior is detected during the determination period, the processor 31 executes step S113. Step S113 is a step for determining whether the driver has performed a secondary preparatory behavior during the determination period. If the driver has not performed a secondary preparatory behavior during the determination period, that is, if the driver's secondary preparatory behavior has not been detected, the processor 31 executes step S114. Step S114 is a step for the processor 31 to set the warning timing to the standard timing. Step S114 may be interpreted as a step for setting the warning threshold to the standard threshold. When step S114 is completed, this flow ends.

[0098] If the driver's secondary preparatory behavior is detected during the determination period, the processor 31 executes step S115. Step S115 is a step in which the processor 31 sets the warning timing later than the standard timing. Step S115 may be interpreted as a step in which the warning threshold is set to a later threshold. When step S115 is completed, this flow ends.

[0099] As described above, when a driver does not take preparatory action despite a dangerous situation, a warning is output earlier than normal. This configuration can prompt the driver to take preparatory action. As a result, safety can be further improved. Note that the timing corresponding to the later threshold value may be rephrased as later timing. Also, the timing corresponding to the earlier threshold value may be rephrased as earlier timing.

[0100] <Adjusting Timing According to Preparatory Action> In the above, the warning timing is changed in two or three stages. However, the warning timing may be changed in four or more stages. The warning threshold may be selected from the first to fifth thresholds shown in FIG. 6 according to the driver's behavior. The multiple thresholds may be rephrased as candidate values ​​for the warning threshold to be actually applied.

[0101] In the figure, "Th1" represents the fifth threshold, "Th2" the fourth threshold, "Th3" the third threshold, "Th4" the fourth threshold, and "Th5" the fifth threshold. The numbers in parentheses in the figure indicate examples of threshold settings. "B_Th" represents the braking threshold. The point in time when TTC becomes 0 represents the point in time when a collision occurs.

[0102] The first threshold is the smallest warning threshold among the multiple candidate values. The first threshold may be set to 1.8 seconds, for example. The second threshold is the second smallest warning threshold among the multiple candidate values. The second threshold may be set to 2.0 seconds, for example. The first threshold and the second threshold correspond to the aforementioned late thresholds.

[0103] The third threshold is the third largest / smallest warning threshold among the multiple candidate values. The third threshold may be set to 2.2 seconds, for example. The third threshold may be an intermediate value between the first threshold and the fifth threshold. The third threshold may correspond to the standard threshold. "STD" in the figure means the standard threshold.

[0104] The fourth threshold is the second largest warning threshold among the multiple candidate values. The fourth threshold may be set to 2.4 seconds, for example. The fifth threshold is the largest warning threshold among the multiple candidate values. The fifth threshold may be set to 2.6 seconds, for example. The fourth and fifth thresholds correspond to the earlier thresholds described above.

[0105] The larger the alarm threshold value, the earlier the alarm timing. Also, the smaller the alarm threshold value, the later the alarm timing. The aforementioned determination end value may be set to a value corresponding to the maximum value of various candidate values ​​(here, the fifth threshold value). Note that the above-mentioned candidate values ​​are merely examples, and the specific number of seconds may be changed as appropriate. The difference between the standard threshold value and the second threshold value / fourth threshold value may be 0.1 seconds, for example. The difference between the standard threshold value and the first threshold value / fifth threshold value may be 0.2 seconds, 0.3 seconds, or the like.

[0106] If the support unit F4 is configured to be able to select an alarm threshold to be actually used from a plurality of candidate values ​​(alarm timing), the support unit F4 may change the alarm threshold depending on the type of preparatory action taken by the driver. If no preparatory action is taken during the determination period, the support unit F4 may set the alarm threshold to the fifth threshold. Furthermore, if the driver reduces the accelerator pedal angle but the right foot remains on the accelerator pedal, the support unit F4 may set the alarm threshold to the fourth threshold.

[0107] The support unit F4 may set the warning threshold to a third threshold if the driver's right foot moves from on the accelerator pedal to an area between the accelerator pedal and the brake pedal during the determination period. The support unit F4 may set the warning threshold to a second threshold if the right foot moves onto the brake pedal and the brake pedal angle is less than a predetermined value. For convenience, the state in which the right foot is placed on the brake pedal and the brake pedal angle is less than a predetermined value is also referred to as a braking preparation state.

[0108] The support unit F4 may set the warning threshold to the second threshold when the driver corrects his / her posture, tightens his / her grip on the steering wheel, or puts his / her hand on the shift lever during the braking preparation state. The support unit F4 may also set the warning threshold to the first threshold when the brake pedal angle becomes equal to or greater than a predetermined value during the determination period or when the shift position is set to the brake position.

[0109] Even when a preparatory action is taken in this way, the timing of the warning may be changed depending on the type of preparatory action taken. The driver may take different behaviors (i.e., preparatory actions) depending on the driver's level of awareness of the hazard. With this configuration, the warning sound is output at a timing depending on the driver's level of awareness of the hazard. As a result, it is possible to reduce the risk of annoyance to the driver and also improve safety. In other words, the support unit F4 can output the warning sound at a more appropriate timing.

[0110] The support unit F4 may evaluate the hazard perception level based on the driver's behavior during the determination period and determine the timing of the warning based on the hazard perception level. The hazard perception level here is a parameter that indicates the degree to which the driver recognizes a hazard or a collision risk that exists in front of the vehicle, etc.

[0111] For example, if the right foot remains on the accelerator pedal after the accelerator pedal angle is reduced, the support unit F4 may determine that the hazard perception level is low. If the support unit F4 evaluates that the hazard perception level is low, the support unit F4 may significantly advance the warning timing by setting the warning threshold to the fifth threshold.

[0112] Furthermore, if the driver exhibits a predetermined hesitant behavior after releasing the accelerator pedal, the support unit F4 may determine that the hazard perception level is at a medium level. If the support unit F4 evaluates that the hazard perception level is at a medium level, the support unit F4 may set the warning threshold to a fourth threshold. This control enables the warning timing to be slightly earlier. Note that the hesitant behavior corresponds to a case where the driver is unsure whether to press the brake pedal. The hesitant behavior may be a behavior of placing the right foot between the accelerator pedal and the brake pedal, or a behavior of repeatedly moving the right foot to the left and then to the right.

[0113] Furthermore, if the driver releases the accelerator pedal and then moves the right foot onto the brake pedal without hesitation, the support unit F4 may determine that the hazard perception level is high. If the support unit F4 evaluates that the hazard perception level is high, the support unit F4 may set the warning threshold to the third threshold and not advance the warning timing. Note that if the support unit F4 evaluates that the hazard perception level is high, the support unit F4 may set the warning threshold to the second threshold or the first threshold, thereby delaying the warning timing from the standard timing.

[0114] A point indicating a danger perception level may be set in advance for each preparatory behavior. The support unit F4 may determine the timing of the warning based on the total value of the danger perception levels set for the preparatory behaviors observed during the determination period. The support unit F4 may be configured to delay the warning timing as the total value of the danger perception levels increases.

[0115] The support unit F4 may evaluate the driver's hazard perception level by taking into consideration not only the driver's behavior but also the driving environment. For example, if the driver reduces the accelerator pedal angle when the current driving speed is lower than the speed limit, the support unit F4 may determine that the hazard perception level is high. Furthermore, the support unit F4 may determine that the hazard perception level is high when the driver reduces the accelerator pedal angle while driving uphill. The support unit F4 may evaluate the hazard perception level as high when the accelerator pedal angle is reduced in a specific situation where the accelerator pedal angle would normally be likely to be maintained or increased.

[0116] Conversely, if the driver reduces the accelerator pedal angle when the current traveling speed exceeds the speed limit, this behavior may be simply to comply with the speed limit, without the driver recognizing the potential hazard. Therefore, the support unit F4 may be configured not to determine that reducing the accelerator pedal angle is a preparatory action when the current traveling speed exceeds the speed limit.

[0117] Furthermore, the support unit F4 may change the criteria for determining whether or not a preparatory action has been taken or the method for evaluating the hazard perception level, depending on the driver's behavioral history or habits. Some drivers may have a habit of placing their right foot between the accelerator pedal and the brake pedal even in a situation where no hazard exists. When the driver has such a habit, the support unit F4 may be configured not to determine that a preparatory action has been taken even if the driver places their right foot between the accelerator pedal and the brake pedal. When the driver has such a habit, the support unit F4 may determine that the hazard perception level is low even if the driver places their right foot between the accelerator pedal and the brake pedal. The support unit F4 may be configured to identify the driver's driving habits from the driver's behavioral history, and exclude actions due to the habit from a list of preparatory actions prepared in advance.

[0118] <Method of Using Sensors> The preparatory action determination unit F3 may measure the depression amount of the accelerator pedal / brake pedal using a pedal sensor. The preparatory action determination unit F3 may determine that the right foot is on the accelerator pedal if the accelerator pedal angle is greater than 0. Furthermore, the preparatory action determination unit F3 may determine that the right foot is on the brake pedal if the brake pedal angle is greater than 0.

[0119] When both the accelerator pedal angle and the brake pedal angle are zero, the preparatory action determination unit F3 may acquire the position of the right foot using another sensor. Here, the other sensor may be a foot position sensor such as the foot camera 13C, a foot sonar, or a surface pressure sensor 13D. In this way, when recognizing a preparatory action, the preparatory action determination unit F3 may be configured to first detect the pedal depression amount, and if the pedal is not depressed, detect the foot position using a foot position sensor. Note that the preparatory action determination unit F3 may also measure the accelerator pedal angle and the brake pedal angle using the foot camera 13C.

[0120] The preparatory action determination unit F3 may determine that the right foot is on the pedal when the accelerator pedal angle or the brake pedal angle is greater than 0, and may stop some or all of the foot position sensors. The preparatory action determination unit F3 may be configured to activate the foot position sensors in response to the accelerator pedal angle and the brake pedal angle becoming 0 or less than a predetermined value. This configuration can reduce the power consumption of the entire system and the processing load on the processor 31.

[0121] The preparatory action determination unit F3 may change the motion sensor 13 to be activated depending on the driving scene. Note that a certain sensor is in an active state when it is powered on and the preparatory action determination unit F3 uses the output value of the sensor to determine the preparatory action. A certain sensor is in an inactive state when it is not powered on or when the preparatory action determination unit F3 does not use the output of the sensor to determine the preparatory action.

[0122] For example, when the vehicle is traveling at a low speed, the change in pedal angle is small. Therefore, it may be difficult to detect preparatory actions from the pedal angle during low-speed traveling. For this reason, when the vehicle is traveling at a low speed, the preparatory action determination unit F3 may be configured to detect preparatory foot actions using the foot camera 13C, foot sonar, or surface pressure sensor 13D, without using the accelerator pedal sensor 13A or the brake pedal sensor 13B. In other words, when the vehicle is traveling at a low speed, the preparatory action determination unit F3 may determine whether preparatory actions have been performed based on time-series data of the right foot position, rather than time-series data of the pedal angle. This configuration can improve the accuracy of determining whether preparatory actions have been performed during low-speed traveling. In other words, it can reduce the risk of erroneously determining that preparatory actions have been performed when they have not been performed. It can also reduce the risk of erroneously determining that preparatory actions have not been performed when they have been performed. Note that, in the present disclosure, "low-speed traveling" may be interpreted as a state in which the traveling speed is less than a predetermined value (e.g., 10 km / h).

[0123] Furthermore, when the perimeter monitoring sensor 11 detects a view obstruction, such as an object installed near an intersection, the preparatory action determination unit F3 may determine whether preparatory action has been taken using the room camera 13F or the DMS 14, without using the foot sensor. The preparatory action determination unit F3 may determine that preparatory action has been taken when detecting, using the room camera 13F or the DMS 14, that the driver's upper body / head has moved forward. The above behavior corresponds to behavior for checking a blind spot. In other driving scenes, the foot sensor may be used to determine whether preparatory action has been taken. As described above, the processor 31 may change the sensor used to determine preparatory action depending on the driving scene. Driving scenes may be classified into low-speed driving, turning right or left, changing lanes, reversing, stopping, starting, and cruising. Cruising refers to a state in which the vehicle is traveling along a road. The processor 31 may determine that the vehicle is cruising when the turn signals are not activated and the vehicle speed is equal to or greater than a predetermined value. The processor 31 may determine the driving scene based on the output signal of the vehicle state sensor 12.

[0124] <Regarding Foot Position Correction> The position and angle of the driver's foot when stepping on the pedal may vary depending on the shape / type of shoes worn by the driver. The results of foot position detection by the foot camera 13C and the surface pressure sensor 13D may be affected by the shape (thickness) of the shoes. For this reason, the processor 31 may be configured to learn the foot positions when the pedal is being pressed and when the pedal is not being pressed from the driving history. The processor 31 may reflect the learning results in the estimation of the foot position as needed.

[0125] A predetermined time (e.g., 5 minutes) after the vehicle power is turned on may be a learning period. The learning period may be a period during which the processor 31 learns (generates) a reference / identification model for estimating the driver's foot position from the pressure distribution acting on the seating surface / images from the foot camera 13C. The processor 31 may learn the pressure distribution / image features when the pedal is depressed based on the pressure distribution / images when the pedal angle is equal to or greater than a predetermined value during the learning period. The processor 31 may also learn the pressure distribution / image features when the pedal is not depressed based on the pressure distribution / images when the pedal angle is 0.

[0126] During the learning period, the driving assistance ECU 30 may stop adjusting the warning timing based on the preparatory action. During the learning period, the driving assistance ECU 30 may be configured to output a warning sound at a standard timing. After the learning period ends, the driving assistance ECU 30 may be configured to perform control to change the warning timing based on whether or not a preparatory action has been taken. Alternatively, during the learning period, the driving assistance ECU 30 may be configured to determine whether or not a preparatory action has been taken without using a foot position sensor. This configuration reduces the risk of erroneous determination of whether or not a preparatory action has been taken due to individual differences, etc.

[0127] The processor 31 may set the warning timing earlier than the standard timing when the driver's line of sight is directed toward a dangerous direction but no preparatory action is taken. The dangerous direction here may be interpreted as the direction in which a dangerous object exists. The processor 31 may acquire the driver's line of sight from the output signal of the DSM 14. When the driver's line of sight is directed toward a dangerous direction but no preparatory action is taken, it is highly likely that the driver is absent-minded or thinking about something else.

[0128] Therefore, in the above case, safety can be improved by advancing the warning timing. Note that, when the driver's line of sight is not directed toward a dangerous direction and the driver is not taking preparatory action, the processor 31 may set the warning timing to the standard timing or to an earlier timing.

[0129] Even when a preparatory action is taken, if the driver's line of sight is not directed toward a danger direction before the preparatory action, the processor 31 may set the warning timing to the standard timing. This is because in such a case, it is unclear whether the detected preparatory action is an action taken after the driver has actually recognized a danger object. The processor 31 may be configured to set the warning timing later only when the driver's line of sight is directed toward a danger direction and the preparatory action is taken.

[0130] <Supplementary Information> Although the above describes an embodiment in which the timing of outputting an alarm sound is changed depending on whether or not a preparatory action has been taken, the timing of initiating automatic braking may also be changed depending on whether or not a preparatory action has been taken. The support unit F4 may be configured to initiate automatic braking earlier when no preparatory action has been taken within the determination period than when preparatory action has been taken within the determination period. Specifically, when no preparatory action by the driver is detected during the determination period, the support unit F4 initiates automatic braking when the TTC reaches a predetermined standard braking threshold. On the other hand, when the preparatory action determination unit F3 detects that a preparatory action has been taken during the determination period, the support unit F4 may initiate automatic braking when the TTC reaches a later braking threshold.

[0131] Both the standard braking threshold and the late braking threshold are thresholds (i.e., braking thresholds) for the TTC that define the timing for starting automatic braking. The standard braking threshold and the late braking threshold may be registered in advance in the storage 33 or the like. The late braking threshold is set to a value that is a predetermined amount smaller than the standard threshold. The difference between the late braking threshold and the standard braking threshold may be set to 0.2 seconds, 0.3 seconds, or the like.

[0132] Furthermore, if a preparatory action is taken during the determination period, the timing to start the automatic brake may be changed depending on the type (content) of the preparatory action taken. The timing to start the automatic brake may also be referred to as the braking start timing or the intervention timing.

[0133] The technical idea related to the control of the timing of outputting the warning sound described above may be applied to the control of the timing of starting automatic braking. The assist control is not limited to outputting a warning sound, but may be automatic braking. Furthermore, the assist control may automatically control the steering angle in a direction to avoid a collision. The processor 31 may be configured to change at least one of the timing of outputting the warning sound, the timing of starting automatic braking, and the timing of starting automatic steering, depending on the behavior of the driver during the determination period. The type / combination of assist controls whose execution timing is changed depending on the behavior of the driver during the determination period may be changed as appropriate. Note that the determination period (particularly the determination end value) may be changed depending on the assist control.

[0134] The processor 31 may change the operation pattern of the behavior control depending on whether or not a preparatory action has been performed during the determination period. The behavior control in the present disclosure may be interpreted as steering control and / or braking control for avoiding a collision of the vehicle with another object. The behavior control may be interpreted as automatic steering or automatic braking for collision avoidance or collision damage mitigation. The behavior control is one form of the above-mentioned assistance control. The behavior control may be rephrased as avoidance control, intervention control, automatic control, or the like.

[0135] Changing the operation pattern of the behavior control may mean changing the start timing of the behavior control, or changing the control amount (initial control amount) at the start of the control. Changing the operation pattern may mean changing both the start timing and the initial control amount. The control amount may be interpreted as the strength of the behavior control. If the behavior control is braking control, the control amount may be interpreted as the strength of the braking force. The strength of the braking force may be expressed by acceleration (deceleration). The strength of the braking force may be expressed by jerk. If the behavior control is steering control, the control amount may be interpreted as steering amount / steering speed.

[0136] The following describes a case where the behavior control is braking control. The following description may also be applied to a case where the behavior control is steering control. When the behavior control is braking control, the initial control amount may be called an initial braking force. Increasing the braking force may be understood as decreasing the acceleration in the negative region, in other words, increasing the deceleration.

[0137] In this disclosure, the standard start timing of braking control is referred to as the standard braking timing, and the standard value of the braking force generated by braking control is referred to as the standard braking force. The standard braking timing may be interpreted as the timing when the TTC reaches a predetermined standard braking threshold. The standard braking timing may be set to a value according to the vehicle type / weight of the vehicle, such as the point when the TTC reaches 1.2 seconds. The standard braking force may be set to -8 m / sec^2, for example.

[0138] The processor 31 may set the initial braking force to the standard braking force if preparatory action is being taken during the determination period, and may set the initial braking force to a value that is a predetermined amount greater than the standard braking force if preparatory action is not being taken. The initial braking force when preparatory action is being taken may be set to -10 m / sec^2. In this way, the processor 31 may increase the initial braking force of the braking control when preparatory action is not being taken during the determination period compared to when preparatory action is being taken. The start timing of the braking control may be the standard braking timing regardless of whether preparatory action is being taken or not.

[0139] Furthermore, when preparatory action is not taken during the determination period, the processor 31 may advance the timing of executing braking control and reduce the initial braking amount compared to when preparatory action is taken. When preparatory action is taken, the processor 31 applies standard braking timing and standard braking force. On the other hand, when preparatory action is not taken, the processor 31 may set the braking start timing to an early timing and the braking force to a weak level. The early timing of braking control may be 0.4 seconds or 0.8 seconds earlier than the standard braking timing. The early timing of braking control may also coincide with the timing of outputting an alarm sound. The weak level of braking force may be set to a value that is a predetermined amount smaller than the standard braking force. The weak level may be -2 m / sec^2, -3 m / sec^2, -4 m / sec^2, etc. A weak level braking force may be rephrased as a weak braking force. Furthermore, braking control at a weak level may be rephrased as weak braking.

[0140] In addition, when the processor 31 is executing weak-level braking control, if the TTC reaches a predetermined emergency value, the processor 31 may switch the braking force to a strong level. The strong level may be set to the same as the standard braking force or a value greater than the standard braking force by a predetermined amount. The emergency value may be set to, for example, 1.2 seconds, 1.0 second, or 0.8 seconds. The strong level braking force may be referred to as forced force. Furthermore, the strong level braking control may be referred to as forced action.

[0141] As described above, the processor 31 may be configured to apply the brakes lightly and early if a preparatory action by the driver is not detected. This configuration can further improve safety.

[0142] Furthermore, the processor 31 may change the warning timing in combination with the change in behavior control. When preparatory action is not being taken, the processor 31 may start gentle braking (i.e., gentle braking) simultaneously with the output of the warning sound. The start timing of the warning sound and gentle braking when preparatory action is not being taken may be the same as the standard timing of the warning sound, or may be set a predetermined amount earlier than the standard timing.

[0143] <Supplementary Remark (1)> The present disclosure also includes the following technical ideas: In addition, the present disclosure also includes a driving assistance ECU, a driving assistance method, and a program corresponding to the driving assistance system described below.

[0144] [Technical Idea 1] A driving assistance system comprising: an external sensor (11) that detects the environment surrounding a vehicle; a motion sensor (13) that detects the movement of a driver; and a control unit (31) that executes assistance control to avoid the vehicle from colliding with another object, wherein the control unit is configured to: determine, based on an output signal of the external sensor, whether or not the current situation is a dangerous situation in which there is a possibility of collision with the other object; execute the assistance control based on the determination that the situation is dangerous; determine, based on the output signal of the motion sensor, whether or not the driver has taken preparatory action to avoid a collision; and change the execution timing of the assistance control depending on whether or not the preparatory action has been taken within a predetermined determination period.

[0145] [Technical Idea 2] The motion sensor is configured to be able to detect a plurality of types of the preparatory behavior, and the control unit is configured to advance the timing of executing the assistance control when the preparatory behavior is not performed during the determination period compared to when the preparatory behavior is performed during the determination period. This is the driving assistance system described in Technical Idea 1.

[0146] [Technical Idea 3] The driving assistance system according to Technical Idea 1, wherein the motion sensor is configured to be able to detect a plurality of types of the preparatory actions, and the control unit is configured to set the execution timing of the assistance control later when any of the preparatory actions has been performed during the determination period than when none of the preparatory actions has been performed during the determination period, and to change the degree to which the execution timing is delayed depending on the type of the preparatory action performed by the driver.

[0147] [Technical Idea 4] A danger perception level is set for each of the preparatory actions, and when the preparatory action is performed during the judgment period, the driving assistance system described in Technical Idea 3 is configured to delay the execution timing as the danger perception level set for the performed preparatory action becomes higher.

[0148] [Technical Idea 5] The driving assistance system described in any one of Technical Ideas 1 to 4, wherein the motion sensor includes a foot sensor (13X) that generates and outputs data regarding the movement of the driver's feet, and the control unit is configured to determine whether the preparatory action has been performed based on the data regarding the movement of the feet output by the foot sensor.

[0149] [Technical Idea 6] The driving assistance system according to Technical Idea 5, wherein the foot sensor is a sensor that generates data indicating the amount of depression of an accelerator pedal and the amount of depression of a brake pedal, and the control unit is configured to determine that the preparatory action has been performed if the amount of depression of the accelerator pedal or the brake pedal changes in a predetermined pattern during the determination period.

[0150] [Technical Idea 7] The driving assistance system according to Technical Idea 6, wherein the foot sensor includes a foot position sensor that detects the position of the foot, and the control unit is configured to determine that the preparatory action has been performed when the position of the foot changes in a predetermined pattern during the determination period.

[0151] [Technical Idea 8] The control unit is configured to: activate the foot position sensors when the depression amounts of the accelerator pedal and the brake pedal are equal to or less than a predetermined value, and determine whether or not the preparatory action has been taken using the detection results of the foot position sensors; and, when the depression amounts of the accelerator pedal or the brake pedal exceed the predetermined value, determine whether or not the preparatory action has been taken based on time-series data of the depression amounts of the accelerator pedal and the brake pedal without using the detection results of the foot position sensors. This is the driving assistance system described in Technical Idea 7.

[0152] [Technical Idea 9] A driving assistance system described in any one of Technical Ideas 5 to 8, wherein the foot sensor includes a foot camera that is a camera that captures an image of an area including a brake pedal and an accelerator pedal, and the control unit is configured to acquire data indicating the movement of the feet by analyzing the image of the foot camera.

[0153] [Technical Idea 10] The driving assistance system described in any one of Technical Ideas 1 to 9, wherein the motion sensor includes a pressure sensor (13D) that detects pressure acting on the seating surface of the driver's seat, and the control unit is configured to determine that the preparatory action has been performed when the pressure acting on the seating surface changes in a predetermined pattern.

[0154] [Technical Idea 11] The driving assistance system described in any one of Technical Ideas 1 to 10, wherein the motion sensor includes a head sensor (13F) that generates and outputs data regarding the movement of the driver's head, and the control unit is configured to determine whether the head has moved in a predetermined pattern during the determination period based on the data regarding the head movement output by the head sensor, and to determine that the preparatory action has been performed if the head has moved in the predetermined pattern during the determination period.

[0155] [Technical Idea 12] The driving assistance system described in any one of Technical Ideas 1 to 11, wherein the motion sensor includes a hand sensor (13E) that generates and outputs data regarding the movement of the driver's hands, and the control unit is configured to determine whether the hands have moved in a predetermined pattern during the determination period based on the data regarding the hand movement output by the hand sensor, and to determine that the preparatory action has been performed if the hands have moved in the predetermined pattern during the determination period.

[0156] [Technical Idea 13] A driving assistance system according to any one of Technical Ideas 1 to 12, further comprising a gaze detector that detects the gaze direction of the driver, wherein the control unit is configured to be able to set the timing for executing the assistance control to standard, delayed, or earlier, and is configured to set the execution timing of the assistance control later when a specific preparatory action has been executed, while setting the execution timing earlier when the gaze direction is directed in a direction in which there is an object that is the other object that may collide with the vehicle and the preparatory action has not been detected.

[0157] [Technical Idea 14] A driving assistance system according to any one of Technical Ideas 1 to 13, comprising: a plurality of types of motion sensors; and a vehicle state sensor (12) that generates and outputs data indicating the state of the vehicle, wherein the control unit is configured to determine a driving scene from the output signal of the vehicle state sensor, and change the motion sensor to be activated depending on the driving scene.

[0158] [Technical Idea 1A] A driving assistance system comprising: an external sensor (11) that detects the environment surrounding a vehicle; a motion sensor (13) that detects the movement of the driver; and a control unit (31) that executes behavior control, which is steering control or braking control, to avoid the vehicle from colliding with another object, wherein the control unit is configured to: determine, based on an output signal from the external sensor, whether or not the current situation is a dangerous situation in which there is a possibility of collision with the other object; execute the behavior control based on the determination that the situation is dangerous; determine, based on the output signal from the motion sensor, whether or not the driver has taken preparatory action to avoid a collision; and change an operation pattern of the behavior control depending on whether or not the preparatory action has been taken within a predetermined determination period.

[0159] [Technical Idea 1B] A driving assistance method for avoiding a collision of a vehicle with another object, comprising: determining whether or not a current situation is a dangerous situation in which there is a possibility of collision with the other object, based on an output signal from an external sensor that detects the surrounding environment of the vehicle; implementing behavior control, which is steering control or braking control, to avoid the vehicle from colliding with the other object, based on the determination that the situation is dangerous; determining whether or not the driver has taken preparatory action to avoid a collision, based on an output signal from a motion sensor that detects movement of the driver; and changing an operation pattern of the behavior control depending on whether or not the preparatory action has been taken within a predetermined determination period.

[0160] [Technical Idea 1C] A program including instructions for causing a computer to execute the following: determining whether the current situation is a dangerous situation where there is a possibility of collision with another object, based on input signals from external sensors that detect the vehicle's surrounding environment; implementing behavior control, which is steering control or braking control, to avoid the vehicle from colliding with another object, based on the determination that the situation is dangerous; determining whether the driver has taken preparatory action to avoid a collision, based on input signals from a motion sensor that detects the driver's movement; and changing the operation pattern of the behavior control depending on whether the preparatory action has been taken within a predetermined determination period.

[0161] <Supplementary Note (2)> The various flowcharts shown in this disclosure are all examples, and the number of steps constituting the flowcharts and the execution order of the processes can be changed as appropriate. The controls shown in each flowchart may be combined / executed in parallel to the extent that there is no contradiction. Expressions such as acquisition, determination, detection, generation, and calculation may be interchangeable. When a device acquires certain data, it also includes the device generating the data based on a signal input from another device / sensor.

[0162] The apparatus, system, and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to execute one or more functions embodied in a computer program. The apparatus and methods described herein may be implemented using dedicated hardware logic circuits. The apparatus and methods described herein may be implemented by one or more special-purpose computers configured by combining a processor executing a computer program with one or more hardware logic circuits. The processor may be any computing core, such as a CPU, MPU, GPU, or DFP (Data Flow Processor). Some or all of the functions of the processor 31 may be implemented as hardware. Some or all of the functions of the processor 31 may be implemented using any of a system-on-chip (SoC), an integrated circuit (IC), and a field-programmable gate array (FPGA).

[0163] The computer program includes instructions that are executed by a computer. The computer program may be stored in a computer-readable non-transitory tangible storage medium. The storage medium for the computer program may be a variety of media, such as a hard-disk drive (HDD), a solid-state drive (SSD), or a flash memory.

Claims

1. An external sensor (11) for detecting the surrounding environment of the vehicle; a plurality of motion sensors (13) for detecting the driver's movements; a control unit (31) that executes assistance control to prevent the vehicle from colliding with another object, The control unit determining whether a current situation is a dangerous situation in which there is a possibility of collision with the other object based on an output signal from the external sensor; performing the assistance control based on the determination of the dangerous situation; determining whether the driver has taken preparatory action to avoid a collision based on an output signal of the motion sensor; changing the execution timing of the assistance control depending on whether the preparatory action has been performed within a predetermined determination period; and changing the combination of the motion sensors used to determine the preparatory behavior depending on the driving scene.

2. The motion sensor is configured to be able to detect a plurality of types of the preparatory behavior, A risk perception level is set for each of the preparatory actions, The control unit 2. The driving assistance system according to claim 1, wherein, when the preparatory action is performed during the determination period, the execution timing is delayed as the danger perception level set for the performed preparatory action becomes higher.

3. The plurality of motion sensors includes a foot sensor (13X) that generates and outputs data regarding the driver's foot movements; The foot sensor includes an accelerator pedal sensor (13A) that generates data indicating the depression amount of an accelerator pedal, a brake pedal sensor (13B) that generates data indicating the depression amount of a brake pedal, and a foot position sensor (13C) that detects the position of the foot, The control unit When the depression amounts of the accelerator pedal and the brake pedal are equal to or less than a predetermined value, the foot position sensors are enabled, and it is determined whether or not the preparatory action has been performed using the detection results of the foot position sensors; 2. The driving assistance system according to claim 1, wherein, when the depression amount of the accelerator pedal or the brake pedal exceeds the predetermined value, it is determined whether or not the preparatory action has been taken based on time-series data of the depression amount of the accelerator pedal and the brake pedal, without using the detection result of the foot position sensor.

4. The motion sensor includes a foot sensor (13X) that generates and outputs data regarding the driver's foot movements; the foot sensor includes a foot camera that is a camera that captures an image of an area including a brake pedal and an accelerator pedal, The control unit acquiring data indicating the movement of the foot by analyzing the image of the foot camera; The driving assistance system according to claim 1 , wherein the driving assistance system is configured to determine whether the preparatory action has been performed based on the data indicating the foot movement.

5. The motion sensor includes a pressure sensor (13D) that detects pressure acting on a seating surface of the driver's seat, The control unit The driving assistance system according to claim 1 , wherein the system is configured to determine that the preparatory action has been taken when the pressure acting on the seating surface changes in a predetermined pattern.

6. the motion sensor includes a head sensor (13F) that generates and outputs data relating to the movement of the driver's head; The control unit determining whether the head has moved in a predetermined pattern during the determination period based on the data relating to the head movement output by the head sensor; The driving assistance system according to claim 1 , wherein the system is configured to determine that the preparatory behavior has been performed when the head moves in the predetermined pattern during the determination period.

7. the motion sensor includes a hand sensor (13E) that generates and outputs data relating to the driver's hand movements; The control unit determining whether the hand moved in a predetermined pattern during the determination period based on the data relating to the hand movement output by the hand sensor; The driving assistance system according to claim 1 , wherein the system is configured to determine that the preparatory behavior has been performed when the hand moves in the predetermined pattern during the determination period.

8. a gaze detector for detecting a gaze direction of the driver; The control unit The timing of executing the assistance control can be set to standard, delayed, or advanced. When the specific preparatory action is performed, the execution timing of the assistance control is set to be later, 2. The driving assistance system according to claim 1, wherein the execution timing is set earlier when the line of sight is directed in a direction in which an object that is the other object with which the vehicle may collide is present and the preparatory behavior is not detected.

9. On the computer, determining whether the current situation is a dangerous situation in which there is a possibility of collision with another object, based on an input signal from an external sensor that detects the surrounding environment of the vehicle; Based on the determination of the dangerous situation, executing an assistance control to avoid the vehicle from colliding with another object; determining whether the driver has taken preparatory action to avoid a collision based on input signals from a plurality of motion sensors that detect driver movements; changing the execution timing of the assistance control depending on whether the preparatory action has been performed within a predetermined determination period; and changing the combination of the motion sensors used to determine the preparatory behavior depending on the driving scene.

10. a communication unit (34) for communicating with other devices; a control unit (31) that executes assistance control to prevent the vehicle from colliding with another object based on the data received by the communication unit, The control unit acquiring a detection result of an external sensor that detects a surrounding environment of the vehicle via the communication unit; determining whether or not a dangerous situation exists in which there is a possibility of a collision with the other object based on an output signal from the external sensor; performing the assistance control based on the determination of the dangerous situation; acquiring detection results of a plurality of motion sensors that detect driver movements via the communication unit; determining whether the driver has taken preparatory action to avoid a collision based on the detection result of the motion sensor; changing the execution timing of the assistance control depending on whether the preparatory action has been performed within a predetermined determination period; and changing the combination of the motion sensors used to determine the preparatory behavior depending on the driving scene.

11. a communication unit (34) for communicating with other devices; a control unit (31) that executes behavior control, such as steering control or braking control, to avoid the vehicle colliding with another object based on the data received by the communication unit, The control unit acquiring a detection result of an external sensor that detects a surrounding environment of the vehicle via the communication unit; determining whether or not a dangerous situation exists in which there is a possibility of a collision with the other object based on an output signal from the external sensor; starting the behavior control based on the determination that the situation is dangerous; acquiring a detection result of a motion sensor that detects a driver's movement via the communication unit; determining whether the driver has taken preparatory action to avoid a collision based on the detection result of the motion sensor; and changing the intensity of the behavior control depending on whether or not the preparatory behavior has been performed within a predetermined determination period.

12. An external sensor (11) for detecting the surrounding environment of the vehicle; a motion sensor (13) for detecting the driver's movement; a control unit (31) that executes behavior control, which is steering control or braking control, to avoid the vehicle from colliding with another object, The control unit determining whether a current situation is a dangerous situation in which there is a possibility of collision with the other object based on an output signal from the external sensor; starting the behavior control based on the determination that the situation is dangerous; determining whether the driver has taken preparatory action to avoid a collision based on an output signal of the motion sensor; and changing the intensity of the behavior control depending on whether or not the preparatory behavior has been performed within a predetermined determination period.

13. On the computer, determining whether the current situation is a dangerous situation in which there is a possibility of collision with another object, based on an input signal from an external sensor that detects the surrounding environment of the vehicle; Based on the determination of the dangerous situation, starting a behavior control such as a steering control or a braking control to avoid the vehicle from colliding with another object; determining whether the driver has taken preparatory action to avoid a collision based on an input signal from a motion sensor that detects a movement of the driver; and changing the intensity of the behavior control depending on whether the preparatory behavior has been performed within a predetermined determination period.