Driver assistance device and method thereof

Abstract

Driver assistance device and method thereof. A driver assistance device includes: an external camera arranged on a vehicle to have an external view of the vehicle, the external camera configured to obtain image data of the external view of the vehicle; an internal camera arranged inside the vehicle to capture the gaze of a driver on the vehicle, the internal camera configured to obtain the gaze data of the driver; and a controller including at least one processor configured to process the image data and the gaze data. The controller may be configured to capture the gaze of the driver based on the gaze data, and based on a determination that the driver is in a negligent condition, control at least one of the control timing of a steering device and a lateral distance limit value for operating the steering device to be changed.

Description

Technical Field

[0001] The present disclosure relates to a driver assistance device, and more particularly, to a driver assistance device capable of reflecting the driver's attention state. Background Art

[0002] Existing advanced driver assistance systems (ADAS) are controlled based on lane information outside the vehicle and object information outside the vehicle. For example, a lane keeping assist system (LKAS) is controlled based on object information and lane information obtained from an image outside the vehicle to keep the lane. In addition, an automatic emergency braking system prevents collisions based on object information outside the vehicle and the driver's vehicle movement information.

[0003] However, because existing ADAS do not consider the driver's condition, they are insufficient to prevent accidents except in typical situations. Summary of the Invention

[0004] One aspect of the present disclosure is to provide a driver assistance device that reflects the driver's condition, and a method of controlling the driver assistance device.

[0005] Other aspects of the disclosure will be partially set forth in the following description, and partially will be obvious from the description, or may be learned from the practice of the disclosure.

[0006] According to one aspect of the present disclosure, a driver assistance device includes: an external camera disposed on a vehicle to have an external view of the vehicle, the external camera configured to obtain image data of the external view of the vehicle; an internal camera disposed inside the vehicle to capture the driver's gaze on the vehicle, the internal camera configured to obtain the driver's gaze data; and a controller including at least one processor configured to process the image data and the gaze data. The controller may be configured to capture the driver's gaze based on the gaze data, and based on a determination that the driver is in a negligent condition, control at least one of a control timing of a steering device and a lateral distance limit value for operating the steering device to be changed.

[0007] When the driver's gaze is not in the forward direction, the controller may be configured to determine that the driver is in the negligent condition, and control the control timing of the steering device to be advanced by a preset time compared to an existing control timing.

[0008] When the driver's gaze is not in the forward direction, the controller may be configured to determine that the driver is in the negligent condition, and change the lateral distance limit value to a value smaller than an existing lateral distance limit value.

[0009] When an object detected from outside the vehicle is present within the driver's line of sight area, the controller may be configured to determine that the driver is not in the inattentive condition and not intervene in the control of the steering device.

[0010] When an object detected from outside the vehicle is not present within the driver's line of sight area, the controller may be configured to determine that the driver is in the inattentive condition.

[0011] The interior camera may be configured to obtain the driver's line of sight data based on the driver's face direction and the driver's pupil direction.

[0012] The driver assistance device may further include a steering device configured to change the driving direction of the vehicle. The steering device may be configured to change the driving direction of the vehicle according to a control signal changed by the controller.

[0013] The driver assistance device may further include a braking device configured to decelerate the vehicle or stop the vehicle. When the driver's line of sight is not in the forward direction, the controller may be configured to determine that the driver is in the inattentive condition and control the control timing of the braking device to be advanced by a preset time compared to the existing control timing.

[0014] The driver assistance device may further include a braking device configured to decelerate the vehicle or stop the vehicle. When the driver's line of sight is not in the forward direction, the controller may be configured to determine that the driver is in the inattentive condition and control the braking torque amount of the braking device to be higher than the existing braking torque amount.

[0015] According to another aspect of the present disclosure, a method of controlling a driver assistance device includes: obtaining image data of an external view of a vehicle by an external camera; obtaining the driver's line of sight data of a driver on the vehicle by an interior camera to capture the driver's line of sight; and capturing the driver's line of sight by a controller based on the line of sight data and controlling at least one of the control timing of a steering device and a lateral distance limit value for operating the steering device to be changed based on a determination that the driver is in an inattentive condition.

[0016] The control may include: when the driver's line of sight is not in the forward direction, determining that the driver is in the inattentive condition and controlling the control timing of the steering device to be advanced by a preset time compared to the existing control timing.

[0017] The control may include: when the driver's gaze is not in the forward direction, determining that the driver is in the negligent condition, and changing the lateral distance limit value to a value smaller than the existing lateral distance limit value.

[0018] The control may include: when an object detected from outside the vehicle is present in the driver's gaze area, determining that the driver is not in the negligent condition, and not intervening in the control of the steering device.

[0019] The control may include: when an object detected from outside the vehicle is not present in the driver's gaze area, determining that the driver is in the negligent condition.

[0020] The gaze data may include information about the driver's face direction and the driver's pupil direction.

[0021] The method may further include a steering device configured to change the traveling direction of the vehicle. The control may include: controlling the steering device according to the changed control signal to change the traveling direction of the vehicle.

[0022] The method may further include a braking device configured to decelerate the vehicle or stop the vehicle. The control may include: when the driver's gaze is not in the forward direction, determining that the driver is in the negligent condition, and controlling the control timing of the braking device to be advanced by a preset time compared to the existing control timing.

[0023] The method may further include a braking device configured to decelerate the vehicle or stop the vehicle. The control may include: when the driver's gaze is not in the forward direction, determining that the driver is in the negligent condition, and controlling the braking torque amount of the braking device to be higher than the existing braking torque amount.

[0024] According to another aspect of the present disclosure, a non-transitory computer-readable medium including program instructions executed by a processor, the computer-readable medium includes: program instructions for obtaining image data of an external view of a vehicle by means of an external camera; program instructions for obtaining gaze data of a driver on the vehicle by means of an internal camera to capture the driver's gaze; and program instructions for capturing the driver's gaze based on the gaze data by means of a controller, and controlling at least one of the control timing of a steering device and a lateral distance limit value for operating the steering device to be changed based on a determination that the driver is in a negligent condition. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] These and / or other aspects of the present disclosure will become apparent and more readily appreciated from the following description taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 is a block diagram showing components of a vehicle according to one embodiment;

[0027] Figure 2 is a block diagram showing a driving assistance system according to one embodiment;

[0028] Figure 3 is a flowchart showing a vehicle control method according to one embodiment;

[0029] Figure 4 is a flowchart showing a vehicle control method according to another embodiment;

[0030] Figure 5 and Figure 6 is a view for describing the calculation of a lateral position error;

[0031] Figure 7 is a view showing an example of changing the control of a braking device. Detailed Embodiments

[0032] Throughout the specification, the same reference numerals refer to the same elements. Not all elements of the disclosed embodiments will be described, and descriptions of what is commonly known in the prior art or what overlaps in the exemplary embodiments will be omitted.

[0033] It will be further understood that the terms "connected" and its derivatives refer to both direct connections and indirect connections, and indirect connections include connections via a wireless communication network.

[0034] The terms "comprising" and "including" are inclusive or open-ended and do not exclude additional, unmentioned elements or method steps unless otherwise stated. It will be further understood that the terms "member" and its derivatives refer to both the case where one member contacts another member and the case where another member exists between two members.

[0035] It is to be understood that, unless the context clearly dictates otherwise, the singular forms include the plural meanings.

[0036] As used herein, the terms "part", "unit", "block", "member", and "module" refer to a unit capable of performing at least one function or operation. These terms may refer to an electrical circuit and may refer to at least one process that is executed by at least one hardware such as a field programmable gate array (FPGA) and an application specific integrated circuit (ASIC) and at least one software stored in a memory or a processor.

[0037] The reference numerals used for the method steps are only for convenience of explanation and are not intended to limit the order of the steps. Therefore, unless the context clearly dictates otherwise, the written order may be practiced in other ways.

[0038] Hereinafter, embodiments of a driver assistance apparatus and a method of controlling the same will be described in detail with reference to the accompanying drawings.

[0039] Figure 1 is a block diagram illustrating components of a vehicle according to one embodiment.

[0040] like Figure 1 As shown in FIG, vehicle 1 may include an engine 10, a transmission 20, a braking device 30, and a steering device 40. Engine 10 may include at least one cylinder and at least one piston and may generate the power required to drive vehicle 1. Transmission 20 may include multiple gears and may transmit the power generated by engine 10 to the wheels of vehicle 1. Braking device 30 may decelerate or stop vehicle 1 by utilizing the friction force on the wheels. Braking device 30 may include one or more brake pads and brake shoes operable to decelerate or stop the vehicle. Steering device 40 may change the direction of travel of vehicle 1.

[0041] The vehicle 1 may include a plurality of electronic components. For example, the vehicle 1 may include an engine management system (EMS) 11, a transmission controller 21 also known as a transmission control unit (TCU), an electronic brake controller 31 also known as an electronic brake control module (EBCM), an electronic power steering (EPS) device 41, a body control module (BCM), and a driver assistance device 100.

[0042] The EMS 11 may control the engine 10 in response to the driver's acceleration intention from the accelerator pedal or a request signal from the driver assistance device 100. For example, the EMS 11 may control the torque of the engine 10.

[0043] The TCU 21 may control the transmission 20 in response to a driver's shift command activated by a shift lever and / or a driving speed of the vehicle 1. For example, the TCU 21 may adjust or regulate a shift ratio from the engine 10 to the wheels of the vehicle 1.

[0044] The EBCM 31 can control the braking device 30 in response to the driver's braking intention from the brake pedal or wheel slip. For example, the EBCM 31 can temporarily release the wheel brakes in response to wheel slip detected in the braking mode of the vehicle 1, thereby implementing an anti-lock braking system (ABS). The EBCM 31 can selectively release the wheel brakes in response to oversteering and / or understeering detected in the steering mode of the vehicle 1, thereby implementing an electronic stability control (ESC). In addition, the EBCM 31 can temporarily brake the wheels in response to wheel slip detected by vehicle travel, thereby implementing a traction control system (TCS).

[0045] The EPS device 41 can assist the steering device 40 in response to the driver's steering intention from the steering wheel, so that the EPS device 41 can assist the driver in easily manipulating the steering wheel. For example, the EPS device 41 can assist the steering wheel (steering device) 40 such that the steering force is reduced in the low-speed driving mode or parking mode of the vehicle 1 and increased in the high-speed driving mode of the vehicle 1.

[0046] The body control module 51 can control various electronic components that can provide user convenience for the driver or ensure the driver's safety. For example, the body control module 51 can control headlights (front lights), windshield wipers, gauges or other instrument panels, multifunction switches, turn signal indicators or similar devices.

[0047] The driver assistance device 100 can assist the driver in easily manipulating (e.g., driving, braking, and steering) the vehicle 1. For example, the driver assistance device 100 can detect the surrounding environment of the vehicle 1 (i.e., the vehicle itself) (e.g., surrounding vehicles, pedestrians, cyclists, lanes, traffic signs, etc.), and can drive, brake, and / or steer the vehicle 1 in response to the detected surrounding environment.

[0048] The driver assistance device 100 can provide various functions for the driver. For example, the driver assistance device 100 can provide a lane departure warning (LDW) function, a lane keeping assist (LKA) function, a high beam assist (HBA) function, an autonomous emergency braking (AEB) function, a traffic sign recognition (TSR) function, a smart cruise control (SCC) function, a blind spot detection (BSD) function, etc.

[0049] The driver assistance device 100 can include: a camera module 101 that operates to obtain image data of the surrounding area of the vehicle 1 (e.g., the exterior and surrounding area of the vehicle 1); and a radar module 102 that operates to obtain data on surrounding objects existing in the surrounding area of the vehicle 1.

[0050] The camera module 101 may include one camera 101a or multiple cameras, and an electronic control unit (ECU) controller 101b. The camera 101a may capture an image including the forward area of the vehicle 1 (e.g., the area in front of the vehicle 1), and may include an image processor that is operated to process the captured image to identify surrounding vehicles, pedestrians, cyclists, lanes, traffic signs, etc. in the captured image.

[0051] The radar module 102 may include one radar 102a or multiple radars, and an electronic control unit (ECU) controller 102b, and may obtain or determine the relative position, relative speed, etc. of surrounding objects (e.g., surrounding vehicles, pedestrians, or cyclists) of the vehicle 1 based on the sensed radar data.

[0052] The above-mentioned electronic components may communicate with each other via a vehicle communication network (NT). For example, the electronic components may perform data communication via Ethernet, media-oriented system transport (MOST), FlexRay, controller area network (CAN), local interconnection network (LIN), etc. For example, the driver assistance device 100 may transmit drive control signals, brake signals, and steering signals to the EMS 11, EBCM 31, and EPS device 41 via the vehicle communication network (NT), respectively.

[0053] Figure 2 is a block diagram showing a driver assistance system according to an embodiment.

[0054] The vehicle 1 may include: an external camera 100 for obtaining an external view of the vehicle 1 and capturing objects and roads in the external view; an internal camera 200 for obtaining the gaze data of the driver; a vehicle information acquirer 300 for obtaining the state of the vehicle 1; a controller 400; a braking device 30; a steering device 40; and a warning device 500.

[0055] The external camera 100 may include a field of view (FOV) 110a facing the forward area of the vehicle 1. The external camera 100 may be mounted at the windshield of the vehicle 1.

[0056] The external camera 100 may capture an image of the forward area of the vehicle 1, and may obtain data of the front view image of the vehicle 1. The front view image data of the vehicle 1 may include information about the positions of surrounding vehicles, pedestrians, cyclists, or lanes located in the forward area of the vehicle 1.

[0057] The external camera 100 may include multiple lenses and multiple image sensors. Each image sensor may include multiple photodiodes to convert light into an electrical signal, and the photodiodes may be arranged in a two-dimensional (2D) matrix.

[0058] The external camera 100 can be electrically connected to the processor or controller 140. For example, the external camera 100 can be connected to the controller 140 via a vehicle communication network (NT), a hard wire, or a printed circuit board (PCB).

[0059] The external camera 100 can transmit the forward view image data of the vehicle 1 to the controller 140.

[0060] The internal camera 200 can be a camera installed inside the vehicle 1 to capture the driver's face. Thus, the internal camera 200 can obtain the gaze data by capturing the driver's face. The internal camera 200 can include various capturing devices capable of providing data for image processing by obtaining the driver's face image. In addition, the internal camera 200 can be an infrared (IR) camera so as to accurately obtain the driver's face image even during night driving. The internal camera 200 can be installed at various positions inside the vehicle 1, but for the convenience of capturing the driver's gaze, it can be installed in a single-row space (such as the dashboard and the center console) having a driver's seat and a passenger seat.

[0061] The internal camera 200 can transmit the driver's gaze data inside the vehicle 1 to the controller 400.

[0062] The vehicle information acquirer 300 can include at least one of various sensors capable of obtaining information related to the speed or posture of the vehicle 1. The vehicle information acquirer 300 can include: a vehicle speed sensor for detecting the speed of the vehicle 1; an acceleration sensor for detecting the acceleration of the vehicle 1; and an angular velocity sensor for detecting the angular velocity of the vehicle 1.

[0063] The vehicle speed sensor can be implemented as a wheel speed sensor for detecting the wheel speed, and the wheel speed sensor can include a wheel speed sensor for detecting the speed of at least one of the front wheels and the rear wheels of the vehicle 1.

[0064] The acceleration sensor can include: a longitudinal acceleration sensor that outputs the acceleration in the height direction of the vehicle 1 (i.e., the acceleration in the Z-axis direction); and a lateral acceleration sensor that outputs the acceleration in the lateral direction of the vehicle 1 (i.e., the acceleration in the Y-axis direction).

[0065] The longitudinal acceleration sensor and the lateral acceleration sensor can be implemented as separate sensor modules, or can be implemented as one sensor module. The angular velocity sensor can be a sensor that measures the attitude of the vehicle 1, and can also be referred to as a gyro sensor. The angular velocity sensor can include: a roll rate sensor that detects the rotational angular velocity of the vehicle 1 in the roll direction; and a yaw rate sensor that detects the rotational angular velocity of the vehicle 1 in the yaw direction. The roll rate sensor and the yaw rate sensor can also be implemented as separate sensor modules, or can be implemented as one sensor module.

[0066] The braking system 32 can include an EBCM 31 (see Figure 1 ) and a braking device 30 (see Figure 1 ). The steering system 42 can include an EPS device 41 (see Figure 1 ) and a steering device 40 (see Figure 1 ).

[0067] The braking device 30 can be connected to the EBCM 31 described with reference to Figure 1 (see Figure 1 ), and the steering device 40 can be connected to the EPS device 41 described with reference to Figure 1 (see Figure 1 ).

[0068] When operating under the control of the controller 400, the warning device 500 can notify the driver to start the control operation. The warning device 500 can notify the driver of the collision risk by stimulating at least one of the driver's vision, hearing, and touch.

[0069] For example, the warning device 500 can provide a display and a warning sound to the driver through the human-machine interface (HMI) provided in the vehicle 1, and can also provide a pre-warning signal or a warning signal to the driver through the vibration generated by the tactile module provided in the vehicle 1.

[0070] The controller 400 can include a processor 410 and a memory 420.

[0071] The processor 410 can process the image data of the external camera 100 and the gaze data of the internal camera 200, and can generate a braking signal for controlling the braking device 30 and a steering signal for controlling the steering device 40.

[0072] The processor 410 can calculate the time to collision (TTC) between the vehicle 1 and the preceding object based on the position information (distance) and speed information (relative speed) of each preceding object, can compare the calculated TTC with a predetermined reference time, and can warn the driver of a high possibility of collision with the preceding object, or can transmit the braking signal to the braking device 30 and the steering device 40.

[0073] In another embodiment, the processor 410 may calculate a collision distance (DTC) based on the speed information of each preceding object, and may warn the driver of a high likelihood of collision with the preceding object according to the comparison result between the calculated DTC and the distance to each preceding object, or may transmit control signals to the braking device 30 and the steering device 40.

[0074] The memory 420 may store programs and / or data that allow the processor 410 to process image data, may store programs and / or data that the processor 410 needs to process radar data, and may store programs and / or data that the processor 410 needs to generate braking signals and / or steering signals.

[0075] The memory 420 may temporarily store the image data received from the external camera 100 and / or the gaze data received from the internal camera 200, and may also temporarily store the processing results of the image data and / or the gaze data processed by the processor 410.

[0076] The memory 420 may include not only volatile memories such as static random access memory (SRAM) or dynamic random access memory (DRAM), but also non-volatile memories such as flash memory, read-only memory (ROM), or erasable programmable read-only memory (EPROM).

[0077] Next, with reference to Figure 3 and Figure 4 the control method performed by the components of the vehicle 1 described above will be described in detail.

[0078] Figure 3 is a flowchart showing a vehicle control method according to an embodiment.

[0079] The controller 400 may control the external camera 100 to capture the external view of the vehicle 1, and may control the internal camera 200 to monitor the gaze of the driver inside the vehicle 1 (301). Then, the controller 400 may receive the image data and the gaze data obtained in operation 301. Here, the internal camera 200 may obtain the driver's gaze data based on the driver's face direction and the driver's pupil direction.

[0080] The controller 400 determines whether there is a risk of lane departure of the vehicle 1 based on the analysis of the image data (302). As a determination result, when there is no risk of lane departure, the controller 400 may continuously monitor the external view and the driver's gaze.

[0081] When there is a risk of lane departure, the controller 400 may determine whether the driver is in a negligent condition (303). Here, the driver's negligent condition may be a situation where the driver's gaze is not facing forward. In addition, in addition to the above situation, the driver's negligent condition may include all situations where it is impossible to normally observe the driver's condition. For example, the driver's pupils are not visible or the eyelid movement is different from usual. In this case, the controller 400 may store the driver's gaze image in the memory 420 and compare the driver's negligent condition with the stored image to determine the driver's negligent condition.

[0082] As a result of determining whether the driver is in a negligent condition, when the driver's condition is normal, the controller 400 may maintain the existing control state without changing the cut-in point or control reference of the steering control (305).

[0083] According to one embodiment, when an object detected from outside the vehicle 1 exists in the driver's gaze area, the controller 400 may determine that the driver is not in a negligent condition and may control the steering device 40 not to intervene. For example, the gaze area may be a region with a constant size centered on the driver's gaze direction. In this case, the gaze area may be various virtual areas such as circular or elliptical. The size of the gaze area may be based on various set dimensions.

[0084] According to another embodiment, when an object detected from outside the vehicle 1 does not exist in the driver's gaze area, the controller 400 may determine that the driver is in a negligent condition.

[0085] When the driver is in a negligent condition, the controller 400 may control to advance the control timing of the steering device 40 (304) and may control the steering control of the steering device 40 to be performed at the advanced control timing (306). At this time, the controller 400 may control to change the control timing of the steering device 40 and may control to change, in addition to the control timing, the lateral distance limit value for operating the steering device 40. The related description will be referred to Figure 4 for detailed description.

[0086] According to this embodiment, when the driver's gaze is not forward, the controller 400 may determine that the driver is in a negligent condition and may control the control timing of the steering device 40 to be advanced by a preset time compared to the existing control timing. At this time, the control timing may be changed according to the setting. The steering device 40 may change the driving direction of the vehicle 1 through the control signal changed by the controller 400. At this time, a control signal is generated for the preset time instead of the existing control cut-in point of the steering device 40, so that the vehicle 1 does not deviate from the lane.

[0087] Figure 4It is a flowchart showing a vehicle control method according to another embodiment. The above control method can reflect the driver's condition and prevent the vehicle 1 from deviating from the lane by advancing the control timing of the steering device 40. According to Figure 4 The control method can reflect the driver's condition, but when the driver is in a negligent condition, the operation standard of the steering device 40 is relaxed to prevent the vehicle 1 from leaving the lane prematurely.

[0088] The controller 400 can control the external camera 100 to capture the external view of the vehicle 1, and can control the internal camera 200 to monitor the driver's gaze inside the vehicle 1 (401). Then, the controller 400 can receive the image data and gaze data obtained in operation 401. Here, the internal camera 200 can obtain the driver's gaze data based on the driver's face direction and the driver's pupil direction.

[0089] The controller 400 determines whether the vehicle 1 has a risk of lane departure based on the analysis of the image data (402). As a determination result, when there is no risk of lane departure, the controller 400 can continuously monitor the external view and the driver's gaze.

[0090] When there is a risk of lane departure, the controller 400 can determine whether the driver is in a negligent condition (403). Here, the driver's negligent condition can be a situation where the driver's gaze is not facing forward. In addition, in addition to the above situation, the driver's negligent condition can include all situations where the driver's condition cannot be observed normally, for example, the driver's pupils are not visible or the eyelid movement is different from usual. In this case, the controller 400 can store the driver's gaze image in the memory 420 and compare the driver's negligent condition with the stored image to determine the driver's negligent condition.

[0091] As a result of determining whether the driver is in a negligent condition, when the driver's condition is normal, the controller 400 can maintain the existing control state without changing the lateral distance limit value of the steering control (408). Here, the lateral distance limit value is a reference value compared with the lateral position error, and the lateral position error is the distance between the center point of the vehicle 1 and the center of the lane. And when the lateral position error exceeds the lateral distance limit value, the controller 400 can perform steering control. And when the lateral position error is less than the lateral distance limit value, the controller 400 can not perform steering control. The method of calculating the lateral position error will be described with reference to Figure 5 and Figure 6 Describe the method of calculating the lateral position error.

[0092] When the driver is in a negligent condition, the controller 400 can change the previously set lateral distance limit value to a smaller value (404). Therefore, since the controller 400 performs steering control even when the deviation of the vehicle 1 from the center of the lane is lower than the existing deviation, sudden lane departure can be prevented.

[0093] When the lateral distance limit value is changed, the controller 400 can calculate the current lateral position error (405). When the lateral position error is greater than the changed lateral distance limit value (406), the controller 400 can perform steering control to align the center of the vehicle 1 with the center of the lane (407).

[0094] According to the present embodiment, when the driver's gaze is not in front, the controller 400 can determine that the driver is in a negligent condition and can change the lateral distance limit value to a value smaller than the existing lateral distance limit value.

[0095] The above references Figure 3 and Figure 4 Each of the control methods described can be performed independently of each other or simultaneously.

[0096] Meanwhile, in Figure 4 a control method based on the lateral position error has been described. Hereinafter, the calculation method of the lateral position error mentioned in the above embodiment will be described with reference to Figure 5 and Figure 6 .

[0097] As Figure 5 shown, it is assumed that the vehicle 1 travels between lanes and a virtual coordinate system is set based on the center of the vehicle 1. At this time, based on the rectangular coordinate system, e represents the lateral position assuming that the vehicle 1 is stationary, ψ represents the angle between the vehicle 1 and the lane, and γ represents the curvature of the lane, represents the change rate of the lane curvature.

[0098] Referring to Figure 6 and the following equation 1, when the vehicle 1 is traveling, the lateral position error of the vehicle 1 can be calculated as follows.

[0099] Equation 1

[0100]

[0101] In particular, the lateral position error (e la ) corresponds to the distance error from the center of the lane corresponding to the future trajectory of the vehicle 1 being driven. In addition, the preset forward distance (d la ) represents the forward distance of the vehicle 1 at the time point (position) when the lateral position error is obtained.

[0102] As Figure 4As described, steering control is performed by comparing the calculated lateral position error with an existing lateral distance limit value or a changed limit value.

[0103] Figure 7 It is a view showing an embodiment of controlling a braking device.

[0104] As described above, the present disclosure can perform steering control by grasping the attention state of the driver. In addition, according to the present disclosure, braking control can be performed by grasping the attention state of the driver.

[0105] The driver assistance device 100 may further include a braking device 30 for decelerating the vehicle 1 or stopping the vehicle 1. Here, when the driver's gaze is not in the front, the controller 400 may determine that the driver is in a negligent state, and may control the control timing of the braking device 30 to be advanced by a preset time compared to the existing control timing. For example, as Figure 7 shown, when the driver attempts to turn right at an intersection, in order to prepare for the situation where the driver only focuses on other vehicles 2 approaching from the left and does not recognize pedestrians on the right turn path, the controller 400 may perform braking control at a time point earlier than the existing braking control timing.

[0106] In addition, according to this embodiment, in addition to advancing the control timing of the braking device 30, the controller 400 may also control the braking torque amount of the braking device 30 to be higher than the existing braking torque amount.

[0107] According to the disclosed embodiment, since the attention state of the driver is reflected, it is possible to prevent lane departure and collision in abnormal situations.

[0108] The disclosed embodiment may be implemented in the form of a recording medium storing computer-executable instructions executable by a processor. The instructions may be stored in the form of program code, and when executed by the processor, the instructions may generate program modules to perform the operations of the disclosed embodiment. The recording medium may be implemented as a non-transitory computer-readable recording medium.

[0109] The non-transitory computer-readable recording medium may include various recording media storing commands interpretable by a computer. For example, the non-transitory computer-readable recording medium may be, for example, ROM, RAM, magnetic tape, magnetic disk, flash memory, optical data storage device, etc.

[0110] So far, embodiments of the present disclosure have been described with reference to the accompanying drawings. It will be apparent to those of ordinary skill in the art that the present disclosure may be implemented in other forms than the above-described embodiments without changing the technical idea or essential features of the present disclosure. The above-described embodiments are merely illustrative and should not be construed in a limiting sense.

[0111] Cross-reference to Related Applications

[0112] This application claims the priority of Korean Patent Application No. 10-2019-0112681, filed on September 11, 2019, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety.

Claims

1. A driver assistance device, the driver assistance device comprising: An external camera disposed on the vehicle to have an external view of the vehicle, the external camera configured to obtain image data of the external view of the vehicle; An internal camera disposed inside the vehicle to capture the gaze of a driver on the vehicle, the internal camera configured to obtain the gaze data of the driver; A braking device configured to decelerate the vehicle or stop the vehicle; And A controller including at least one processor configured to process the image data and the gaze data, wherein the controller is configured to capture the gaze of the driver based on the gaze data, and based on a determination that the driver is in a negligent condition, control at least one of a control timing of a steering device and a lateral distance limit value for operating the steering device to be changed, wherein when the driver's gaze is not in the forward direction, the controller is configured to determine that the driver is in the negligent condition, change the lateral distance limit value to a value smaller than an existing lateral distance limit value, and control a braking torque amount of the braking device to be higher than an existing braking torque amount, and wherein the lateral distance limit value is a reference value compared with a lateral position error, the lateral position error being a distance between a center point of the vehicle and a center of a lane.

2. The driver assistance device according to claim 1, wherein, When the driver's gaze is not in the forward direction, the controller is further configured to determine that the driver is in the negligent condition, and control the control timing of the steering device to be advanced by a preset time compared with an existing control timing.

3. The driver assistance device according to claim 1, wherein When an object detected from outside the vehicle exists in the driver's gaze area, the controller is further configured to determine that the driver is not in the negligent condition and not interfere with the control of the steering device.

4. The driver assistance device according to claim 1, wherein, When an object detected from outside the vehicle does not exist in the driver's gaze area, the controller is further configured to determine that the driver is in the negligent condition.

5. The driver assistance device according to claim 1, wherein, The internal camera is configured to obtain the gaze data of the driver based on a face direction of the driver and a pupil direction of the driver.

6. The driver assistance device according to claim 1, the driver assistance device further comprising: A steering device configured to change a traveling direction of the vehicle, wherein the steering device is configured to change the traveling direction of the vehicle according to a control signal changed by the controller.

7. The driver assistance device according to claim 1, Among them, when the driver's gaze is not in the forward direction, the controller is further configured to determine that the driver is in the negligent condition, and control the control timing of the braking device to be advanced by a preset time compared with an existing control timing.

8. A method of controlling a driver assistance device, the method comprising: Obtaining, by an external camera, image data of an external view of a vehicle; Obtaining, by an internal camera, gaze data of a driver on the vehicle to capture the gaze of the driver; And The controller captures the driver's gaze based on the gaze data, and based on the determination that the driver is in a negligent condition, controls at least one of the control timing of the steering device and the lateral distance limit value for operating the steering device to be changed. Wherein, the control includes: When the driver's gaze is not in the forward direction, it is determined that the driver is in the negligent condition, and the lateral distance limit value is changed to a value smaller than the existing lateral distance limit value, and the braking torque amount of the braking device configured to decelerate or stop the vehicle of the driver assistance device is controlled to be higher than the existing braking torque amount, and Wherein, the lateral distance limit value is a reference value compared with the lateral position error, and the lateral position error is the distance between the center point of the vehicle and the center of the lane.

9. The method according to claim 8, wherein, The control further includes: When the driver's gaze is not in the forward direction, it is determined that the driver is in the negligent condition, and the control timing of the steering device is controlled to be advanced by a preset time compared with the existing control timing.

10. The method according to claim 8, wherein, The control further includes: When an object detected from outside the vehicle exists in the driver's gaze area, it is determined that the driver is not in the negligent condition, and the control of the steering device is not intervened.

11. The method according to claim 8, wherein The control further includes: When an object detected from outside the vehicle does not exist in the driver's gaze area, it is determined that the driver is in the negligent condition.

12. The method according to claim 8, wherein The gaze data includes information about the driver's face direction and the driver's pupil direction.

13. The method according to claim 8, the method further includes: A steering device configured to change the driving direction of the vehicle, Wherein, the control includes: Controlling the steering device according to the changed control signal to change the driving direction of the vehicle.

14. The method according to claim 8, Among them, The control further includes: When the driver's gaze is not in the forward direction, it is determined that the driver is in the negligent condition, and the control timing of the braking device is controlled to be advanced by a preset time compared with the existing control timing.

15. A non-transitory computer-readable medium including program instructions executed by a processor, the computer-readable medium includes: Program instructions for obtaining image data of the external view of the vehicle by means of an external camera; Program instructions for obtaining the gaze data of the driver on the vehicle by means of an internal camera to capture the driver's gaze; And Program instructions for the controller to capture the driver's gaze based on the gaze data, and based on the determination that the driver is in a negligent condition, control at least one of the control timing of the steering device and the lateral distance limit value for operating the steering device to be changed. Wherein, when the driver's line of sight is not in the forward direction, the controller is configured to determine that the driver is in the inattentive condition, change the lateral distance limit value to a value smaller than the existing lateral distance limit value, and control the braking torque amount of a braking device configured to decelerate the vehicle or stop the vehicle to be higher than the existing braking torque amount, and Wherein, the lateral distance limit value is a reference value compared with a lateral position error, and the lateral position error is the distance between the center point of the vehicle and the center of the lane.

Images