System and procedure for handling a driving hazard index for a vehicle

Abstract

System (100) for handling a driving hazard index for a vehicle, wherein the system comprises: a driving trajectory generator (130-1) for a driver's vehicle, configured to generate a driving trajectory of the driver's vehicle based on driving information from the driver's vehicle, input from an internal sensor (110) which senses a driving situation of the driver's vehicle; a peripheral vehicle trajectory generator (130-1) configured to generate a trajectory of a peripheral vehicle based on environmental information input from an external sensor (120) which senses the driving situation of the driver's vehicle; a trajectory load calculation device (130-5) configured to calculate a trajectory load representing a comparison result obtained by comparing a predetermined threshold with a trajectory distance, which is the difference between the peripheral vehicle trajectory and the driving trajectory of the vehicle driven by the driver; and a Hazard Index Manager (130-11) configured to generate a driving hazard index according to the calculated trajectory load.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority under 35 USC §119 to Korean Patent Application No. 10-2014-0072280, filed on June 13, 2014, the disclosure of which is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0002] The present invention relates to a system for handling a driving hazard index for vehicles, the system handling a driving hazard index generated based on information detected from the vehicle. BACKGROUND

[0003] Generally, an autonomous driving control system is installed in vehicles to improve the protection of the driver or a passenger, driving assistance, and driving comfort. The autonomous driving control system detects a lane using a camera and controls the vehicle's steering based on the detection result.

[0004] With regard to autonomous driving control systems, prior art documents (e.g., Korean Patent Application KR 10 2012 0 022 305 A) disclose an autonomous driving technology in which a driving characteristic of a driver is taken into account with respect to a path search and a driving operation.

[0005] An autonomous vehicle disclosed in prior art documents (e.g., Korean Patent Application KR 10 2012 0 022 305 A) includes: an image photography unit that captures image information about a road in the vicinity of the vehicle; a receiver that receives path information transmitted from a peripheral vehicle in the vicinity of the vehicle; a memory that stores a past driving pattern of the vehicle; and a control device that determines a driving pattern of the vehicle based on image information about the road, path information transmitted from the peripheral vehicle, and the past driving pattern of the vehicle. The autonomous vehicle learns the characteristics of a driver and controls autonomous driving of the vehicle.

[0006] In prior art documents (e.g., Korean patent application KR 10 2012 0 022 305 A), the autonomous vehicle collects past driving patterns to examine a driver's driving habit (a previous driving pattern). In this case, if an unforeseen situation results in abnormal collection of past driving data that differs from the current driving data, the driver will face a stronger counteraction due to the autonomous driving operation of a vehicle based on the abnormally collected past driving data than in general autonomous driving. DE 10 2006 036 363 A1 discloses a method and apparatus for predicting a movement trajectory, in which the trajectory of the host vehicle and also the trajectory of a peripheral vehicle are determined.From the movement trajectories, a distance between the vehicles can be predicted, which can be used to adjust the speed of the host vehicle within the framework of an ACC system. DE 10 2009 016 568 B4, DE 10 2010 005 290 A1, and DE 10 2013 018 967 A1 also disclose systems in which the driving trajectory of the host vehicle and that of a peripheral vehicle are determined.

[0007] Thus, with regard to autonomous driving of a vehicle, there is a need to develop a system that correctly reflects the driving habits of a driver. OVERVIEW

[0008] Accordingly, the present invention provides a system and method for managing a driving hazard index for vehicles that accurately reflects a driver's driving habits.

[0009] According to a general aspect, a system for managing a driving hazard index for vehicles comprises: a generator for generating a driving trajectory of a driver's vehicle, the generator configured to generate the driving trajectory of the driver's vehicle based on driving information of the driver's vehicle input from an internal sensor that senses a driving situation of the driver's vehicle; a peripheral vehicle trajectory generator configured to generate a trajectory of a peripheral vehicle based on environmental information input from an external sensor that senses a driving situation of the driver's vehicle;a trajectory load calculation device configured to calculate a trajectory load representing a comparison result obtained by comparing a predetermined threshold with a trajectory distance, which is the difference between the peripheral vehicle trajectory and the driver's vehicle driving trajectory; and a hazard index manager configured to generate a driving hazard index according to the calculated trajectory load.

[0010] According to another general aspect, a method for handling a driving danger index for vehicles comprises the steps of: generating a driving trajectory of a driver's vehicle based on driving information of the driver's vehicle input from an internal sensor that senses a driving situation of the driver's vehicle; generating a trajectory of a peripheral vehicle based on environmental information input from an external sensor that senses a driving situation of the driver's vehicle; calculating a trajectory load representing a comparison result obtained by comparing a predetermined threshold value with a trajectory distance that is the difference between the peripheral vehicle trajectory and the driving trajectory of the driver's vehicle; and generating a driving danger index corresponding to the calculated trajectory load.

[0011] Further features and aspects will become apparent from the following detailed description, drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a block diagram of a system for handling a driving hazard index for vehicles according to an embodiment of the present invention; Fig. 2 shows a block diagram of the detailed configuration of an environment detection device according to Fig. 1; Fig. Figure 3 is a diagram schematically illustrating three dangerous sections calculated by a peripheral vehicle load calculation device according to Fig. 2 are divided on the basis of a time to collision (TTC); Fig. Figure 4 is a diagram showing a configuration of a sensor for detecting peripheral vehicles located in each of the dangerous sections according to Fig. 3 are arranged; Fig. Fig. 5 is a diagram showing an example of detecting peripheral vehicles by the sensor according to Fig. 4; Fig. 6 is a flowchart illustrating a method for handling a driving hazard index for vehicles according to an embodiment of the present invention; Fig. 7 shows a block diagram illustrating another embodiment of an environment detection device according to Fig. 2; Fig. Fig. 8 is a block diagram showing the detailed configuration of a detection section generator according to Fig. 7; Fig. 9 shows a flowchart illustrating a method for optimizing a detection section according to an embodiment of the present invention; Fig. 10 shows a block diagram illustrating the detailed configuration of a driver status sensing system according to Fig. 1; Fig. 11A and Fig. 11B is a diagram illustrating a detected information device according to an embodiment of the present invention; Fig. 12A and Fig. 12B is a diagram illustrating a method for checking the closing of a driver's eyes according to an embodiment of the present invention; Fig. 13A to 13F are diagrams illustrating a fatigue load of a driver in a driving interruption load according to an embodiment of the present invention; Fig. 14 is a flowchart illustrating an operation for checking the closing of a driver's eyes according to an embodiment of the present invention; Fig. 15A to 15C are diagrams illustrating a driver's observation neglect load in the driving interruption load according to an embodiment of the present invention; Fig. 16 is a diagram illustrating an output of a screen according to an embodiment of the present invention; and Fig. 17 shows a flowchart illustrating a driver status sensing method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] Embodiments of the present invention are described in detail below with reference to the accompanying drawings. Regarding the allocation of reference numerals to the elements in each figure, it should be noted that the same reference numerals that have already been used in other figures to designate like elements will continue to be used for these elements wherever possible. Furthermore, no detailed descriptions are given here regarding well-known functions or configurations in order not to unnecessarily detract from the subject matter of the present invention.

[0013] Fig. 1 shows a block diagram of a system 100 for managing a driving hazard index for vehicles according to an embodiment of the present invention.

[0014] According to Fig. 1, the system 100 for managing a driving hazard index is capable of precisely analyzing a driving habit based on information acquired in real time by various sensors, instead of analyzing a driving habit of a driver based on a previously learned past driving pattern.

[0015] The driving danger index management system 100 according to an embodiment of the present invention can use the information acquired in real time by the various sensors and thus analyzes a driving habit of a driver more precisely than a driving habit analysis method of the relevant prior art that analyzes a driving habit based on a learned past driving pattern.

[0016] The system 100 for managing a driving hazard index may include an internal sensor 110, an external sensor 120, an environment detection device 130, a driving situation sensing interface 140, a driver status sensing system 160, and an output unit 150.

[0017] The internal sensor 110 can sense a driver's vehicle driving situation to capture the driver's vehicle driving information. The driver's vehicle driving information can include the vehicle speed information 11-1, the yaw rate information 11-3, the steering angle information 11-5, the acceleration information 11-7, and the wheel speed information 11-9.

[0018] To detect the driver's vehicle driving information, the internal sensor 110 may include a vehicle speed sensor 110-1 that detects the vehicle speed 11-1, a yaw rate sensor 110-3 that detects the yaw rate information 11-3, a steering angle sensor 110-5 that detects the steering angle information 11-5, and a wheel speed sensor 110-9 that detects the wheel speed information 11-9.

[0019] The external sensor 120 can sense the surroundings of the driver's vehicle to capture environmental information. The environmental information can include front / rear radar information 12-1, front / rear image information 12-3, side ultrasonic information 12-5, surround view monitoring (AVM) image information 12-7, and navigation information 12-9.

[0020] To acquire the surrounding information, the external sensor 120 may include a front / rear radar 120-1 that acquires the front / rear radar information 12-1, a front / rear camera 120-3 that acquires the front / rear image information 12-3, a side ultrasonic generator 120-5 that acquires the side ultrasonic information 12-5, an AVM camera 120-7 that acquires the AVM image information 12-7, a navigation device (NAV) 120-9 that acquires the navigation information 12-9, a vehicle-to-infrastructure communication system, and a vehicle-to-vehicle communication system. A vehicle-to-infrastructure communication system and a vehicle-to-vehicle communication system may contain information that is, for example, a traffic control feature (e.g., stop lights, stop signs), current weather-related information, information on temporary anomalies and the locations of temporary anomalies (e.g.,Construction zones, temporary speed limits), unforeseen situations (e.g., accident scenes, roadblocks, etc.), positions of other vehicles, routes of other vehicles (e.g., origin, destination, expected trajectory), lane status, and a merge sign (e.g., turn signal). The lane status (information) may be a single-lane rural road, a freeway with a median, a boulevard, a multi-lane road, a one-way street, a dual-traffic street, or a city street. All variations of the above teachings are within the scope of this patent application.

[0021] The environment recognition device 130 can calculate a trajectory load by using the driver's vehicle driving information supplied from the internal sensor 110 and the environmental information supplied from the external sensor 120, and can handle a driving danger index based on the result of the calculation. This is described in connection with Fig. 2 described in detail.

[0022] The driving situation sensing interface 140 may perform the interfacing with the output unit 150 and transmit the driver status information supplied from the driver status sensing system 160 and the driving danger index supplied from the environment recognition device 130 to the driver via the output unit 150.

[0023] The output unit 150 can output the driving hazard index reflected from the driving status information in the form of audible or visual information to communicate a driving hazard index to the driver. For this purpose, the output unit 150 can include a speaker 150-1, an audio / video navigation device (AVN) 150-3, and a head-up display 150-5.

[0024] Further, the output unit 150 may include an engine control system 150-7, an automatic control system 150-9, and a steering control system 150-11 for setting the time at which the lateral control of the vehicle starts.

[0025] The driver status sensing system 160 can detect a driving condition such as driving while drowsy, etc. The driver status sensing system 160 will be described later in connection with Fig. 10 to 17 described.

[0026] Fig. 2 shows a block diagram of the detailed configuration of an environment detection device 130 according to Fig. 1.

[0027] According to Fig. 2, as described above, the surroundings recognition device 130 can calculate the trajectory load by using the driver's vehicle driving information supplied from the internal sensor 110 and the surroundings information supplied from the external sensor 120, and calculate the driving danger index (or a peripheral risk index) based on the calculated trajectory load.

[0028] In order to handle the driving danger index even more precisely, the environment recognition device 130 may calculate the driving danger index by additionally taking into account a peripheral vehicle load and a road load in addition to the trajectory load.

[0029] The environment recognition device 130 may include a driver-vehicle traveling trajectory generator 130-1, a peripheral-vehicle trajectory generator 130-3, a trajectory load calculator 130-5, a peripheral-vehicle calculator 130-7, a road load calculator 130-9, and a traveling danger index manager 130-11.

[0030] The driver-vehicle driving trajectory generator 130-1 may acquire a driver-vehicle driving trajectory using vehicle speed information, steering angle information, deceleration / acceleration information, and yaw rate information supplied from the internal sensor 110.

[0031] The peripheral vehicle trajectory generator 130-3 can acquire a peripheral vehicle traveling trajectory 13-3 using the surrounding information including the front / rear radar information 12-1, the front / rear image information 12-3, the side ultrasonic information 12-5, and the AVM image information 12-7 supplied from the external sensor 120.

[0032] The front / rear radar information 12-1 has low precision in determining an object, but enables the acquisition of precise distance information (longitudinal direction). On the other hand, since the image information 12-3 and 12-7 is used to acquire a monocular image, the image information 12-3 and 12-7 has low precision in determining the distance (longitudinal direction). However, the image information 12-3 and 12-7, on the other hand, enables the precise identification of an object and can be used to obtain lateral information.

[0033] In a target angle model equation, the longitudinal distance information can be acquired by the front / rear radar 120-1, and the lateral distance information can be acquired by the front / rear camera 120-3, the AVM camera 120-7, and the lateral ultrasonic generator 120-5.

[0034] The following equation (1) can be used as the target vehicle module equation by which the peripheral vehicle trajectory generator 130-3 calculates the peripheral vehicle trajectory. A=[1Δt000100001Δt0001],H=[10000010]x=[xvxyvy] Xk=Axk+wkZk=Hxk+vk where x, V x , y, and V y Denotes status variables of a target vehicle, and x and y denote a position of the target vehicle and are measured by an image camera. V x and V ydenotes a speed of the target vehicle. A denotes a vehicle model equation, H denotes a measured value model equation, and the state variables denote a distance and a speed in an x-axis direction and a distance and a speed in a y-axis direction.

[0035] A system noise and a measured value noise indicate a white Gaussian bell.

[0036] The trajectory load calculation device 130-5 may calculate a trajectory load “W Trj ” The trajectory load “W Trj “ may be a comparison result obtained by comparing a predetermined threshold value with a trajectory distance value which is a difference between the peripheral vehicle traveling trajectory 13-3 and the driver vehicle traveling trajectory 13-1.

[0037] The driver-vehicle travel trajectory 13-1 and the peripheral vehicle travel trajectory 13-3 can be predicted, and a situation where there is a risk of collision may be a situation requiring high driver attention. The trajectory load calculation device 130-5 can calculate the situation as the trajectory load "W". Trj “ digitize.

[0038] The trajectory load “W Trj “ can be calculated and expressed in the following equation (2): WTrj(i)= / TTnj(i)−DTnj(i) / if WTrj(i)>threshold,0WTrj(i) <Schwellenwert,1 where D Trj the driver-vehicle driving trajectory and T Trj denotes a peripheral vehicle trajectory. Furthermore, i (1, 2, ..., n) denotes a detected peripheral vehicle.

[0039] According to equation (1), trajectories of detected peripheral vehicles can be compared with a trajectory of the driver's vehicle, and if a trajectory distance is smaller than a threshold, the trajectory load “W Trj ” can be set to 1. Furthermore, if the trajectory distance is greater than the threshold, the trajectory load “W Trj “ should be set to 0.

[0040] The peripheral vehicle calculation device 130-7 can analyze the number of front / rear / side peripheral vehicles and, based on the surrounding information, determine whether the peripheral vehicles are changing lanes. It can calculate a peripheral vehicle load "Ws" based on the analysis result. The number of peripheral vehicles and a trajectory change of each of the peripheral vehicles can act as a load requiring the driver's attention.

[0041] To calculate the peripheral vehicle load, three hazard sections 1, 2, and 3 can be calculated based on the time to collision (TTC). The three hazard sections 1, 2, and 3 are shown in Fig. 3. Here, the TTC can be defined as the time it takes for a corresponding vehicle to collide with a target vehicle when the catching-up speed of a vehicle is constant. The TTC can be calculated based on the vehicle speed information 11-1 and the steering angle information 11-5.

[0042] For example, according to Fig. 4 and Fig. 5 the three danger sections ①, ② and ③ are set by calculating a time obtained by multiplying a relative speed value of a detected vehicle by means of a TTC value for each of the peripheral vehicles (20, 30, 40 and 50 in Fig. 5) which are determined from the detection sections 43 and 47 detected by the front / rear radar 120-1, detection sections detected by the front / rear camera 120-3, and a detection section 45 detected by the side ultrasonic generator 120-5.

[0043] When the three danger sections ①, ② and ③ have been set, the peripheral vehicle calculation device 130-7 may analyze the number of peripheral vehicles detected in the sections and analyze whether the peripheral vehicles change lanes, and may calculate a peripheral vehicle load “Ws” based on the analysis result.

[0044] As the number of vehicles detected in section 1 increases and the frequency with which the detected vehicles change lanes increases, the peripheral vehicle load "Ws" may increase. On the other hand, if no detected peripheral vehicle exists, or even though a detected peripheral vehicle exists, the peripheral vehicle is detected in section 3, or the trajectory change of the peripheral vehicle detected in section 2 is not significant, the peripheral vehicle load "Ws" may decrease.

[0045] The peripheral vehicle load “Ws” can be expressed by the following equation: Ws=α∑i=1nSi+β∑i=1nLi where α denotes a weighting factor, β denotes a weighting factor, S denotes a position (the section ①, ② or ③) of the detected peripheral vehicle, and L indicates whether the detected peripheral vehicle changes lanes. If the detected peripheral vehicle has changed lanes, L can be set to 1, and if the lane has not been changed, L can be set to 0. With i (1 <i<n, wobei n eine natürliche Zahl ist) ist ein detektiertes Peripheriefahrzeug bezeichnet.

[0046] The road load calculation device 130-9 can calculate a road load using a road shape, a road surface status, and a traffic status included in the environmental information. For example, a short-distance road load can be calculated using image information obtained from an ultrasonic device, a radar, and / or a lidar among the external sensors, and a long-distance road load can be calculated using vehicle-to-vehicle and vehicle-to-infrastructure sensors among the external sensors.

[0047] The driver's attention is more demanding on a curve than on a straight road, and at an intersection than on a normal road, and when the traffic situation ahead deteriorates, the driver's attention is also required. Therefore, it is necessary to calculate the road load.

[0048] The road load can be calculated based on the navigation information 12-9 supplied from the navigation device 120-9, which includes road status information and road surface status information acquired by the front / rear camera 120-3. The road load can be calculated according to the following equation (4): WR=α×A+β×B+γ×C where A denotes a value representing a road status. For example, when a curvature value of a frontage road increases, A may have a large value, and when a traffic light changes, a pedestrian appears, a speed limit is displayed, or a child safety zone occurs, A may have a large value. B denotes a road surface status value, and B can reflect a paved road or an unpaved road. C denotes traffic on a frontage road, and when traffic increases, C may have a large value. A, B, and C can all be normalized to a range of 0 to 5.

[0049] The driving hazard index manager 130-11 can handle the driving hazard index (the peripheral risk index) calculated by summing the multiple loads “W Trj “, “W S “ and “W R“ which are calculated in respective steps.

[0050] The driving hazard index manager 130-11 can calculate the trajectory load “W Trj “ , the peripheral vehicle load “W S “ and the road load “W R ” to calculate a summed result value as a driving hazard index.

[0051] If the calculated driving hazard index is higher than a predetermined threshold, the driving hazard index manager 130-11 may provide the calculated driving hazard index to the vehicle control systems 150-7, 150-9, and 150-11 via the driving situation sensing interface 140.

[0052] The vehicle control systems 150-7, 150-9, and 150-11 that have received the calculated driving hazard index above the predetermined threshold may calculate an engine operation, a braking operation, and a steering operation to restrict a driving function of the vehicle.

[0053] The driving hazard index can be expressed in the form of the following equation (5): Driving hazard index=WTrj+WS+WR

[0054] Fig. 6 shows a flowchart illustrating a method for handling a driving hazard index for vehicles according to an embodiment of the present invention.

[0055] According to Fig. 6, in step S610, an operation for generating the driving trajectory of the driver's vehicle may be performed, and in step S612, an operation for predicting a position of the driver's vehicle may be performed using the generated driving trajectory of the driver's vehicle. The driving trajectory of the driver's vehicle may be generated based on vehicle speed information, steering angle information, acceleration / deceleration information, and yaw rate information.

[0056] Subsequently, in step S614, an operation for generating a peripheral vehicle travel trajectory may be performed, and in step S616, an operation for predicting a position of a peripheral vehicle may be performed. The peripheral vehicle travel trajectory may be generated based on longitudinal distance information obtained from a radar, lateral distance information obtained from a camera, and lateral distance information obtained from an ultrasonic generator. Here, the longitudinal distance information may be information about the longitudinal distance to the peripheral vehicle with respect to the driver's vehicle, and the longitudinal distance information may be information about the longitudinal distance to the peripheral vehicle with respect to the driver's vehicle.

[0057] Subsequently, in step S618, an operation for calculating a trajectory load may be performed. The trajectory load may be calculated based on the driving trajectory of the driver's vehicle and the driving trajectory of the peripheral vehicle. For example, driving trajectories of detected peripheral vehicles may be compared with the driving trajectory of the driver's vehicle, and if a trajectory distance, which is a difference between these trajectories, is less than a threshold, the trajectory load may be calculated as 1. On the other hand, if the trajectory distance is greater than the threshold, the trajectory load may be calculated as 0.

[0058] Subsequently, in step S620, an operation for calculating a peripheral vehicle load may be performed. The peripheral vehicle load may be calculated by considering the number of vehicles located in each of a plurality of hazardous sections divided based on a TTC and whether the vehicle changes lanes. The peripheral vehicles located in the plurality of hazardous sections may be detected using a radar, a camera, and an ultrasonic generator, and the plurality of hazardous sections may be obtained by calculating a time determined by dividing a relative speed value by a relative distance (which is a TTC value) to a detected vehicle.

[0059] Subsequently, in step S622, an operation for calculating a road load may be performed. The road load may be calculated based on navigation information, road surface status information, traffic information, etc.

[0060] Subsequently, in step S624, an operation for calculating a driving hazard index may be performed. The driving hazard index may be calculated by summing the trajectory load, the peripheral vehicle load, and the road load.

[0061] Subsequently, in step S626, an operation may be performed to compare the calculated driving danger index with a threshold. If the driving danger index is equal to or greater than the threshold, in step S628, the system 100 may warn the driver of a dangerous driving situation in stages based on a threshold level. If the driving danger index is below the threshold, steps S610 to S624 may be performed again.

[0062] In step S630, an operation may be performed in which the driving danger index indicating the dangerous driving situation is notified to the driver via a speaker, a HUD, and an AVN installed in the vehicle.

[0063] Fig. 7 shows a block diagram illustrating another embodiment of the environment detection device according to Fig. 2.

[0064] According to Fig. 7, an environment recognition device 130 according to another embodiment of the present invention may include a detection section generator 132 for detecting an optimized section area (hereinafter referred to as a detection section) and for detecting an environment in the optimized detection section. The remaining elements except for the detection section generator 132 are the same as the elements used in the environment recognition device 130 according to Fig. 2 are included, and therefore the above-mentioned Fig. 2 in connection with the environment detection device 130 also apply to the other elements.

[0065] The detection section generator 132 can optimize a detection section containing a target object so that an actual obstacle can be precisely identified among the target objects in the vicinity of the vehicle.

[0066] Fig. Fig. 8 is a block diagram showing the detailed configuration of the detection section generator 132 according to Fig. 7.

[0067] According to Fig. 8, the detection section generator 132 for optimizing a detection section according to an embodiment of the present invention may include a remote sensor 132-1, a target tracker 132-3, a trajectory manager 132-5, and a memory 132-9.

[0068] The remote sensor 132-1 can detect the position of an object in the surroundings of the vehicle to output a detection signal. The remote sensor 132-1 can comprise a lidar sensor, a radar sensor, a camera sensor, or a plurality of these sensors.

[0069] The target tracker 132-3 may identify a target based on the detection signal and generate a trajectory having a covariance of an error and a position estimate corresponding to the identified obstacle, thereby tracking the position.

[0070] In general, a Kalman filter can be used to eliminate the error of a sensor and sense the position of a moving object.

[0071] The Kalman filter may employ a technique that repeats an operation of calculating an estimated position of an object based on an estimated value of an object's position at a previous time and a measured value of an object's position, thus counteracting an error that occurs when measuring the object's position and measuring a precise position of the object. In this case, an estimated value based only on a measured value from a previous time may be calculated at a current time based on an estimated value of an object's position at a previous time.

[0072] Subsequently, an estimate at a current time based only on a measurement value from a previous time can be corrected based on a covariance at the current time calculated only on the basis of the measurement value from the previous time and a measurement value of the position of the object at the current time, and an estimate of the object at the current time can be calculated.

[0073] The target tracker 132-3 can set a number of trajectories, a detection section, and an initial position of an obstacle corresponding to each of the trajectories based on the position of the object indicated by the detection signal. The detection section and the initial position of the obstacle can be set by the Kalman filter according to the following equation (6): x^(k|k−1)=F(k−1)x^(k−1|k−1)z^(k|k−1)=H(k)x^(k|k−1) where x̂(k|k - 1) denotes an estimate of a status value of the object at a time "k" based on the information at a time "k-1", x̂(k -1|k - 1) denotes an estimate of a status value of the object at a time "k-1" based on the information at a time "k-1", and ẑ(k|k - 1) denotes an estimate of a position of the object at a time "k" based on the information at a time "k-1".

[0074] Here, each trajectory may include the Kalman filter to track a specific object sensed by the remote sensor 132-1. This means that each trajectory may include a covariance of an error for correcting a position based on a measured value and an estimated value of a position of a tracked object, and an estimated value, a measured value, and a covariance calculated per time may be stored as histories in the memory 132-9.

[0075] A configuration of the Kalman filter contained in each lane can be expressed by the following equation (7): x(k)=F(k−1)x(k−1)+v(k−1)z(k)=H(k)x(k)+w(k)P(k|k)∑iβ(k,i)[P(k|k,i)+(x^(k| k,i)−x^(k|k))(x^(k|k,i)−x^(k|k))T]P(k|k−1)=F(k−1)P(k−1|k1)F(k−1)r+Q(k−1) where x(k) denotes a status value of the object at time "k", F(k-1) denotes a status change model that represents a change when a time point changes from time "k-1" to time "k", z(k) denotes a position of the object at time "k", H(k) denotes an observation model that represents a change from a status of the object to the position of the object, v(k-1) denotes a processing noise at time "k", and w(k) denotes a measurement noise at time "k". Furthermore, P(k|k) denotes a covariance of an error of the Kalman filter at time "k" calculated based on the information at time "k", and P(k|k-1) denotes a covariance of an error of the Kalman filter at time "k" calculated based on the information at time "k-1". Q(k-1) denotes a prediction covariance at time “k-1”.

[0076] The target tracker 132-3 can determine whether the position of the object indicated by the detection signal is included in the detection section corresponding to a trajectory based on errors of a measurement value, an estimated value of the position of an object, and a covariance of each of the errors.

[0077] The target tracker 132-3 can set a detection section range based on a status value of the Kalman filter included in a current trajectory and update a status value of the Kalman filter based on the measured values ​​occurring in the detection section. The target tracker 132-3 can calculate a residual value based on the measured value and the estimated value of the object's position, calculate a residual covariance based on the observation model and a covariance of an estimation error included in the Kalman filter, and determine whether the target enters the detection section based on the residual value and the residual covariance.

[0078] Here, the detection interval can be set as a interval representing a specific probability value or a smaller value in the Gaussian probability distribution, which has the residual covariance as the variance, and the probability value can be considered a gate probability. Thus, the detection interval can be calculated by calculating the residual covariance and setting a gate probability value. The residual covariance and the gate probability value can be optimized over time using the Kalman filter, thereby optimizing the detection interval over time.

[0079] A method for calculating the residual value and the residual covariance and a state in which the object is placed in the detection section can be expressed by the following equation (8): v(k,i)=z(k,i)−z^(k,i)−z^(k|k−1)S(k)=H(k)P(k|k−1)H(k)T+R(k)v(k,i)TS(k)−1v(k,i) <r where v(k, i) denotes a residual value of an object "i" at time "k," and z(k, i) denotes a measured value at a position of the object "i." Furthermore, P(k|k-1) denotes a covariance of an estimation error of the Kalman filter, R(k) denotes a measurement noise at time "k," S(k) denotes the residual covariance at time "k," and r denotes a range of a detection section.

[0080] The trajectory manager 132-5 may update a position estimate included in a trajectory based on a detection signal included in the trajectory corresponding to an object located in a detection section.

[0081] To detect the position estimate, the trajectory manager 132-5 may calculate a Kalman gain based on a residual covariance and a covariance of an estimation error, and may calculate the position estimate using information at a current time based on an estimated position value estimated based on the Kalman gain, a position measurement value of the object, and information from a previous time. The update of the position estimate can be expressed by the following equation (9): K(k)=P(k|k−1)HTS(k)−1x^(k|k,i)={x^(k|k−1)i=0x^(k|k−1|)+K(k)v(k,i)i>0x^(k|k)=∑iβ(k,i)x^(k|k,i) where K(k) denotes the Kalman gain factor.

[0082] As described above, the path manager 132-5 can update the position measurement value over time based on the measurement value, thereby calculating a precise position measurement value.

[0083] When a distance between an object position estimate included in a first trajectory and an object position estimate included in a second trajectory is smaller than a predetermined reference value, the trajectory manager 132-5 may initialize the first trajectory and the second trajectory based on a history stored in the memory 132-9.

[0084] The memory 132-9 can store a history obtained by updating a trajectory. The history stored in the memory 132-9 can store a position estimate, a position measurement, and a covariance value of a time-dependent estimation error of the Kalman filter contained in the trajectory.

[0085] When a position estimate is updated as described above, objects indicated by two trajectories may collide with each other, depending on the specific case. If a position estimate indicated by an object is reduced to a value below a prestored reference value, the trajectory manager 132-5 may determine that the objects indicated by the two trajectories are colliding and may initialize the trajectory based on the data contained in the histories of two colliding trajectories.

[0086] Furthermore, if all the object position estimates included in a trajectory are not included in a detection section corresponding to the trajectory, the trajectory manager 132-5 may initialize the trajectory based on a history of the trajectory stored in the memory 132-9. This means that if an object tracked by a trajectory deviates from the detection section, or the object tracked by the trajectory is considered noise or error and thus disappears, the trajectory may not be able to track the object, and thus the trajectory manager 132-5 may initialize the trajectory and track a new object.

[0087] As described above, a track can track a moving obstacle by using the Kalman filter. When a track tracking an object fails to track the object, or when two tracks collide, a track can be initialized and a new object can be tracked. Accordingly, the object identification performance of a peripheral status detection system is improved.

[0088] As described above, the target tracker 132-3 and the trajectory manager 132-5 may track an obstacle to generate or update a trajectory, and data included in the generated or updated trajectory may be transmitted to the vehicle control device 132-7 and may be used to control a vehicle so that the vehicle avoids an obstacle or a warning is issued.

[0089] Fig. 9 shows a flowchart illustrating a method for optimizing a detection section according to an embodiment of the present invention.

[0090] According to Fig. 9, in step S910, the target tracker 132-3 may generate a trajectory including a covariance of an error and a position estimate corresponding to an identified vehicle based on a detection signal that the remote sensor 132-1 has generated and outputs from a position of an object.

[0091] As described above, the remote sensor 132-1 can comprise a lidar sensor, a radar sensor or several of these sensors.

[0092] Furthermore, a trajectory generated by the target tracker 132-3 may include a Kalman filter containing a position estimate and a covariance of an error. A covariance of the Kalman filter included in the trajectory is configured as described by equations (1) and (2).

[0093] Subsequently, in step S920, the target tracker 132-3 may calculate a detection section, which is the area in which an obstacle to the path has been detected.

[0094] Based on the position of the object specified by the detection signal, the size of a detection section can be set to an initial value. Furthermore, the detection section can be set as a section that represents a gate probability value or a lower value in the Gaussian probability distribution, which has a residual covariance as the variance.

[0095] Subsequently, in step S930, the target tracker 132-3 may select a valid detection signal in which the position of the object indicated by the detection signal is included in the detection section from the detection signal.

[0096] As described above, the detection signal may include a measurement of the position of the object being tracked by a peripheral search system of a vehicle, and the target tracker 132-3 may select a valid measurement of the measurement value included in the detection section to update the Kalman filter and use the updated Kalman filter to track an object.

[0097] At this time, the target tracker 132-3 can determine whether the position of the object indicated by the detection signal is included in the detection section corresponding to the trajectory based on errors of a measured value and an estimated value of the position of an object and a covariance of each of the errors.

[0098] The target tracker 132-3 can set a detection section range based on a status value of a Kalman filter included in a current trajectory, and update a status value of the Kalman filter using measured values ​​included in the detection section. Here, the target tracker 132-3 can calculate a residual value based on the measured value and the estimated value of the object's position, calculate a residual covariance based on the observation model and a covariance of an estimation error included in the Kalman filter, and determine whether the object enters the detection section based on the residual value and the residual covariance. A method for calculating the residual value and the residual covariance and a state in which the object is located in the detection section is given in Equation (8).The residual covariance and a gate probability value can be optimized over time by the Kalman filter, and thus the detection section can be optimized over time.

[0099] Subsequently, in step S940, when a distance between an object position estimate included in a first path and an object position estimate included in a second path is smaller than a predetermined reference value, the path manager 132-5 may initialize the first path and the second path based on a history stored in the memory 132-9.

[0100] The history stored in the memory 132-9 may include a position estimate, a position measurement, and a covariance value of an estimation error of the Kalman filter included in the trajectory over time.

[0101] When a position estimate is updated as described above, objects indicated by two trajectories may collide with each other, depending on the specific case. If a position estimate indicated by an object is reduced to a value below a prestored reference value, the trajectory manager 132-5 may determine that the objects indicated by two trajectories are colliding with each other, and may initialize the trajectory based on data contained in the histories of the two colliding trajectories.

[0102] Subsequently, in step S950, the trajectory manager 132-5 may update the selected detection signal and a position estimate included in a trajectory corresponding to an object whose position is located in the detection section.

[0103] To update the position estimate, the trajectory manager 132-5 can generate a Kalman gain based on a residual covariance and a covariance of an estimation error, and calculate the position estimate using information about the current time, namely, based on an estimated position value estimated based on the Kalman gain, a position measurement value of an object, and information about a previous time. The update of the position estimate is expressed in Equation (9).

[0104] Subsequently, when all the object position estimates included in a trajectory are not included in a detection section corresponding to the trajectory, the trajectory manager 132-5 may initialize the trajectory and terminate an operation based on a history of the trajectory stored in the memory 132-9.

[0105] This means that if an object tracked by a track deviates from the detection section or the object tracked by the track is considered noise or error and thus disappears, the track may not be able to track the object, and thus the track manager 132-5 may initialize the track and track a new object.

[0106] As described above, a track can track a moving obstacle by using the Kalman filter. When a track tracking an object fails to track the object, or when two tracks collide, a track can be initialized and a new object can be tracked. Accordingly, the object identification performance of a peripheral status detection system is improved.

[0107] Data contained in the trajectory generated or updated by the method described above may be transmitted to the vehicle controller 132-7 and used to control a vehicle so that the vehicle avoids an obstacle or a warning is issued.

[0108] As described above, in the vehicle control device and vehicle control method according to an embodiment of the present invention, a valid gate representing a section of interest (SOI) tracked by the peripheral status detection system of a vehicle is dynamically updated to sense an obstacle, thus accurately tracking an obstacle located in the vicinity of the vehicle. Thus, in the vehicle control device and vehicle control method, a distance to an obstacle whose position is accurately tracked using only a lidar sensor or a radar sensor is increased, thereby preventing an accident.

[0109] Fig. 10 shows a block diagram showing the detailed configuration of the driver status sensing system 160 according to Fig. 1.

[0110] According to Fig. 10, the driver status sensing system 160 may include a detection unit 161, a control unit 163, and an output unit 165.

[0111] The detection unit 161 may be an element for detecting driving manipulation information of a vehicle and carelessness status information of a driver. The detection unit 161 may include a driving manipulation sensing unit 161A and a carelessness status sensing unit 161B. The detection unit 161 may detect the driving manipulation information using the driving manipulation sensing unit 161A and detect the carelessness status information using the carelessness status sensing unit 161B.

[0112] The driving tampering sensing unit 161A may be an element for detecting tampering with a control unit whose operation is required for driving the vehicle. For example, the driving tampering sensing unit 161A may be an electronic control unit (ECU) of the vehicle or a separate module included in the ECU.

[0113] The driving manipulation sensing unit 161A may include a plurality of manipulation sensing units such as an accelerator pedal manipulation sensing unit d1, a brake pedal manipulation sensing unit d2, a multi-function manipulation sensing unit d3, and a control handle manipulation sensing unit d4.

[0114] According to Fig. 11A, the plurality of manipulation sensing units included in the driving manipulation sensing unit 161A can sense manipulations for driving the vehicle and, for example, can sense at least one of accelerator pedal (ACC pad) manipulation, brake pedal manipulation, multifunction switch manipulation, and control handle manipulation.

[0115] Furthermore, the driving manipulation sensing unit 161A can detect driving manipulation information, which further includes clutch pedal manipulation or transmission manipulation of a manual transmission vehicle. In this case, the driving manipulation sensing unit 161A can check the vehicle speed, and if the vehicle speed is equal to or higher than a certain speed (e.g., 10 km / h), the driving manipulation sensing unit 161A can detect driving manipulation information.

[0116] For example, the driving manipulation sensing unit 161A may detect the number “n A "Sense the operation of an accelerator pedal manipulated by the driver during each specific period within a predetermined time period. For example, the driving manipulation sensing unit 161A may continuously check whether the accelerator pedal is depressed (ON) every 50-second period within a 200-second period.

[0117] Likewise, the driving manipulation sensing unit 161A can detect the numbers “n B ' and 'n C “ of brake pedal and clutch pedal operations manipulated during each specific time period.

[0118] As another example, the driving manipulation sensing unit 161A may detect the number “n M"of actuations of each multi-function switch manipulated by the driver during a specific period of time. Here, each multi-function switch may be a switch for actuating a vehicle windshield wiper or a vehicle lamp such as a turn signal indicator. For example, the driver manipulation sensing unit 161A may count the number of actuations of a multi-function switch manipulated during a period of 200 ms.

[0119] As another example, the driving manipulation sensing unit 161A may sense an angular velocity of a steering wheel manipulated by the driver during a certain period of time. In this case, the driving manipulation sensing unit 161A may measure an angular change amount of the steering wheel to calculate the angular velocity without separately measuring the angular velocity. For example, the driving manipulation sensing unit 161A may measure the angular change amount of the steering wheel to calculate the angular velocity every 50 ms within a 200 ms period.

[0120] The carelessness status sensing unit 161B may be an element for sensing driver movement and manipulation of a control unit that is additionally operated when the vehicle is running. The carelessness status sensing unit 161B may include a plurality of sensing units such as an audio signal input sensing unit T1, an air conditioning signal input sensing unit T2, a navigation signal input sensing unit T3, etc.

[0121] Furthermore, the carelessness status sensing unit 161B may include a drowsiness index measuring unit P, a gaze direction measuring unit E, and a speech sensing unit V for sensing a movement (a visual factor or a verbal factor) of the driver. The plurality of sensing units and the plurality of measuring units configured according to Fig. 10 included in the carelessness status sensing unit 161B may be provided at a specific position of the vehicle and may detect carelessness status information based on information about at least one aspect of a manipulation of a peripheral device, the driver's facial expression, and the driver's voice.

[0122] For example, the carelessness status sensing unit 161B may detect the number “n T"of manipulations of peripheral devices performed by the driver while the vehicle is traveling. Here, the peripheral devices may not be devices necessary for the driving operation performed by the driver, such as an AVN or a vehicle air conditioning system, but may be control units manipulated for the convenience of the driver or a vehicle interior. For example, to obtain carelessness status information, the carelessness status sensing unit 161B may sense the number of input operations performed via operation switches of peripheral devices manipulated by the driver during a certain period of time while the vehicle is traveling at a certain speed (e.g., 10 km / h) or faster.

[0123] As another example, the carelessness status sensing unit 161B may sense the driver's voice via a microphone arranged at a specific position in the vehicle. For example, the carelessness status sensing unit 161B may check a utterance generation time and a pulse level (a voice pitch) of speech data received from the microphone during a specific time. For example, the carelessness status sensing unit 161B may detect a time during which speech data having a pulse level equal to or higher than a pulse threshold is received within a specific time period (e.g., 200 ms) to obtain carelessness status information using the pulse threshold stored in a memory.

[0124] As another example, the carelessness status sensing unit 161B may receive a facial image of the driver from a camera arranged at a specific position on the vehicle to capture eye-closing information and observation neglect information of the driver. Fig. 11B the camera may have a light-emitting diode (LED) for near-infrared light to record an image during the day and night.

[0125] For example, the carelessness status sensing unit 161B may separately extract an eye area image of the driver from a face image of the driver. In this case, the eye area image may be extracted from the driver's face image through image processing. This means that the carelessness status sensing unit 161B may detect carelessness status information, such as the driver's eyes closing or observation neglect, based on the driver's face image captured by the camera and the eye area image included in the face image.

[0126] The carelessness status sensing unit 161B can obtain the driver's eye-closing information based on the eye area image extracted from the driver's face image.

[0127] According to the Fig. 12A and Fig. 12B, the carelessness status sensing unit 161B may sense an eyelid area from the eye area image, and when a sum of ∠A and ∠B, which are angles of eyelids, is equal to or less than ∠C, which is a predetermined angle threshold (∠A + ∠B ≤ ∠C), the carelessness status sensing unit 161B may determine that the driver closes his eyes.

[0128] The carelessness status sensing unit 161B may check the angles of the eyes for a predetermined period of time to detect the driver's eye closure frequency, and calculate a specific time value and the driver's eye closure frequency, thereby obtaining a driver's drowsiness time (a drowsiness index). For example, the carelessness status sensing unit 161B may measure (count) the driver's eye closure frequency by dividing one second into 250 ms intervals. If the count value is 3, the driver's eye closure time (a drowsiness index) may be detected as 750 ms.

[0129] Further, the carelessness status sensing unit 161B can detect the driver's observation neglect based on the driver's face image and the eye area image included in the face image.

[0130] According to Fig. 15A to 15C, in an observation neglect area, observation neglect information can be acquired by checking (viewing direction) whether a driver's visual range is within a visual distance “η” in which no load exists when the vehicle is running, based on an angle of a vehicle wheel (ie, an angle change amount (ΘA or ΘB in Fig. 15A) of a wheel in a center of the vehicle) instead of an angle of a steering wheel. The field of view can be checked by performing an operation in which an angle of a face (α in Fig. 15B) is calculated from the driver’s face image received from the camera and then a position (pupil position) of a pupil (β in Fig. 15C) is measured in the eye area image.

[0131] Specifically, in a case where a shift lever is in D-shift or N-shift with an angle of the steering wheel within a certain angle range, in a situation where the driver's visual range does not enter a driving no-load visual range (a, b) for a certain time, the carelessness status sensing unit 161B may determine an observation neglect, check a corresponding time, and acquire the observation neglect information.For example, in a case where an angle of a steering wheel is less than ± 15 degrees and a speed of the vehicle is 10 km / h or more and the shift lever is in D-shift or N-shift, in a situation where the driver's visual range does not enter a driving no-load visual range (a, b) for 1.5 seconds or more, the carelessness status sensing unit 161B may detect attention neglect with respect to the area in front of the vehicle.

[0132] The control unit 163 may be an element that controls the overall operation of the driver status detection system 160, and it may be embodied as an electronic unit. For example, the control unit 163 may be referred to as a driving load calculation unit (DWC).

[0133] Specifically, the control unit 163 may calculate a driving manipulation load and a driving interruption load based on the driving manipulation information and the carelessness status information obtained from the detection unit 161.

[0134] Further, the control unit 163 may compare the driving manipulation load with the driving interruption load to determine whether a driving status of the driver is a safety driving status, and if it is determined that the driving status of the driver is not the safety driving status, the control unit 163 may control the output unit 165 to output warning information.

[0135] First, the control unit 163 may calculate the driving manipulation load and the driving interruption load based on the driving manipulation information and the carelessness status information obtained from the detection unit 161.

[0136] For example, the control unit 163 can control the driving manipulation load “W d ” based on the driving manipulation information obtained from the driving manipulation sensing unit 161A of the detecting unit 161.

[0137] Specifically, the control unit 163 may acquire the driving manipulation information from the driving manipulation sensing unit 161A, read an item-based weight value of the driving manipulation information from the memory, and calculate the driving manipulation load.

[0138] The weighting value here can be a value extracted and predetermined for each item of driving manipulation information through various experiments. Alternatively, the weighting value can be a value arbitrarily predetermined for each item by a worker. The memory can also be a storage device for storing data and can be, for example, a non-volatile memory.

[0139] Furthermore, the control unit 163 can calculate a load for each item based on an item included in the driving manipulation information and a weight value corresponding to the item. For example, when driving manipulation information in the form of accelerator pedal manipulation, brake pedal manipulation, multifunction switch manipulation, and control handle manipulation has been received from the detection unit 161, the control unit 163 can retrieve a weight value "D" from the memory. A“ of the accelerator pedal manipulation, a weighting value “D B “ of the brake pedal manipulation, a weighting value “D M “ of the multifunction switch manipulation and a weighting value “Θ” of the control handle manipulation.

[0140] For example, if the time information and the frequency information of each item have been acquired by the acquisition unit 161 in units of 50 ms per second period of 200 ms, the driving manipulation load “W” calculated by the control unit 163 for a second period of 200 ms may be d “ can be expressed by the following equation (10): Wd=DA×nA×50ms+DB×nB×50ms+DC×nC×50ms+DM×nM+θ×n0×50ms

[0141] Each item included in the driving manipulation information can be added or subtracted as appropriate. For example, if clutch pedal manipulation and gearshift manipulation are included in the driving manipulation information depending on the vehicle type, the control unit 163 can further calculate the driving manipulation load by considering a weight value for the clutch pedal manipulation and a weight value for the gearshift manipulation.

[0142] As another example, the control unit 163 may control the travel interrupt load “W i" based on the carelessness status information obtained from the carelessness status sensing unit 161B of the detection unit 161. Specifically, the control unit 163 may calculate loads of the objects included in the carelessness status information acquired from the carelessness status sensing unit 161B and sum the calculated loads of the objects to calculate the travel interruption load.

[0143] The control unit 163 may calculate a peripheral device manipulation load “T(n)” based on the number of manipulations of peripheral devices obtained from the carelessness status sensing unit 161B of the detection unit 161.

[0144] For example, the control unit 163 may further calculate the peripheral device manipulation load by considering a peripheral device manipulation weight value stored in the memory. For example, when the number of peripheral device manipulation inputs performed by the driver for a period of 20 ms is acquired from the acquisition unit 161, the control unit 163 may perform an arithmetic operation on the number of peripheral device manipulation inputs and a weight value to calculate the peripheral device manipulation load "T(n)".

[0145] Furthermore, the control unit 163 may calculate a speech load "V(n)" based on the driver's speech data received from the carelessness status sensing unit 161B of the detection unit 161. Specifically, the control unit 163 may calculate the speech load based on a pulse threshold of the speech data stored in the memory. For example, the control unit 163 may calculate a time at which speech data with the pulse threshold or higher is received in the driver's speech data received from the carelessness status sensing unit 161B of the detection unit 161 within a certain time (e.g., 200 ms), thereby calculating a speech load "V(t)".

[0146] Further, the control unit 163 may calculate a drowsiness load “P(t)” and a driving observation neglect load “E(t)” based on the eye-closing information and the observation neglect information included in the carelessness status information obtained from the carelessness status sensing unit 161B of the detection unit 161.

[0147] According to the diagrams according to the Fig. 13A to 13F, when the driver is in an inertial state, the change range in the diagram can be very stable and narrow. However, during a driver inertial state, a PERCLOS value can experience a very dramatic change and be higher in its overall numerical value than the inertial state. The PERCLOS value can be expressed here by the following equation (11): PERCLOS(%)Accumulation of eye closure timeSpecified measurement time for accumulation×100

[0148] Based on the diagram according to Fig. 13A to 13F, when the control unit 163 has determined that the PERCLOS value has reached a certain percentage (30%) or higher, namely, when the carelessness status sensing unit 161B has determined that the driver has closed their eyes for a certain time (e.g., 75 seconds) relative to a certain period of time (e.g., 250 seconds), the control unit 163 can control the output unit 165 to output a vehicle warning sound. The purpose of this is that if, following the determination that the driver has closed their eyes for a certain time or longer, the driver's driving is judged to be drowsy driving, a warning sound is immediately output to the driver, since drowsy driving is the most dangerous factor when driving a vehicle.

[0149] In the following, an operation in which the control unit 163 checks the closing of the driver's eyes will be described using the flowchart according to Fig. 14 described.

[0150] Specifically, in step S1410, the control unit 163 may determine whether a vehicle speed "V" is a specified speed (e.g., V ≥ 10 km / h) or higher. If it is determined that the vehicle speed "V" is the specified speed or higher, the control unit 163 may, using the carelessness status sensing unit 161B, check the number "y" of eye closures based on an angle of an eyelid per a specified time "x" (e.g., 250 ms or 0.25 s) within a specified measurement period "N" to calculate an eye closure time (a drowsiness time) of the driver.

[0151] In step S1420, the control unit 163 may determine whether the eyelid angle satisfies ∠A+∠B≤∠C. If the eyelid angle satisfies ∠A+∠B≤∠C, the control unit 163 may determine the driver's eye closure by counting the number of eye closures (y++) in step S1430.

[0152] The control unit 163 may count the number "y" of eye closures per certain time "x" (x+=0.25, y++), and if in step S1440 the number of eye closures for one second is three (x=1, y=3), in step S1450 the control unit 163 may count a tolerance factor value "P" (P++) and repeat the operation while increasing (N++) by one second each time.

[0153] For example, if the tolerance factor value "P" is 100 or more within a measurement time of 250 seconds in step S1460, the control unit 163 may control the output unit 165 to generate a warning event in step S1470. At this time, the control unit 163 may change the tolerance factor value "P" to 99, reduce the measurement value "N" by one second (P-99, N-=1), repeat the above steps, and calculate an eye-closure time (a drowsiness time) of the driver. Furthermore, the control unit 163 may consider a case where the PERCLOS value is 30% or more for a certain time, thereby calculating the drowsiness load "P(t)".

[0154] Further, the control unit 163 may calculate the observation neglect load “E(t)” based on the neglect information obtained from the carelessness status sensing unit 161B.

[0155] According to Fig. 15A, when a visual range deviates from the visual distance “η” in which no load exists during driving, a factor value of the load factor “P” can be changed according to a deviation range.

[0156] If a deviation range (a',b') is predetermined, the control unit can read a factor value of a load factor from the memory for each range corresponding to a current field of view of the driver and thereby calculate the observation neglect load "E(t)".

[0157] The control unit 163 may perform an arithmetic operation on the calculated peripheral device manipulation load “T(n)”, the speech load “V(t)”, the drowsiness load “P(t)” and the observation neglect load “E(t)” to calculate the driving interruption load “W i “ to calculate.

[0158] Furthermore, depending on the status of the vehicle and the case, at least one of the loads detected based on the observation status information may not be used to calculate the driving interruption load.

[0159] The control unit 163 can control the driving manipulation load “W d “ with the driving interruption load “W i “ to determine the driver’s safety driving status.

[0160] In detail, if a value of the driving interruption load “W i ”, which is dependent on the driving manipulation load “W d “ is subtracted, equal to a predetermined equivalent load “W l " or less than this, the control unit 163 determines that the driver is not in a safe driving state. The predetermined substitute load "W l“Here, “W” may be a value extracted from a driving manipulation load and driving information (including a gear change status, a vehicle acceleration, a steering operation, and / or the like) through an experiment based on a state of an experienced driver and stored in the memory, and may be varied depending on a driving state (W d ∝ W l ). This means that a condition expressed by the following equation (12) can be satisfied to determine a driver's driving status as a safety driving status: ∑l=l1l=i2{Wd(t,n)⋅(v(t)+T(n)+E(l)+P(t))}≥W,Wd=d1l1+d2l2+d3t3+d4t4 where t denotes a time value and n is a frequency value.

[0161] The output unit 165 may be an element for outputting an image and a warning sound and may include a liquid crystal display (LCD) and a speaker. For example, according to Fig. 16(b), the output unit 165 outputs the image and the warning sound through a cluster. Alternatively, the output unit 165 may display the image using an audio display device. Furthermore, according to Fig. 16(a), the output unit 165 displays a safe driving status determination result and its result value, which are obtained by the control unit 163, on the screen in bar chart form. In this case, the bar chart may be displayed in different colors.

[0162] For example, the control unit 163 may control the output unit 165 to output information about a current load based on a difference between the travel manipulation load and the travel interruption load in the form of a bar chart on a screen.

[0163] In detail, a result value “C” obtained by calculating a difference between the driving manipulation load “W d“ and the driving interruption load “W i “, which is calculated by the control unit 163, by the equivalent load “W l “ will be shown as the percentage unit of the bar chart.

[0164] For example, a case where driving the vehicle is difficult is set to 75%. If the result value "C" is 75% or higher, the control unit 163 can control the output unit 165 to display the bar chart in red. Furthermore, by controlling the output unit 165 to flicker the bar chart, the control unit 163 can warn the driver of a danger that may occur while driving the vehicle.

[0165] In this case, the control unit 163 may control the output unit 165 to output the warning sound so as to warn the driver.

[0166] If the result value “C” is less than 75%, the bar chart can be displayed in green, informing the driver that a safety status exists.

[0167] Furthermore, if the result value “C” obtained by dividing a difference between the driving manipulation load “W d “ and the driving interruption load “W i “ by the equivalent load “W l “ is equal to or above a certain level (e.g., 85%), the control unit 163 forcibly cuts off the power supply to an AVN that has no difficulty in handling the safe driving of the vehicle.

[0168] Furthermore, the control unit 163 can increase the volume of the warning tone output by the output unit 165 so that it is proportional to the energy of the result value “C” so that the driver receives a strong warning of a dangerous driving situation.

[0169] As described above, by means of the present invention, a driving load occurring during vehicle driving can be compared with a driving interruption load that interrupts the driving of the vehicle, such as a peripheral device, driver drowsiness, and / or the like, and thus the driver's safe driving status can be checked. By means of the present invention, a warning can be issued to the driver when a load for interrupting the driving of the vehicle is high, and, as appropriate, by means of the present invention, the operation of an element (e.g., a peripheral device) that is not essential to the driving of the vehicle can be forcibly stopped to reduce the driving interruption load that interrupts the driving of the vehicle, thus enabling the driver to drive the vehicle safely.

[0170] Fig. 17 shows a flowchart illustrating a driver status sensing method according to an embodiment of the present invention.

[0171] First, in step S1710, the driver status sensing system 160 may acquire driving manipulation information and carelessness status information of a driver.

[0172] Specifically, the driver status sensing system 160 may sense driving manipulations from driving control units essential to driving the vehicle to obtain driving manipulation information.

[0173] For example, the driver status detection system 160 may obtain driving manipulation information by sensing at least one of accelerator pedal (ACC pad) manipulation, brake pedal manipulation, multifunction switch manipulation, and steering handle manipulation. In this case, the driver status sensing system 160 may check the vehicle speed, and if the vehicle speed is equal to or higher than a certain speed (e.g., 10 km / h), the driver status sensing system 160 may detect the driving manipulation information.

[0174] For example, the driver status sensing system 160 may detect the number “n A " of operations of an accelerator pedal manipulated by the driver during each specific period within a predetermined time period. Likewise, the driver status sensing system 160 may sense the numbers "n B ' and 'n C“ of brake pedal and clutch pedal operations manipulated during each specific time period.

[0175] As another example, the driver status sensing system 160 may detect the number “n M "of actuations of each multifunction switch manipulated by the driver during a specific period of time. Here, each multifunction switch can be a switch for activating a vehicle's windshield wiper or a vehicle lamp, such as a turn signal indicator.

[0176] As another example, the driver status sensing system 160 may sense an angular velocity of a steering wheel manipulated by the driver during a certain period of time. In this case, the driver status sensing system 160 may measure an angular change amount of the steering wheel to calculate the angular velocity without separately measuring the angular velocity.

[0177] Further, the driver status sensing system 160 may detect carelessness status information based on a peripheral device controlled separately by the driver while the vehicle is running, based on the driver's voice, and based on facial information.

[0178] For example, the driver status sensing system 160 may detect the number “n T "of manipulations of peripheral devices performed by the driver while the vehicle is moving. Here, the peripheral devices may not be devices essential to the driving process performed by the driver, such as an AVN or a vehicle climate control system, but may be control units manipulated for the convenience of the driver or a vehicle interior.

[0179] As another example, the driver status sensing system 160 may sense the driver's voice via a microphone located at a specific location in the vehicle. For example, the driver status sensing system 160 may check a utterance generation time and an impulse level (voice pitch) of speech data received from the microphone during a specific time.

[0180] As another example, the driver status sensing system 160 may receive a facial image of the driver from a camera arranged at a specific position on the vehicle to capture eye closure information and observation neglect information of the driver. Fig. 11B the camera may have a light-emitting diode (LED) for near-infrared light to record an image during the day and night.

[0181] For example, the driver status sensing system 160 may separately extract an eye area image of the driver from a face image of the driver. In this case, the eye area image may be extracted from the driver's face image through image processing. This means that the driver status sensing system 160 may obtain carelessness status information, such as the driver's eyes closing or observation neglect, based on the driver's face image captured by the camera and the eye area image included in the face image.

[0182] The driver status sensing system 160 can detect the driver's eye closure information based on the eye area image extracted from the driver's face image. According to the Fig. 12A and Fig. 12B, the driver status sensing system 160 may sense an eyelid area from the eye area image, and when a sum of ∠A and ∠B, which are angles of eyelids, is equal to or less than ∠C, which is a predetermined angle threshold (∠A + ∠B ≤ ∠C), the carelessness status sensing unit 161B may determine that the driver closes his eyes.

[0183] The driver status sensing system 160 may check the angles of the eyes during a predetermined period of time to sense the number of times the driver closes his eyes, and calculate a certain time value and the number of times the driver closes his eyes, thereby obtaining a drowsiness time (a drowsiness index) of the driver.

[0184] Furthermore, the driver status sensing system 160 can detect the driver's observation neglect based on the driver's face image and the eye area image included in the face image.

[0185] According to Fig. 15A to 15C, in an observation neglect area, observation neglect information can be acquired by checking (viewing direction) whether a driver's visual range is within a visual distance “η” in which no load exists when the vehicle is running, based on an angle of a vehicle wheel (ie, an angle change amount (ΘA or ΘB in Fig. 15A) of a wheel in a center of the vehicle) instead of an angle of a steering wheel. The field of view can be checked by performing an operation in which an angle of a face (α in Fig. 15B) is calculated from the driver’s face image received from the camera and then a position (pupil position) of a pupil (β in Fig. 15C) is measured in the eye area image.

[0186] Specifically, in a case where a shift lever is in D-shift or N-shift with an angle of the steering wheel within a certain angle range, in a situation where the driver's visual range does not enter a driving no-load visual range (a, b) for a certain time, the driver status sensing system 160 may determine an observation neglect, check a corresponding time, and acquire the observation neglect information.

[0187] First, in step S1720, the driver status sensing system 160 may calculate the driving tampering load and the driving interruption load based on the driving tampering information and the carelessness status information acquired in step S1710.

[0188] For example, the driver status sensing system 160 may detect the driving manipulation load “W d " based on the driving manipulation information. Specifically, the driver status sensing system 160 may acquire the driving manipulation information, read an item-based weight value of the driving manipulation information from the memory, and calculate the driving manipulation load.

[0189] The weighting value here can be a value extracted and predetermined for each item of driving manipulation information through various experiments. Alternatively, the weighting value can be a value arbitrarily predetermined for each item by a worker.

[0190] Furthermore, the driver status sensing system 160 can calculate a load for each item based on an item included in the driving manipulation information and a weight value corresponding to the item. For example, when driving manipulation information in the form of accelerator pedal manipulation, brake pedal manipulation, multifunction switch manipulation, and control handle manipulation has been received, the driver status sensing system 160 can retrieve a weight value "D" from the memory. A “ of the accelerator pedal manipulation, a weighting value “D B “ of the brake pedal manipulation, a weighting value “D M" of the multifunction switch manipulation and a weight value "Θ" of the control handle manipulation. For example, if the time information and the frequency information of each item have been acquired in units of 50 ms within a 200 ms period, the driving manipulation load "W d “ for a duration of 200 ms can be expressed by equation (10).

[0191] As another example, the driver status sensing system 160 may detect the driving interrupt load “W i " based on the carelessness status information. Specifically, the driver status sensing system 160 may calculate loads of the objects included in the carelessness status information and sum the calculated loads of the objects to calculate the driving interruption load.

[0192] The driver status sensing system 160 may calculate a peripheral device manipulation load "T(n)" based on the number of peripheral device manipulations. For example, the driver status sensing system 160 may further calculate the peripheral device manipulation load by considering a peripheral device manipulation weight value stored in the memory. For example, when the number of peripheral device manipulation inputs performed by the driver for a second duration of 20 ms is detected, the driver status sensing system 160 may perform an arithmetic operation on the number of peripheral device manipulation inputs and a weight value to calculate the peripheral device manipulation load "T(n)".

[0193] Furthermore, the driver status sensing system 160 may calculate a speech load "V(n)" based on detected driver speech data. Specifically, the driver status sensing system 160 may calculate the speech load based on an impulse threshold of the speech data stored in the memory. For example, the driver status sensing system 160 may calculate a time at which speech data with the impulse threshold or higher is received among the driver speech data received within a certain time (e.g., 200 ms), thereby calculating a speech load "V(t)".

[0194] Further, the driver status sensing system 160 may calculate a drowsiness load “P(t)” and a driving observation neglect load “E(t)” based on the eye-closing information and the observation neglect information included in the carelessness status information.

[0195] In this case, the driver status sensing system 160 may perform a control according to the flowchart of Fig. 14 for checking the driver's eye closure. The driver status sensing system 160 may check the number "y" of eye closures based on an eyelid angle per a certain time "x" within a certain measurement period "N" to calculate an eye closure time (a drowsiness time) of the driver. Furthermore, the driver status sensing system 160 may convert a value obtained by counting (P) in a case where the PERCLOS value is at or above 30% during a certain time into a load factor to calculate the drowsiness load "P(t)".

[0196] Furthermore, the driver status sensing system 160 may calculate the observation neglect load “E(t)” based on the observation neglect information contained in the carelessness status information. According to Fig. 15A to 15C, if the field of view deviates from the visual distance “η” at which no load exists during driving, a factor value of the load factor “P” can be changed according to a deviation range. According to Fig. 15A, when a deviation range (a',b') is predetermined, the driver status sensing system 160 may read a factor value of a load factor from the memory for each range corresponding to a current driver visibility range, and thereby calculate the observation neglect load "E(t)".

[0197] The driver status sensing system 160 may perform an arithmetic operation on the calculated peripheral device manipulation load “T(n)”, the speech load “V(t)”, the drowsiness load “P(t)”, and the observation neglect load “E(t)” to calculate the driving interruption load “W i “ to calculate.

[0198] In step S1730, the driver status sensing system 160 may determine a predetermined substitute load “W l “ with a difference between the driving manipulation load “W d “ and the driving interruption load “W i " compare.

[0199] In detail, if a value of the driving interruption load “W i ”, which is dependent on the driving manipulation load “W d “ is subtracted, equal to a predetermined equivalent load “W l" or less than this, the driver status sensing system 160 determines that the driver is not in a safe driving state. The predetermined substitute load "W l “Here, “W” may be a value extracted from a driving manipulation load and driving information (including a gear change status, a vehicle acceleration, a steering operation, and / or the like) through an experiment based on a state of an experienced driver and stored in the memory, and may be varied depending on a driving state (W d ∝ W l ). In this case, the driver status sensing system 160 may calculate a result value “C” obtained by multiplying the difference between the driving manipulation load “W d “ and the driving interruption load “W i “ by the predetermined equivalent load “W l “ is divided.

[0200] For example, a case where driving the vehicle is difficult may be set to a percentage threshold, and when the result value "C" calculated in step S1730 is equal to or more than the percentage threshold, the driver status sensing system 160 may issue a warning to the driver in step S1740.

[0201] If 75% or higher is set as a situation where driving the vehicle is difficult, the driver status sensing system 160 may display a red graph showing the current load in red. Furthermore, the driver status sensing system 160 may alert the driver to a dangerous driving situation by flickering the red graph. In this case, the status sensing system 160 may emit a warning sound to alert the driver.

[0202] For example, if the result value "C" calculated in step S1730 is less than 75%, in step S1730, the status sensing system 160 may display the red graph history, but with the current load shown in green, thus informing the driver of a safe status.

[0203] As described above, by means of the present invention, a driving load occurring during vehicle driving can be compared with a driving interruption load that interrupts the driving of the vehicle, such as a peripheral device, driver drowsiness, and / or the like, and thus the driver's safe driving status can be checked. By means of the present invention, a warning can be issued to the driver when a load for interrupting the driving of the vehicle is high, and, as appropriate, by means of the present invention, the operation of an element (e.g., a peripheral device) that is not essential to the driving of the vehicle can be forcibly stopped to reduce the driving interruption load that interrupts the driving of the vehicle, thus enabling the driver to drive the vehicle safely.

[0204] Furthermore, if the driving habit reveals that a vehicle is driven by several different drivers, a learning process is carried out without taking into account that different drivers have different learned driving patterns and driving situations, so that a driver pattern recognition effect is prevented.

[0205] Furthermore, the present invention solves the problem that lanes are not recognized in the side control when a backlight occurs and a vehicle enters a tunnel, so that an available area for the side control device is increased.

[0206] Furthermore, the present invention can be used in conjunction with an autonomous vehicle control device that is still being developed.

[0207] As described above, according to embodiments of the present invention, by precisely examining a driving habit of a driver, a start timing for lateral control of a vehicle used in autonomous driving is adjusted so that a steering uncertainty feeling felt by the driver is eliminated.

[0208] While some embodiments have been described above, it should be understood that various modifications may be made. For example, suitable results may also be achieved if the described techniques are performed in a different order and / or if components in a described system, architecture, device, or circuit are combined in different ways and / or are replaced or supplemented by other components or their equivalents. Thus, other implementations are also within the scope of the claims.

Claims

[1] A system (100) for managing a driving hazard index for a vehicle, the system comprising: a driving trajectory generator (130-1) for a driver's vehicle, configured to generate a driving trajectory of the driver's vehicle based on driving information of the driver's vehicle input from an internal sensor (110) that senses a driving situation of the driver's vehicle; a peripheral vehicle trajectory generator (130-1) configured to generate a trajectory of a peripheral vehicle based on environmental information input from an external sensor (120) that senses the driving situation of the driver's vehicle; a trajectory load calculation device (130-5) configured to calculate a trajectory load representing a comparison result obtained by comparing a predetermined threshold value with a trajectory distance, which is the difference between the peripheral vehicle trajectory and the driving trajectory of the driver's vehicle; and a hazard index manager (130-11) configured to generate a driving hazard index according to the calculated trajectory load. [2] The system according to claim 1, further comprising a peripheral vehicle load calculating device (130-7) configured to calculate a peripheral vehicle load based on the number of peripheral vehicles included in the surrounding information and whether the peripheral vehicles are changing lanes. [3] The system according to claim 2, wherein the peripheral vehicle load calculating device (130-7) divides the vehicle into a plurality of danger sections with respect to the driver's vehicle based on a time to collision (TTC), and calculates the peripheral vehicle load based on the number of peripheral vehicles included in each of the plurality of divided danger sections and whether the peripheral vehicles perform a lane change. [4] The system according to claim 2, wherein the danger index manager (130-11) calculates the driving danger index according to a value obtained based on the trajectory load and the peripheral vehicle load. [5] The system according to claim 1, further comprising a road load calculating device (130-9) configured to calculate a road load based on at least one of road shape, road surface status and traffic status included in the environmental information. [6] The system according to claim 5, wherein the danger index manager (130-11) calculates the driving danger index according to a value obtained based on the trajectory load and the road load. [7] The system of claim 1, further comprising: a peripheral vehicle load calculation device (130-7) configured to calculate a peripheral vehicle load based on the number of peripheral vehicles included in the surrounding information and on whether the peripheral vehicles perform a lane change; and a road load calculation device (130-9) configured to calculate a road load based on at least one aspect of road shape, road surface status and traffic status included in the environmental information, wherein the hazard index manager (130-11) calculates the driving hazard index according to a value obtained based on the trajectory load, the peripheral vehicle load and the road load. [8] A method for handling a driving hazard index for vehicles, the method comprising: Generating a driving trajectory of a driver's vehicle based on driving information of the driver's vehicle input from an internal sensor (110) that senses a driving situation of the driver's vehicle; Generating a trajectory of a peripheral vehicle based on environmental information input from an external sensor (120) that senses a driving situation of the driver's vehicle; Calculating a trajectory load representing a comparison result obtained by comparing a predetermined threshold with a trajectory distance that is a difference between the peripheral vehicle trajectory and the driving trajectory of the driver's vehicle; and Generate a driving hazard index according to the calculated trajectory load. [9] The method of claim 8, further comprising calculating a peripheral vehicle load based on the number of peripheral vehicles included in the surrounding information and whether the peripheral vehicles are changing lanes. [10] The method of claim 9, wherein calculating the peripheral vehicle load comprises: Performing a division into several hazard sections with respect to the driver’s vehicle based on a time to collision (TTC); and Calculating the peripheral vehicle load based on the number of peripheral vehicles contained in each of the multiple divided hazard sections and whether the peripheral vehicles change lanes. [11] The method of claim 8, further comprising calculating a road load based on at least one aspect of road shape, road surface status, and traffic status included in the environmental information. [12] The method of claim 11, wherein generating the driving hazard index comprises generating the driving hazard index according to a value obtained based on the trajectory load and the road load.

Images