Driving assistance device and driving assistance method

Abstract

Based on the sequence of safety checks performed by the driver, the system evaluates the driver's driving skills and provides driving assistance tailored to those skills. [Solution] A driver assistance device that assists a driver in driving, comprising: a driver information acquisition unit that acquires driver information including the driver's gaze information; a vehicle information acquisition unit that acquires vehicle information of the vehicle being driven by the driver; a transition state generation unit that generates transition states of the driver's actions using the driver information and the vehicle information; a driving score calculation unit that calculates a driving score for the driver's actions based on a correspondence table of the action transition states and scores generated by the transition state generation unit; a calculation reason creation unit that creates a reason for calculating the driving score calculated by the driving score calculation unit using the action transition states; and an output unit that outputs the driving score and the reason for calculating the driving score.

Description

Technical Field

[0001] The present invention relates to a driving support device that evaluates a driver's driving skills and supports driving according to the evaluated driving skills, and a driving support method.

Background Art

[0002] In recent years, vehicles equipped with ADAS (Advanced Driving Assistance System) (hereinafter referred to as "ADAS vehicles") have been rapidly spreading. In addition, as a function of ADAS, there has been proposed a system that changes the mode of driving support according to the driver's safety confirmation behavior.

[0003] For example, the abstract of Patent Document 1 states that "The safety confirmation diagnosis system of the present invention includes a camera (2) that photographs the driver's face, a face orientation detection unit (3) that detects the orientation of the driver's face based on the image data of the driver's face, a driving scene detection unit (9) that detects the driving scene of the vehicle, an evaluation section division unit (10) that divides the driving scene into evaluation sections with different evaluation criteria, and a safety confirmation behavior evaluation unit (11) that evaluates the driver's safety confirmation behavior based on the orientation of the driver's face and the evaluation section of the vehicle's driving scene."

[0004] Also, in paragraph 0057 of the same document, "Even when performing autonomous driving such as auto cruise or lane keep assist, as long as the responsibility for safe driving lies with the driver, the driver always needs to perform safety confirmation. Therefore, in the second embodiment as well, the safety confirmation behavior is evaluated for each evaluation section of each driving scene, and when it is determined that the driver's safety confirmation behavior is not being appropriately performed, advice is presented to perform the safety confirmation behavior, or the safety confirmation direction is presented. Further, in the second embodiment, when it is repeatedly determined that the driver's safety confirmation behavior is not being appropriately performed, the operation mode of the autonomous driving device is changed, for example, the vehicle speed is controlled to be below the set value, or lane change of the vehicle is made impossible." Thus, Patent Document 1 discloses a safety confirmation diagnostic system that, if it is determined that safety confirmation actions have not been performed appropriately, prompts the driver to perform safety checks or restricts the mode of automated driving. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2017-151694 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, the safety confirmation diagnostic system described in Patent Document 1 does not evaluate safety confirmation behavior based on the time-series changes in driver behavior. Therefore, in the safety confirmation diagnostic system of Patent Document 1, if there is inappropriate behavior in any of the evaluation intervals, the safety confirmation behavior as a whole may be judged as inappropriate, regardless of the importance of that evaluation interval, making it difficult to perform a flexible evaluation of the order of the driver's safety confirmation behaviors.

[0007] Therefore, the present invention aims to provide a driver assistance device and a driver assistance method that evaluate the driver's driving skills based on the sequence of the driver's safety confirmation actions, and that provide driver assistance according to the evaluated driving skills. [Means for solving the problem]

[0008] To solve the above problems, one driver assistance device is a driver assistance device that assists a driver in driving, and comprises: a driver information acquisition unit that acquires driver information including the driver's gaze information; a vehicle information acquisition unit that acquires vehicle information of the vehicle being driven by the driver; a transition state generation unit that generates transition states of the driver's actions using the driver information and the vehicle information; a driving score calculation unit that calculates a driving score for the driver's actions based on a correspondence table of the action transition states and scores generated by the transition state generation unit; a calculation reason creation unit that creates a reason for calculating the driving score calculated by the driving score calculation unit using the action transition states; and an output unit that outputs the driving score and the reason for calculating the driving score. [Effects of the Invention]

[0009] The driver assistance device and driver assistance method of the present invention make it possible to evaluate the driver's driving skills based on the sequence of the driver's safety confirmation actions, and to provide driver assistance according to the evaluated driving skills. [Brief explanation of the drawing]

[0010] [Figure 1A] Hardware configuration diagram of the driver assistance device in Example 1. [Figure 1B] Functional block diagram of the driver assistance system in Example 1. [Figure 2] A plan view illustrating the vehicle's operating environment. [Figure 3] A flowchart illustrating an example of the operation of the driver assistance device in Example 1. [Figure 4] A diagram illustrating the state transition score of Example 1. [Figure 5] A flowchart illustrating the score calculation in Example 1. [Figure 6] Functional block diagram of the driver assistance system in Example 2 [Figure 7] A flowchart illustrating an example of the operation of the driver assistance device in Example 2. [Figure 8A]In Example 2, the behavior transition diagram at the time of performing the entrainment confirmation. [Figure 8B] In Example 2, the behavior transition diagram when the entrainment confirmation is not performed. [Figure 9] In Example 2, the flowchart for explaining the score calculation.

Mode for Carrying Out the Invention

[0011] Hereinafter, examples of the present invention will be described with reference to the drawings.

Example

[0012] First, the driving support device 1 according to Example 1 of the present invention mounted on an ADAS vehicle will be described using FIGS. 1A to 5.

[0013] <ADAS vehicle> First, the ADAS vehicle will be described. The ADAS vehicle is a vehicle equipped with an advanced driving support system (ADAS), and has an external sensor, a vehicle sensor, and an HMI (Human Machine Interface) as components for realizing driving support control.

[0014] The external sensor is a sensor capable of recognizing the surrounding environment of the vehicle, and examples thereof include a monocular camera, a stereo camera, a lidar, a millimeter-wave radar, an ultrasonic sensor, an infrared sensor, and the like.

[0015] The vehicle sensor is a sensor capable of acquiring the position information, longitudinal and lateral accelerations, yaw rate, etc. of the vehicle, and examples thereof include a wheel speed sensor, an acceleration sensor, an angular velocity sensor, a terminal of a global navigation satellite system (GNSS), and the like.

[0016] The HMI is a device for interaction with the driver, and examples thereof include an eye tracker and an in-vehicle camera for acquiring the driver's line-of-sight direction, etc., a touch-type display operable by the driver, and an audio device for outputting warnings.

[0017] <Driving support device 1> Figure 1A is a hardware configuration diagram of the driver assistance device 1 of this embodiment. As illustrated here, the driver assistance device 1 is a computer system in which the hardware, including an input interface for receiving information from external sensors, an output interface for outputting information to the HMI, a storage device such as a semiconductor memory for storing information, and a computing device such as a CPU (Central Processing Unit) for processing information, are interconnected via an internal bus. Note that some or all of this hardware may be replaced with dedicated devices, general-purpose machine learning machines, DSPs (Digital Signal Processors), FPGAs (Field-Programmable Gate Arrays), GPUs (Graphics Processing Units), PLDs (Programmable Logic Devices), etc.

[0018] Furthermore, Figure 1B is a functional block diagram of the driver assistance device 1 of this embodiment, which is realized by the hardware of Figure 1A working together to execute a predetermined driver assistance program. As shown here, the driver assistance device 1 includes the following functional units: a driver information acquisition unit 11, a vehicle information acquisition unit 12, a transition state generation unit 13, a driving score calculation unit 14, a calculation reason creation unit 15, and an output unit 16.

[0019] <Example of operation of driver assistance device 1> Figure 2 is a plan view illustrating the driving environment of an ADAS vehicle (hereinafter referred to as "vehicle 21") equipped with the driver assistance device 1 of this embodiment, showing a situation where vehicle 21 is about to turn left at an intersection. Figure 3 is a flowchart illustrating an example of the operation of the driver assistance device 1 while vehicle 21 is in motion. The following explanation will proceed sequentially, based on the driving environment shown in Figure 2, and each step in Figure 3.

[0020] <<Step S31>> First, in step S31, the driver information acquisition unit 11 acquires driver information from the HMI or the like. This driver information includes information such as the driver's line of sight and head rotation angle. By utilizing this information, the driver information acquisition unit 11 can determine what the driver is looking at. If the object to be looked at has already been calculated by another device, that information may be acquired as driver information.

[0021] Eye-line information is generally represented as a single point (viewpoint), but the surrounding area is often also visible. Therefore, eye-line information can be represented as a field of view, formed from the viewpoint and the surrounding area. Eye-line information is information indicating the direction in which the driver is actually looking with their eyes. This eye-line information can be acquired directly by an eye tracker worn by the driver, or it can be estimated using images captured by an in-car camera. Eye-line information can also be estimated from the driver's gaze information and head turn angle.

[0022] <<Step S32>> In step S32, the vehicle information acquisition unit 12 acquires vehicle information from vehicle sensors and external sensors. This vehicle information includes vehicle motion information such as vehicle speed and steering angle, position information acquired from GNSS, and various external information obtained from external sensors. Generally, vehicle motion information that can be acquired from vehicle sensors includes, but is not limited to, vehicle speed, longitudinal acceleration, lateral acceleration, yaw rate, steering angle, and steering angular velocity. Furthermore, longitudinal acceleration can be calculated from the time derivative of vehicle speed, and lateral acceleration can be calculated from the relationship between yaw rate and velocity, so it is not always necessary to directly obtain physical quantities.

[0023] <<Step S33>> In step S33, the vehicle information acquisition unit 12 estimates the driving state of the vehicle 21 based on the vehicle information acquired in step S32. Specifically, the estimated driving state is the type of driving scene. In the driving environment shown in Figure 2, the driving state is estimated to be a "left turn at an intersection scene". Alternatively, the driving state may be estimated by dividing it into more detailed driving scenes. For example, the "left turn at an intersection scene" in Figure 2 may be divided into three scenes: before entering the intersection 22, inside the intersection 23, and exiting the intersection 24.

[0024] <<Step S34>> In step S34, the vehicle information acquisition unit 12 determines whether the driving state has changed. If the driving state has changed, the process proceeds to step S35; otherwise, it returns to step S31. Here, a change in driving state would be, in the example in Figure 2, when the "left turn at an intersection scene" ends and the process transitions to the "straight driving scene".

[0025] <<Step S35>> In step S35, the transition state generation unit 13 generates a state transition score. Here, the state transition score represents the quality of the action transitions that the driver can take in various driving scenarios, expressed as a score of magnitude.

[0026] Figure 4 shows an example of a state transition score table referenced in this step. The score table shown here is a set of information representing the correspondence between the sequence of actions a driver can take in a "left turn at an intersection" scenario and the state transition scores for those actions.

[0027] For example, if a driver's sequence of actions is a combination of past state "paying attention ahead" and current state "paying attention ahead," the driver is only focusing on what is in front of them (in other words, not paying attention to anything else), so this sequence of actions will receive a low rating (score 0). On the other hand, if the sequence of actions is a combination of past state "paying attention ahead" and current state "checking the rearview mirror," the driver is paying attention to both what is in front and what is behind, so this sequence of actions will receive a high rating (score 1). Similarly, if the sequence of actions is a combination of past state "checking for blind spots" and current state "turning left," the driver is turning left after checking for blind spots, so this sequence of actions will receive a higher rating (score 2).

[0028] Needless to say, the states (actions) in the state transition score table in Figure 4 are changed according to the conditions of the road. For example, at the intersection in Figure 2, since there is a traffic light 25 ahead, the action of "checking the traffic light" may be added as a past state or present state.

[0029] Note that in the score table in Figure 4, state transition scores were generated based on combinations of two consecutive states, but state transition scores may also be generated based on combinations of three or more consecutive states. In that case, state transition scores may also be generated according to the order of the three or more states.

[0030] Figure 5 is a flowchart illustrating the details of step S35.

[0031] First, in step S35a, the driver information acquisition unit 11 acquires driver information in the same manner as in step S31.

[0032] In step S35b, the vehicle information acquisition unit 12 acquires vehicle information in the same manner as in step S32.

[0033] In step S35c, the transition state generation unit 13 estimates the driver's actions based on the driver information and vehicle information acquired in steps S35a and S35b.

[0034] In step S35d, the transition state generation unit 13 determines whether the driver's behavior has changed. If the driver's behavior has changed, the process proceeds to step S35e; otherwise, it returns to step S35a.

[0035] In step S35e, the driving score calculation unit 14 adds state transition scores based on the current driver behavior, while referring to the state transition score table illustrated in Figure 4.

[0036] In step S35f, the driving score calculation unit 14 updates the state transition score table based on the actions the driver has taken up to that point. For example, after checking the rearview mirror once, the scores in the row where the current state in Figure 4 is checking the rearview mirror may all be set to 0. By repeating these steps, the score for each driving state is calculated, and the driver's driving skill is determined.

[0037] Furthermore, the method for calculating the score in step S35 may be further refined. For example, in the safety check under the environment shown in Figure 2, performing a check for vehicles getting in the way, which should ideally be done before entering the intersection 22, inside the intersection 23 is not considered appropriate action. Therefore, a high evaluation (score 1) may be calculated only when a safe action is performed within a predetermined area, and a low evaluation (score 0) may be calculated if the same action is performed at an inappropriate time. To determine whether the vehicle is traveling within a predetermined area, map information and vehicle position information, such as those found in a navigation system, can be utilized.

[0038] Alternatively, the state transition score may be calculated separately for visual skills and operational skills. Examples of visual skills include the actions shown at the top of Figure 4. For operational skills, the ability to perform actions such as deceleration, acceleration, and steering in a sequential manner is evaluated.

[0039] <<Step S36>> In step S36, the driving score calculation unit 14 calculates a driving score based on the driver's behavior history. For example, as shown in Figure 2, when vehicle 21 turns left at an intersection, the driver performs the left turn while keeping an eye on the road ahead, checking the rearview mirror, checking the side mirrors, and checking for blind spots. Therefore, the driving score calculation unit 14 obtains a driving score by summing up the individual state transition scores generated by the transition state generation unit 13 in step S35.

[0040] <<Step S37>> In step S37, the calculation reason generation unit 15 stores time-series data of the driver's driving behavior. At this time, the calculation reason generation unit 15 may not only store time-series data of actual driving behavior, but may also estimate and store exemplary behavioral transitions that would result in a higher score. By presenting these exemplary behavioral transitions to the driver, the driver can reflect on their driving afterward, which can help improve their driving skills.

[0041] <<Step S38>> In step S38, the output unit 16 outputs a warning to the driver according to the results of steps S36 and S37. There are various methods for outputting the warning, such as a warning sound, voice guidance, or drawing on an LCD panel or HUD (Head-Up Display), but in this embodiment, the method is not limited.

[0042] Furthermore, the level of warning may be adjusted according to the driving score. It is also possible to estimate dangerous driving based on the driving score. For example, there are three levels of dangerous driving: "accident," "near miss," and "near miss precursor." In this case, if the situation exceeds the probability of dangerous driving occurring at level 1, it is classified as an "accident"; if it exceeds the probability of dangerous driving occurring at level 2, it is classified as a "near miss"; and if it exceeds the probability of dangerous driving occurring at level 3, it is classified as a "near miss precursor," and the level of warning may be adjusted for each. A "near miss" is an instance where sudden braking or steering occurs due to a delay in seeing an obstacle or signal indication. A "near miss precursor" is driving that did not involve sudden braking or steering but had the potential to create danger. An example of this is when blind spot checks were not performed.

[0043] Based on the above, by utilizing the driver assistance device described in Example 1, it is possible to appropriately output warnings to the driver and contribute to safe driving and improvement of the driver's driving skills. To improve driver skills, it is also possible to review the time-series data of the actions taken by the driver afterward.

[0044] Furthermore, this driver assistance system can not only be installed in vehicles, but can also be used to improve drivers' driving skills by being installed in driving simulators. [Examples]

[0045] Next, the driver assistance device 1 according to Embodiment 2 of the present invention will be described using Figures 6 to 9. Note that common points with Embodiment 1 will not be explained again.

[0046] Figure 6 is a functional block diagram of the driver assistance device 1 of this embodiment. As shown here, the driver assistance device 1 of this embodiment differs from Embodiment 1 in that the transition state generation unit 13 creates a transition state diagram that represents the transition states as diagrams.

[0047] Figure 7 is a flowchart illustrating an example of the operation of the driver assistance device 1 while the vehicle 21 is in motion. The following explanation will proceed sequentially, assuming the driving environment shown in Figure 2, for each step in Figure 7. Note that steps S71 to S74 are equivalent to steps S31 to S34 in Figure 3; therefore, the explanation will focus on steps S75 onwards.

[0048] <<Step S75>> In step S75, the transition state generation unit 13 generates a transition state diagram. Here, a transition state diagram is a diagram that graphically represents the driver behavior over time, with driving actions as nodes and transitions between each action as edges.

[0049] Figure 8A is an example of a transition diagram in the "left turn at an intersection" scenario shown in Figure 2, where a blind spot check is performed at an appropriate time. In this example, the driver, who was paying attention to the road ahead, checks the rearview mirror, the side mirrors, and then a blind spot check before turning left. In this case, according to the state transition score table in Figure 4, the scores are added sequentially as +1, +1, +1, and +2, so the total driving score is 5.

[0050] On the other hand, Figure 8B is an example of a transition diagram in the "left turn at an intersection" scene shown in Figure 2, where the driver fails to check for blind spots at an appropriate time. In this example, the driver, who was paying attention to the road ahead, checks the rearview mirror and the side mirrors, but then turns left without checking for blind spots. In this case, according to the state transition score table in Figure 4, the score is added sequentially as +1, +1, +1, so the total driving score is 3. In other words, since the driving score in Figure 8B is lower than that in Figure 8A, the driver's driving skill in Figure 8B can be evaluated as being low.

[0051] <<Step S76>> In step S76, the driving score calculation unit 14 calculates the driving score based on the transition state diagram.

[0052] Here, we will explain the method for calculating scores using transition state diagrams, with reference to Figures 8A to 9.

[0053] Steps S76a to S76e perform the same operations as steps S35a to S35e in Figure 5 of Example 1.

[0054] In step S76f, the transition state diagram is dynamically modified based on the driver's past operating state. For example, as shown in Figure 8A, if an engagement check is performed, the score attached to the edge is updated. In addition to updating the edge score, the graph structure may also be changed according to the time series by deleting nodes, deleting edges, etc.

[0055] <<Step S77>> In step S77, the calculation reason creation unit 15 stores the reason for calculating the score. At this time, exemplary driving behavior transitions may be stored using graph search theory or the like. Presenting exemplary driving behavior transitions to the driver contributes to improving driving skills.

[0056] <<Step S78>> In step S78, the calculation reason generation unit 15 outputs a warning to the driver based on the driving score.

[0057] Based on the above, in this embodiment, by using a transition state diagram, it is possible to easily handle whether the driver is appropriately performing safe actions, and to output warnings with high accuracy according to the probability of dangerous driving.

[0058] While embodiments of the vehicle control device according to this disclosure have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and any design changes, etc., that do not depart from the gist of this disclosure are also included in this disclosure. [Explanation of Symbols]

[0059] 1. Driving support system, 11. Driver information acquisition unit, 12 Vehicle Information Acquisition Unit, 13 Transition state generation unit, 14. Driving score calculation unit, 15. Calculation Reasoning Creation Department, 16 Output section, 21 vehicles, 22 Scene before entering the intersection, 23 Intersection Scene, 24. Scene of leaving the intersection. 25 traffic lights,

Claims

1. A driver assistance device that assists the driver in driving, A driver information acquisition unit that acquires driver information including the driver's gaze information, A vehicle information acquisition unit that acquires vehicle information of the vehicle being driven by the aforementioned driver, A transition state generation unit generates transition states of the driver's driving actions using the driver information and the vehicle information, A driving score calculation unit calculates a driving score for the driver's actions based on a correspondence table of the transition states and scores of the actions generated by the transition state generation unit. A calculation reason creation unit creates the reason for calculating the driving score calculated by the driving score calculation unit using the transition state of the aforementioned actions, An output unit that outputs the aforementioned driving score and the reason for calculating the aforementioned driving score, A driver assistance device characterized by being equipped with the following features.

2. In the driving support device according to claim 1, The driver information acquisition unit acquires the driver's gaze information and head rotation angle, A driving assistance device characterized by calculating a target to be viewed from the aforementioned gaze information and the aforementioned head-turning angle.

3. In the driving support device according to claim 1, A driver assistance device characterized in that the aforementioned driving score is calculated based on the driver's past behavior.

4. In the driving support device according to claim 1, The aforementioned output unit is a driving support device characterized by outputting exemplary driving in a time series.

5. In the driving support device according to claim 1, The driver assistance device is characterized in that the output unit changes the warning level according to the magnitude of the driving score.

6. In the driving support device according to claim 1, The transition state generation unit generates a transition state diagram that represents the transition state as a figure, The driving support device is characterized in that the driving score calculation unit calculates a driving score for the driver's actions based on the transition state diagram and the score correspondence table.

7. In the driving support device according to claim 6, The states in the aforementioned transition state diagram are defined as nodes, and transitions as edges. A driving assistance device characterized in that each edge has a driving score.

8. In the driving support device according to claim 7, The score of the aforementioned edge is dynamically changed according to the transition state. A driver assistance system characterized by determining driving skills by adding dynamically changed driving scores in a time series.

9. A driver assistance method in which a computer system assists the driver in driving, A driver information acquisition step, which acquires driver information including the driver's gaze information, A vehicle information acquisition step in which vehicle information of the vehicle being driven by the aforementioned driver is acquired, A transition state generation step that generates transition states of the driver's driving actions using the driver information and the vehicle information, A driving score calculation step, which calculates a driving score for the driver's actions based on a correspondence table between the transition states of the actions and the scores, A calculation reason creation step to create the reason for calculating the driving score calculated using the transition state of the aforementioned behavior, An output step that outputs the aforementioned driving score and the reason for calculating the aforementioned driving score, A driving assistance method characterized by comprising the following:

Images