Lane departure intention estimation device

Abstract

A lane departure intention estimation device includes: an estimation unit that estimates that a driver does not have a lane departure intention when the driver of a host vehicle is in a distracted state or a non-awake state; and a prediction unit that, when the driver is neither in a distracted state nor in a non-awake state, predicts whether the driver has a lane departure intention on the basis of the time-series signal and the first classification signal using the learned machine learning model. The time series signal includes vehicle information, lane information, and target information. The first classification signal includes a signal indicating that the driver is neither in a distracted state nor in a non-awake state. The learned machine learning model is obtained by learning using a data set that learns a time series signal and learns a first classification signal and a tag.

Description

Technical Field

[0001] The present disclosure relates to a lane departure intention estimation device. Background Art

[0002] Japanese Unexamined Patent Application Publication No. 2022-077931 describes a driving assistance device that, if an intention to change lanes is determined during lane departure prevention, does not activate the lane departure prevention function. Furthermore, JP 2022-077931A describes a device that detects the driver's line of sight, gestures, and the like, acquired through image input devices such as various input switches and cameras, as an intention determination unit for determining the driver's intention to change lanes. Summary of the Invention

[0003] However, JP2022-077931A does not describe a specific technique for determining whether the driver intends to change lanes based on the results of detecting the driver's line of sight, gestures, etc. Therefore, it is impossible to appropriately estimate whether the driver intends to leave the lane using the technique described in JP2022-077931A.

[0004] In view of this problem, an object of the present disclosure is to provide a lane departure intention estimation apparatus that enables appropriate estimation of whether the driver of a host vehicle has a lane departure intention.

[0005] (1) One aspect of the present disclosure is a lane departure intention estimation device including an estimation unit and a prediction unit. The estimation unit estimates that the driver of the host vehicle does not have a lane departure intention when the driver of the host vehicle is in a distracted state or an unconscious state. The prediction unit predicts whether the driver of the host vehicle has a lane departure intention based on a time series signal and a first classification signal using a learned machine learning model when the driver of the host vehicle is neither distracted nor unconscious. The time series signal includes vehicle information, lane information, and target information, wherein the vehicle information is information related to the host vehicle, the lane information is information related to the lane in which the host vehicle is traveling, and the target information is information related to targets existing around the host vehicle. The first classification signal includes a signal indicating that the driver of the host vehicle is neither distracted nor unconscious. The learned machine learning model is obtained by learning using the learning time series signal and learning data. The learned time series signal includes learned vehicle information, learned lane information, and learned target information, wherein the learned vehicle information is information related to the learning vehicle, the learned lane information is information related to the lane in which the learning vehicle is traveling, and the learned target information is information related to targets existing around the learning vehicle. The learning data is a dataset of a first classification signal and a label. The first classification signal indicates that the driver of the learning vehicle is neither distracted nor unconscious. The label indicates whether the driver of the learning vehicle has a lane departure intention.

[0006] (2) The lane departure intention estimation device according to (1) may include a control unit that limits output of a lane departure warning when the prediction unit predicts that the driver of the host vehicle has a lane departure intention. The lane departure warning is a warning of lane departure of the host vehicle.

[0007] (3) The lane departure intention estimation device according to (1) may include a control portion that causes execution of the lane keeping assist to be restricted in a case where the prediction portion predicts that the driver of the host vehicle has a lane departure intention.

[0008] (4) The lane departure intention estimation device according to (1) may include a driver monitoring unit that outputs a first classification signal and a second classification signal based on an image captured by a driver monitoring camera. The image includes the driver of the main vehicle. The first classification signal includes a signal indicating that the driver of the main vehicle is neither in a distracted state nor in an unconscious state. The second classification signal includes a signal indicating that the driver of the main vehicle is in a distracted state or an unconscious state. In the case where the driver monitoring unit outputs the second classification signal including a signal indicating that the driver of the main vehicle is in a distracted state or an unconscious state and the prediction unit predicts that the driver of the main vehicle has a lane departure intention, the estimation result of the estimation unit may be given priority over the prediction result of the prediction unit. The estimation result indicates that the driver of the main vehicle does not have a lane departure intention. The prediction result indicates that the driver of the main vehicle has a lane departure intention.

[0009] (5) In the lane departure intention estimation device according to (4), a first classification signal output from the driver monitoring unit may be input to the prediction unit. The first classification signal may include a signal indicating an internal state of the driver of the host vehicle. The internal state is estimated by the driver monitoring unit based on an image captured by a driver monitoring camera. The image includes the driver of the host vehicle.

[0010] According to the present disclosure, it is possible to appropriately estimate whether the driver of the host vehicle has a lane departure intention. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described below with reference to the accompanying drawings, in which like numerals represent like elements, and in which: Figure 1 is a diagram showing an example of a host vehicle 1 to which the lane departure intention estimation device 16 according to the first embodiment is applied; Figure 2A is a diagram showing an example of the configuration and the like of the prediction section 3E; Figure 2B is a diagram showing an example of the configuration and the like of the prediction section 3E; Figure 3 is a diagram for describing an example of processing performed by the processor 163 of the lane departure intention estimation device 16 according to the first embodiment when the host vehicle 1 deviates from the lane in which the host vehicle 1 is traveling; Figure 4A 3G is a diagram for describing an example in which the control unit 3G causes the HMI 13 to limit the output of the lane departure warning; Figure 4B 3G is a diagram for describing an example in which the control unit 3G causes the HMI 13 to limit the output of the lane departure warning; Figure 4C3G is a diagram for describing an example in which the control unit 3G causes the HMI 13 to limit the output of the lane departure warning; Figure 4D is a diagram for describing an example in which the control portion 3G causes the HMI 13 to limit output of a lane departure warning; and Figure 5 1 is a diagram showing an example of data processing and flow in the host vehicle 1 to which the lane departure intention estimation device 16 according to the third embodiment is applied. DETAILED DESCRIPTION

[0012] Hereinafter, embodiments of a lane departure intention estimation apparatus according to the present disclosure will be described with reference to the accompanying drawings. First embodiment

[0013] Figure 1 1 is a diagram showing an example of a host vehicle 1 to which the lane departure intention estimation device 16 according to the first embodiment is applied. Figure 1 In the illustrated example, host vehicle 1 includes a surrounding situation sensor 11, a vehicle state sensor 12, a human-machine interface (HMI) 13, a driver monitoring camera 14, a vehicle control unit 15, a steering actuator 15A, a brake actuator 15B, a drive actuator 15C, and a lane departure intention estimation device 16. Surrounding situation sensor 11 detects lane markings defining the lane in which host vehicle 1 is traveling, as well as objects surrounding host vehicle 1 (e.g., nearby vehicles, obstacles, etc.), and transmits detection results to vehicle control unit 15 and lane departure intention estimation device 16. Surrounding situation sensor 11 includes, for example, a camera that images the area in front of host vehicle 1, a light detection and ranging (LiDAR) system, a radar, a sonar system, and the like. Detection results from surrounding situation sensor 11 include, for example, lane information related to the lane in which host vehicle 1 is traveling, and object information related to objects surrounding host vehicle 1. Lane information includes, for example, information indicating the horizontal position of lane lines in the camera image, the curvature of the lane in which the host vehicle 1 is traveling, etc. Target information includes, for example, information indicating the position and speed of a target relative to the host vehicle 1 .

[0014] The vehicle state sensor 12 detects the state of the host vehicle 1 and transmits the detection result to the vehicle control device 15 and the lane departure intention estimation device 16. The vehicle state sensor 12 includes, for example, a vehicle speed sensor, a steering torque sensor, etc. The detection result of the vehicle state sensor 12 includes, for example, vehicle information related to the host vehicle 1, etc. The vehicle information includes, for example, vehicle speed, steering torque, etc. The HMI 13 has functions such as receiving various operations by the driver of the host vehicle 1, and outputting information such as warnings by displaying information or reproducing information sounds to the driver of the host vehicle 1. The HMI 13 transmits signals indicating the operations of the driver of the host vehicle 1 to the vehicle control device 15. Warnings output by the HMI 13 include, for example, a lane departure alert (LDA), which is a warning in case the host vehicle 1 deviates from the lane in which the host vehicle 1 is currently traveling. The driver monitoring camera 14 is, for example, located above the steering column of the host vehicle 1 and captures images of the face and a portion of the upper body of the driver of the host vehicle 1. Furthermore, the driver monitoring camera 14 transmits the captured image data to the lane departure intention estimation device 16. In another example, the driver monitoring camera 14 may be located, for example, on the center cluster, rearview mirror, instrument panel, or instrument cover of the host vehicle 1 instead of the steering column.

[0015] exist Figure 1 In the example shown, a vehicle control device 15 controls the travel of the host vehicle 1. The vehicle control device 15 includes, for example, a driving assistance electronic control unit (ECU), and controls a steering actuator 15A, a brake actuator 15B, and a drive actuator 15C based on information (data and signals) sent from the surrounding situation sensor 11, the vehicle state sensor 12, the HMI 13, and the lane departure intention estimation device 16.

[0016] The lane departure intention estimation device 16 includes a microcomputer including a communication interface (I / F) 161, a memory 162, and a processor 163. The communication interface 161 includes an interface circuit. The memory 162 stores programs and various types of data used in the processing performed by the processor 163. The processor 163 has the functions of an acquisition unit 3A, a driver monitoring unit 3B, a first determination unit 3C, an estimation unit 3D, a prediction unit 3E, a second determination unit 3F, and a control unit 3G. The acquisition unit 3A acquires the detection results of the surrounding situation sensor 11 (such as lane information or target information), the detection results of the vehicle state sensor 12 (such as vehicle information), and data of images captured by the driver monitoring camera 14 (driver monitoring camera images).

[0017] The driver monitoring unit 3B measures the state of the driver of host vehicle 1 based on images captured by the driver monitoring camera 14 and other means. Specifically, the driver monitoring unit 3B senses the position and orientation of the driver's face and the open or closed state of the eyes of the driver of host vehicle 1, and determines whether the driver of host vehicle 1 is in a state where the driver can check surrounding conditions and perform driving operations. For example, the driver monitoring unit 3B estimates the drowsiness level of the driver of host vehicle 1 by using driver monitoring camera images to detect face areas, detect faces, estimate head posture, detect eye openness, and analyze eyelid behavior. In the case where the driver monitoring unit 3B determines that the driver of the main vehicle 1 is in an unconscious state, the driver monitoring unit 3B outputs an unconscious signal indicating that the driver of the main vehicle 1 is in an unconscious state as a driver monitoring signal. At the same time, in the case where the driver monitoring unit 3B determines that the driver of the main vehicle 1 is in an eye-open state, the driver monitoring unit 3B outputs an eye-open signal indicating that the driver of the main vehicle 1 is in an eye-open state as a driver monitoring signal. In addition, in the case where the driver monitoring unit 3B determines that the driver of the main vehicle 1 is in a distracted state, the driver monitoring unit 3B outputs a distraction signal (the distraction signal is one of the face direction signals) indicating that the driver of the main vehicle 1 is in a distracted state as a driver monitoring signal. The unconscious signal, the eye-open signal and the distraction signal (face direction signal) belong to "strong signals (for example, such as signals with high reliability output by the artificial intelligence (AI) used as the driver monitoring unit 3B)". If the driver of host vehicle 1 looks outside the field of view where safety needs to be checked (e.g., toward the passenger seat, car navigation, etc.), driver monitoring unit 3B outputs a distraction signal. Furthermore, if the driver of host vehicle 1 only looks to the left for a predetermined period of time while host vehicle 1 is traveling in the left lane of a two-lane straight road, driver monitoring unit 3B outputs a distraction signal. Specifically, driver monitoring unit 3B outputs a distraction signal if the driver of host vehicle 1 is not looking toward a target that needs to be checked.

[0018] Furthermore, if the driver monitoring unit 3B determines, based on the image captured by the driver monitoring camera 14 and the detection results of the surrounding situation sensor 11, that the driver of the host vehicle 1 is gazing at an object around the host vehicle 1, the driver monitoring unit 3B outputs a signal indicating the direction of the driver's gaze of the host vehicle 1 (a signal indicating the object the driver of the host vehicle 1 is gazing at (gaze target information)) as a driver monitoring signal. Furthermore, if the driver monitoring unit 3B determines that the driver of the host vehicle 1 has a high stress level, the driver monitoring unit 3B outputs a signal indicating the high stress level of the driver of the host vehicle 1 as a driver monitoring signal. Furthermore, if the driver monitoring unit 3B determines that the driver of the host vehicle 1 is inattentive, the driver monitoring unit 3B outputs a signal indicating the inattentive state of the driver of the host vehicle 1 as a driver monitoring signal. The signal indicating the direction of the driver's gaze of the host vehicle 1, the signal indicating the high stress level of the driver of the host vehicle 1, and the signal indicating the inattentive state of the driver of the host vehicle 1 are considered "weak signals (e.g., signals whose reliability as the AI output of the driver monitoring unit 3B is low)." The driver monitoring unit 3B outputs a weak signal as a "first classification signal," such as a signal indicating the direction of the driver's gaze of the host vehicle 1 (a signal indicating the target the driver of the host vehicle 1 is looking at), a signal indicating that the driver of the host vehicle 1 has a high stress level, or a signal indicating that the driver of the host vehicle 1 is in a state of inattention. Furthermore, the driver monitoring unit 3B outputs a strong signal as a "second classification signal," such as an unconscious signal, an eye-closing signal, or a distracted signal (a face direction signal). For example, the signal indicating the direction of the driver's gaze of the host vehicle 1 (a signal indicating the target the driver of the host vehicle 1 is looking at) or other signals included in the "first classification signal" each correspond to a signal indicating that the driver of the host vehicle 1 is neither distracted nor unconscious.

[0019] The first determining portion 3C determines whether the driver of the host vehicle 1 is in a distracted state based on the strong signal (second classification signal) output from the driver monitoring portion 3B. Furthermore, the first determining portion 3C determines whether the driver of the host vehicle 1 is in an unconscious state based on the strong signal (second classification signal) output from the driver monitoring portion 3B. When the main vehicle 1 deviates from the lane in which the main vehicle 1 is traveling, the estimation unit 3D estimates whether the driver of the main vehicle 1 has a lane departure intention. If described in detail, in the case where the first determination unit 3C determines that the driver of the main vehicle 1 is in a distracted state, the estimation unit 3D estimates that the driver of the main vehicle 1 does not have a lane departure intention. In addition, in the case where the first determination unit 3C determines that the driver of the main vehicle 1 is in a non-awake state, the estimation unit 3D estimates that the driver of the main vehicle 1 does not have a lane departure intention. Conversely, in the case where the first determination unit 3C determines that the driver of the main vehicle 1 is neither in a distracted state nor in a non-awake state (for example, such as in the case where the driver of the main vehicle 1 is looking forward but it is unknown whether the driver of the main vehicle 1 intends to cause the main vehicle 1 to deviate from the lane), the estimation unit 3D outputs an estimation result indicating that it is unknown whether the driver of the main vehicle 1 has a lane departure intention.

[0020] Prediction unit 3E predicts whether the driver of host vehicle 1 intends to leave the lane based on the weak signal (first classified signal) output from driver monitoring unit 3B. For example, if driver monitoring unit 3B outputs the first classified signal (weak signal) indicating that the driver of host vehicle 1 is neither distracted nor unconscious, prediction unit 3E predicts whether the driver of host vehicle 1 intends to leave the lane. Specifically, prediction unit 3E uses a learned machine learning model to predict whether the driver of host vehicle 1 intends to leave the lane based on a time-series signal including vehicle information, lane information, and target information, as well as the first classified signal. A learned machine learning model is obtained by learning using a learning time series signal and learning data. The learning time series signal includes learning vehicle information as information related to a learning vehicle (not shown), learning lane information as information related to the lane in which the learning vehicle is traveling, and learning target information as information related to targets existing around the learning vehicle. The learning data is a data set of a learning first classification signal and a label. The learning first classification signal indicates that the driver of the learning vehicle is neither in a distracted state nor in an unconscious state. The label indicates whether the driver of the learning vehicle has a lane departure intention. For example, in a case where the main vehicle 1 travels by avoiding an obstacle in the lane in which the main vehicle 1 is traveling (for example, such as a vehicle parked on the road), in a case where the main vehicle 1 travels by avoiding a heavy vehicle when passing a heavy vehicle traveling in an opposite lane, or in a case where the main vehicle 1 changes lanes, the prediction unit 3E predicts that the driver of the main vehicle 1 has a lane departure intention.

[0021] Figure 2A and Figure 2B Each of them is a diagram showing an example of the configuration of the prediction section 3E. If described in detail, Figure 2AAn example of the configuration of the prediction section 3E is shown, and Figure 2B The probability output from the prediction unit 3E is shown ( Figure 2B The vertical axis in the figure) and time ( Figure 2B An example of the relationship between the horizontal axis in Figure 2A and Figure 2B In the example shown in , the prediction section 3E uses a hidden Markov model as the machine learning model, but in another example, the prediction section 3E may use a machine learning model other than the hidden Markov model as the machine learning model. exist Figure 2A and Figure 2B In the example shown, the prediction section 3E uses the learned machine learning model to output the likelihood of the validity of the prediction indicating that the driver of the host vehicle 1 has a lane departure intention as a result of the calculation processing in the hidden layer. Figure 2B In the case where the probability is less than or equal to the threshold value (in the case where the probability is less than or equal to the threshold value), the prediction section 3E predicts that the driver of the host vehicle 1 has a lane departure intention and sets the flag to "1" (indicating lane departure intention). Figure 2B t1 and the time period after time t2 in the lane departure state), the prediction portion 3E predicts that the driver of the host vehicle 1 has no lane departure intention and sets the flag to “0” (indicating no lane departure intention).

[0022] exist Figure 1 In the illustrated example, the first classified signal (weak signal) output from the driver monitoring unit 3B is input to the prediction unit 3E described above. The first classified signal includes a signal indicating the internal state of the driver of the host vehicle 1 estimated by the driver monitoring unit 3B based on images captured by the driver monitoring camera 14 (e.g., information indicating the direction of the driver's gaze of the host vehicle 1, information indicating that the driver of the host vehicle 1 has a high level of stress, etc.). These signals are combined with another first classified signal (e.g., a signal indicating that the driver of the host vehicle 1 is in a state of inattention) and input to the prediction unit 3E, thereby increasing the prediction accuracy of the prediction unit 3E.

[0023] The second determination portion 3F determines whether the prediction portion 3E predicts that the driver of the host vehicle 1 has a lane departure intention. If the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to leave the lane when the host vehicle 1 deviates from the lane in which the host vehicle 1 is currently traveling, the control unit 3G causes the HMI 13 to output a lane departure warning. Furthermore, if the prediction unit 3E predicts that the driver of the host vehicle 1 does not intend to leave the lane when the host vehicle 1 deviates from the lane in which the host vehicle 1 is currently traveling, the control unit 3G causes the HMI 13 to output a lane departure warning. Meanwhile, if the prediction unit 3E predicts that the driver of the host vehicle 1 does intend to leave the lane when the host vehicle 1 deviates from the lane in which the host vehicle 1 is currently traveling, the control unit 3G causes the HMI 13 to limit the output of the lane departure warning.

[0024] exist Figure 1 In the example shown, when the main vehicle 1 deviates from the lane in which the main vehicle 1 is traveling, in some cases, the estimation unit 3D may estimate that the driver of the main vehicle 1 does not have the lane departure intention based on the "second classification signal" output from the driver monitoring unit 3B, and the prediction unit 3E may simultaneously predict that the driver of the main vehicle 1 has the lane departure intention based on the "first classification signal" output from the driver monitoring unit 3B. In this case, the control portion 3G prioritizes the estimation result of the estimation portion 3D indicating that the driver of the host vehicle 1 has no lane departure intention over the prediction result of the prediction portion 3E indicating that the driver of the host vehicle 1 has a lane departure intention, and causes the HMI 13 to output a lane departure warning.

[0025] Figure 3 is a schematic diagram for describing an example of processing performed by the processor 163 of the lane departure intention estimation device 16 according to the first embodiment when the host vehicle 1 deviates from the lane in which the host vehicle 1 is traveling. Figure 3 In the example shown, when driver monitoring unit 3B outputs a "first classified signal (weak signal)" and a "second classified signal (strong signal)" such as a face direction signal or an eye-opening signal, for example, prediction unit 3E uses the learned machine learning model to predict whether the driver of host vehicle 1 has a lane departure intention based on the time series signal including vehicle information, lane information, and target information, as well as the "first classified signal (weak signal)," and outputs a prediction result. In other words, information based on the "weak signal" indicating that the driver of host vehicle 1 may have a lane departure intention is indirectly used. In step S10, first determination unit 3C determines whether the driver of host vehicle 1 is distracted or unconscious based on the "second classified signal (strong signal)" output from driver monitoring unit 3B. In the case of "yes" (when the driver of host vehicle 1 is distracted or unconscious), information based on the strong signal and indicating that "the driver of host vehicle 1 does not intend to leave the lane" is directly used (i.e., estimation unit 3D estimates that the driver of host vehicle 1 does not intend to leave the lane), and processing proceeds to step S12. In the case of "no" (when the driver of host vehicle 1 is neither distracted nor unconscious), the driver of host vehicle 1 is looking forward, but it is unknown whether the driver of host vehicle 1 has an intention to leave the lane, and processing proceeds to step S11.

[0026] In step S11, the second determination unit 3F determines whether the prediction unit 3E predicts that the driver of the host vehicle 1 has a lane departure intention. If the prediction unit 3E predicts that the driver of the host vehicle 1 has a lane departure intention, the process proceeds to step S13. If the prediction unit 3E predicts that the driver of the host vehicle 1 has a lane departure intention, the process proceeds to step S12. In step S12 , the control unit 3G causes the HMI 13 to output a lane departure warning and activate the LDA. In step S13, control unit 3G causes HMI 13 to limit the output of the lane departure warning and deactivate LDA. As described above, when estimation unit 3D estimates that the driver of host vehicle 1 does not intend to depart from a lane, and prediction unit 3E simultaneously predicts that the driver of host vehicle 1 does intend to depart from a lane, the estimation result of estimation unit 3D is given priority, and control unit 3G causes HMI 13 to output a lane departure warning.

[0027] The definitions of strong signals (second-classified signals) and weak signals (first-classified signals) are organized as follows. A strong signal is a signal that directly enables a determination of whether the driver of the host vehicle 1 can take evasive or collision avoidance action in response to a lane departure or target collision. A weak signal is a signal that cannot, on its own, determine whether evasive action should be taken. Based on this organization, an eye-open signal can be considered a strong signal. Specifically, if the driver is in an eye-closed state due to drowsiness, the driver cannot be expected to take evasive action as a vehicle behavior of the host vehicle 1 even if the host vehicle 1 is about to deviate from the lane in which the host vehicle 1 is traveling. At the same time, even if the driver of the host vehicle 1 is looking in that direction or visually identifying the target, it is not possible to know whether the driver of the host vehicle 1 can take evasive driving action to avoid the target, for example, from the gaze direction signal of the driver of the host vehicle 1. Such a signal is regarded as a weak signal and is used as one of the signals input to the prediction unit 3E.

[0028] Figure 4A 、 Figure 4B 、 Figure 4C and Figure 4D Each of them is a diagram for describing an example in which the control unit 3G causes the HMI 13 to limit the output of the lane departure warning. If described in detail, Figure 4A shows the temporal variation of the distraction signal output from the driver monitoring unit 3B, Figure 4B shows the temporal change of the prediction result of the prediction unit 3E. Figure 4C shows the temporal variation of the LDA internal flags, and Figure 4D The time variation of the LDA actual activation flag (the control result of the control unit 3G) is shown. Figure 4A In the example, "1" on the vertical axis indicates a state where a distraction signal is output, and "0" on the vertical axis indicates a state where a distraction signal is not output. Figure 4B , “1” on the vertical axis indicates a prediction result indicating that the driver of the host vehicle 1 has a lane departure intention, and “0” on the vertical axis indicates a prediction result indicating that the driver of the host vehicle 1 does not have a lane departure intention. Figure 4C In FIG, “1” on the vertical axis indicates a state where the LDA activation condition is satisfied (a state where the output of the lane departure alarm is restricted), and “0” on the vertical axis indicates a state where the LDA activation condition is not satisfied (a state where the output of the lane departure alarm is not restricted). Figure 4D , “1” on the vertical axis indicates a state where LDA actual control exists (a state where the output of the lane departure alarm is actually limited), and “0” on the vertical axis indicates a state where LDA actual control does not exist (a state where the output of the lane departure alarm is not actually limited). exist Figure 4A 、 Figure 4B 、 Figure 4C and Figure 4D In the example shown in , during the time period from time t11 to time t12, the host vehicle 1 deviates from the lane in which the host vehicle 1 is traveling, and the prediction section 3E predicts that the driver of the host vehicle 1 has a lane departure intention. Therefore, the HMI 13 attempts to limit the output of the lane departure warning ( Figure 4CThe vertical axis in FIG1 becomes "1". Meanwhile, during the period from time t11 to time t12, the driver monitoring portion 3B outputs a distraction signal (strong signal), and the estimation portion 3D estimates that the driver of the host vehicle 1 does not have a lane departure intention. The control portion 3G gives priority to the estimation result of the estimation portion 3D over the prediction result of the prediction portion 3E, and causes the HMI 13 to output a lane departure warning ( Figure 4D The vertical axis in becomes "0").

[0029] As described above, in the lane departure intention estimation device 16 according to the first embodiment, the signal output from the driver monitoring unit 3B is classified into both a direct signal (a strong signal or a second-classified signal) and an indirect signal (a weak signal or a first-classified signal). This improves the accuracy of the estimation of whether the driver of the host vehicle 1 has a lane departure intention. Specifically, direct information (such as the driver of the host vehicle 1 being distracted and not looking forward or the driver of the host vehicle 1 being unconscious and not awake) is a strong signal that directly indicates the state of the driver of the host vehicle 1. Therefore, this strong signal is not input to the prediction unit 3E, but is instead used to override the prediction result of the prediction unit 3E. Specifically, if the driver monitoring unit 3B outputs a distracted signal or an unconscious signal, even if the prediction unit 3E predicts that the driver of the host vehicle 1 has a lane departure intention, the estimation result of the estimation unit 3D indicating that the driver of the host vehicle 1 does not have a lane departure intention is given priority. That is, the lane departure intention estimation device 16 according to the first embodiment adopts an arbitration structure. Estimation portion 3D directly uses the strong signal to estimate whether the driver of host vehicle 1 has a lane departure intention, thereby enabling highly accurate estimation. Second embodiment

[0030] The host vehicle 1 to which the lane departure intention estimation apparatus 16 according to the second embodiment is applied is configured similarly to the host vehicle 1 to which the lane departure intention estimation apparatus 16 according to the first embodiment described above, except for the following points.

[0031] In an example of a host vehicle 1 employing the lane departure intention estimation device 16 according to the second embodiment, if the estimation unit 3D estimates that the driver of the host vehicle 1 does not have a lane departure intention when the host vehicle 1 deviates from the lane in which the host vehicle 1 is currently traveling, the control unit 3G causes the vehicle control device 15 to perform lane keeping assistance (e.g., steering assistance, etc.). Furthermore, if the prediction unit 3E predicts that the driver of the host vehicle 1 does not have a lane departure intention when the host vehicle 1 deviates from the lane in which the host vehicle 1 is currently traveling, the control unit 3G causes the vehicle control device 15 to perform lane keeping assistance. Meanwhile, if the prediction unit 3E predicts that the driver of the host vehicle 1 has a lane departure intention when the host vehicle 1 deviates from the lane in which the host vehicle 1 is currently traveling, the control unit 3G causes the vehicle control device 15 to limit the execution of lane keeping assistance. In the example of the main vehicle 1 applying the lane departure intention estimation device 16 according to the second embodiment, when the estimation unit 3D estimates that the driver of the main vehicle 1 does not have a lane departure intention and the prediction unit 3E simultaneously predicts that the driver of the main vehicle 1 has a lane departure intention, the control unit 3G gives priority to the estimation result of the estimation unit 3D indicating that the driver of the main vehicle 1 does not have a lane departure intention over the prediction result of the prediction unit 3E indicating that the driver of the main vehicle 1 has a lane departure intention, and causes the vehicle control device 15 to perform lane keeping assist. Third embodiment

[0032] The host vehicle 1 to which the lane departure intention estimation apparatus 16 according to the third embodiment is applied is configured similarly to the host vehicle 1 to which the lane departure intention estimation apparatus 16 according to the first or second embodiment described above, except for the following points.

[0033] Figure 5 1 is a diagram showing an example of data processing and flow in the host vehicle 1 to which the lane departure intention estimation device 16 according to the third embodiment is applied. In the third embodiment, a data collection system is used to collect online driving data of host vehicle 1 used by the driver (user) of host vehicle 1 (i.e., host vehicle 1 is used as a learning vehicle) to improve the accuracy of assigning true value to the data collected by the data collection system. The data collected by the data collection system includes front camera images, recognition processing results, and vehicle controller area network (CAN) information, and further includes driver monitoring information (information input to and output from driver monitoring unit 3B). exist Figure 5 In the example shown, the data collected by the data collection system is subjected to scenario-based data classification. In this step, the task of cutting out the scene to be learned is performed, and a classification is arranged for each scene based on whether the scene is intended by the driver of the host vehicle 1 or not. Generally, the related art classifies the learning dataset into only two types of datasets: the scenario intended by the driver of the host vehicle 1 and the scenario not intended. Figure 5In the example shown in , data classification is further subdivided by using the above-mentioned "first classification signal (weak signal)" and "second classification signal (strong signal)". For example, data accompanied by a closed-eye state or a distracted state (strong signal) is classified as an "unintentional scene with reliability (high)". For example, in a scene where the main vehicle 1 overtakes a parked vehicle, if it is accompanied by a driver monitoring signal (weak signal) indicating that the driver of the main vehicle 1 is visually identifying the state of the parked vehicle, the data is classified as an "intentional scene with reliability (medium)". For example, in a scene where the main vehicle 1 overtakes a parked vehicle, if it is accompanied by a driver monitoring signal (weak signal) indicating that the driver of the main vehicle 1 is visually identifying the state of the parked vehicle and is accompanied by the driver of the main vehicle 1 operating the flash lights, the data is classified as an "intentional scene with reliability (high)". Finally, the machine learning model learns by using a dataset corresponding to reliability. For example, considering the frequency of occurrence in actual scenarios, the machine learning model uses "intentional scenarios with high reliability (high)", "intentional scenarios with medium reliability (medium)", "unintentional scenarios with medium reliability (medium)", and "unintentional scenarios with high reliability (high)" at a ratio of 1:2:2:1. Generally, related art classifies flash light operation as an "intentional scenario" as a direct driving operation behavior by the driver of host vehicle 1. However, there is a possibility that the flash light operation is not an operation for host vehicle 1 to avoid a parked vehicle, but rather an operation performed because it seems more advantageous for host vehicle 1 to move to the next lane as a result of host vehicle 1 swinging relative to the lane in which host vehicle 1 is currently traveling. exist Figure 5 In the example shown in FIG, the flasher operation is combined with information indicating that the driver of host vehicle 1 visually recognized a parked vehicle. Therefore, it can be determined that the flasher operation is directed at host vehicle 1 to avoid the parked vehicle. As a result, the reliability of the intentional scenario can be set to (high).

[0034] That is, in Figure 5 In the example shown in , in order to improve the accuracy of giving a true value to the data collected by the data collection system that collects the driving online data of the main vehicle 1 used by the driver (user) of the main vehicle 1, a mark of the presence of the intention of the driver of the main vehicle 1 / the absence of the intention of the driver of the main vehicle 1 is given depending on the state of the output "first classification signal (weak signal)" and "second classification signal (strong signal)". exist Figure 5 The example shown in proposes rules for giving the true value of intent and proposes an improvement in the accuracy of learning by the machine learning model. When data acquired by the host vehicle 1 is used as learning data for a machine learning model, the learning data requires a label (flag) indicating whether the driver of the host vehicle 1 has a lane departure intention. It is preferable to give the flag by directly asking the driver of the host vehicle 1 whether he or she has a lane departure intention, but in practice it is difficult to ask the driver of the host vehicle 1 whether the driver of the host vehicle 1 has a lane departure intention. Therefore, the related art generally performs scene determination. If the scene to be determined is, for example, a parked vehicle avoidance scene or a lane change scene, it is determined (considered) that the driver of the host vehicle 1 has a lane departure intention. In contrast, in Figure 5 In the example shown, the accuracy of the labels assigned to the intent estimation level is improved by applying labels combined with driver monitoring information. Furthermore, by improving the hierarchical classification of intent scenarios with high reliability (high) and intent scenarios with medium reliability (medium) corresponding to two types of signals (strong and weak), such as by changing the learning order or ratio, the accuracy of learning and evaluation can be further improved.

[0035] As described above, an embodiment of the lane departure intention estimation device according to the present disclosure has been described with reference to the accompanying drawings, but the lane departure intention estimation device according to the present disclosure is not limited to the above-mentioned embodiment. Changes can be made appropriately within the scope of not departing from the main purpose of the present disclosure. The configurations of the various examples of the above-mentioned embodiments can be appropriately combined. In the various examples of the above-mentioned embodiments, the processing performed by the lane departure intention estimation device 16 has been described as software processing performed by executing a program, but the processing performed by the lane departure intention estimation device 16 may be processing performed by hardware. Alternatively, the processing performed by the lane departure intention estimation device 16 may be processing provided by a combination of both software and hardware. In addition, the program stored in the memory 162 of the lane departure intention estimation device 16 (the program that implements the functions of the processor 163 of the lane departure intention estimation device 16) may be recorded, for example, in a computer-readable storage medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium, and, for example, provided or distributed.

Claims

1. A lane departure intention estimation device, comprising: an estimating unit that estimates that the driver of the host vehicle does not have a lane departure intention if the driver of the host vehicle is in a distracted state or an unconscious state; as well as a prediction unit that, when the driver of the host vehicle is neither in the distracted state nor in the unconscious state, predicts whether the driver of the host vehicle has a lane departure intention based on a time series signal and a first classification signal using a learned machine learning model, the time series signal including vehicle information, lane information, and target information, the vehicle information being information related to the host vehicle, the lane information being information related to a lane in which the host vehicle is traveling, and the target information being information related to targets present around the host vehicle, the first classification signal including a signal indicating that the driver of the host vehicle is neither in the distracted state nor in the unconscious state, wherein The learned machine learning model is obtained by learning using a learning time series signal and learning data, wherein the learning time series signal includes learning vehicle information, learning lane information and learning target information, wherein the learning vehicle information is information related to the learning vehicle, the learning lane information is information related to the lane in which the learning vehicle is traveling, and the learning target information is information related to targets existing around the learning vehicle. The learning data is a data set of a learning first classification signal and a label, wherein the learning first classification signal indicates that the driver of the learning vehicle is neither in the distracted state nor in the unconscious state, and the label indicates whether the driver of the learning vehicle has a lane departure intention.

2. The lane departure intention estimation device according to claim 1 includes a control unit that limits the output of a lane departure alarm when the prediction unit predicts that the driver of the host vehicle has a lane departure intention, the lane departure alarm being an alarm of lane departure of the host vehicle. 3 . The lane departure intention estimation device according to claim 1 , comprising a control portion that causes execution of lane keeping assist to be restricted if the prediction portion predicts that the driver of the host vehicle has a lane departure intention.

4. The lane departure intention estimation device according to claim 1 , comprising a driver monitoring portion that outputs the first classification signal and the second classification signal based on an image captured by a driver monitoring camera, the image including the driver of the host vehicle, the first classification signal including a signal indicating that the driver of the host vehicle is neither in the distracted state nor in the unconscious state, and the second classification signal including a signal indicating that the driver of the host vehicle is in the distracted state or in the unconscious state, wherein In a case where the driver monitoring unit outputs the second classification signal including a signal indicating that the driver of the main vehicle is in the distracted state or the unconscious state and the prediction unit predicts that the driver of the main vehicle has a lane departure intention, the estimation result of the estimation unit indicating that the driver of the main vehicle does not have a lane departure intention is given priority over the prediction result of the prediction unit indicating that the driver of the main vehicle has a lane departure intention.

5. The lane departure intention estimation device according to claim 4, wherein: The first classification signal output from the driver monitoring portion is input to the prediction portion; and The first classification signal includes a signal indicative of an interior state of a driver of the host vehicle, the interior state estimated by the driver monitoring portion based on an image captured by the driver monitoring camera, the image including the driver of the host vehicle.

Images