Lane departure intention estimation device

Abstract

To accurately estimate whether a driver of an own vehicle intends to depart from a lane.SOLUTION: A lane departure intention estimation device 16 includes: an estimation unit 3D that estimates that a driver of an own vehicle 1 does not intend to depart from a lane when the driver is in inattentive nor unaware; and a prediction unit 3E that predicts whether the driver intends to depart the lane based on a time-series signal including vehicle information, lane information, and target object information and a first classification signal including a signal indicating that the driver is neither inattentive nor unaware, by using a trained machine learning model when the driver is neither inattentive nor unaware, in which the trained machine learning model is obtained by training using a dataset composed of a training time-series signal containing the training vehicle information, training lane information, and training target information, training first classification signal indicating that the driver of the training vehicle is neither inattentive nor unaware, and a label indicating whether the driver of the training vehicle intends to depart the lane.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a lane departure intention estimation device. [Background technology]

[0002] Patent Document 1 describes a driving assistance device that deactivates the lane departure prevention function on the condition that it is determined that the driver intends to change lanes while lane departure prevention is being performed. Patent Document 1 also describes that an intention determination unit that determines the driver's intention to change lanes can use a device that detects the driver's line of sight and gestures obtained by various input switches or image input means such as a camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-077931 Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not describe a specific method for determining whether or not the driver intends to change lanes based on the detection results of the driver's line of sight, gestures, etc. Therefore, the technology described in Patent Document 1 may not be able to properly estimate whether or not the driver intends to depart from the lane.

[0005] In view of the above, an object of the present disclosure is to provide a lane departure intention estimation device that can appropriately estimate whether or not the driver of a host vehicle intends to depart from a lane. [Means for solving the problem]

[0006] (1) One aspect of the present disclosure includes an estimation unit that estimates that a driver of a host vehicle does not intend to depart from a lane when the driver of the host vehicle is in an inattentive state or in an unaware state, and an estimation unit that estimates that a driver of the host vehicle does not intend to depart from a lane when the driver of the host vehicle is in an inattentive state or in an unaware state, and a first classification signal that includes a time-series signal including vehicle information that is information about the host vehicle, lane information that is information about the lane in which the host vehicle is traveling, and target information that is information about targets present around the host vehicle, and a signal indicating that the driver of the host vehicle is not in an inattentive state or in an unaware state, by using a trained machine learning model when the driver of the host vehicle is not in an inattentive state or in an unaware state. and a prediction unit that predicts whether the driver of the training vehicle has a lane departure intention, and the trained machine learning model is obtained by learning using training data that is a data set of a training time series signal including training vehicle information that is information about the training vehicle, training lane information that is information about the lane in which the training vehicle is traveling, and training target information that is information about targets present around the training vehicle, and a training first classification signal that indicates that the driver of the training vehicle is not in an inattentive state or awake, and a label that indicates whether the driver of the training vehicle has a lane departure intention.

[0007] (2) The lane departure intention estimation device of (1) may be provided with a control unit that limits the output of a lane departure warning, which is a warning that the vehicle has departed from its lane, when the prediction unit predicts that the driver of the vehicle has the intention to depart from the lane.

[0008] (3) The lane departure intention estimation device of (1) may include a control unit that limits the execution of lane keeping assistance when the prediction unit predicts that the driver of the host vehicle has an intention to depart from the lane.

[0009] (4) The lane departure intention estimation device of (1) includes a driver monitor unit that outputs the first classification signal including a signal indicating that the driver of the vehicle is not inattentive or not in a state of drowsiness, and the second classification signal including a signal indicating that the driver of the vehicle is inattentive or not in a state of drowsiness, based on an image including the driver of the vehicle captured by a drive monitor camera. When the driver monitor unit outputs the second classification signal including a signal indicating that the driver of the vehicle is inattentive or not in a state of drowsiness, and the prediction unit predicts that the driver of the vehicle has the intention to depart from the lane, the estimation result of the estimation unit indicating that the driver of the vehicle does not have the intention to depart from the lane may be given priority over the prediction result of the prediction unit indicating that the driver of the vehicle has the intention to depart from the lane.

[0010] (5) In the lane departure intention estimation device of (4), the first classification signal output from the driver monitor unit may be input to the prediction unit, and the first classification signal may include a signal indicating the internal state of the driver of the vehicle estimated by the driver monitor unit based on an image including the driver of the vehicle captured by the drive monitor camera. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to appropriately estimate whether or not the driver of the host vehicle intends to depart from the lane. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram showing an example of a host vehicle 1 to which a lane departure intention estimation device 16 according to a first embodiment is applied. [Figure 2] FIG. 2 is a diagram showing an example of the configuration of a prediction unit 3E. [Figure 3] 10 is a diagram illustrating an example of processing executed by the processor 163 of the lane departure intention estimating device 16 of the first embodiment when the host vehicle 1 departs from the lane in which the host vehicle 1 is traveling. FIG. [Figure 4]10 is a diagram for explaining an example in which the control unit 3G causes the HMI 13 to limit the output of a lane departure warning. FIG. [Figure 5] 10 is a diagram showing an example of data processing and flow in a host vehicle 1 to which a lane departure intention estimation device 16 according to a third embodiment is applied. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of a lane departure intention estimation device according to the present disclosure will be described with reference to the drawings.

[0014] First Embodiment FIG. 1 is a diagram illustrating an example of a host vehicle 1 to which a lane departure intention estimation device 16 according to a first embodiment is applied. In the example illustrated in FIG. 1, the host vehicle 1 includes a surrounding condition sensor 11, a vehicle state sensor 12, an HMI (Human Machine Interface) 13, a driver monitor camera 14, a vehicle control device 15, a steering actuator 15A, a braking actuator 15B, a drive actuator 15C, and a lane departure intention estimation device 16. The surrounding condition sensor 11 detects markings that define the lane in which the host vehicle 1 is traveling and objects (e.g., nearby vehicles, obstacles, etc.) present around the host vehicle 1, and transmits the detection results to the vehicle control device 15 and the lane departure intention estimation device 16. The surrounding condition sensor 11 includes, for example, a camera that captures images ahead of the host vehicle 1, a LiDAR (Light Detection and Ranging), radar, sonar, etc. The detection results of the surrounding condition sensor 11 include, for example, lane information, which is information about the lane on which the host vehicle 1 is traveling, and target information, which is information about targets present around the host vehicle 1. The lane information includes, for example, information indicating the horizontal position of a lane marking in a camera image, the curvature of a curve in the lane on which the host vehicle 1 is traveling, etc. The target information includes, for example, information indicating the relative position and relative speed of a target with respect to the host vehicle 1.

[0015] The vehicle state sensor 12 detects the state of the host vehicle 1 and transmits the detection result to the vehicle control device 15 and the lane departure intention estimation device 16. The vehicle state sensor 12 includes, for example, a vehicle speed sensor, a steering torque sensor, etc. The detection result of the vehicle state sensor 12 includes, for example, vehicle information that is information about the host vehicle 1, and the vehicle information includes, for example, vehicle speed, steering torque, etc. The HMI 13 has functions such as accepting various operations from the driver of the vehicle 1 and outputting information such as warnings to the driver of the vehicle 1 by display, voice, etc., and transmits a signal indicating the operation of the driver of the vehicle 1 to the vehicle control device 15. The warnings output by the HMI 13 include, for example, a lane departure warning (LDA (Lane Departure Alert)) that is a warning that the vehicle 1 has deviated from the lane in which the vehicle 1 is traveling. The driver monitor camera 14 is disposed, for example, above the steering column of the host vehicle 1, and captures an image of the face and part of the upper body of the driver of the host vehicle 1. The driver monitor camera 14 also transmits the captured image data to the lane departure intention estimation device 16. In another example, the driver monitor camera 14 may be disposed in a position other than the steering column of the host vehicle 1, such as a center cluster, a room mirror, a meter panel, or a meter hood.

[0016] 1, a vehicle control device 15 controls the traveling of the host vehicle 1. The vehicle control device 15 is configured by, for example, a driving assistance ECU (Electronic Control Unit), and controls a steering actuator 15A, a braking actuator 15B, and a driving actuator 15C based on information (data, signals) transmitted from a surrounding condition sensor 11, a vehicle state sensor 12, an HMI 13, and a lane departure intention estimation device 16.

[0017] The lane departure intention estimation device 16 is configured by a microcomputer equipped with a communication interface (I / F) 161, a memory 162, and a processor 163. The communication interface 161 has an interface circuit. The memory 162 stores programs and various data used in the processing executed by the processor 163. The processor 163 has functions as an acquisition unit 3A, a driver monitor unit 3B, a first determination unit 3C, an estimation unit 3D, a prediction unit 3E, a second determination unit 3F, and a control unit 3G. The acquisition unit 3A acquires detection results (lane information, target information, etc.) from the surrounding situation sensor 11, detection results (vehicle information, etc.) from the vehicle state sensor 12, data on images captured by the driver monitor camera 14 (driver monitor camera images), etc.

[0018] The driver monitor unit 3B measures the state of the driver of the vehicle 1 based on images captured by the driver monitor camera 14, etc. Specifically, the driver monitor unit 3B detects the position and orientation of the face of the driver of the vehicle 1 and whether the eyes are open or closed, and determines whether the driver of the vehicle 1 is in a state where he or she can check the surrounding situation and perform driving operations. For example, the driver monitor unit 3B uses the driver monitor camera image to perform face area detection, facial part detection, head posture estimation, eye openness detection, and eyelid behavior analysis to estimate the level of drowsiness of the driver of the vehicle 1. When the driver monitor unit 3B determines that the driver of the vehicle 1 is in a waking state, it outputs, as a driver monitor signal, a non-waking signal indicating that the driver of the vehicle 1 is in a waking state. On the other hand, when the driver monitor unit 3B determines that the driver of the vehicle 1 is in an eyes-open state, it outputs, as a driver monitor signal, an eyes-open signal indicating that the driver of the vehicle 1 is in an eyes-open state. Furthermore, when the driver monitor unit 3B determines that the driver of the vehicle 1 is in an inattentive state, it outputs, as a driver monitor signal, a looking-at-side signal (the looking-at-side signal is part of the face direction signal) indicating that the driver of the vehicle 1 is in an inattentive state. The non-waking signal, eye-open signal, and looking-at-side signal (face direction signal) belong to the category of "strong signals (for example, signals that an AI (artificial intelligence) used as the driver monitor unit 3B outputs with high reliability)." When the driver of the vehicle 1 is looking outside the field of view where safety should be confirmed, such as toward the passenger seat or a car navigation system, the driver monitor unit 3B outputs an inattentive signal. Also, when the driver of the vehicle 1 is traveling in the left lane of a straight road with two lanes on each side, and the driver's line of sight is directed only to the left for a certain period of time, the driver monitor unit 3B outputs an inattentive signal. In other words, when the driver of the vehicle 1 is not looking toward an object that needs to be confirmed, the driver monitor unit 3B outputs an inattentive signal.

[0019] Furthermore, when the driver monitor unit 3B determines, based on the image captured by the driver monitor camera 14 and the detection results of the surrounding condition sensor 11, that the driver of the vehicle 1 is gazing at a target present around the vehicle 1, it outputs, as a driver monitor signal, a signal indicating the line of sight of the driver of the vehicle 1 (a signal indicating the target that the driver of the vehicle 1 is gazing at (gazing target information)). Furthermore, when the driver monitor unit 3B determines that the stress level of the driver of the vehicle 1 is high, it outputs, as a driver monitor signal, a signal indicating that the stress level of the driver of the vehicle 1 is high. Furthermore, when the driver monitor unit 3B determines that the driver of the vehicle 1 is in an absentminded state, it outputs, as a driver monitor signal, a signal indicating that the driver of the vehicle 1 is in an absentminded state. The signal indicating the line of sight of the driver of the vehicle 1, the signal indicating that the stress level of the driver of the vehicle 1 is high, and the signal indicating that the driver of the vehicle 1 is in an absentminded state are classified as "weak signals (for example, signals for which an AI used as the driver monitor unit 3B outputs low reliability)." The driver monitor unit 3B outputs weak signals such as a signal indicating the line of sight of the driver of the vehicle 1 (a signal indicating a target that the driver of the vehicle 1 is gazing at), a signal indicating a high stress level of the driver of the vehicle 1, and a signal indicating that the driver of the vehicle 1 is absentminded as the "first classification signal." In addition, the driver monitor unit 3B outputs strong signals such as a non-awakening signal, an eye-closure signal, and an aside signal (face direction signal) as the "second classification signal." For example, a signal indicating the line of sight of the driver of the vehicle 1 (a signal indicating a target that the driver of the vehicle 1 is gazing at) included in the "first classification signal" corresponds to a signal indicating that the driver of the vehicle 1 is neither in an aside state nor in an aside state.

[0020] The first determination unit 3C determines whether the driver of the vehicle 1 is in an inattentive state based on the strong signal (second classification signal) output from the driver monitor unit 3B. The first determination unit 3C also determines whether the driver of the vehicle 1 is in a waking state based on the strong signal (second classification signal) output from the driver monitor unit 3B. The estimation unit 3D estimates whether the driver of the host vehicle 1 intends to depart the lane when the host vehicle 1 departs from the lane in which the host vehicle 1 is traveling. Specifically, the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart the lane when the first determination unit 3C determines that the driver of the host vehicle 1 is in an inattentive state. Furthermore, the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart the lane when the first determination unit 3C determines that the driver of the host vehicle 1 is in an inattentive state. On the other hand, when the first determination unit 3C determines that the driver of the host vehicle 1 is neither in an inattentive state nor in an inattentive state (for example, when the driver of the host vehicle 1 is looking ahead but it is unclear whether or not the driver intentionally caused the host vehicle 1 to depart the lane), the estimation unit 3D outputs an estimation result indicating that it is unclear whether the driver of the host vehicle 1 intends to depart the lane.

[0021] The prediction unit 3E predicts whether or not the driver of the host vehicle 1 intends to depart from the lane based on a weak signal (first classification signal) output from the driver monitor unit 3B. The prediction unit 3E predicts whether or not the driver of the host vehicle 1 intends to depart from the lane when, for example, the driver monitor unit 3B outputs a first classification signal (weak signal) indicating that the driver of the host vehicle 1 is neither inattentive nor awake. In detail, the prediction unit 3E uses a trained machine learning model to predict whether or not the driver of the host vehicle 1 intends to depart from the lane based on a time-series signal including vehicle information, lane information, and target information, and the first classification signal. The trained machine learning model is obtained by performing training using training data, which is a data set of a training time series signal including training vehicle information, which is information about a training vehicle (not shown), training lane information, which is information about the lane in which the training vehicle is traveling, and training target information, which is information about targets present around the training vehicle, a training first classification signal indicating that the driver of the training vehicle is not inattentive or awake, and a label indicating whether the driver of the training vehicle intends to depart from the lane. The prediction unit 3E predicts that the driver of the vehicle 1 intends to deviate from the lane, for example, when the vehicle 1 is traveling while avoiding an obstacle (such as a vehicle parked on the road) in the lane in which the vehicle 1 is traveling, when the vehicle 1 is traveling while avoiding a large vehicle traveling in the oncoming lane when passing the large vehicle, or when the vehicle 1 is changing lanes.

[0022] Fig. 2 is a diagram showing an example of the configuration of the prediction unit 3E. In detail, Fig. 2(A) shows an example of the configuration of the prediction unit 3E, and Fig. 2(B) shows an example of the relationship between the likelihood (vertical axis of Fig. 2(B)) output from the prediction unit 3E and time (horizontal axis of Fig. 2(B)). In the example shown in Fig. 2, the prediction unit 3E uses a hidden Markov model as the machine learning model, but in other examples, the prediction unit 3E may use a machine learning model other than a hidden Markov model as the machine learning model. In the example shown in FIG. 2, the prediction unit 3E uses a trained machine learning model to output a likelihood indicating the validity of predicting that the driver of the host vehicle 1 intends to depart from the lane as a result of calculation processing of the hidden layer. If the likelihood is greater than a threshold (during the period from time t1 to t2 in FIG. 2(B)), the prediction unit 3E predicts that the driver of the host vehicle 1 intends to depart from the lane and sets the flag to "1 (indicating that the lane departure is intentional)." If the likelihood is equal to or less than the threshold (during the period before time t1 and the period after time t2 in FIG. 2(B)), the prediction unit 3E predicts that the driver of the host vehicle 1 does not intend to depart from the lane and sets the flag to "0 (indicating that the lane departure is unintentional)."

[0023] 1, the first classification signal (weak signal) output from the driver monitor unit 3B is input to the prediction unit 3E as described above. The first classification signal includes a signal indicating the internal state of the driver of the vehicle 1 (e.g., information indicating the line of sight of the driver of the vehicle 1, information indicating that the stress level of the driver of the vehicle 1 is high, etc.) estimated by the driver monitor unit 3B based on an image captured by the driver monitor camera 14. By combining these signals with other first classification signals (e.g., a signal indicating that the driver of the vehicle 1 is absentminded, etc.) and inputting them to the prediction unit 3E, the prediction accuracy of the prediction unit 3E can be improved.

[0024] The second determination unit 3F determines whether or not the prediction unit 3E has predicted that the driver of the host vehicle 1 intends to depart from the lane. When the host vehicle 1 deviates from the lane in which it is traveling and the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart from the lane, the control unit 3G causes the HMI 13 to output a lane departure warning. Also, when the host vehicle 1 deviates from the lane in which it is traveling and the prediction unit 3E predicts that the driver of the host vehicle 1 does not intend to depart from the lane, the control unit 3G causes the HMI 13 to output a lane departure warning. On the other hand, when the host vehicle 1 deviates from the lane in which it is traveling and the prediction unit 3E predicts that the driver of the host vehicle 1 intends to depart from the lane, the control unit 3G causes the HMI 13 to limit the output of the lane departure warning.

[0025] In the example shown in Figure 1, when vehicle 1 deviates from the lane in which it is traveling, it is possible that, based on the "second classification signal" output from driver monitor unit 3B, estimation unit 3D estimates that the driver of vehicle 1 does not intend to deviate from the lane, and at the same time, based on the "first classification signal" output from driver monitor unit 3B, prediction unit 3E predicts that the driver of vehicle 1 intends to deviate from the lane. In such a case, the control unit 3G prioritizes the estimation result of the estimation unit 3D, which indicates that the driver of the vehicle 1 does not intend to depart from the lane, over the prediction result of the prediction unit 3E, which indicates that the driver of the vehicle 1 does intend to depart from the lane, and causes the HMI 13 to output a lane departure warning.

[0026] 3 is a diagram illustrating an example of processing executed by the processor 163 of the lane departure intention estimation device 16 of the first embodiment when the host vehicle 1 deviates from the lane in which the host vehicle 1 is traveling. In the example shown in FIG. 3, when the driver monitor unit 3B outputs a "first classification signal (weak signal)" and a "second classification signal (strong signal)" such as a face direction signal or an eye open signal, the prediction unit 3E uses a trained machine learning model to predict whether the driver of the host vehicle 1 intends to depart from the lane based on a time-series signal including vehicle information, lane information, and target information, and the "first classification signal (weak signal)," and outputs the prediction result. In other words, information indicating that "the driver of the host vehicle 1 may intend to depart from the lane" based on the "weak signal" is indirectly used. In step S10, the first determination unit 3C determines whether the driver of the host vehicle 1 is in an inattentive state or in a drowsy state based on the "second classification signal (strong signal)" output from the driver monitor unit 3B. If the result is YES (if the driver of the host vehicle 1 is in an inattentive state or in a drowsy state), the information indicating that "the driver of the host vehicle 1 does not intend to depart from the lane" based on the strong signal is directly used (i.e., the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart from the lane), and the process proceeds to step S12. If the result is NO (if the driver of the host vehicle 1 is not in an inattentive state and not in a drowsy state), although the driver of the host vehicle 1 is looking ahead, it is not possible to determine whether the driver of the host vehicle 1 intends to depart from the lane, and the process proceeds to step S11.

[0027] In step S11, the second determination unit 3F determines whether the prediction unit 3E has predicted that the driver of the host vehicle 1 intends to depart from the lane. If the result is YES (if the prediction unit 3E has predicted that the driver of the host vehicle 1 intends to depart from the lane), the process proceeds to step S13, and if the result is NO (if the prediction unit 3E has predicted that the driver of the host vehicle 1 does not intend to depart from the lane), the process proceeds to step S12. In step S12, the control unit 3G causes the HMI 13 to output a lane departure warning, and the LDA is activated. In step S13, the control unit 3G causes the HMI 13 to limit the output of the lane departure warning, and the LDA is deactivated. As described above, if the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart from the lane and the prediction unit 3E predicts that the driver of the host vehicle 1 intends to depart from the lane, the estimation result of the estimation unit 3D takes priority, and the control unit 3G causes the HMI 13 to output the lane departure warning.

[0028] The definitions of strong signals (class 2 signals) and weak signals (class 1 signals) can be stated another way as follows: A strong signal is a signal that allows the driver of the vehicle 1 to directly determine whether or not he or she can take action to avoid deviation or collision in the event of lane departure or collision with an object. A weak signal is a signal that does not allow the driver to determine, from the signal alone, whether or not he or she can take the above-mentioned avoidance action. Based on the above, an eye-open signal can be said to be a strong signal. Specifically, if the driver has their eyes closed due to drowsiness or the like, even if the vehicle 1 is about to deviate from the lane in which it is traveling, the vehicle behavior is such that the driver cannot be expected to take evasive action. On the other hand, for example, a gaze direction signal of the driver of the vehicle 1 does not indicate whether the driver of the vehicle 1 can take driving action to avoid an object even if the driver is looking in that direction or has visually recognized the object. Such a signal is treated as a weak signal and is used as one of the input signals to the prediction unit 3E.

[0029] 4A and 4B are diagrams illustrating an example in which the control unit 3G causes the HMI 13 to limit the output of a lane departure warning. Specifically, FIG. 4A shows the time change in the inattentive signal output from the driver monitor unit 3B, FIG. 4B shows the time change in the prediction result of the prediction unit 3E, FIG. 4C shows the time change in the LDA internal flag, and FIG. 4D shows the time change in the LDA actual operation flag (control result of the control unit 3G). In FIG. 4A, "1" on the vertical axis indicates a state in which the inattentive signal is output, and "0" on the vertical axis indicates a state in which the inattentive signal is not output. In FIG. 4B, "1" on the vertical axis indicates a prediction result indicating that the driver of the vehicle 1 intends to depart the lane, and "0" on the vertical axis indicates a prediction result indicating that the driver of the vehicle 1 does not intend to depart the lane. In Figure 4(C), "1" on the vertical axis indicates a state in which the LDA activation condition is satisfied (a state in which the output of the lane departure warning is restricted), and "0" on the vertical axis indicates a state in which the LDA activation condition is not satisfied (a state in which the output of the lane departure warning is not restricted). In Figure 4(D), "1" on the vertical axis indicates a state in which LDA actual control is in effect (a state in which the output of the lane departure warning is actually restricted), and "0" on the vertical axis indicates a state in which LDA actual control is not in effect (a state in which the output of the lane departure warning is not actually restricted). In the example shown in Fig. 4, during the period from time t11 to t12, the prediction unit 3E predicts that the host vehicle 1 deviates from the lane in which it is traveling and that the driver of the host vehicle 1 intends to depart from the lane, so the HMI 13 attempts to limit the output of a lane departure warning (the vertical axis of Fig. 4(C) becomes "1"). On the other hand, during the period from time t11 to t12, the driver monitor unit 3B outputs an inattentive signal (a strong signal), and the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart from the lane. The control unit 3G prioritizes the estimation result of the estimation unit 3D over the prediction result of the prediction unit 3E, and causes the HMI 13 to output a lane departure warning (the vertical axis of Fig. 4(D) becomes "0").

[0030] As described above, in the lane departure intention estimation device 16 of the first embodiment, signals output from the driver monitor unit 3B are classified into two categories: direct signals (strong signals, second category signals) and indirect signals (weak signals, first category signals). This improves the accuracy of estimating whether the driver of the vehicle 1 intends to depart from the lane. Specifically, direct information, such as the driver of the vehicle 1 being in an inattentive state (not looking ahead) or the driver being unconscious (not awake), is a strong signal that directly indicates the state of the driver of the vehicle 1. Therefore, such strong signals are not input to the prediction unit 3E but are used to override the prediction result of the prediction unit 3E. Specifically, if the driver monitor unit 3B outputs an inattentive signal or a non-awake signal, the estimation result of the estimation unit 3D, which indicates that the driver of the vehicle 1 does not intend to depart from the lane, takes precedence, even if the prediction unit 3E predicts that the driver of the vehicle 1 intends to depart from the lane. In other words, the lane departure intention estimation device 16 of the first embodiment employs an arbitration structure. By directly using a strong signal, the estimation unit 3D estimates whether the driver of the vehicle 1 intends to depart from the lane, thereby enabling highly accurate estimation.

[0031] Second Embodiment The vehicle 1 to which the lane departure intention estimation device 16 of the second embodiment is applied is configured in the same manner as the vehicle 1 to which the lane departure intention estimation device 16 of the first embodiment described above is applied, except for the points described below.

[0032] In one example of the host vehicle 1 to which the lane departure intention estimation device 16 of the second embodiment is applied, the control unit 3G causes the vehicle control device 15 to execute lane keeping assist (e.g., steering assist) when the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart the lane when the host vehicle 1 departs the lane in which the host vehicle 1 is traveling. The control unit 3G also causes the vehicle control device 15 to execute lane keeping assist when the prediction unit 3E predicts that the driver of the host vehicle 1 does not intend to depart the lane when the host vehicle 1 departs the lane in which the host vehicle 1 is traveling. On the other hand, the control unit 3G causes the vehicle control device 15 to restrict the execution of lane keeping assist when the prediction unit 3E predicts that the driver of the host vehicle 1 intends to depart the lane when the host vehicle 1 departs the lane in which the host vehicle 1 is traveling. In an example of a host vehicle 1 to which the lane departure intention estimation device 16 of the second embodiment is applied, when the estimation unit 3D estimates that the driver of the host vehicle 1 does not intend to depart the lane and at the same time the prediction unit 3E predicts that the driver of the host vehicle 1 does intend to depart the lane, the control unit 3G prioritizes the estimation result of the estimation unit 3D indicating that the driver of the host vehicle 1 does not intend to depart the lane over the prediction result of the prediction unit 3E indicating that the driver of the host vehicle 1 does intend to depart the lane, and causes the vehicle control device 15 to perform lane keeping assistance.

[0033] <Third embodiment> The host vehicle 1 to which the lane departure intention estimation device 16 of the third embodiment is applied is configured in the same manner as the host vehicle 1 to which the lane departure intention estimation device 16 of the first or second embodiment described above is applied, except for the points described below.

[0034] FIG. 5 is a diagram showing an example of data processing and flow in the host vehicle 1 to which the lane departure intention estimating device 16 of the third embodiment is applied. In the third embodiment, a data collection system is used to collect online driving data of the vehicle 1 used by the driver (user) of the vehicle 1 (i.e., the vehicle 1 is used as a learning vehicle), and the accuracy of the true value of the data collected by the data collection system is improved. The data collected by the data collection system includes front camera images, recognition processing results, and vehicle CAN (Controller Area Network) information, as well as driver monitor information (information input to the driver monitor unit 3B and information output from the driver monitor unit 3B). In the example shown in Figure 5, scene-based data classification is performed on the data set collected by the data collection system. In this process, scenes to be learned are extracted and each scene is classified as either intentional or unintentional by the driver of the vehicle 1. In conventional general technologies, the learning dataset consisted of only two types of datasets: intentional and unintentional scenes by the driver of the vehicle 1. However, in the example shown in FIG. 5, the data classification is further refined using the above-mentioned "first classification signal (weak signal)" and "second classification signal (strong signal)." For example, data accompanied by a closed-eye state or a distracted state (strong signal) is classified as an "unintentional scene with high reliability." For example, in a scene in which the vehicle 1 overtakes a parked vehicle, if the scene is accompanied by a driver monitor signal (weak signal) indicating that the driver of the vehicle 1 is viewing the parked vehicle, the scene is classified as an "intentional scene with medium reliability." For example, in a scene in which the vehicle 1 overtakes a parked vehicle, if the scene is accompanied by a driver monitor signal (weak signal) indicating that the driver of the vehicle 1 is viewing the parked vehicle and if the driver of the vehicle 1 operates the turn signal, the scene is classified as an "intentional scene with high reliability." Finally, a machine learning model is trained using a dataset based on the reliability. For example, taking into account the frequency of occurrence in real scenes, the machine learning model is trained using a 1:2:2:1 ratio of "high reliability intentional scenes," "medium reliability intentional scenes," "medium reliability unintentional scenes," and "high reliability unintentional scenes." In conventional general technology, the turn signal operation was classified as an "intentional scene" as a direct driving operation behavior of the driver of the vehicle 1. However, there remained the possibility that this turn signal operation was not a turn signal operation for the vehicle 1 to avoid a parked vehicle, but rather a turn signal operation performed because the vehicle 1 had staggered relative to the lane in which it was traveling and it seemed better for the vehicle 1 to move to the adjacent lane. 5, this turn signal operation is paired with information that the driver of the vehicle 1 is viewing the parked vehicle, so it can be determined that this turn signal operation is an operation by the vehicle 1 to avoid the parked vehicle. As a result, the reliability of the intention can be set to (high).

[0035] In other words, in the example shown in Figure 5, in order to increase the accuracy of the true value of the data collected in a data collection system that collects online driving data of the vehicle 1 used by the driver (user) of the vehicle 1, tags indicating whether the driver of the vehicle 1 intended the data or not are assigned depending on the output state of the ``first classification signal (weak signal)'' and ``second classification signal (strong signal).'' The example shown in Figure 5 proposes rules for determining the truth of intentions, and proposes ways to improve the accuracy of learning in machine learning models. When data obtained in the host vehicle 1 is used as learning data for a machine learning model, the learning data needs to have a label (tag) indicating whether or not the driver of the host vehicle 1 intends to depart from the lane. It would be preferable to directly confirm with the driver of the host vehicle 1 whether or not the driver of the host vehicle 1 intends to depart from the lane and attach a tag to the label, but in reality, it is difficult to confirm with the driver of the host vehicle 1 whether or not the driver of the host vehicle 1 intends to depart from the lane. Therefore, in conventional general technology, a scene determination is performed, and if the scene to be determined is, for example, a scene of avoiding a parked vehicle or a lane change scene, it is determined (or assumed) that the driver of the vehicle 1 has the intention to depart from the lane. In contrast, in the example shown in Fig. 5, tagging accuracy at the intention estimation level can be improved by combining driver monitor information. In addition, by stratifying scenes into intention-present scenes with high reliability and intention-present scenes with medium reliability according to two types of strong and weak signals, the accuracy of learning and evaluation can be further improved (by changing the order of learning or changing the ratio).

[0036] As described above, embodiments of the lane departure intention estimation device of the present disclosure have been described with reference to the drawings. However, the lane departure intention estimation device of the present disclosure is not limited to the above-described embodiments and may be modified as appropriate without departing from the spirit and scope of the present disclosure. The configurations of the above-described embodiments may be combined as appropriate. In the above-described embodiments, the processing performed by the lane departure intention estimation device 16 has been described as software processing performed by executing a program. However, the processing performed by the lane departure intention estimation device 16 may be hardware processing. Alternatively, the processing performed by the lane departure intention estimation device 16 may be a combination of both software and hardware processing. Furthermore, the program stored in the memory 162 of the lane departure intention estimation device 16 (the program that realizes the functions of the processor 163 of the lane departure intention estimation device 16) may be recorded on a computer-readable storage medium such as a semiconductor memory, a magnetic recording medium, an optical recording medium, or the like and provided, distributed, etc. [Explanation of symbols]

[0037] 1...Own vehicle, 11...Surrounding condition sensor, 12...Vehicle state sensor, 13...HMI, 14...Driver monitor camera, 15...Vehicle control device, 15A...Steering actuator, 15B...Braking actuator, 15C...Driving actuator, 16...Lane departure intention estimation device, 161...Communication interface, 162...Memory, 163...Processor, 3A...Acquisition unit, 3B...Driver monitor unit, 3C...First determination unit, 3D...Estimation unit, 3E...Prediction unit, 3F...Second determination unit, 3G...Control unit

Claims

1. an estimation unit that estimates that the driver of the host vehicle does not intend to depart from the lane when the driver of the host vehicle is in an inattentive state or in a drowsy state; a prediction unit that, when the driver of the host vehicle is not in an inattentive state and not in an awake state, predicts whether the driver of the host vehicle intends to depart from the lane by using a trained machine learning model, based on a time-series signal including vehicle information that is information about the host vehicle, lane information that is information about the lane in which the host vehicle is traveling, and target information that is information about targets present around the host vehicle, and a first classification signal including a signal indicating that the driver of the host vehicle is not in an inattentive state and not in an awake state; The trained machine learning model is a lane departure intention estimation device obtained by performing training using training data which is a dataset of a training time series signal including training vehicle information which is information about a training vehicle, training lane information which is information about the lane in which the training vehicle is traveling, and training target information which is information about targets present around the training vehicle, a training first classification signal which indicates that the driver of the training vehicle is not in an inattentive state and is not awake, and a label which indicates whether the driver of the training vehicle has the intention to depart from a lane.

2. 2. The lane departure intention estimation device according to claim 1, further comprising: a control unit that limits output of a lane departure warning, which is a warning for the vehicle having departed from its lane, when the prediction unit predicts that the driver of the vehicle has an intention to depart from its lane.

3. The lane departure intention estimation device according to claim 1 , further comprising a control unit that limits execution of lane keeping assist when the prediction unit predicts that the driver of the host vehicle has an intention to depart from a lane.

4. a driver monitor unit that outputs, based on an image including the driver of the vehicle captured by a drive monitor camera, the first classification signal including a signal indicating that the driver of the vehicle is not inattentive and not in a drowsy state, and the second classification signal including a signal indicating that the driver of the vehicle is inattentive or in a drowsy state; 2. The lane departure intention estimation device according to claim 1, wherein, when the driver monitor unit outputs the second classification signal including a signal indicating that the driver of the host vehicle is in an inattentive state or a drowsy state, and when the prediction unit predicts that the driver of the host vehicle has an intention to depart from the lane, the estimation result of the estimation unit indicating that the driver of the host vehicle does not have an intention to depart from the lane is given priority over the prediction result of the prediction unit indicating that the driver of the host vehicle has an intention to depart from the lane.

5. The first classification signal output from the driver monitor unit is input to the prediction unit, 5. The lane departure intention estimation device according to claim 4, wherein the first classification signal includes a signal indicating an internal state of the driver of the host vehicle estimated by the driver monitor unit based on an image including the driver of the host vehicle captured by the drive monitor camera.

Images