Lane departure suppression control device, lane departure suppression control method, and program

Abstract

The present invention addresses the problem of effectively suppressing lane departure of a vehicle when it is detected that a driver is about to sneeze or sneeze. A lane departure suppression control device is characterized by implementing lane departure suppression control capable of suppressing lane departure of a vehicle when the vehicle deviates from a traveling lane or is likely to deviate from the traveling lane, the lane departure suppression control device being provided with: a detection means for detecting the lane departure of the vehicle; a sneezing detection unit that detects whether a driver is about to sneeze or sneeze on the basis of a driver image of the driver of the vehicle; and a control means for performing control so as to increase the steering torque or rudder angle of the lane departure suppression control or suppress the interruption of the lane departure suppression control when the detection means detects that the driver is about to sneeze or has sneezed.

Description

Technical Field

[0001] This invention relates to a lane departure suppression control device, a lane departure suppression control method, and a program. Background Technology

[0002] For example, Patent Document 1 discloses a technology that, when the sound of a driver sneezing is detected, determines whether the steering wheel has been rotated by a predetermined amount, and if it is determined that the steering wheel has been rotated by a predetermined amount, restricts the turning of the wheel corresponding to the steering operation of that predetermined amount of rotation.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2016-005933 Summary of the Invention

[0004] The technology described in Patent Document 1 is limited to the amount of steering control. Therefore, while it can cope with situations where the vehicle is traveling on a straight road, it cannot suppress lane departure when traveling on a curved road where the amount of steering control must be increased.

[0005] The technology of the present invention was made to solve the above-mentioned problems, and its purpose is to effectively suppress lane departure of the vehicle when the driver is detected to be about to sneeze or has already sneezed.

[0006] The present invention relates to a lane departure suppression control device, characterized in that it implements lane departure suppression control capable of suppressing lane departure of the vehicle when the vehicle deviates from or is likely to deviate from its lane. The lane departure suppression control device comprises:

[0007] A detection unit that detects whether the driver is about to sneeze or has already sneezed based on a captured image of the vehicle's driver; and

[0008] The control unit controls the lane departure mitigation control by increasing the steering torque or rudder angle, or by suppressing the interruption of the lane departure mitigation control, when the detection unit detects that the driver is about to sneeze or has already sneezed. Attached Figure Description

[0009] Figure 1 This is a schematic diagram illustrating the hardware structure of the vehicle involved in this embodiment.

[0010] Figure 2 This is a schematic diagram illustrating the software structure of the control device involved in this embodiment.

[0011] Figure 3 This is a schematic diagram illustrating lane departure suppression control.

[0012] Figure 4 This is a timing diagram illustrating the processing flow of LDA enhancement control involved in this embodiment.

[0013] Figure 5 This is a flowchart illustrating the processing routine of LDA enhancement control involved in this embodiment. Detailed Implementation

[0014] Hereinafter, the lane departure suppression control device, lane departure suppression control method and procedure involved in this embodiment will be described with reference to the accompanying drawings.

[0015] [Hardware Structure]

[0016] Figure 1 This is a schematic diagram illustrating the hardware structure of the vehicle VH involved in this embodiment.

[0017] The vehicle (VH) has an Electronic Control Unit (ECU) 10. The ECU 10 includes a Central Processing Unit (CPU) 11, Read Only Memory (ROM) 12, Random Access Memory (RAM) 13, and an interface device 14. The CPU 11 is a processor that executes various programs stored in the ROM 12. The ROM 12 is non-volatile memory that stores data required by the CPU 11 to execute various programs. The RAM 13 is volatile memory that provides the working area for various programs to be executed by the CPU 11. The interface device 14 is a communication device for communicating with external devices.

[0018] ECU10 is the central device for driving assistance functions such as Lane Departure Alert Control (LDA). Driving assistance includes the concept of autonomous driving. Within ECU10, the drive unit 20, braking unit 21, steering unit 22, internal sensor unit 30, external sensor unit 40, driver monitoring unit 50, and Human Machine Interface (HMI) 60 are communicatively connected.

[0019] The drive unit 20 generates driving force that is transmitted to the drive wheels of the vehicle VH. Examples of drive units 20 include electric motors and engines. In this embodiment, the vehicle VH can be any of a hybrid electric vehicle (HEV), a plug-in hybrid electric vehicle (PHEV), a fuel cell electric vehicle (FCEV), a battery electric vehicle (BEV), or a motor-driven vehicle. The braking unit 21 applies braking force to the wheels of the vehicle VH.

[0020] The steering system 22 applies steering force to the wheels of the vehicle VH. The steering system 22 can be either a rack and pinion type or a steer-by-wire type. The steering system 22 has a steering operating unit 23 including a steering wheel SW. Furthermore, the steering system 22 includes a steering motor 25 that applies steering torque to the steering shaft 24. The steering motor 25 generates steering torque according to instructions from the ECU 10. This steering torque enables the left and right steering wheels of the vehicle VH to be turned. Additionally, the steering operating unit 23 is not limited to a steering wheel SW and can be a shape other than a steering wheel, such as a steering stick.

[0021] The internal sensor device 30 is a type of sensor that detects the vehicle's VH (vehicle speed, steering angle, and torque) state. Specifically, the internal sensor device 30 includes a vehicle speed sensor 31, a steering angle sensor 32, a steering torque sensor 33, a yaw rate sensor 34, and an acceleration sensor 35.

[0022] Vehicle speed sensor 31 detects the vehicle speed (V) of vehicle VH. Steering angle sensor 32 detects the rotation angle of the steering wheel SW or steering shaft 24 of vehicle VH, i.e., steering angle σD. Steering torque sensor 33 detects the rotational torque of the steering wheel SW or steering shaft 24, i.e., steering torque Tq. Yaw rate sensor 34 detects the yaw rate of vehicle VH. Acceleration sensor 35 detects the acceleration of vehicle VH. The internal sensor device 30 sends the status of vehicle VH detected by each sensor 31 to 35 to ECU 10 at a predetermined period.

[0023] The external sensor device 40 is a type of sensor that identifies target information related to objects around the vehicle VH. Specifically, the external sensor device 40 includes a radar sensor 41, a camera sensor 42, etc. Here, target information can be exemplified by, for example, surrounding vehicles, white lines painted on the road surface, curbs, guardrails, walls, etc.

[0024] Radar sensor 41 detects targets present around the vehicle VH. Radar sensor 41 includes millimeter-wave radar and / or lidar. The millimeter-wave radar radiates radio waves (millimeter waves) in the millimeter-wave frequency band and receives millimeter waves (reflected waves) reflected by targets within its radiation range. Based on the phase difference between the transmitted millimeter waves and the received reflected waves, the attenuation level of the reflected waves, and the time from transmitting the millimeter waves to receiving the reflected waves, the millimeter-wave radar obtains the relative distance and relative speed between the vehicle VH and the target. The lidar sequentially scans pulsed laser beams with wavelengths shorter than millimeter waves in multiple directions and receives reflected light from targets, thereby obtaining the shape of targets detected in front of the vehicle VH, the relative distance and relative speed between the vehicle VH and the target.

[0025] Camera sensor 42 captures images of the area around the vehicle VH and processes the captured image data to obtain information about objects around the vehicle VH. For example, a digital camera with an imaging element such as a CMOS or CCD can be used as camera sensor 42. Object information includes information such as the type of object detected around the vehicle VH, the relative distance between the vehicle VH and the object, and the relative speed between the vehicle VH and the object. The type of object can be identified, for example, through machine learning such as pattern matching.

[0026] The external sensor device 40 repeatedly transmits the acquired target information to the ECU 10 at predetermined intervals. The ECU 10 determines the relative relationship between the vehicle's VH and the target by synthesizing the relative relationship between the vehicle's VH and the target obtained by the radar sensor 41 and the relative relationship between the vehicle's VH and the target obtained by the camera sensor 42. Furthermore, the external sensor device 40 does not necessarily need to include both the radar sensor 41 and the camera sensor 42; for example, it may only include the camera sensor 42.

[0027] The driver monitoring device 50 is a device for acquiring the driver's status in the vehicle (VH), and for example, includes a driver camera 51. The driver camera 51 mainly captures the driver's face and sends the captured facial images of the driver to the ECU 10 at predetermined intervals.

[0028] HMI60 is an interface for inputting and outputting information between ECU10 and the driver, and includes input and output devices. Input devices may include a touch panel, a switch, or a microphone. Output devices may include a display device 61 and a speaker 62. The display device 61 may be a central display, multi-information display, head-up display, or navigation system display, such as one mounted on an instrument panel. The speaker 62 may be a speaker from an audio system or navigation system.

[0029] [Software Structure]

[0030] Figure 2 This is a schematic diagram illustrating the software structure of the ECU10 involved in this embodiment. For example... Figure 2As shown, the ECU 10 includes a lane recognition unit 100, an LDA control unit 110, a driver state determination unit 120, and an LDA enhancement control unit 130 as functional elements. These functional elements 100-130 are implemented by the CPU 11 of the ECU 10 reading programs stored in the ROM 12 into the RAM 13 and executing them. Furthermore, while each of the functional elements 100-130 is described in this embodiment as an element included in the integrated hardware, i.e., the ECU 10, any part of them can also be provided in another ECU separate from the ECU 10. Moreover, all or part of the functional elements 100-130 of the ECU 10 can also be provided in an information processing device of a facility that can communicate with the vehicle's VH (e.g., a management center).

[0031] The lane recognition unit 100 identifies the driving lane (VH) of the vehicle in motion based on the detection results of the external sensor device 40. Here, the driving lane refers not only to the dividing lines such as white or yellow lines painted on the road surface, but also to the driving area defined by structures such as curbs, guardrails, and walls. In addition, for convenience, the boundaries of the driving area defined by these dividing lines or structures will be referred to as "boundary lines" below.

[0032] like Figure 3 As shown, the lane recognition unit 100 recognizes the left boundary line LL and the right boundary line LR. Furthermore, the lane recognition unit 100 calculates the radius of curvature R of the center line LC, which is the center position of these left and right boundary lines LL and LR, and calculates the deviation angle (hereinafter, yaw angle θy) between the direction of the center line LC and the direction in which the vehicle VH is facing. Moreover, the lane recognition unit 100 calculates the distance (hereinafter, lateral position Δx) in each road width direction between the vehicle VH (e.g., the left front wheel) and the left boundary line LL, and between the vehicle VH (e.g., the right front wheel) and the right boundary line LR. Figure 3 Only the lateral position Δx between the vehicle VH and the left boundary LL is shown. At this time, there are two lateral positions Δx, one on the left and one on the right. However, when performing LDA control as described later, the lateral position Δx estimated as the direction in which the vehicle VH deviates from the driving lane, i.e., the direction represented by the yaw angle θy, is used. Hereinafter, the lateral position Δx, yaw angle θy, and radius of curvature R calculated by the lane recognition unit 100 will also be collectively referred to as "driving lane information".

[0033] When the vehicle speed V detected by the vehicle speed sensor 31 is above a predetermined threshold speed V0, and the vehicle VH is about to deviate from the driving lane, the LDA control unit 110 executes an alarm via the HMI 60 or a lane departure warning by imparting vibration to the steering wheel SW, and performs LDA control to suppress the vehicle VH from deviating from the driving lane by controlling the operation of the steering device 22 or the braking device 21. The LDA control unit 110 calculates the target steering angle (hereinafter, LDA target steering angle σB) for LDA control based on the driving lane information (Δx, θY, R) obtained by the lane recognition unit 100. The LDA target steering angle σB is set to the steering angle at which the vehicle VH does not deviate to the outside of the boundary lines LL, LR (left boundary LL in the example).

[0034] The LDA control unit 110 determines whether the LDA start condition is met. For example, an LDA start condition could be defined as the case where the predicted arrival time TR before the vehicle VH reaches the boundary lines LL and LR (left boundary LL in the example diagram) is shorter than a predetermined threshold TR0 (TR < TR0). The predicted arrival time TR can be calculated, for example, based on a known formula assuming that the vehicle VH undergoes constant acceleration linear motion before reaching the boundary lines LL and LR.

[0035] When the LDA start condition is met, the LDA control unit 110 calculates the LDA auxiliary torque Ts based on the difference Δσ between the target LDA rudder angle σB and the steering angle σD obtained by the steering angle sensor 32. Furthermore, while calculating the LDA auxiliary torque Ts, the LDA control unit 110 sends a command signal including information representing the LDA auxiliary torque Ts to the steering device 22. As a result, the LDA auxiliary torque Ts is transmitted from the steering motor 25 to the steering shaft 24, causing the steering wheels of the vehicle VH to steer, thereby preventing the vehicle VH from deviating from its driving lane.

[0036] After initiating LDA control, the LDA control unit 110 terminates LDA control when the lateral position Δx of the vehicle VH acquired by the lane recognition unit 100 reaches or exceeds the predetermined lateral position Δx2 for termination determination, assuming the lane departure state has been eliminated. Furthermore, during the execution of LDA control, if the steering torque Tq (hereinafter also referred to as driver input torque) detected by the steering torque sensor 33 exceeds a predetermined cancellation torque threshold Tq0, the LDA control unit 110 interrupts, i.e., cancels, the executing LDA control.

[0037] The driver state determination unit 120 is an example of the detection and determination unit of the present invention, which determines the driver's state based on the driver's facial image sent from the driver monitoring device 50. In this embodiment, the driver state determination unit 120 determines (1) whether the driver is in a state of anticipation of sneezing, (2) whether the driver has already sneezed, and (3) whether the driver is in a distracted state. Regarding whether the driver is in a state of anticipation of sneezing, for example, it can be determined by recognizing the driver's closed eyes, facial or head movements, etc., from the driver's facial image. Regarding whether the driver has already sneezed, for example, it can be determined by recognizing the driver's mouth movements, facial or head movements, etc., from the driver's facial image. Regarding whether the driver is in a distracted state, for example, if the driver's gaze direction or facial orientation, as recognized by the driver's facial image, continues for a predetermined threshold time T0 or more without being oriented towards a predetermined range including the front of the vehicle VH, it can be determined that the driver is in a distracted state. Furthermore, if the driver status determination unit 120 determines that the driver is in a state of anticipation of sneezing or determines that the driver has already sneezed, even if the driver subsequently makes a large head movement and the face deviates from the fixed position, it will not treat it as a state that cannot be identified by driver monitoring.

[0038] The LDA enhancement control unit 130 is an example of the control unit of the present invention. It performs LDA enhancement control, which enhances the LDA control performed by the LDA control unit 110, based on the determination result of the driver state determination unit 120. Specifically, the LDA enhancement control unit 130 performs LDA enhancement control when the driver state determination unit 120 determines (1) that the driver is in a state of anticipation of sneezing, (2) that the driver has already sneezed, and (3) that the driver is in a state of distraction. For example, LDA enhancement control can be performed by increasing the LDA auxiliary torque Ts or the LDA target steering angle σB used by the LDA control unit 120 for steering control compared to its normal value (a value set when no sneezing is detected). Hereinafter, based on... Figure 4 The timing diagram illustrates the processing flow of LDA enhancement control.

[0039] At time t1, when the driver state determination unit 120 determines that the driver is in a state of anticipation of sneezing, the LDA enhancement control unit 130 executes LDA enhancement control, which enhances LDA control even if the driver's face is not in a fixed position. That is, when the driver is about to sneeze and unconsciously wants to cause the vehicle VH to deviate from the driving lane, LDA control is executed based on a larger than normal LDA assist torque Ts or LDA target steering angle σB. As a result, lane departure of the vehicle VH can be effectively suppressed in the anticipation stage before the driver actually sneezes. Furthermore, by increasing the LDA assist torque Ts or LDA target steering angle σB through LDA enhancement control, lane departure of the vehicle VH can be effectively suppressed not only when the vehicle VH is traveling on a straight road, but also when traveling on a curved road.

[0040] At time t2, when the driver state determination unit 120 determines that the driver has sneezed, the LDA enhancement control unit 130 reduces the threshold time T0 used by the driver state determination unit 120 to determine the distraction state within a predetermined period Tt. That is, when the driver has sneezed, the criteria for determining the distraction state are relaxed. As a result, when the driver is distracted after sneezing, for example, by looking for a tissue or wiping his hands, this behavior of the driver can be easily identified as a distraction state. When the driver is determined to be in a distraction state, by enhancing LDA control, lane departure of the vehicle VH due to the driver's actions after sneezing can be effectively suppressed. The predetermined period Tt is not particularly limited; for example, it can be set based on the average time (e.g., tens of seconds) required for the driver to look for a tissue or wipe his hands after sneezing. Furthermore, the amount of reduction in the threshold time T0 is not particularly limited; it can be a fixed value or a variable value corresponding to the vehicle speed V of the vehicle VH.

[0041] Suppose that after the driver sneezes, at time t3, the rate of change Tqv of the driver's input torque Tq detected by the steering torque sensor 33 exceeds a threshold speed. At this time, even if the driver's input torque Tq exceeds the LDA cancellation torque threshold Tq0, the LDA enhancement control unit 130 prohibits the interruption (cancellation) of LDA control. Therefore, even if the driver makes an unintended steering operation due to a sneeze, LDA control can reliably continue. That is, the influence of unintentional steering operations accompanying a driver's sneeze can be eliminated, and lane departure of the vehicle's VH can be effectively suppressed.

[0042] Figure 5 This is a flowchart illustrating the processing routine of LDA enhanced control performed by CPU 11 of ECU 10. This routine begins, for example, with the movement of the vehicle VH.

[0043] In step S100, ECU10 determines whether the vehicle speed V of vehicle VH is above the threshold speed V0 based on the detection result of vehicle speed sensor 31. If the vehicle speed V is above the threshold speed V0 (yes), ECU10 proceeds to step S110. On the other hand, if the vehicle speed V is below the threshold speed V0 (no), ECU10 returns to the current routine.

[0044] In step S110, ECU 10 determines whether the driver is in a state of anticipation of sneezing based on the facial image of the driver captured by driver camera 51. If it is determined that the driver is in a state of anticipation of sneezing (yes), ECU 10 proceeds to step S120. On the other hand, if it is determined that the driver is not in a state of anticipation of sneezing (no), ECU 10 returns to the current routine.

[0045] In step S120, ECU 10 enhances LDA control by increasing the LDA assist torque Ts or LDA target steering angle σB during lane departure compared to normal values. Next, in step S130, ECU 10 determines whether the driver has sneezed based on an image of the driver's face captured by driver camera 51. If the driver has sneezed (yes), ECU 10 proceeds to step S140. Otherwise, if the driver has not sneezed (no), ECU 10 returns to the current routine.

[0046] In step S140, ECU 10 relaxes the criteria for determining a distraction state by reducing the threshold time T0 used to determine the distraction state within a specified period Tt. Next, in step S150, ECU 10 determines whether the rate of change Tqv of the driver's input torque Tq exceeds a threshold speed based on the detection result of the steering torque sensor 33. If the rate of change Tqv of the driver's input torque Tq exceeds the threshold speed (yes), ECU 10 proceeds to step S160. Conversely, if the rate of change Tqv of the driver's input torque Tq does not exceed the threshold speed (no), ECU 10 returns to the current procedure.

[0047] In step S160, even if the driver input torque Tq exceeds the LDA cancellation torque threshold Tq0, ECU10 performs interruption prohibition processing to disable LDA control. Then, ECU10 returns to the current routine.

[0048] The lane departure suppression control device, lane departure suppression control method and procedure involved in this embodiment have been described above. However, the present invention is not limited to the above embodiment. Various modifications can be made as long as they do not depart from the purpose of the present invention.

[0049] For example, the technology of this invention can also be applied to autonomous vehicles. In this case, when switching from autonomous to manual driving, only the control of this invention needs to be implemented.

Claims

1. A lane departure suppression control device, characterized in that, Lane departure suppression control, which suppresses lane departure of a vehicle when the vehicle deviates from or is likely to deviate from its lane, includes: A detection unit that detects whether the driver is about to sneeze or has already sneezed based on a captured image of the vehicle's driver; and The control unit controls the lane departure mitigation control by increasing the steering torque or rudder angle, or by suppressing the interruption of the lane departure mitigation control, when the detection unit detects that the driver is about to sneeze or has already sneezed.

2. The lane departure suppression control device according to claim 1, characterized in that, The control unit is configured as follows: If the driver steering torque input by the driver exceeds a predetermined torque threshold during the execution of the lane departure suppression control, the execution of the lane departure suppression control is interrupted. and If the detection unit detects that the driver is about to sneeze or has already sneezed, and the rate of change of the driver's steering torque exceeds a predetermined threshold speed, then the interruption of the lane departure suppression control is prohibited.

3. The lane departure suppression control device according to claim 1, characterized in that, It also includes a determination unit that determines, based on the driver image, whether the driver is in a distracted state and not looking ahead at the vehicle. The control unit is configured as follows: If the determination unit determines that the driver is in the distracted state, the steering torque or rudder angle of the lane departure suppression control is increased. And if the detection unit detects that the driver is about to sneeze or has already sneezed, the determination unit can easily determine that the driver is in the distracted state by relaxing the conditions used for determination.

4. A lane departure suppression control method, characterized in that, When a vehicle deviates from or is likely to deviate from its lane, lane departure suppression control is implemented to suppress the vehicle's lane departure. The lane departure suppression control method includes the following steps: The system detects whether the driver is about to sneeze or has already sneezd based on an image of the vehicle's driver taken by camera; and If the driver is detected to be about to sneeze or has already sneezed, the steering torque or rudder angle of the lane departure suppression control is increased, or the interruption of the lane departure suppression control is suppressed.

5. A program, characterized in that, The computer of the lane departure suppression control device, which implements lane departure suppression control to suppress lane departure when the vehicle deviates from or is likely to deviate from its lane, performs the following processing: The system detects whether the driver is about to sneeze or has already sneezd based on an image of the vehicle's driver taken by camera; and If the driver is detected to be about to sneeze or has already sneezed, the lane departure suppression control is activated by increasing the steering torque or rudder angle of the lane departure suppression control, or by suppressing the interruption of the lane departure suppression control.

Images