Control device for driving support system

Abstract

To provide a control device for a driving support system, capable of controlling driving support on the basis of a driving behavior of a driver when the driver feels stressed.SOLUTION: A control device for a driving support system 1000 for supporting driving of a driver, includes: a feeling determination section 152b for determining whether the driver feels stressed, on the basis of a face image of the driver; a driving tendency determination section 152e for determining whether current driving of the driver tends to be unstable driving as compared to usual driving, on the basis of a result obtained by comparison between a driving behavior of the driver which has been acquired when it has been determined that the driver feels stressed, and a usual driving behavior of the driver; and a change section 152g for changing control of driving support performed by the driving support system 1000 when it is determined that the driver tends to perform the unstable driving.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a control device for a driving support system.

Background Art

[0002] Conventionally, when it is estimated that a passenger on an autonomous vehicle is stressed based on biometric information indicating the emotion of the passenger, the stress target, which is the cause of stress outside the vehicle, is identified from the direction of the passenger's line of sight, and control is performed to increase the safety margin by making it difficult to approach the stress target. (See Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above patent document, the stress target is limited to the cause of stress outside the vehicle, such as the intrusion of another vehicle. However, the causes of stress vary from person to person, and if the cause cannot be specifically identified, it is assumed that the driver cannot be assisted in driving. For this reason, in the technology described in the above patent document, when the driver feels stress due to factors other than external factors outside the vehicle, such as temporal pressure (time pressure) or psychological factors such as accident anxiety that the driver feels during driving, there is a problem that a safety margin cannot be ensured based on these stress factors.

[0005] In addition, a driver in a stressed state tends to take unstable driving actions such as a narrow inter-vehicle distance, sudden acceleration and deceleration, and driving at a high vehicle speed. Since the technology described in the above patent document does not consider such a tendency of driving behavior, there is a limit to ensuring a safety margin.

[0006] Therefore, an object of the present invention is to provide a control device for a driving support system capable of controlling driving support based on a driver's driving behavior when the driver is feeling stress.

Means for Solving the Problems

[0007] The gist of the present disclosure is as follows.

[0008] (1) A control device for a driving support system that performs driving support for a driver, An emotion determination unit that determines whether the driver is feeling stress based on the driver's face image, Based on the result of comparing the driving behavior of the driver obtained when it is determined that the driver is feeling stress with the driver's normal driving behavior, a driving tendency determination unit that determines whether the current driving of the driver has an unstable driving tendency compared to the normal driving, A change unit that changes the control of the driving support by the driving support system when it is determined that the driver has the unstable driving tendency, A control device for a driving support system, comprising:

[0009] (2) The control device for a driving support system according to (1) above, wherein when it is determined that the driver has the unstable driving tendency, the change unit advances the timing of performing the previous driving support.

[0010] (3) The driving support is performed when the distance to an object around the vehicle reaches a predetermined threshold value, The control device for a driving support system according to (1) or (2) above, wherein when it is determined that the driver has the unstable driving tendency, the change unit increases the threshold value.

[0011] (4) The driving support is performed so that the inter-vehicle distance from the preceding vehicle becomes a predetermined value, The control device of the driving assistance system according to (1) above, wherein the changing unit increases the predetermined value when the driver is determined to have the unstable driving tendency.

Effect of the Invention

[0012] According to the present invention, there is provided a control device of a driving assistance system capable of controlling driving assistance based on a driver's driving behavior when the driver is feeling stressed.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Mode for Carrying Out the Invention

[0014] Hereinafter, some embodiments according to the present invention will be described with reference to the drawings. However, these descriptions are intended merely as examples of preferred embodiments of the present invention and are not intended to limit the present invention to such specific embodiments.

[0015] FIG. 1 is a schematic configuration diagram of an operation support system 1000 according to one embodiment. The operation support system 1000 is mounted on a vehicle such as an automobile, and includes an in-vehicle camera 110, a driver monitor camera 120, a vehicle control device 130, one or more sensors 140, an electronic control unit (ECU) 150, and a warning device 160. Each of the in-vehicle camera 110, the driver monitor camera 120, the vehicle control device 130, the sensor 140, the ECU 150, and the warning device 160 is communicably connected via an in-vehicle network compliant with a standard such as a Controller Area Network (CAN).

[0016] The in-vehicle camera 110 and the driver monitor camera 120 include a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or a C-MOS, and an imaging optical system that forms an image of an area to be photographed on the two-dimensional detector. The in-vehicle camera 110 is provided near the dashboard inside the vehicle or near the windshield, photographs the surroundings of the vehicle (for example, the front of the vehicle), and generates an image representing the environment around the vehicle. Similarly, the driver monitor camera 120 is provided toward the assumed position of the driver near the dashboard, the steering column, or the windshield inside the vehicle, photographs the driver's face, and generates an image (face image) in which the driver's face is shown. The in-vehicle camera 110 and the driver monitor camera 120 perform photographing at a predetermined photographing cycle (for example, 1 / 30 second to 1 / 10 second). The in-vehicle camera 110 may be composed of a stereo camera and may be configured to acquire the distance from each structure on the image based on the parallax between the left and right images. Each time the in-vehicle camera 110 and the driver monitor camera 120 generate an image, they output the generated image to the ECU 150 via the in-vehicle network.

[0017] The vehicle control device 130 is various devices related to vehicle control, and includes a driving device such as an internal combustion engine or an electric motor as a driving source for driving the vehicle, a transmission, a braking device for braking the vehicle, a steering device for turning the vehicle, and the like. When the driving source for driving the vehicle is an electric motor, the vehicle control device 130 may include a battery for storing electric power, a fuel cell for supplying electric power to the electric motor, and the like. Note that when the vehicle is an electric vehicle (EV), the internal combustion engine may not be included in the vehicle control device 130. Further, when the vehicle is an engine vehicle, the electric motor may not be included in the vehicle control device 130.

[0018] The one or more sensors 140 include sensors for peripheral monitoring for monitoring the periphery of the vehicle 100, for example, sensors such as a lidar (Light Detection and Ranging) and a radar. The radar includes a front side radar sensor inside the front bumper and a rear side radar sensor inside the rear bumper. Further, the one or more sensors 140 include a vehicle speed sensor for detecting the vehicle speed and an acceleration sensor for detecting the acceleration of the vehicle.

[0019] The ECU 150 is an aspect of a control device that controls the driving assistance system 1000. The ECU 150 includes a processor 152, a memory 154, and a communication interface 156. The processor 152 includes one or more CPUs (Central Processing Units) and peripheral circuits thereof. The processor 152 may further include other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphic processing unit. The memory 154 includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory, and stores data related to the processing according to the present embodiment as necessary. The communication interface 156 includes an interface circuit for connecting the ECU 150 to an in-vehicle network.

[0020] The warning device 160 includes a display device and a speaker. The display device is composed of, for example, a liquid crystal display (LCD), is provided near a meter panel or a dashboard, and displays and outputs a warning in response to an instruction from the ECU 150. The speaker outputs a warning audibly in response to an instruction from the ECU 150.

[0021] Figure 2 is a schematic diagram for explaining the processing performed by the driving support system 1000. First, in the first stage of the processing, the driver's emotion is estimated based on the face image of the driver captured by the driver monitor camera 120, and it is determined whether the driver is feeling stress. In the second stage, when it is determined that the driver is feeling stress, the driving behavior feature amount C1 of the vehicle by the driver is acquired. Also, in the second stage, the driving behavior feature amount C2 in the normal driving in which the driver is not feeling stress, which is stored in advance in the memory 154, is acquired. In the third stage, the driving behavior feature amount C1 of the driver acquired when it is determined that the driver is feeling stress is compared with the driving behavior feature amount C2 of the driver's normal driving. When the driving behavior feature amount C1 and the driving behavior feature amount C2 deviate by a predetermined value or more, the control by the driving support system is changed to the safer side. For example, warning a driver in a stressed state about speeding or the like may cause concern that the psychological state will deteriorate further. Therefore, by changing the set value of the driving support system to the safer side, more appropriate driving support can be provided without the driver being aware of it.

[0022] FIG. 3 is a schematic diagram showing the functional blocks of the processor 152 of the ECU 150 for realizing the above-described processing. The processor 152 of the ECU 150 includes a face image acquisition unit 152a, an emotion determination unit 152b, a driving behavior acquisition unit 152c, a comparison unit 152d, a driving tendency determination unit 152e, a driving support unit 152f, and a modification unit 152g. Each of these units included in the processor 152 is a functional module realized by, for example, a computer program operating on the processor 152. That is, the functional blocks of the processor 152 are composed of the processor 152 and a program (software) for operating the same. Further, the program may be recorded in the memory 154 provided in the ECU 150 or a recording medium connected externally. Alternatively, each of these units included in the processor 152 may be a dedicated arithmetic circuit provided in the processor 152.

[0023] The face image acquisition unit 152a of the processor 152 acquires the face image of the driver generated by the driver monitoring camera 120.

[0024] The emotion determination unit 152b of the processor 152 estimates the emotion of the driver based on the face image of the driver, and determines whether the driver is feeling stress based on the estimated emotion. At this time, for example, the emotion of the driver is estimated by template matching between a template image representing various facial expressions corresponding to emotions and the face image acquired by the face image acquisition unit 152a, or by inputting the image acquired by the face image acquisition unit 162a to an identifier learned by machine learning for detecting the facial expression of the face corresponding to the emotion. Then, the emotion determination unit 152b determines that the driver is feeling stress when the stress level of the estimated emotion exceeds a predetermined threshold based on, for example, a stress level predetermined for each emotion.

[0025] Note that the emotion determination unit 152b can use, as the above-described identifier, for example, a segmentation identifier that has been pre-trained to output, for each pixel of the input image, the probability that an object that may be represented by that pixel belongs to each type of object, and to identify that the object with the highest probability is being represented. As such an identifier, the emotion determination unit 152b can use, for example, a deep neural network (DNN) having an architecture of a convolutional neural network type (CNN) for segmentation, such as a Fully Convolutional Network (FCN).

[0026] The driving behavior acquisition unit 152c of the processor 152 acquires the driving behavior of the vehicle by the driver. More specifically, when it is determined that the driver is feeling stress, the driving behavior acquisition unit 152c acquires the driving behavior feature amount C1 of the driver and the driving behavior feature amount C2 of the driver's normal driving. The driving behavior feature amounts C1 and C2 include the inter-vehicle distance from the preceding vehicle, the acceleration / deceleration of the vehicle, and the vehicle speed.

[0027] When it is determined that the driver is feeling stress, the driving behavior acquisition unit 152c acquires the inter-vehicle distance from the preceding vehicle from an image representing the environment around the vehicle generated by the in-vehicle camera 110. Also, the driving behavior acquisition unit 152c acquires the acceleration / deceleration of the vehicle from an acceleration sensor and the vehicle speed from a vehicle speed sensor.

[0028] On the other hand, the driving behavior feature amount C2 of the driver's normal driving is acquired in advance and stored in the memory 154 when it is determined that the driver is not feeling stress. The driving behavior acquisition unit 152c acquires the driving behavior feature amount C2 stored in the memory 154 in advance when it is determined that the driver is feeling stress.

[0029] When it is determined that the driver is feeling stress, the comparison unit 152d of the processor 152 compares the driving behavior of the driver (driving behavior feature amount C1) acquired at that time with the driving behavior of the driver's normal driving (driving behavior feature amount C2).

[0030] Based on the determination result of whether the driver is feeling stress and the comparison result by the comparison unit 152d, the driving tendency determination unit 152e of the processor 152 determines whether the current driving of the driver has an unstable driving tendency compared to the normal driving. Specifically, when it is determined that the driver is feeling stress, if the driving behavior feature amount C1 and the driving behavior feature amount C2 deviate by a predetermined value or more, the driving tendency determination unit 152e determines that the current driving of the driver has an unstable driving tendency.

[0031] Based on the image representing the environment around the vehicle generated by the in-vehicle camera 110 and the information detected by the peripheral monitoring sensor, the driving support unit 152f of the processor 152 controls the vehicle control device 130 and also controls the warning device 160 to support the driving of the driver.

[0032] The driving support unit 152f provides driving support when the distance to an object around the vehicle reaches a predetermined threshold. For example, when the driving support unit 152f detects an object such as another vehicle or a person in front of the vehicle based on the image generated by the in-vehicle camera 110 or the information detected by the peripheral monitoring sensor and determines that there is a possibility of collision with the object, it causes the warning device 160 to output a warning and also activates the emergency brake of the braking device. At this time, when the distance to the object becomes equal to or less than the first threshold, the driving support unit 152f outputs a warning and activates the emergency brake.

[0033] In addition, when the driving support unit 152f detects an object such as a pedestrian or the curb of a curve ahead based on the image generated by the in-vehicle camera 110 or the information detected by the peripheral monitoring sensor, it controls the braking device or the steering device so as not to approach the object too closely. At this time, when the distance to the object becomes equal to or less than the second threshold, the driving support unit 152f intervenes in the control of the braking device or the steering device.

[0034] In addition, based on the image generated by the in-vehicle camera 110, the driving support unit 152f controls the steering device to avoid deviating from the lane or road. At this time, when there is another vehicle traveling adjacent to it, if the distance to the other vehicle becomes equal to or less than a third threshold value, the driving support unit 152f intervenes in the control of the steering device and controls the steering to avoid approaching the other vehicle.

[0035] Furthermore, the driving support unit 152f uses a rear side radar sensor inside the rear bumper, and when there is another vehicle on the rear side when the driver changes lanes, it causes the warning device 160 to output a warning to assist the driver in making a judgment when changing lanes. At this time, when the distance to the other vehicle becomes equal to or less than a fourth threshold value, the driving support unit 152f outputs a warning.

[0036] Also, based on the image generated by the in-vehicle camera 110 or the information detected by the surrounding monitoring sensor, the driving support unit 152f performs driving support so that the inter-vehicle distance from the preceding vehicle becomes a predetermined value, and controls the internal combustion engine, transmission, electric motor, and braking device.

[0037] When it is determined that the driver has an unstable driving tendency, the change unit 152g of the processor 152 changes the control of the driving support by the driving support system 1000. When it is determined that the driver has an unstable driving tendency, the change unit 152g advances the timing of performing the driving support.

[0038] For example, when the driver is determined to have an unstable driving tendency, the change unit 152g increases the first threshold value, and when there is a possibility of colliding with an object, it advances the timing of the activation of the emergency brake or the output of a warning. Also, the change unit 152g increases the second threshold value to advance the timing of intervention in the braking device or the steering device so as not to approach an object such as a pedestrian or the curb of a forward curve too much. Further, the change unit 152g increases the third threshold value to advance the timing at which the steering device intervenes when the vehicle approaches an adjacent vehicle when controlling the steering device to avoid deviating from the lane or the road.

[0039] Also, the change unit 152g increases the fourth threshold value to expand the operating range of the warning device 160 when warning of the presence of a rear vehicle during a lane change when assisting the driver's judgment during a lane change using the rear side radar sensor inside the rear bumper.

[0040] Also, when driving support is performed so that the inter-vehicle distance from the preceding vehicle becomes a predetermined value, if the driver is determined to have an unstable driving tendency, the change unit 152g increases the predetermined value to increase the inter-vehicle distance from the preceding vehicle.

[0041] As described above, when estimating the stress state only from the face image, there is a concern that the stress state cannot be estimated appropriately and may be misdetected. However, by comparing normal driving with driving in a state determined to be the stress state and implementing control changes only when an unsafe driving tendency is observed, driving support is performed more appropriately.

[0042] FIG. 4 is a flowchart showing the processing performed by the processor 152 of the ECU 150 at each predetermined control cycle. First, the face image acquisition unit 152a acquires a face image of the driver captured by the driver monitoring camera 120 (step S10). Next, the emotion determination unit 152b estimates the driver's emotion based on the face image of the driver (step S12) and determines whether the driver is feeling stress (step S14).

[0043] When it is determined in step S14 that the driver is feeling stress, the image generated by the in-vehicle camera 110 and the information detected by the sensor 140 are acquired (step S16). On the other hand, when it is determined in step S14 that the driver is not feeling stress, the process returns to step S10.

[0044] After step S16, the driving behavior acquisition unit 152c acquires the driving behavior of the vehicle by the driver in a state where the driver is feeling stress (step S18).

[0045] Next, the comparison unit 152d compares the driving behavior of the driver acquired when it is determined that the driver is feeling stress with the driving behavior of the driver's normal driving, and the driving tendency determination unit 152e determines whether the current driving of the driver has an unstable driving tendency compared to the normal driving (step S20).

[0046] When it is determined in step S20 that the current driving of the driver has an unstable driving tendency compared to the normal driving, the change unit 152g changes the control of the driving support by the driving support system 1000 (step S22). After step S22, the process in this control cycle ends. On the other hand, when it is not determined in step S20 that the current driving of the driver has an unstable driving tendency compared to the normal driving, the process in this control cycle ends without going through step S22.

[0047] As described above, according to the present embodiment, the control of the driving support is changed based on the stress state of the driver generated by various psychological factors and the driving behavior when the driver is feeling stress. Thereby, it becomes possible to perform more appropriate driving support when the driver is feeling stress without specifying the stress factors.

Explanation of Reference Numerals

[0048] 110 In-vehicle camera 120 Driver monitoring camera 130 Vehicle control device 140 Sensor 150 Electronic control unit (ECU) 152 Processor 152a Facial image acquisition unit 152b Emotion determination unit 152c Driving behavior acquisition unit 152d Comparison unit 152e Driving tendency determination unit 152f Driving support unit 152g Change unit 154 Memory 156 Communication interface 160 Warning device 1000 Driving support system

Claims

1. A control device for a driving support system that provides driving support for a driver of a vehicle, an emotion determination unit that determines whether the driver is feeling stress based on a facial image of the driver, a driving tendency determination unit that determines whether the current driving of the driver has an unstable driving tendency compared to the normal driving based on the result of comparing the driving behavior of the driver obtained when it is determined that the driver is feeling stress with the normal driving behavior of the driver, a change unit that changes the control of the driving support by the driving support system when it is determined that the driver has the unstable driving tendency, A control device for a driving support system, comprising:

2. The control device for a driving support system according to claim 1, wherein when it is determined that the driver has the unstable driving tendency, the change unit advances the timing of providing the pre-driving support.

3. The driving support is provided when the distance to an object around the vehicle reaches a predetermined threshold value, The control device for a driving support system according to claim 1 or 2, wherein when it is determined that the driver has the unstable driving tendency, the change unit increases the threshold value.

4. The driving support is provided so that the inter-vehicle distance from the preceding vehicle becomes a predetermined value, The control device for a driving support system according to claim 1, wherein when it is determined that the driver has the unstable driving tendency, the change unit increases the predetermined value.

Images