Vehicle control system

Abstract

This system prevents the settings from being changed repeatedly based on the sleep status of the vehicle's passengers. [Solution] The ECU 150's processor 152 includes a sleep state determination unit 152a that determines the sleep state of passengers other than the driver in the vehicle, and a setting change unit 152b that changes the settings of the driving assistance function that allows the vehicle to autonomously perform at least a part of the driving operations or the settings of the warnings output from the HMI device 160 based on the sleep state of the passengers. The setting change unit 152b maintains the changed settings until the vehicle's power is turned off.

Description

Technical Field

[0001] The present invention relates to a vehicle control device.

Background Art

[0002] Conventionally, it is known to determine an acceleration / deceleration value based on the psychological states of vehicle occupants including the driver and passengers during autonomous driving, and to correct the numerical value based on whether each occupant is sleeping when quantifying the psychological state (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the sleep state of an occupant may change moment by moment, and may become deeper or shallower. In the technology described in the above patent document, the acceleration / deceleration value is changed many times according to the sleep state of the occupant, so there is a possibility that the driver may misrecognize the setting state of the acceleration / deceleration value of the driving support function. And if the driver misrecognizes the setting state of the driving support function, there is a problem that the driver may overtrust the driving support function and it may affect the driving operation of the driver.

[0005] Therefore, an object of the present invention is to provide a vehicle control device capable of suppressing the situation where the vehicle settings are changed many times according to the sleep state of a passenger in the vehicle when changing the vehicle settings according to the sleep state.

Means for Solving the Problems

[0006] The gist of the present disclosure is as follows.

[0007] (1) A sleep state determination unit that determines the sleep state of passengers other than the driver in the vehicle, A setting change unit that changes the settings of a driver assistance function that allows the vehicle to autonomously perform at least a portion of the driving operations or the settings of a warning output from a warning device, based on the sleep state of the passenger, Equipped with, The setting change unit is a vehicle control device that maintains the changed setting after the setting has been changed until the vehicle's power is turned off.

[0008] (2) The vehicle control device according to (1) above, wherein the setting change unit changes the setting of the driving assistance function between a first mode of driving operations performed autonomously by the vehicle under normal circumstances and a second mode of driving operations that are less rigorous than the first mode, and when it is determined that the passenger is asleep, the setting of the driving assistance function changes from the first mode to the second mode.

[0009] (3) The vehicle control device according to (1) or (2) above, wherein the setting change unit determines that the passenger is asleep, and further reduces the vehicle acceleration / deceleration to maintain the distance to the preceding vehicle by the driving assistance function or the amount of steering when the vehicle deviates from its lane by the driving assistance function.

[0010] (4) The control device for the vehicle as described in (1) or (2) above, wherein the setting change unit changes the timing of steering intervention by the driving assistance function when lane departure occurs to an earlier timing when it is determined that the passenger is asleep.

[0011] (5) The control device for the vehicle according to (1) or (2) above, wherein the setting change unit reduces the volume of the warning output from the warning device when it is determined that the passenger is asleep. [Effects of the Invention]

[0012] According to the present invention, a vehicle control device is provided that can prevent the settings from being changed repeatedly depending on the sleep state of the passengers in the vehicle when the vehicle settings are changed according to the sleep state of the passengers. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of a vehicle driver assistance system according to one embodiment. [Figure 2] This is a schematic diagram showing the functional blocks of the ECU's processor. [Figure 3] This is a flowchart showing the processing performed by the ECU's processor. [Modes for carrying out the invention]

[0014] Several embodiments of the present invention will be described below with reference to the drawings. However, these descriptions are intended to be merely illustrative of preferred embodiments of the present invention and are not intended to limit the present invention to such specific embodiments.

[0015] Figure 1 is a schematic diagram of a vehicle driver assistance system 1000 according to one embodiment. The driver assistance system 1000 is mounted on a vehicle such as an automobile and includes a front camera 110, an interior camera 120, one or more sensors 130, a vehicle control device 140, an electronic control unit (ECU, hereinafter referred to as ECU) 150, and an HMI device 160. The front camera 110, the interior camera 120, one or more sensors 130, the vehicle control device 140, the ECU 150, and the HMI device 160 are each connected to communicate via an in-vehicle network compliant with a standard such as a Controller Area Network (CAN).

[0016] The front camera 110 and the interior camera 120 each have a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The front camera 110 is installed inside the vehicle, near the dashboard or windshield, and photographs the area around the vehicle (for example, in front of the vehicle) to generate an image representing the environment around the vehicle. The interior camera 120 is installed inside the vehicle, near the dashboard, steering column, or windshield, facing the assumed position of a passenger in the vehicle, and photographs the passenger's face to generate an image of the passenger's face (face image). The front camera 110 and the interior camera 120 take photographs at predetermined shooting intervals (for example, 1 / 30 second to 1 / 10 second). The front camera 110 may be composed of a stereo camera and may be configured to acquire the distance to each structure on the image from the parallax of the left and right images. The front camera 110 and the interior camera 120 output the generated image to the ECU 150 via the in-vehicle network each time they generate an image.

[0017] One or more sensors 130 include surrounding monitoring sensors for monitoring the area around the vehicle, such as lidar (light detection and randomizing) and radar sensors.

[0018] Vehicle control equipment 140 consists of various devices related to vehicle control, including a drive system such as an internal combustion engine or electric motor as a drive source for driving the vehicle, a transmission, a braking system for braking the vehicle, and a steering system for turning the vehicle. If the drive source for driving the vehicle is an electric motor, vehicle control equipment 140 may also include a battery for storing electricity, a fuel cell for supplying electricity to the electric motor, etc. However, if the vehicle is an electric vehicle (EV), an internal combustion engine is not required to be included in vehicle control equipment 140. Also, if the vehicle is a gasoline-powered vehicle, an electric motor is not required to be included in vehicle control equipment 140.

[0019] The ECU 150 is an aspect of the vehicle determination device according to the present disclosure. The ECU 150 has a processor 152, a memory 154, and a communication interface 156. The processor 152 has one or more CPUs (Central Processing Units) and its peripheral circuits. The processor 152 may further have other arithmetic circuits such as a logical arithmetic unit, a numerical arithmetic unit, or a graphic processing unit. The memory 154 has, for example, a volatile semiconductor memory and a non-volatile semiconductor memory, and stores data related to the processing according to the present embodiment. In particular, the memory 154 stores data related to the setting of the driving support function or the setting of the warning output from the HMI device 160. Specifically, the memory 154 stores both the set value when it is determined that the passenger is sleeping and the set value (initial value) when it is not determined that the passenger is sleeping for these data. The communication interface 156 has an interface circuit for connecting the ECU 150 to the in-vehicle network.

[0020] The HMI device 160 is an aspect of the warning device and includes a display device, a speaker, and a vibration device. The display device is composed of, for example, a liquid crystal display (LCD), and is provided near the meter panel or the dashboard. The display device displays and outputs a warning according to an instruction from the ECU 150. The speaker outputs a warning by sound according to an instruction from the ECU 150. The vibration device is a device that vibrates the steering wheel, seat belt, etc., and outputs a warning by vibration according to an instruction from the ECU 150.

[0021] The driving support system 1000 provides driving support by means of a driving support function in which the vehicle autonomously performs at least a part of the driving operation. Specifically, the driving support system 1000 provides driving support by means of an adaptive cruise control (ACC) function and a lane departure prevention assist (LDA) function based on the image of the front camera 110 and the information detected by the sensor 130. The adaptive cruise control function, for example when driving on a highway, detects a white line and a vehicle ahead, and supports the steering operation and driving at a constant inter-vehicle distance. Further, the lane departure prevention assist function warns of the possibility of deviating from the lane or road, and supports a part of the steering operation to avoid deviating from the lane or road. Further, the driving support system 1000 issues a warning by the HMI device 1630 to inform the driver of dangers encountered during vehicle driving.

[0022] In these driving support functions, it is preferable to set an optimal driving state that does not interfere with sleep according to the sleep state of the passengers sitting in the passenger seat or the rear seat of the vehicle. Further, it is preferable to optimize the settings of the HMI device 160 so as not to interfere with sleep according to the sleep state of the passengers in the vehicle.

[0023] For this reason, the driving support system 1000 determines the sleep state of the passengers from the image generated by the in-vehicle camera 120, and automatically changes these settings based on the sleep state. As a result, an optimal setting considering the sleep of the passengers becomes possible.

[0024] On the other hand, if the passengers repeatedly switch between the sleep state and the awake state during driving, if the set value is changed each time, there is a possibility that the driver cannot determine the current setting of the driving support function, and there is a possibility that the driver may misunderstand the mode of the driving support function. And if the driver misunderstands that the setting of the driving support function is set to the normal mode even though the setting of the driving support function is set to the mode when the passengers are sleeping, the driver may overtrust the driving support function.

[0025] Therefore, in this embodiment, during a single trip from when the vehicle's power is turned on (ignition switch turned on) to when it is turned off, the setting changes according to the sleep state are limited to only once. Then, when the driver assistance system 1000 detects that the power has been turned off, these settings are returned to their initial state. This prevents the settings from switching frequently during a single trip, and prevents the driver from overestimating or misunderstanding the system's behavior.

[0026] Figure 3 is a schematic diagram showing the functional blocks of the processor 152 of the ECU 150. The processor 152 of the ECU 150 includes a sleep state determination unit 152a, a setting change unit 152b, a driving support unit 152c, and an HMI output unit 152d. Each of these parts of the processor 152 is a functional module realized, for example, by a computer program running on the processor 152. In other words, the functional blocks of the processor 152 consist of the processor 152 and a program (software) to make it function. The program may also be recorded in the memory 154 of the ECU 150 or on an externally connected recording medium. Alternatively, each of these parts of the processor 152 may be a dedicated arithmetic circuit provided on the processor 152.

[0027] The sleep state determination unit 152a determines the sleep state of passengers other than the driver in the vehicle. Specifically, the sleep state determination unit 152a determines the sleep state of passengers based on the degree of eye opening. When the sleep state determination unit 152a determines the degree of eye opening, it first detects the passenger's eyes from the face image generated by the in-vehicle camera 120 and detects the outline of the eyes. At this time, the outline of the eyes is detected from the image, for example, by template matching between a template image and an image generated by the in-vehicle camera 120, or by inputting the image generated by the in-vehicle camera 120 into a classifier trained on machine learning for object detection. After determining the three-dimensional position of the outline of the eyes, the sleep state determination unit 152a determines the degree of eye opening from the distance between the upper eyelid and the lower eyelid. Then, the sleep state determination unit 152a determines that the passenger is asleep if the proportion of time the passenger's eyes are closed within a predetermined time exceeds a threshold.

[0028] As described above, the sleep state determination unit 152a may determine the position and contours of various facial components, such as the contours of the passenger's eyes, by inputting the image into a pre-trained classifier. For example, the sleep state determination unit 152a may use a segmentation classifier as such a classifier, which is pre-trained to output the likelihood that an object is represented in each pixel of the input image, for each type of object that could potentially be represented in that pixel, and to identify the object with the highest likelihood as being represented. The sleep state determination unit 152a may use a deep neural network (DNN) with a segmentation convolutional neural network (CNN) architecture, such as a Fully Convolutional Network (FCN), as such a classifier. Alternatively, the sleep state determination unit 152a may use a segmentation classifier that follows other machine learning methods, such as a random forest or a support vector machine. In this case, the sleep state determination unit 152a inputs the image into a segmentation classifier to identify pixels in the image that contain various components. The sleep state determination unit 152a then defines the set of images containing the same type of element as the region representing that element.

[0029] The setting change unit 152b changes the settings of the driver assistance function, which allows the vehicle to autonomously perform at least a portion of the driving operations, or the settings of the warnings output from the HMI device 160, based on the sleep state of the passenger. The setting change unit 152b changes the settings of the driver assistance function between a first mode, in which the vehicle autonomously performs driving operations under normal circumstances, and a second mode, in which the driving operations are less rigorous than the first mode. Then, if the setting change unit 152b determines that the passenger is asleep, it changes the settings of the driver assistance function from the first mode to the second mode. This suppresses sudden acceleration, sudden deceleration, or sudden steering of the vehicle, and also prevents the passenger's sleep from being disturbed.

[0030] Specifically, if the setting change unit 152b determines that a passenger is asleep, it further reduces the vehicle's acceleration and deceleration used to maintain the distance to the preceding vehicle by the preceding vehicle following function. This suppresses sudden acceleration or deceleration of the vehicle. Also, if the setting change unit 152b determines that a passenger is asleep, it further increases the set value for the distance to the preceding vehicle by the preceding vehicle following function. This suppresses sudden acceleration or deceleration of the vehicle in response to the movement of the preceding vehicle.

[0031] Furthermore, if the setting change unit 152b determines that a passenger is asleep, it changes the timing of steering intervention by the lane departure prevention steering support function to an earlier timing. Also, if the setting change unit 152b determines that a passenger is asleep, it further reduces the amount of steering input by the lane departure prevention steering support function when a vehicle deviates from its lane. This suppresses sudden steering inputs when a vehicle deviates from its lane.

[0032] Furthermore, the setting change unit 152b modifies the warning settings output from the HMI device 160 based on the passenger's sleep state. For example, if the setting change unit 152b determines that the passenger is asleep, it changes the warning output from the HMI device 160 from sound to vibration. Also, if the setting change unit 152b determines that the passenger is asleep, it reduces the volume of the warning output from the HMI device 160. This suppresses the warning sound and prevents the passenger's sleep from being disturbed.

[0033] The setting change unit 152b may change the settings based on data related to the settings of the driving assistance function or the settings of the warnings output from the HMI device 160, which are stored in the memory 154 in advance.

[0034] When the settings for the driver assistance function or warning are changed by the setting change unit 152b, the HMI device 160 notifies the driver accordingly. For example, the HMI device 160 notifies the driver of an announcement such as, "The driver assistance function mode has been changed from normal mode to passenger sleep mode." Here, "normal mode" corresponds to the first mode described above, and "passenger sleep mode" corresponds to the second mode described above.

[0035] The setting change unit 152b maintains the changed setting after it has been changed until the vehicle's power is turned off. When the vehicle's power is turned off, the setting change unit 152b reverts the setting to its original state based on the initial value of the data related to the settings of the driver assistance function or the settings of the warnings output from the HMI device 160, which is stored in the memory 154 in advance, when it is determined that the passenger is not asleep.

[0036] Furthermore, the setting change unit 152b may change the settings of the driving assistance function or the warnings to avoid disturbing the passenger's sleep as the passenger's sleep is deeper, that is, as the passenger's level of alertness decreases. For example, the setting change unit 152b may further reduce the vehicle's acceleration and deceleration as the passenger's sleep is deeper. Also, the setting change unit 152b may further reduce the volume of the warnings as the passenger's sleep is deeper. In this case, the sleep state determination unit 152a may determine that the passenger's sleep is deeper if the proportion of time the passenger's eyes are closed within a predetermined period of time is high.

[0037] The driver assistance unit 152c provides driver assistance through functions such as following a preceding vehicle and steering assistance to prevent lane departure. Specifically, the driver assistance unit 152c controls vehicle control equipment 140, such as the internal combustion engine, transmission or electric motor, and braking system, so that the distance to the preceding vehicle is set to a predetermined value, based on images generated by the front camera 110 or information detected by surrounding monitoring sensors. In addition, the driver assistance unit 152c controls vehicle control equipment 140, such as the steering system, to avoid lane departure or road departure, based on images generated by the front camera 110.

[0038] If the settings of the driver assistance function are changed by the setting change unit 152b, the driver assistance unit 152c will provide driver assistance using the driver assistance function based on the changed settings.

[0039] The HMI output unit 152d causes the HMI device 160 to output a warning. For example, the HMI output unit 152d causes the HMI device 160 to output a warning if the vehicle deviates from a lane or road based on an image generated by the front camera 110 or information detected by the surrounding monitoring sensors. If the setting of the warning output from the HMI device 160 is changed by the setting change unit 152b, the HMI output unit 152d causes the HMI device 160 to output a warning based on the changed setting.

[0040] Figure 4 is a flowchart showing the processing performed by the processor 152 of the ECU 150 at predetermined control cycles. First, the sleep state determination unit 152a acquires a facial image of the passenger (step S10) and determines the passenger's sleep state from the facial image (step S12). If the sleep state determination unit 152a determines that the passenger is asleep (YES in step S14), the setting change unit 152b changes the settings of the driving assistance function or the warning settings output from the HMI device 160 according to the passenger's sleep state (step S16). The driver is then notified that these settings have been changed (step S18). On the other hand, if it is not determined in step S14 that the passenger is asleep, the process returns to step S10.

[0041] Next, it is determined whether or not the vehicle is still being driven (step S20). If it is determined that the vehicle is not still being driven (NO in step S20), that is, if it is determined that the vehicle's power is off, the setting change unit 152b changes the vehicle settings back to the original settings (step S22). On the other hand, if it is determined that the vehicle is still being driven (YES in step S20), the determination in step S20 is performed again.

[0042] As explained above, according to this embodiment, if it is determined that a passenger is asleep, the setting of the driver assistance function is changed from a first mode, which is the driving operation that the vehicle normally performs autonomously, to a second control mode, which is a gentler driving operation than the first mode. Therefore, it is possible to avoid disturbing the passenger's sleep. In addition, since the changed mode is maintained until the vehicle's power is turned off, the setting is not changed repeatedly depending on the passenger's sleep state, and the driver is less likely to misinterpret the setting of the driver assistance function. [Explanation of symbols]

[0043] 150...Electronic Control Unit (ECU), 152...Processor, 152a...Sleep State Determination Unit, 152b...Setting Change Unit, 152c...Driving Support Unit, 152d...HMI Output Unit, 1000...Driving Support System

Claims

1. A sleep state determination unit that determines the sleep state of passengers other than the driver in the vehicle, A setting change unit that changes the settings of a driver assistance function that allows the vehicle to autonomously perform at least a portion of the driving operations or the settings of a warning output from a warning device, based on the sleep state of the passenger, Equipped with, The setting change unit is a vehicle control device that maintains the changed setting after the setting has been changed until the vehicle's power is turned off.

2. The vehicle control device according to claim 1, wherein the setting change unit changes the setting of the driving assistance function between a first mode of driving operations performed autonomously by the vehicle under normal circumstances and a second mode of driving operations that are less rigorous than the first mode, and when it is determined that the passenger is asleep, the setting of the driving assistance function is changed from the first mode to the second mode.

3. The vehicle control device according to claim 1 or 2, wherein the setting change unit, when it is determined that the passenger is asleep, further reduces the vehicle acceleration / deceleration used by the driving assistance function to maintain the distance to the preceding vehicle or the amount of steering used by the driving assistance function when the vehicle deviates from its lane.

4. The vehicle control device according to claim 1 or 2, wherein the setting change unit, when it is determined that the passenger is asleep, changes the timing of steering intervention by the driving assistance function when the vehicle deviates from its lane to an earlier timing.

5. The vehicle control device according to claim 1 or 2, wherein the setting change unit reduces the volume of the warning output from the warning device when it is determined that the passenger is asleep.

Images