Vehicle control device

Abstract

The present invention addresses the problem of suppressing a plurality of changes in settings in accordance with a sleep state when the settings of a vehicle are changed in accordance with the sleep state of a passenger in the vehicle. A processor of an ECU is provided with: a sleep state determination unit that determines the sleep state of a passenger other than a driver in a vehicle; and a setting changing unit that changes, on the basis of the sleep state of the passenger, the setting of the driving assistance function for at least a portion of the driving operation autonomously performed by the vehicle or the setting of the warning output from the HMI device, and after the setting is changed, the setting changing unit maintains the setting changed until the power of the vehicle is turned off.

Description

Technical Field

[0001] This invention relates to a vehicle control device. Background Technology

[0002] Previously, it was known to determine acceleration and deceleration values ​​based on the psychological state of the occupants, including the driver and passengers during autonomous driving, and to make corrections to the values ​​based on whether each occupant was asleep when quantifying the psychological state (for example, see Patent Document 1).

[0003] Patent Document 1: Japanese Patent Application Publication No. 2018-052160 Summary of the Invention

[0004] However, an occupant's sleep state can change constantly, sometimes becoming deeper and sometimes lighter. In the technology described in the aforementioned patent documents, because the deceleration value changes repeatedly based on the occupant's sleep state, the driver may misinterpret the acceleration / deceleration settings of the driving assistance function. Furthermore, if the driver misinterprets the settings of the driving assistance function, there is a problem that the driver may become overly reliant on the function, affecting their driving operation.

[0005] Therefore, the object of the present invention is to provide a vehicle control device that can suppress multiple changes in settings caused by sleep status when the vehicle settings are changed according to the sleep status of the passengers in the vehicle.

[0006] The main points of this invention are as follows:

[0007] (1) A vehicle control device comprising:

[0008] The sleep state determination unit determines the sleep state of passengers other than the driver while riding in the vehicle; and

[0009] The setting modification unit, based on the sleep state of the passenger, modifies the settings of at least some of the driving assistance functions that are autonomously operated by the vehicle, or the settings of warnings output from the warning device.

[0010] The setting change unit maintains the changed setting until the vehicle's power is turned off after the setting is changed.

[0011] (2) The vehicle control device according to (1) above, wherein,

[0012] The setting change unit normally changes the setting of the driving assistance function between a first mode of driving operation performed autonomously by the vehicle and a second mode of driving operation that is slower than the first mode. If it is determined that the passenger is sleeping, the setting of the driving assistance function is changed from the first mode to the second mode.

[0013] (3) The vehicle control device according to (1) or (2) above, wherein,

[0014] If it is determined that the passenger is asleep, the setting change unit further reduces the vehicle acceleration / deceleration based on the distance to the vehicle in front based on the driving assistance function, or the steering amount when deviating from the lane based on the driving assistance function.

[0015] (4) The vehicle control device according to (1) or (2) above, wherein,

[0016] If it is determined that the passenger is asleep, the setting change unit will change the timing of the steering intervention when the driver assistance function deviates from the lane to an earlier time.

[0017] (5) The vehicle control device according to (1) or (2) above, wherein,

[0018] If it is determined that the passenger is asleep, the setting change unit reduces the volume of the warning output from the warning device.

[0019] Invention Effects

[0020] According to the present invention, a vehicle control device is provided that, when changing the vehicle settings based on the sleep state of the passengers in the vehicle, can suppress multiple changes to the settings caused by the sleep state. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a vehicle's driver assistance system based on one implementation method.

[0022] Figure 2 This is a schematic diagram representing the functional modules of the ECU's processor.

[0023] Figure 3 It is a flowchart representing the processing performed by the processor of the ECU. Detailed Implementation

[0024] Hereinafter, several embodiments of the present invention will be described with reference to the accompanying drawings. However, these descriptions are merely illustrative of preferred embodiments of the invention and are not intended to limit the invention to such specific embodiments.

[0025] Figure 1 This is a schematic structural diagram of a vehicle driver assistance system 1000 based on one embodiment. The driver assistance system 1000, for example, is installed in a vehicle such as an automobile, and includes a front-facing camera 110, an in-vehicle camera 120, one or more sensors 130, a vehicle control device 140, an electronic control unit (ECU, hereinafter referred to as ECU) 150, and an HMI device 160. The front-facing camera 110, the in-vehicle camera 120, the one or more sensors 130, the vehicle control device 140, the ECU 150, and the HMI device 160 are communicatively connected via an in-vehicle network conforming to a standard such as a Controller Area Network (CAN).

[0026] The front-facing camera 110 and the interior camera 120 have an imaging optical system comprising a two-dimensional detector composed of an array of photoelectric conversion elements such as CCD or C-MOS that are sensitive to visible light, and an image of the area to be photographed on the two-dimensional detector. The front-facing camera 110 is positioned near the dashboard or windshield inside the vehicle and captures images of the surroundings of the vehicle (e.g., the area in front of the vehicle), generating an image representing the environment around the vehicle. The interior camera 120 is positioned near the dashboard, steering column, or windshield inside the vehicle and, facing a hypothetical position of a passenger, captures the passenger's face, generating an image reflecting the passenger's face (a facial image). The front-facing camera 110 and the interior camera 120 capture images at predetermined shooting intervals (e.g., 1 / 30 to 1 / 10 of a second). The front-facing camera 110 may be a stereo camera or configured to obtain the distances of various structures in the image based on the parallax of the left and right images. Each time the front-facing camera 110 and the interior camera 120 generate an image, they output the generated image to the ECU 150 via the in-vehicle network.

[0027] One or more sensors 130 include sensors for monitoring the surroundings of the vehicle, such as LiDAR (Light Detection and Ranging) sensors, radar sensors, etc.

[0028] Vehicle control equipment 140 includes various devices related to vehicle control, including drive units such as internal combustion engines or electric motors that serve as the drive source for driving the vehicle, transmissions, braking devices for braking the vehicle, and steering devices for rotating the vehicle. When the drive source for driving the vehicle is an electric motor, vehicle control equipment 140 may include batteries that store electricity, fuel cells that supply electricity to the electric motor, etc. Furthermore, when the vehicle is an electric vehicle (EV), vehicle control equipment 140 may not include an internal combustion engine. And, when the vehicle is a motor-driven vehicle, vehicle control equipment 140 may not include an electric motor.

[0029] ECU 150 is one embodiment of the vehicle determination device according to the present invention. ECU 150 includes a processor 152, a memory 154, and a communication interface 156. The processor 152 has one or more central processing units (CPUs) and their peripheral circuitry. The processor 152 may also include other arithmetic circuits such as logic units, numerical processing units, or graphics processing units. The memory 154 includes, for example, volatile and non-volatile semiconductor memories, storing data related to the processing involved in this embodiment. In particular, the memory 154 stores data related to the setting of driver assistance functions or the setting of warnings output from the HMI device 160. Specifically, the memory 154 stores both the setting values ​​for this data when it is determined that the passenger is asleep and the setting values ​​(initial values) when it is not determined that the passenger is asleep. The communication interface 156 has interface circuitry for connecting ECU 150 to an in-vehicle network.

[0030] HMI device 160 is one type of warning device, including a display device, a speaker, and a vibration device. The display device, for example, is a liquid crystal display (LCD) and is located near the instrument panel or dashboard. The display device displays and outputs warnings according to instructions from ECU 150. The speaker outputs warnings audibly according to instructions from ECU 150. The vibration device is a device that vibrates the steering wheel, seat belt, etc., and outputs warnings via vibration according to instructions from ECU 150.

[0031] The driver assistance system 1000 provides driver assistance based on at least a portion of the driver assistance functions performed autonomously by the vehicle. Specifically, the driver assistance system 1000 provides driver assistance based on the images from the front-facing camera 110 and the information detected by the sensor 130, using functions such as Adaptive Cruise Control (ACC) and Lane Departure Mitigation Steering Assist (LDA). The Adaptive Cruise Control function, for example, detects white lines or vehicles ahead while driving on a highway, supports steering wheel operation, or supports maintaining a constant following distance. The Lane Departure Mitigation Steering Assist function warns of the possibility of deviating from the lane or driving path and supports a portion of the steering wheel operation to avoid deviating from the lane or driving path. Furthermore, the driver assistance system 1000 provides warnings based on the HMI device 1630 to inform the driver of hazards encountered while driving the vehicle.

[0032] Among these driver assistance functions, it is preferable to set the driving state to the optimal state that does not interfere with the sleep of passengers in the front passenger seat or rear seat of the vehicle. Furthermore, it is preferable to optimally set the HMI device 160 according to the sleep state of the vehicle's passengers so as not to interfere with their sleep.

[0033] Therefore, the driver assistance system 1000 determines the sleep state of the passenger based on images generated by the in-vehicle camera 120, and automatically changes these settings based on the sleep state. This allows for optimal settings that take into account the passenger's sleep.

[0034] On the other hand, if a passenger repeatedly switches between sleep and wakefulness while driving, and the settings are changed each time, the driver may not be able to determine the current setting of the driver assistance function, and may misunderstand the mode of the driver assistance function. Moreover, if the driver assistance function is set to the normal mode even when the passenger is asleep, it may be mistaken for the driver assistance function being set to the normal mode, which may sometimes lead the driver to over-rely on the driver assistance function.

[0035] Therefore, in this embodiment, the setting change corresponding to the sleep state is limited to only once during one trip, from when the vehicle's power is turned on (ignition switch is turned on) to when the power is turned off. Furthermore, when the power is detected to be off, the driver assistance system 1000 restores these settings to their initial state. Thus, settings are not switched multiple times during one trip, thereby preventing the driver from overly relying on the system's behavior or causing misunderstandings.

[0036] Figure 3This is a schematic diagram showing the functional modules of the processor 152 of the ECU 150. The processor 152 of the ECU 150 includes a sleep state determination unit 152a, a setting change unit 152b, a driving assistance unit 152c, and an HMI output unit 152d. These units of the processor 152 are, for example, functional modules implemented by a computer program running on the processor 152. That is, the functional blocks of the processor 152 consist of the processor 152 and the program (software) used to enable it to perform its functions. Furthermore, this program can be recorded in the memory 154 of the ECU 150 or in a recording medium connected to an external source. Alternatively, these units of the processor 152 can be dedicated arithmetic circuits provided in the processor 152.

[0037] The sleep state determination unit 152a determines the sleep state of passengers other than the driver while riding in the vehicle. Specifically, the sleep state determination unit 152a determines the sleep state of the passengers based on the degree of eye opening of the passengers. When determining the degree of eye opening, the sleep state determination unit 152a first detects the eyes of the passengers in the facial image generated by the in-vehicle camera 120 and detects the contours of the eyes. At this time, for example, the contours of the eyes are detected from the image by matching the template image with the image generated by the in-vehicle camera 120, or by inputting the image generated by the in-vehicle camera 120 into a machine learning recognizer for object detection. If the sleep state determination unit 152a calculates the three-dimensional position of the eye contours, the degree of eye opening is determined based on the distance between the upper and lower eyelids. Furthermore, if the proportion of time the passengers' eyes are closed within a specified time exceeds a threshold, the sleep state determination unit 152a determines that the passengers are sleeping.

[0038] As described above, the sleep state determination unit 152a can determine the position and contour of various facial features, such as the outline of the eyes, of a passenger by inputting an image into a pre-learned recognizer. For example, the sleep state determination unit 152a can use a segmentation recognizer that, from the input image, outputs the probability that an object might be displayed in each pixel according to the type of object that might be displayed in that pixel, and pre-learns to identify the object with the highest display probability. As such a recognizer, the sleep state determination unit 152a can, for example, use a deep neural network (DNN) with a segmentation convolutional neural network (CNN) architecture, such as a fully convolutional neural network (FCN). Alternatively, the sleep state determination unit 152a can also utilize a segmentation recognizer based on other machine learning methods, such as a random forest or support vector machine. In this case, the sleep state determination unit 152a determines the pixels that represent various facial features in the image by inputting the image into the segmentation recognizer. Furthermore, the sleep state determination unit 152a sets the collection of images that reflect the same type of elements as the area for displaying those elements.

[0039] Based on the passenger's sleep state, the setting change unit 152b changes the settings of at least some of the driving assistance functions that are driven autonomously by the vehicle, or the settings of warnings output from the HMI device 160. The setting change unit 152b changes the settings of the driving assistance functions between a first mode of driving operation that is normally performed autonomously by the vehicle and a second mode of driving operation that is slower than the first mode. Furthermore, if the setting change unit 152b determines that the passenger is sleeping, it changes the settings of the driving assistance functions from the first mode to the second mode. This suppresses sudden acceleration, sudden deceleration, or sudden steering of the vehicle, and also prevents disturbance to the passenger's sleep.

[0040] Specifically, when the setting modification unit 152b determines that the passenger is asleep, it further reduces the vehicle's acceleration and deceleration used to maintain the distance to the vehicle in front based on the follow-the-car function. This suppresses sudden acceleration or deceleration of the vehicle. Furthermore, when the setting modification unit 152b determines that the passenger is asleep, it increases the set value of the distance to the vehicle in front based on the follow-the-car function. This suppresses sudden acceleration or deceleration of the vehicle corresponding to the movement of the vehicle in front.

[0041] Furthermore, when the setting modification unit 152b determines that the passenger is asleep, it changes the timing of the steering intervention during lane departure based on the lane departure suppression steering assist function to an earlier time. Also, when the setting modification unit 152b determines that the passenger is asleep, it further reduces the steering amount during lane departure based on the lane departure suppression steering assist function. This suppresses sharp steering during lane departure.

[0042] Furthermore, the setting change unit 152b changes the setting of the warning output from the HMI device 160 based on the sleep state of the passenger. For example, if the setting change unit 152b determines that the passenger is sleeping, it changes the warning output from the HMI device 160 from sound to vibration. Also, if the setting change unit 152b determines that the passenger is sleeping, it lowers the volume of the warning output from the HMI device 160. This suppresses the warning sound, thereby preventing disturbance to the passenger's sleep.

[0043] The setting change unit 152b can change the settings based on data related to the settings of driving assistance functions or the settings of warnings output from the HMI device 160 that are stored in the memory 154 in advance.

[0044] If the settings of the driving assistance function or warning are changed via the setting change unit 152b, the driver is notified of this change from the HMI device 160. For example, the HMI device 160 may broadcast a message to the driver stating, "The driving assistance function mode has been changed from normal mode to passenger sleep mode." Here, normal mode corresponds to the first mode described above, and passenger sleep mode corresponds to the second mode described above.

[0045] After changing the settings, the setting change unit 152b maintains the changed settings until the vehicle's power is turned off. If the vehicle's power is turned off, the setting change unit 152b restores the settings to their original state based on the initial values ​​in the data stored in the memory 154 related to settings of driver assistance functions or warnings output from the HMI device 160, where it is not determined that the passenger is asleep.

[0046] Furthermore, the deeper the passenger's sleep, i.e., the lower their level of alertness, the more easily the setting modification unit 152b can change the settings of the driving assistance functions or warnings without disturbing the passenger's sleep. For example, the deeper the passenger's sleep, the more the setting modification unit 152b can further reduce the vehicle's acceleration and deceleration. Also, the deeper the passenger's sleep, the more the warning volume can be further reduced. In this case, the sleep state determination unit 152a can determine that the higher the proportion of time the passenger's eyes are closed within a specified time, the deeper the passenger's sleep.

[0047] The driver assistance unit 152c provides driver assistance based on a follow-the-car function and a lane departure prevention steering assist function. Specifically, the driver assistance unit 152c controls vehicle control equipment 140, such as the internal combustion engine, transmission or electric motor, and braking device, based on images generated by the front-facing camera 110 or information detected by surrounding monitoring sensors, to maintain a predetermined distance from the vehicle in front. Furthermore, the driver assistance unit 152c controls vehicle control equipment 140, such as the steering device, based on images generated by the front-facing camera 110, to prevent deviation from the lane or driving path.

[0048] When the settings of the driving assistance function are changed by the setting change unit 152b, the driving assistance unit 152c performs driving assistance based on the changed settings.

[0049] HMI output unit 152d outputs a warning to HMI device 160. For example, HMI output unit 152d outputs a warning to HMI device 160 when the vehicle deviates from its lane or driving route based on an image generated by front camera 110 or information detected by surrounding monitoring sensors. If the setting of the warning output from HMI device 160 is changed by setting change unit 152b, HMI output unit 152d outputs a warning to HMI device 160 based on the changed setting.

[0050] Figure 3 This is a flowchart illustrating the processing performed by the processor 152 of the ECU 150 according to a predetermined control cycle. First, the sleep state determination unit 152a acquires a facial image of the passenger (step S10) and determines the passenger's sleep state based on the facial image (step S12). If the sleep state determination unit 152a determines that the passenger is asleep ("Yes" in step S14), the setting change unit 152b changes the settings of the driver assistance functions or the settings of warnings output from the HMI device 160 according to the passenger's sleep state (step S16). Furthermore, the driver is notified of these setting changes (step S18). On the other hand, if it is not determined in step S14 that the passenger is asleep, the process returns to step S10.

[0051] Next, it is determined whether to continue driving the vehicle (step S20). If it is determined that driving will not continue ("No" in step S20), that is, if it is determined that the vehicle's power is off, the setting change unit 152b changes the vehicle settings back to the original settings (step S22). On the other hand, if it is determined that driving will continue ("Yes" in step S20), the determination in step S20 is performed again.

[0052] As explained above, according to this embodiment, when it is determined that a passenger is asleep, the setting of the driving assistance function is changed from the first mode, which is the driving operation normally performed autonomously by the vehicle, to a second control mode, which is a slower driving operation than the first mode. Therefore, it does not disturb the passenger's sleep. Furthermore, since the changed mode is maintained until the vehicle's power is turned off, the setting is not changed multiple times based on the passenger's sleep state, thereby preventing the driver from mistakenly identifying the setting state of the driving assistance function.

[0053] Symbol Explanation

[0054] 150 - Electronic Control Unit (ECU), 152 - Processor, 152a - Sleep State Determination Unit, 152b - Setting Change Unit, 152c - Driving Assistance Unit, 152d - HMI Output Unit, 1000 - Driving Assistance System.

Claims

1. A vehicle control device, characterized in that, have: The sleep state determination unit determines the sleep state of passengers other than the driver while riding in the vehicle; and The setting modification unit, based on the sleep state of the passenger, modifies the settings of at least some of the driving assistance functions that are autonomously operated by the vehicle, or the settings of warnings output from the warning device. The setting change unit maintains the changed setting until the vehicle's power is turned off after the setting is changed.

2. The vehicle control device according to claim 1, characterized in that, The setting change unit normally changes the setting of the driving assistance function between a first mode of driving operation performed autonomously by the vehicle and a second mode of driving operation that is slower than the first mode. If it is determined that the passenger is sleeping, the setting of the driving assistance function is changed from the first mode to the second mode.

3. The vehicle control device according to claim 1 or 2, characterized in that, If it is determined that the passenger is asleep, the setting change unit further reduces the vehicle acceleration / deceleration based on the distance to the vehicle in front based on the driving assistance function, or the steering amount when deviating from the lane based on the driving assistance function.

4. The vehicle control device according to claim 1 or 2, characterized in that, If it is determined that the passenger is asleep, the setting change unit will change the timing of the steering intervention when the driver assistance function deviates from the lane to an earlier time.

5. The vehicle control device according to claim 1 or 2, characterized in that, If it is determined that the passenger is asleep, the setting change unit reduces the volume of the warning output from the warning device.

Images