Safe driving support device

Abstract

To enable appropriate awakening means to be selected and the degree of awakening treatment to be adjusted in accordance with the physical and mental fatigue of a driver and so as to reduce the risk of accident occurrence.SOLUTION: A safety driving assistance device comprises: an imaging unit 10 for imaging an occupant in a vehicle; an information acquisition unit(s) 20 for acquiring biological information of the occupant inside and outside of the vehicle; a first assessment unit 30 for assessing physical fatigue of occupant in the vehicle on the basis of images of the occupant in the vehicle imaged by the imaging unit or information that includes the biological information of the occupant inside and outside of the vehicle acquired by the information acquisition unit; a second assessment unit 40 for assessing a mental fatigue level of the occupant in the vehicle on the basis of the images of occupant in the vehicle imaged by the imaging unit or the information that includes the biological information of the occupant inside and outside of the vehicle acquired by the information acquisition unit; and a control unit 50 for controlling external vehicle devices 60, 70, 80, 90 on the basis of at least one of assessment information in the first assessment unit 30 and assessment information in the second assessment unit 40.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a safe driving support device. [Background technology]

[0002] Generally, it is difficult to determine the physical and mental fatigue of the occupants, including the driver, in the vehicle and to determine the degree of fatigue at the start of driving the vehicle. On the other hand, a technology has been disclosed in which the level of drowsiness of a driver is detected, and whether or not wakefulness assistance for the driver is necessary is determined based on the detected level. If necessary, the driver's seat is vibrated for at least a portion of the time period with a vibration waveform that superimposes a first frequency that promotes muscle tension and a second frequency that suppresses muscle tension (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-82906 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 uses only drowsiness as a parameter for determining the driver's state, and determines the need for alertness assistance solely based on whether the drowsiness level is greater than a predetermined threshold level. As a result, there is a problem in that it is not possible to select an appropriate alerting method or adjust the level of alertness treatment according to the occupant's physical and mental fatigue level based on multiple determination factors.

[0005] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide a safe driving support device that can select appropriate awakening means and adjust the degree of awakening treatment according to the occupant's physical and mental fatigue level based on multiple judgment factors, thereby reducing the risk of accidents caused by fatigue. [Means for solving the problem]

[0006] Form 1: One or more embodiments of the present invention include an imaging unit that images an occupant inside a vehicle, an information acquisition unit that acquires information including biological information of the occupant inside and outside the vehicle, a first determination unit that determines a physical fatigue level of the occupant inside the vehicle based on the image of the occupant inside the vehicle captured by the imaging unit or the information including biological information of the occupant inside and outside the vehicle acquired by the information acquisition unit, and a second determination unit that determines a mental fatigue level of the occupant inside the vehicle based on the image of the occupant inside the vehicle captured by the imaging unit or the information including biological information of the occupant inside and outside the vehicle acquired by the information acquisition unit. a third determination unit that determines the resting state of the occupant in the vehicle from the video of the occupant in the vehicle taking a rest captured by the imaging unit; and at least one of the determination information in the first determination unit and the determination information in the second determination unit, and the determination information in the third determination unit. and a control unit that controls a vehicle device based on the vehicle speed.

[0007] Form 2: One or more embodiments of the present invention include: The aforementioned The control unit The aforementioned May stimulate the visual, auditory, tactile and olfactory senses of vehicle occupants The aforementioned We propose a safe driving support device that controls vehicle devices.

[0009] Form 3: One or more embodiments of the present invention comprise: The aforementioned The control unit Based on the determination information in the second determination unit and the determination information in the third determination unit, When it is determined that the mental fatigue level of the occupants in the vehicle is equal to or higher than a predetermined level and that the occupants have not been able to take a predetermined amount of rest, The aforementioned We propose a safe driving support device that controls vehicle devices.

[0010] Form 4: One or more embodiments of the present invention may comprise: The aforementioned The vehicle device includes a driving assistance device that assists a driver in driving the vehicle, The aforementioned The control unit The aforementioned For driving assistance devices At least one of the determination information in the first determination unit and the determination information in the second determination unit, and the determination information in the third determination unit and outputs a control signal based on the above.

[0011] Form 5: One or more embodiments of the present invention may comprise: The aforementioned The control unit The aforementioned The judgment information in the first judgment unit The aforementioned In the third judgment section The aforementioned Based on the judgment information, The aforementioned When it is determined that the physical fatigue level of the occupants in the vehicle is equal to or greater than a predetermined level and that they have not been able to get the required amount of rest, The aforementioned On vehicle devices The aforementioned The present invention proposes a safe driving support device that outputs a control signal based on the determination information. [Effects of the Invention]

[0015] According to one or more embodiments of the present invention, it is possible to select appropriate awakening means and adjust the degree of awakening treatment depending on the physical and mental fatigue level of the occupant based on multiple judgment factors, thereby reducing the risk of accidents caused by fatigue. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a diagram showing an arrangement state of a safe driving support device according to a first embodiment of the present invention. [Figure 2] 1 is a diagram showing a configuration of a safe driving support device according to a first embodiment of the present invention. [Figure 3] 2 is a process flow diagram of the safe driving support device according to the first embodiment of the present invention. FIG. [Figure 4]3 is a diagram showing the relationship between output items of an information acquisition unit and physical fatigue or mental fatigue in the safe driving support device according to the first embodiment of the present invention. FIG. [Figure 5] 3 is a diagram illustrating an example of determination information of a first determination unit or a second determination unit in the safe driving support device according to the first embodiment of the present invention. FIG. [Figure 6] 2 is a diagram showing an example of operation of a vehicle device in the safe driving support device according to the first embodiment of the present invention. FIG. [Figure 7] FIG. 4 is a diagram showing the configuration of a safe driving support device according to a second embodiment of the present invention. [Figure 8] FIG. 6 is a diagram showing the configuration of a control unit of a safe driving support device according to a second embodiment of the present invention. [Figure 9] FIG. 6 is a process flow diagram of a safe driving support device according to a second embodiment of the present invention. [Figure 10] FIG. 10 is a process flow diagram of a machine learning unit of a safe driving support device according to a second embodiment of the present invention. [Figure 11] FIG. 10 is a diagram showing the configuration of a safe driving support device according to a third embodiment of the present invention. [Figure 12] FIG. 10 is a process flow diagram of a safe driving support device according to a third embodiment of the present invention. [Figure 13] 10 is a diagram illustrating an example of determination information of a second determination unit or a third determination unit in a safe driving support device according to a third embodiment of the present invention. FIG. [Figure 14] FIG. 10 is a diagram showing the configuration of a safe driving support device according to a fourth embodiment of the present invention. [Figure 15] FIG. 10 is a process flow diagram of a safe driving support device according to a fourth embodiment of the present invention. [Figure 16] 10 is a diagram illustrating an example of determination information of a first determination unit or a third determination unit in a safe driving support device according to a fourth embodiment of the present invention. FIG. [Figure 17] 10A and 10B are diagrams illustrating an example of operation of the driving support device in the safe driving support device according to the fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, an embodiment of the present invention will be described with reference to FIGS.

[0018] First Embodiment A safe driving support device 1 according to this embodiment will be described with reference to FIGS. 1 to 6. FIG.

[0019] As shown in FIG. 1, the safety driving support device 1 of this embodiment is provided, for example, in the center front of the driver's seat in a vehicle, and is a device in which a control unit controls vehicle devices based on at least one of the physical fatigue level or mental fatigue level of the occupant in the vehicle determined by a first determination unit or a second determination unit based on an image of the occupant captured by an imaging unit 10 described later or information including biometric information of the occupant inside and outside the vehicle acquired by an information acquisition unit. The detailed configuration will be described separately.

[0020] <Configuration of safe driving support device> As shown in FIG. 2, the safe driving support device 1 of this embodiment is configured to include an imaging unit 10, an information acquisition unit 20, a first judgment unit 30, a second judgment unit 40, a control unit 50, a visual stimulation device 60, an auditory stimulation device 70, a tactile stimulation device 80, and an olfactory stimulation device 90. In this embodiment, the vehicle devices 60, 70, 80, and 90 are described as components of a safe driving support device, but the vehicle devices 60, 70, 80, and 90 may be individual devices that can be operated according to the intention of the occupant.

[0021] The imaging unit 10 includes an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor), and outputs images (including moving images and still images) of passengers in the vehicle captured by the imaging element. The imaging unit 10 may be a dedicated camera, but if an occupant recognition device is provided, it is preferable to use the imaging unit of the occupant recognition device. Also, it is preferable that the imaging unit is composed of both an optical camera and a near-infrared camera.

[0022] The information acquisition unit 20 includes units provided inside and outside the vehicle, and acquires, for example, biological information of the occupants. Here, the biological information includes, for example, vital data of each occupant. Examples of the information acquisition unit 20 provided in the vehicle include a unit equipped in the vehicle and a unit worn by the occupant, such as a smart watch or a smartphone. The information acquisition unit 20 provided outside the vehicle includes both fixed and mobile units used in the home. Examples of fixed units include those installed on the ceiling or wall of a bedroom, stationary units, and units embedded in a bed mattress. Examples of mobile units include those equipped on a robot or the like. It also includes an information acquisition unit 20 that acquires outside vehicle environment information such as weather, outside temperature, humidity, date and time, and an information acquisition unit 20 that acquires traffic congestion information, construction zone information, accident information, broken-down vehicle information, etc. from external devices, etc.

[0023] The first determination unit 30 determines the physical fatigue level of the occupant in the vehicle based on information obtained from at least one of the imaging unit 10 and the information acquisition unit 20. Specifically, the first determination unit 30 quantifies, for example, a plurality of levels ranging from mild to severe (for example, levels 1 to 5 shown in FIG. 5 ) for each piece of information obtained from the imaging unit 10 or the information acquisition unit 20, and determines the physical fatigue level of the occupant in the vehicle. Note that a specific threshold is used to quantify the level, and it is preferable to use a value indicated by many academic studies, etc., as this specific threshold. As shown in FIG. 4, examples of information obtained from the imaging unit 10 include the degree of eye opening, the number of blinks, the sleeping time, the resting time, the number of turns in sleep, and the like. Examples of information that can be obtained from the information acquisition unit 20 include heart rate, blood oxygen saturation, heart rate variability, respiratory rate, body temperature, blood pressure, hemoglobin level, activity level, number of turns in bed, sleep state, sleep time, and information about the external environment of the vehicle. Here, the degree of eye opening can be used as an indicator for detecting drowsiness, the number of blinks can be used as an indicator for detecting the degree of eye fatigue, and the number of turns in sleep, REM sleep, non-REM sleep, etc. can be used as indicators for detecting the sleep state including the quality of sleep.

[0024] The second determination unit 40 determines the mental fatigue level of the occupant in the vehicle based on information obtained from at least one of the imaging unit 10 and the information acquisition unit 20. Specifically, the second determination unit 40 quantifies, for example, a number of levels ranging from mild to severe (for example, levels 1 to 5 shown in FIG. 5) for each piece of information obtained from the imaging unit 10 or the information acquisition unit 20, and determines the mental fatigue level of the occupant in the vehicle. Note that a specific threshold is used to quantify the level, and it is preferable to use a value indicated by many academic studies, etc., as this specific threshold.

[0025] The control unit 50 includes a known CPU, RAM, ROM, an I / O bus, etc. (not shown), and controls the entire safe driving support device 1 according to a control program stored in the ROM. In addition, in this embodiment, in particular, control is performed of vehicle devices (for example, a visual stimulation device 60, an auditory stimulation device 70, a tactile stimulation device 80, and an olfactory stimulation device 90, hereinafter referred to as vehicle devices 60, 70, 80, and 90) that are installed in the vehicle and that stimulate the awakening and recovery of occupants from fatigue. More specifically, the vehicle devices 60, 70, 80, 90 that encourage the occupant to wake up or recover from fatigue are controlled based on at least one of the determination information from the first determination unit and the determination information from the second determination unit. The control method will be described in detail later.

[0026] The visual stimulation device 60 is one of the vehicle devices that wakes up the occupant and helps them recover from fatigue, and includes light sources within the vehicle (light sources for interior lights, meters, HUD (Head-Up Display), monitors, etc.) and a sunroof opening and closing mechanism. Specifically, in response to a control signal from the control unit 50, the visual stimulation device 60 turns on the room light at night and changes the brightness and chromaticity. During the daytime, the sunroof is operated to let outside light into the vehicle interior. Note that the amount of outside light that is let into the vehicle interior may be controlled by controlling the light intensity of the window glass.

[0027] The auditory stimulation device 70 is one of the vehicle devices 60, 70, 80, 90 that stimulate the occupant to wake up and recover from fatigue, and includes an in-vehicle sound device or the like. Specifically, it outputs warning sounds, warning messages, music, environmental sounds, pre-registered human voices, and sounds that encourage ASMR (Autonomous Sensory Meridian Response). Here, ASMR (autonomic sensory meridian response) is a reaction that humans feel from hearing or seeing, and refers to a pleasant sensation that makes the brain tingle.In addition, sounds that stimulate ASMR can be exemplified by, for example, the sound of chewing that records the crunching sound when eating food, the sound that uses binaural recording to make you feel as if you are actually being ear-cleaned, the tapping sound of tapping on a keyboard or desk, the sound of rain, the murmur of a river, the sound of the sea, and other natural sounds.

[0028] The tactile stimulation device 80 is one of the vehicle devices 60, 70, 80, 90 that wakes up occupants and helps them recover from fatigue. It includes a vibration mechanism that vibrates the seat surface and back, and a vibration mechanism that vibrates the headrest, and also has the function of outputting images and audio on how to perform a fatigue recovery massage according to the target area.

[0029] The olfactory stimulation device 90 is one of the vehicle devices 60, 70, 80, 90 that stimulate the occupant to wake up or recover from fatigue, and includes an aroma emitter or the like. Specifically, it emits the scent of mint or aromatic oils that help wake you up from fatigue. For example, it emits the scent of rosemary, bergamot, geranium, peppermint, eucalyptus, lemon, etc. that improve concentration, the scent of lavender, chamomile, juniper, neroli, yuzu, ylang-ylang, rosewood, etc. that promote relaxation, and the scent of sweet marjoram, rosemary, ginger, juniper, marjoram, lavender, chamomile, etc. that promote recovery from fatigue.

[0030] <Handling of safe driving support devices> The processing of the safe driving support device 1 according to this embodiment will be described with reference to FIGS.

[0031] First, as shown in FIG. 3, the control unit 50 determines whether or not the ACC (accessory power supply) is turned on (step S110). When the control unit 50 determines that the ACC is not on ("NO" in step S110), the control unit 50 transitions to a standby mode.

[0032] On the other hand, if the control unit 50 determines that the ACC is on ("YES" in step S110), the first judgment unit 30 and the second judgment unit 40 execute judgment processing based on the information from the information acquisition unit 20 and output the judgment information to the control unit 50 (steps S120, S130). Here, the first determination section 30 and the second determination section 40 perform a determination process on the output items of the information acquisition section 20, as shown in FIG. Specifically, the first judgment unit 30 judges the physical fatigue level of the occupant based on at least one of the following information: blood oxygen saturation, respiratory rate, body temperature, blood pressure, hemoglobin level, activity level, number of blinks, eye opening, sleep time, number of turns in bed, sleep state, rest time, and external vehicle environment information. In addition, the second judgment unit 40 judges the occupant's mental fatigue level based on at least one of the following information: heart rate, heart rate variability, respiratory rate, body temperature, blood pressure, hemoglobin level, number of blinks, eye opening, sleep time, number of turns in bed, sleep state, rest time, and external vehicle environment information. 5, the first determination unit 30 and the second determination unit 40 use five levels of evaluation, with the heart rate, blood oxygen saturation, heart rate variability, respiratory rate, body temperature, blood pressure, hemoglobin level, activity level, and blink count indicating higher levels as the values ​​increase. The levels of eye opening, sleep time, number of turns in sleep, sleep state, and rest time indicate higher levels as the values ​​decrease. Furthermore, the worse the situation is, the higher the level of the external environment information is.

[0033] The control unit 50 determines whether the occupant is experiencing at least one or more physical fatigue or mental fatigue based on at least one or more pieces of determination information from the first determination unit 30 or the second determination unit 40 (step S140). Specifically, if the judgment information obtained from the first judgment unit 30 contains an information acquisition unit output item that is equal to or above a predetermined threshold level (for example, level 4 on a 5-level level shown in Figure 5), the control unit 50 judges that the occupant is physically fatigued, and if the judgment information obtained from the second judgment unit 40 contains an information acquisition unit output item that is equal to or above a predetermined threshold level (for example, level 4 on a 5-level level shown in Figure 5), the control unit 50 judges that the occupant is mentally fatigued. Furthermore, if the judgment information obtained from the first judgment unit 30 and the second judgment unit 40 includes an information acquisition unit output item that is equal to or above a predetermined threshold level (for example, level 4 on a 5-level scale shown in FIG. 5), the control unit 50 judges that the vehicle occupant is physically and mentally fatigued. In the example of Figure 5, the judgment information from the first judgment unit 30, including blood oxygen saturation, eye opening, sleep time, and rest state, is above the threshold level, and the judgment information from the second judgment unit 40, including eye opening, sleep time, and rest state, is above the threshold level, so the control unit 50 judges that the vehicle occupant is physically and mentally fatigued. Furthermore, when determining the fatigue level by calculating the number of output items from the information acquisition unit that are at or above a predetermined level (for example, level 4 on a five-level scale shown in Figure 5) on a five-point scale, or by calculating the simple average or weighted average of all output items from the information acquisition unit, it is preferable to use the judgment level found based on analysis of past data or academic knowledge as the predetermined fatigue level (predetermined level). The threshold level can be changed as appropriate, and even if an information acquisition unit output item does not meet the threshold level, if there is an effective method for improving that information acquisition unit output item, as a preventive measure, the effective method for improving that information acquisition unit output item may be executed.

[0034] In step S140, if the control unit 50 determines that the occupant does not have at least one of physical fatigue and mental fatigue ("NO" in step S140), the control unit 50 shifts the processing to step S110.

[0035] On the other hand, when the control unit 50 determines that the occupant has at least one of physical fatigue and mental fatigue ("YES" in step S140), the control unit 50 shifts the processing to step S150.

[0036] The control unit 50 executes control processing for the vehicle devices 60, 70, 80, 90 that prompt the occupant to wake up or recover from fatigue, based on at least one determination information from the first determination unit 30 or the second determination unit 40 (step S150). Specifically, when the judgment information of the first judgment unit 30 and the second judgment unit 40 is as shown in Figure 5, the judgment information of the first judgment unit 30 indicates that the items of blood oxygen saturation, eye opening, sleep time, and rest state are above the threshold level (level 4 on a 5-level level shown in Figure 5), and the judgment information of the second judgment unit 40 indicates that the items of eye opening, sleep time, and rest state are above the threshold level (level 4 on a 5-level level shown in Figure 5), so the control unit 50 controls the visual stimulation device 60, for example, as shown in Figure 6, to open the sunroof during the day and change the interior light to blue light at night. Furthermore, the control unit 50 controls the auditory stimulation device 70 to play natural sounds, for example. Furthermore, the control unit 50 controls the tactile stimulation device 80 to notify, for example, by image or sound, how to massage the user to promote recovery from fatigue. Furthermore, the control unit 50 controls the olfactory stimulation device 90 to emit the scent of aromatic oil that promotes fatigue recovery. After executing the control process for the vehicle devices 60, 70, 80, and 90, the control unit 50 ends the series of processes.

[0037] In addition to the visual stimulation device 60, the auditory stimulation device 70, the tactile stimulation device 80, and the olfactory stimulation device 90, the ventilation in the vehicle cabin may be switched to an outside air introduction mode to introduce air with a high oxygen content into the vehicle cabin, thereby promoting wakefulness.

[0038] <Actions and Effects> As described above, the safe driving support device 1 of this embodiment is configured to include an imaging unit 10 that images occupants in the vehicle, an information acquisition unit 20 that acquires biometric information of the occupants inside and outside the vehicle, a first judgment unit 30 that judges the physical fatigue level of the occupants in the vehicle, a second judgment unit 40 that judges the mental fatigue level of the occupants in the vehicle, and a control unit 50 that controls vehicle devices 60, 70, 80, 90 that promote awakening and fatigue recovery of the occupants based on at least one of the judgment information in the first judgment unit and the judgment information in the second judgment unit. In other words, the first determination unit 30 determines the physical fatigue level of the occupants in the vehicle based on information acquired by the imaging unit 10 that images the occupants in the vehicle and the information acquisition unit 20 that acquires biometric information of the occupants inside and outside the vehicle, and the second determination unit 40 determines the mental fatigue level of the occupants in the vehicle based on information acquired by the imaging unit 10 that images the occupants in the vehicle and the information acquisition unit 20 that acquires biometric information of the occupants inside and outside the vehicle. The control unit 50 controls the vehicle devices 60, 70, 80, 90 that encourage the occupant to wake up or recover from fatigue based on at least one of the determination information from the first determination unit 30 and the determination information from the second determination unit 40. Therefore, by encouraging occupants to wake up and recover from fatigue according to their level of physical and mental fatigue, the risk of accidents caused by fatigue can be reduced.

[0039] In addition, the control unit 50 quantifies the judgment information from the first judgment unit 30 and the judgment information from the second judgment unit 40, and controls the vehicle devices 60, 70, 80, and 90 that encourage the occupant to wake up and recover from fatigue based on at least one of the judgment information.This makes it possible to select an appropriate awakening means and adjust the degree of awakening treatment depending on the fatigue level, thereby reducing the risk of accidents caused by fatigue.

[0040] Furthermore, the control unit 50 of the safe driving support device 1 according to this embodiment controls a plurality of vehicle devices 60, 70, 80, and 90 that can stimulate the visual, auditory, tactile, and olfactory senses of the occupants in the vehicle. Therefore, the control unit 50 controls the vehicle devices 60, 70, 80, 90 that stimulate the visual, auditory, tactile, and olfactory senses of the occupants in the vehicle based on at least one of the judgment information from the first judgment unit and the judgment information from the second judgment unit, thereby making it possible to select an appropriate awakening means and adjust the degree of awakening treatment, thereby reducing the risk of accidents caused by fatigue. Furthermore, if there is a method of awakening and recovering from fatigue that is effective for a specific item among the items acquired by the information acquisition unit 20, and the judgment level of that item is high, control can be performed to execute the method of awakening and recovering from fatigue for that item regardless of the overall evaluation of all items, thereby enabling the implementation of a more appropriate awakening and fatigue recovery device according to the fatigue level of the occupant.

[0041] Furthermore, if the imaging unit 10 is configured with both an optical camera and a near-infrared camera, images of the occupant can be acquired day or night. Therefore, the resting state of the occupant can be accurately grasped, and more appropriate measures for waking up or relieving fatigue can be taken according to the occupant's fatigue level.

[0042] <Second embodiment> A safe driving support device 1A according to this embodiment will be described with reference to FIGS.

[0043] <Configuration of safe driving support device 1A> As shown in Figure 7, the safe driving support device 1A of this embodiment is configured to include an imaging unit 10, an information acquisition unit 20, a first judgment unit 30, a second judgment unit 40, a control unit 50A, a visual stimulation device 60, an auditory stimulation device 70, a tactile stimulation device 80, and an olfactory stimulation device 90. Note that components with the same reference numerals as those in the first embodiment have the same functions, and therefore detailed descriptions thereof will be omitted.

[0044] The control unit 50A uses machine learning to determine a control method for the vehicle devices 60, 70, 80, and 90 that can stimulate the visual, auditory, tactile, and olfactory senses of the occupants in the vehicle, and controls the vehicle devices 60, 70, 80, and 90. Details will be described later.

[0045] <Configuration of control unit 50A> As shown in FIG. 8, the control unit 50A is configured to include a dataset generation unit 51, a trained model 52, a machine learning unit 53, a machine learning result database (DB) 54, and a control signal generation unit 55.

[0046] The data set generation unit 51 generates a data set to be used by the machine learning unit 53 based on the judgment information input from at least one of the first judgment unit 30 and the second judgment unit 40.

[0047] The trained model 52 is a neural network (NN) that uses specific training data in advance. It is generated by inputting the data into the input and performing deep learning. The trained model may be generated based on data stored in a machine learning result database (DB) 54, which will be described later.

[0048] The machine learning unit 53 includes a neural network (NN) and uses the dataset generated by the dataset generation unit 51 as input data, and performs deep learning using the trained model 52 to output a control method for the vehicle devices 60, 70, 80, and 90. More specifically, the machine learning unit 53 calculates the degree of deviation, for example, of the score for the similarity between the dataset generated by the dataset generation unit 51 and the trained model 52, and if this degree of deviation is equal to or less than a predetermined value, detects the control method of the vehicle devices 60, 70, 80, 90 included in the trained model 52. Then, the machine learning unit 53 outputs the data to the control signal generation unit 55 and stores it in the machine learning result database (DB) 54. The occupant's unique ID and personal information may be linked to the dataset generated by the dataset generation unit 51 and input to the machine learning unit 53. The information may also include information about the season, weather, temperature, humidity, time of day, or environmental information outside the vehicle obtained from an external device, such as information about construction areas, accident information, and broken-down vehicles. Furthermore, a trained model 52 may be generated and updated as appropriate using data stored in the machine learning result database (DB) 54, which is data generated by the dataset generation unit 51 after a predetermined time has elapsed and linked to the occupant's unique ID, personal information, etc.

[0049] The machine learning result database (DB) 54 stores a dataset that combines a dataset generated by the dataset generation unit 51 via the machine learning unit 53 with control method information for the vehicle devices 60, 70, 80, and 90 output from the machine learning unit 53. The data may include an ID unique to the occupant and personal information.

[0050] The control signal generation unit 55 generates and outputs, for example, a control signal specific to the vehicle devices 60, 70, 80, and 90 based on the control method information for the vehicle devices output from the machine learning unit 53.

[0051] <Processing of the safe driving support device 1A> The processing of the safe driving support device 1A according to this embodiment will be described with reference to FIGS. 9, the process of the safe driving support device 1A according to this embodiment differs from that of the first embodiment only in the process of step S200. Therefore, only the process of step S200 will be described in detail below.

[0052] <Vehicle Device Control Process by Control Unit 50A> As shown in FIG. 10, when at least one of the first judgment unit 30 or the second judgment unit 40 judges that the occupant has at least one or more physical fatigue or mental fatigue ("YES" in step S140 of FIG. 9), the control unit 50A inputs judgment information from at least one of the first judgment unit 30 or the second judgment unit 40 to the data set generation unit 51 (step S210).

[0053] The data set generation unit 51 generates a data set based on the judgment information input from at least one of the first judgment unit 30 or the second judgment unit 40, and outputs the data set to the machine learning unit 53. Meanwhile, the machine learning unit 53 imports the trained model 52 (step S220).

[0054] The machine learning unit 53 uses the dataset generated by the dataset generation unit 51 as input data and performs machine learning using the trained model 52, thereby outputting the control method for the vehicle devices 60, 70, 80, and 90 as a learning result to the control signal generation unit 55. Meanwhile, the machine learning unit 53 associates the control method for the vehicle devices 60, 70, 80, and 90 as a learning result with the dataset input from the dataset generation unit 51 and stores it in the machine learning result database (DB) 54 (step S230).

[0055] The control signal generating unit 55 generates and outputs a control signal that is unique to each of the vehicle devices 60, 70, 80, and 90 (step S240).

[0056] <Actions and Effects> As described above, the control unit 50A of the safe driving support device 1A of this embodiment uses machine learning to determine a control method for the vehicle devices 60, 70, 80, and 90 that can stimulate the visual, auditory, tactile, and olfactory senses of the occupants in the vehicle, and controls the vehicle devices 60, 70, 80, and 90. Therefore, the control unit 50A performs machine learning based on at least one of the judgment information from the first judgment unit 30 and the judgment information from the second judgment unit 40, and controls the vehicle devices 60, 70, 80, and 90 that stimulate the vision, hearing, touch, and smell of the occupants in the vehicle, so that it is possible to select appropriate awakening means and adjust the degree of awakening treatment, thereby reducing the risk of accidents caused by fatigue.

[0057] Furthermore, if the occupant's unique ID and personal information are linked to the dataset generated by the dataset generation unit 51 and used as input information for the machine learning unit 53, control tailored to the occupant becomes possible, making it possible to select appropriate awakening means and adjust the degree of awakening treatment uniquely and appropriately for each occupant, thereby reducing the risk of accidents caused by fatigue.

[0058] Furthermore, if environmental information outside the vehicle, such as the season, weather, temperature, humidity, time of day, and traffic congestion, is included in the above, more appropriate control can be achieved according to the occupant. This will further enable the selection of appropriate awakening means and the degree of awakening treatment to be individually and appropriately adjusted according to the occupant, thereby reducing the risk of accidents caused by fatigue.

[0059] Furthermore, by using the data stored in the machine learning result database (DB) 54, which is a dataset generated by the dataset generation unit 51 after a predetermined time has elapsed and data linked to the occupant's unique ID, personal information, etc., as learning data, and appropriately updating the trained model 52, it is possible to select more effective awakening measures and adjust the degree of awakening treatment more effectively, thereby reducing the risk of accidents caused by fatigue.

[0060] <Third embodiment> A safe driving support device 1B according to this embodiment will be described with reference to FIGS.

[0061] <Configuration of safe driving support device 1B> As shown in FIG. 11, the safe driving support device 1B of this embodiment is configured to include an imaging unit 10, an information acquisition unit 20, a first judgment unit 30, a second judgment unit 40B, a control unit 50B, a visual stimulation device 60, an auditory stimulation device 70, a tactile stimulation device 80, an olfactory stimulation device 90, and a third judgment unit 100. Note that components with the same reference numerals as those in the first and second embodiments have the same functions, and therefore detailed descriptions thereof will be omitted.

[0062] The second determination unit 40B determines the mental fatigue level of the occupant in the vehicle based on information obtained from at least one of the imaging unit 10 and the information acquisition unit 20. Specifically, the second determination unit 40B determines the mental fatigue level of the occupant based on the heart rate, heart rate variability, respiratory rate, body temperature, blood pressure, hemoglobin level, number of blinks, eye opening degree, external environment information, etc. 13, the second determination unit 40B evaluates the vehicle's exterior environment information in five stages, with the heart rate, heart rate variability, respiratory rate, body temperature, blood pressure, hemoglobin level, and blink rate being evaluated at higher levels as the values ​​increase. The eye opening degree is also evaluated at higher levels as the values ​​decrease. Furthermore, the exterior environment information is evaluated at higher levels as the conditions worsen.

[0063] The third determination unit 100 determines the resting state of an occupant in a vehicle based on information obtained from at least one of the imaging unit 10 and the information acquisition unit 20. Specifically, the third determination unit 100 determines the resting state of the occupant based on the amount of activity, sleeping time, number of turns, sleeping state, resting time, etc. In particular, the third determination unit 100 determines the resting state from sleep state information including the sleeping time and sleep quality of the occupant acquired by the information acquisition unit 20. Furthermore, the third determination unit 100 determines the resting state from, for example, moving images of the occupant taking a rest acquired by the imaging unit 10. 13, the evaluation level of the third determination unit 100 is 5 levels, and the levels of the amount of activity, number of turns, and sleep state are higher as the values ​​are higher. Also, the levels of the sleeping time and resting time are higher as the values ​​are lower. Examples of information obtained from the imaging unit 10 include rest time, sleeping time, number of times of turning over in sleep, etc. Examples of information obtained from the information acquisition unit 20 include rest time, sleeping time, number of times of turning over in sleep, sleep state including quality of sleep, etc.

[0064] The control unit 50B controls the vehicle devices 60, 70, 80, and 90 when it determines, based on the judgment information from the second judgment unit 40B and the judgment information from the third judgment unit 100, that the mental fatigue level of the occupants in the vehicle is at or above a predetermined level and that they have not been able to get a predetermined amount of rest. Specifically, if the judgment information obtained from the second judgment unit 40B includes an information acquisition unit output item that is rated at a predetermined level or higher (for example, level 4 on a 5-point scale shown in Figure 13) on a 5-point scale, the control unit 50B judges that the occupant has a high level of mental fatigue. In the example of FIG. 13, the eye opening item in the determination information of the second determination unit 40B is equal to or greater than the threshold level, so the control unit 50B determines that the vehicle occupant is mentally fatigued. On the other hand, if the judgment information obtained from the third judgment unit 100 includes an information acquisition unit output item that is rated at or above a predetermined level on a five-point scale (for example, level 4 on the five-point scale shown in Figure 13), the control unit 50B judges that the occupant is not getting enough rest. In the example of Figure 13, the judgment information of the third judgment unit 100, which includes the activity level, sleep time, and sleep state, is above the threshold level, so the control unit 50B judges that the vehicle occupant is not getting enough rest. Regarding rest, it is preferable to make a comprehensive judgment based not only on the rest state during the trip, but also on the rest state from the day before the trip or several weeks before. Specifically, it is preferable to make a judgment based mainly on the rest state around the trip day, and also taking into account whether regular rest has been taken in a predetermined period before that.

[0065] <Processing of safe driving support device 1B> The processing of the safe driving support device 1B according to this embodiment will be described with reference to FIGS.

[0066] First, as shown in FIG. 12, the control unit 50B determines whether or not the ACC (accessory power supply) is turned on (step S110). When the control unit 50B determines that the ACC is not on ("NO" in step S110), the control unit 50B transitions to a standby mode.

[0067] On the other hand, if the control unit 50B determines that the ACC is on ("YES" in step S110), the second judgment unit 40B and the third judgment unit 100 execute judgment processing based on information from at least one of the imaging unit 10 or the information acquisition unit 20, and output the judgment information to the control unit 50B (steps S120, S310). Here, the second determination section 40B and the third determination section 100 perform a determination process on the output items of the information acquisition section 20 as shown in FIG.

[0068] The control unit 50B determines whether the occupant's mental fatigue level is equal to or higher than a predetermined level and whether the occupant has not had enough rest, based on the second determination unit 40B and the third determination unit 100 (step S320). When the control unit 50B determines that the occupant's mental fatigue level is not equal to or higher than a predetermined level and that the occupant has had sufficient rest ("NO" in step S320), the control unit 50B shifts the process to step S110.

[0069] On the other hand, if the control unit 50B determines that the occupant is mentally fatigued and has not had enough rest ("YES" in step S320), it shifts the process to step S330. Specifically, in the example of Fig. 13, the control unit 50B determines that the occupant is mentally fatigued because the eye opening degree is at level 4 or higher on a five-level scale shown in Fig. 13, and determines that the occupant is not resting sufficiently because the activity level, sleep time, and sleep state are at level 4 or higher on a five-level scale shown in Fig. 13. Therefore, the control unit 50B shifts the process to step S330 to take measures to alleviate and recover the occupant's mental fatigue level.

[0070] The control unit 50B executes control processing for the vehicle devices 60, 70, 80, 90 that promote the alleviation and recovery of the occupant's mental fatigue level based on the second determination unit 40B and the third determination unit 100 (step S330). Specifically, when the judgment information of the second judgment unit 40B and the third judgment unit 100 is as shown in FIG. 13, the control unit 50B controls the visual stimulation device 60, for example, as shown in FIG. 6, to open the sunroof during the day and change the interior light to blue light at night. Furthermore, the control unit 50B controls the auditory stimulation device 70 to play natural sounds, for example. Furthermore, the control unit 50B controls the tactile stimulation device 80 to notify, by image or sound, how to massage the user to promote recovery from fatigue, for example. Furthermore, the control unit 50B controls the olfactory stimulation device 90 to emit the scent of aromatic oil that promotes fatigue recovery. After executing the control process for the vehicle devices 60, 70, 80, and 90, the control unit 50B ends the series of processes.

[0071] In addition to the vehicle devices 60, 70, 80, and 90, the ventilation in the vehicle compartment may be switched to an outside air introduction mode to introduce air with a high oxygen content into the vehicle compartment, thereby promoting wakefulness.

[0072] <Actions and Effects> As described above, the safety driving support device 1B of this embodiment has a third judgment unit 100 that judges the rest state of the occupants in the vehicle, and the control unit 50B controls the vehicle devices 60, 70, 80, 90 when it judges, based on the judgment information in the second judgment unit 40B and the judgment information in the third judgment unit 100, that the mental fatigue level of the occupants in the vehicle is at or above a predetermined level and that they have not been able to get a predetermined amount of rest. In other words, if the mental fatigue level of the occupants in the vehicle is determined to be above a predetermined level based on the determination information from the second determination unit 40B, and they have not had enough sleep the day before the trip, or have not had enough and regular sleep in the days or weeks prior to that, or have not had enough rest during the trip, the occupants may be feeling fatigued to a level greater than the mental fatigue level derived from the determination information from the second determination unit 40B. Therefore, when the above conditions are met, the control unit 50B can control the vehicle devices 60, 70, 80, and 90 to select an appropriate awakening means and adjust the degree of awakening treatment according to the fatigue level actually felt by the occupant, thereby reducing the risk of accidents caused by fatigue.

[0073] In the first embodiment, the second determination unit 40 makes a determination including items for determining a resting state, and the control unit 50 determines the overall mental fatigue level of the occupant based on the determination information of the second determination unit 40. In the present embodiment, the second determination unit 40B makes a determination regarding items excluding items for determining a resting state, and the third determination unit 100 makes a determination regarding items for determining a resting state, and when the control unit 50B determines, based on the determination information of the second determination unit 40B and the determination information of the third determination unit 100, that the mental fatigue level of the occupant in the vehicle is equal to or higher than a predetermined level and that the occupant has not been able to take a predetermined amount of rest, the control unit 50B controls the vehicle devices 60, 70, 80, and 90. Therefore, by more accurately assessing the actual level of mental fatigue felt by the occupants, it is possible to select appropriate awakening measures and adjust the degree of awakening treatment, thereby reducing the risk of accidents caused by fatigue.

[0074] According to this embodiment, the third determination unit 100 determines the resting state from information acquired by the information acquisition unit 20, the information including the sleeping time and the quality of the sleep of the occupant. That is, the sleep state before the start of driving is determined based on information from sensors installed on the bed, ceiling, etc. in the home. Therefore, since the control unit 50B makes a multifaceted judgment of the occupant's mental fatigue level on the day of driving based on the judgment information from the second judgment unit 40B and the judgment information from the third judgment unit 100, it is possible to accurately determine the actual mental fatigue level felt by the occupant, select more appropriate awakening means, and adjust the degree of awakening treatment, thereby reducing the risk of accidents occurring due to fatigue.

[0075] According to this embodiment, the third determination unit 100 determines the resting state from the moving image of the occupant captured by the imaging unit 10 when the occupant is taking a rest. Therefore, since the control unit 50B makes a multifaceted judgment of the occupant's mental fatigue level on the day of driving based on the judgment information from the second judgment unit 40B and the judgment information from the third judgment unit 100, it is possible to accurately determine the mental fatigue level actually felt by the occupant, select appropriate awakening means, and adjust the degree of awakening treatment, thereby reducing the risk of accidents occurring due to fatigue.

[0076] <Fourth embodiment> A safe driving support device 1C according to this embodiment will be described with reference to FIGS.

[0077] <Configuration of safe driving support device 1C> As shown in Figure 14, the safety driving support device 1C of this embodiment is configured to include an imaging unit 10, an information acquisition unit 20, a first judgment unit 30C, a second judgment unit 40B, a control unit 50C, a visual stimulation device 60, an auditory stimulation device 70, a tactile stimulation device 80, an olfactory stimulation device 90, and a third judgment unit 100. Note that components with the same reference numerals as those in the first to third embodiments have the same functions, and therefore detailed descriptions thereof will be omitted.

[0078] The first determination unit 30C determines the physical fatigue level of the occupant in the vehicle based on information obtained from at least one of the imaging unit 10 and the information acquisition unit 20. Specifically, the first determination unit 30C determines the mental fatigue level of the occupant based on blood oxygen saturation, respiratory rate, body temperature, blood pressure, hemoglobin level, activity level, blink frequency, eye opening degree, external environment information, etc.

[0079] The control unit 50C outputs a control signal based on at least one determination information of the first determination unit 30C, the second determination unit 40B, or the third determination unit 100 to the driving support device 200 outside the safe driving support device 1C. Specifically, when the control unit 50C determines, for example, based on the judgment information from the first judgment unit 30C and the judgment information from the third judgment unit 100, that the occupant's physical fatigue level is at or above a predetermined level and that the occupant has not been able to get a predetermined amount of rest, it outputs a control signal based on the judgment information to the external driving assistance device 200. Here, examples of control of the external driving assistance device 200 by the control unit 50C include collision mitigation brake control, sleep detection alarm control that issues a warning earlier than usual by changing the sleep detection warning threshold, steering control that suppresses sudden steering of the electric power steering system more than the actual steering operation, false start prevention control, lane keeping control, active cruise control (ACC), etc., as shown in Figure 17.

[0080] In addition, when the control unit 50C determines, for example, based on the judgment information from the second judgment unit 40B and the judgment information from the third judgment unit 100, that the mental fatigue level of the occupants in the vehicle is at or above a predetermined level and that they have not been able to get a predetermined amount of rest, it controls the vehicle devices 60, 70, 80, 90 that promote recovery from the mental fatigue level.

[0081] <Processing of the safe driving support device 1C> The processing of the safe driving support device 1C according to this embodiment will be described with reference to FIGS.

[0082] First, as shown in FIG. 15, the control unit 50C determines whether or not the ACC (accessory power supply) is turned on (step S110). When the control unit 50C determines that the ACC is not on ("NO" in step S110), the control unit 50C transitions to a standby mode.

[0083] On the other hand, if the control unit 50C determines that the ACC is on ("YES" in step S110), the first judgment unit 30C, the second judgment unit 40B, and the third judgment unit 100 execute judgment processing based on information from at least one of the imaging unit 10 or the information acquisition unit 20, and output the judgment information to the control unit 50C (steps S120, S130, and S410). Here, the first determination unit 30C, the second determination unit 40B, and the third determination unit 100 perform determination processing on the output items of the information acquisition unit 20, as shown in FIGS. Specifically, the first determination unit 30C determines the occupant's physical fatigue level based on blood oxygen saturation, respiratory rate, body temperature, blood pressure, hemoglobin level, activity level, blinking frequency, eye opening degree, external environment information, etc. The second determination unit 40B determines the occupant's mental fatigue level based on heart rate, heart rate variability, respiratory rate, body temperature, blood pressure, hemoglobin level, blinking frequency, eye opening degree, external environment information, etc. The third determination unit 100 determines the resting state of the occupant based on the amount of activity, the sleeping time, the number of times the occupant has turned over in their sleep, the sleeping state including the quality of sleep, the resting time, and the like.

[0084] The control unit 50C determines whether the physical fatigue level of the occupant is equal to or greater than a predetermined level based on the first determination unit 30C (step S420). If the control unit 50C determines that the occupant's physical fatigue level is not above a predetermined level ("NO" in step S420), it determines whether the occupant's mental fatigue level is above a predetermined level based on the second judgment unit 40B (step S430). If the control unit 50C determines that the mental fatigue level of the occupant is not equal to or higher than a predetermined level ("NO" in step S430), the control unit 50C shifts the process to step S110.

[0085] On the other hand, when the control unit 50C determines that the occupant's mental fatigue level is equal to or higher than the predetermined level ("YES" in step S430), it executes processing based on the determination information by the third determination unit 100 (step S440) and determines whether the occupant's mental fatigue level is equal to or higher than the predetermined level and whether the occupant has had a predetermined amount of rest (step S450). When the control unit 50C determines that the occupant's mental fatigue level is equal to or higher than the predetermined level but has had a predetermined amount of rest ("NO" in step S450), it shifts the processing to step S110.

[0086] On the other hand, if the control unit 50C determines that the occupant's mental fatigue level is above a predetermined level and that the occupant has not been able to rest for a predetermined amount of time ("YES" in step S450), it executes control processing of the vehicle devices 60, 70, 80, and 90 to promote the alleviation and recovery of the occupant's mental fatigue level based on the judgment information of the second judgment unit 40B (step S470). Specifically, when the judgment information of the second judgment unit 40B is as shown in Figure 16, the control unit 50C controls the visual stimulation device 60, for example, as shown in Figure 6, to open the sunroof during the day, and to reduce the brightness of the interior light or change the chromaticity to a warm color during the night. Furthermore, the control unit 50C controls the auditory stimulation device 70 to play natural sounds, for example. Furthermore, the control unit 50C controls the tactile stimulation device 80 to notify, by image or sound, how to massage the user to promote recovery from fatigue, for example. Furthermore, the control unit 50C controls the olfactory stimulation device 90 to emit the scent of aromatic oil that promotes recovery from fatigue. After the control unit 50C has executed the control process for the vehicle devices 60, 70, 80, and 90, the control unit 50C ends the series of processes.

[0087] If the control unit 50C determines in step S420 that the physical fatigue level of the occupant is equal to or higher than a predetermined level ("YES" in step S420), it executes processing based on the determination information of the third determination unit 100 (step S440).

[0088] Based on the processing result of step S440, the control unit 50C determines whether the occupant's physical fatigue level is equal to or higher than a predetermined level and whether the occupant has had a predetermined amount of rest (step S460). If the control unit 50C determines that the occupant's physical fatigue level is equal to or higher than a predetermined level but that the occupant has had a predetermined amount of rest ("NO" in step S460), the control unit 50C shifts the processing to step S110.

[0089] On the other hand, if the control unit 50C determines that the occupant's physical fatigue level is above a predetermined level and that the occupant has not been able to rest for a predetermined amount of time ("YES" in step S460), it executes control processing of the vehicle devices 60, 70, 80, and 90 to promote the alleviation and recovery of the occupant's physical fatigue level based on the judgment information of the second judgment unit 40B (step S470).

[0090] In addition, the control unit 50C outputs a control signal based on the determination information from the third determination unit 100 to the driving assistance device 200, and controls the driving assistance device 200 (step S480). Specifically, the control unit 50C controls the driving assistance device 200 to perform, for example, collision mitigation brake control or sleep detection warning control that issues a warning earlier than usual by changing the warning threshold for sleep detection, steering control that suppresses sudden steering beyond the actual steering operation for the electric power steering system, false start prevention control, lane keeping control, active cruise control (ACC), etc.

[0091] <Actions and Effects> As described above, the safety driving support device 1C of this embodiment has a third judgment unit 100 that judges the rest state of the occupants in the vehicle, and the control unit 50C controls the vehicle devices 60, 70, 80, 90 when it judges, based on the judgment information in the second judgment unit 40B and the judgment information in the third judgment unit 100, that the mental fatigue level of the occupants in the vehicle is at or above a predetermined level and that they have not been able to get a predetermined amount of rest. In other words, if the mental fatigue level of the occupants in the vehicle is determined to be above a predetermined level based on the determination information from the second determination unit 40B, and they have not had sufficient sleep the day before the trip, or have not had sufficient and regular sleep in the days or weeks prior to that, or have not had sufficient rest during the trip, the occupants may be feeling fatigued to a level greater than the mental fatigue level derived from the determination information from the second determination unit 40B. Therefore, when the above conditions are met, the control unit 50C can control the vehicle devices 60, 70, 80, and 90 to select an appropriate awakening means and adjust the degree of awakening treatment according to the fatigue level actually felt by the occupant, thereby reducing the risk of accidents caused by fatigue.

[0092] Furthermore, when the control unit C of the safety driving support device 1C of this embodiment determines, based on the judgment information from the first judgment unit 30C and the judgment information from the third judgment unit 100, that the occupant's physical fatigue level is at or above a predetermined level and that the occupant has not been able to rest for a predetermined period of time, the control unit C controls the vehicle devices 60, 70, 80, 90 that promote recovery from the physical fatigue level, and outputs a control signal based on the judgment information to the driving support device 200. In other words, if the judgment information from the first judgment unit 30C indicates that the physical fatigue level of the occupants in the vehicle is equal to or higher than a predetermined level, and that they have not had sufficient sleep the day before the driving day, or have not had sufficient and regular sleep in the preceding few days or weeks, or have not had sufficient rest during the driving, the vehicle devices 60, 70, 80, 90 are controlled to promote recovery from the physical fatigue level, and a control signal based on the judgment information is output to the driving assistance device 200 to reduce the physical load. Therefore, when the above conditions are met, the control unit 50C controls the vehicle devices 60, 70, 80, 90 and the driving assistance device 200, thereby reducing the physical burden on the occupants and preventing the physical fatigue level from worsening, while selecting appropriate awakening means and adjusting the degree of awakening treatment according to the fatigue level actually felt by the occupants, thereby reducing the risk of accidents caused by fatigue.

[0093] In the first embodiment, at least one of the first determination unit 30 or the second determination unit 40 makes a determination including items for determining a resting state, and the control unit 50 determines the overall mental fatigue level of the occupant based on the determination information of at least one of the first determination unit 30 or the second determination unit 40. In this embodiment, at least one of the first determination unit 30C or the second determination unit 40B makes a determination regarding items excluding items for determining a resting state, and the third determination unit 100 makes a determination regarding items for determining a resting state, and when the control unit 50C determines, based on the determination information of the second determination unit 40B and the determination information of the third determination unit 100, that the mental fatigue level of the occupant in the vehicle is equal to or higher than a predetermined level and that the occupant has not been able to take a predetermined amount of rest, it controls the vehicle devices 60, 70, 80, and 90. Alternatively, when the control unit 50C determines, based on the judgment information from the first judgment unit 30C and the judgment information from the third judgment unit 100, that the occupant's physical fatigue level is at or above a predetermined level and that the occupant has not been able to get a predetermined amount of rest, it outputs a control signal based on the judgment information to the driving assistance device 200. Therefore, by accurately determining the level of mental fatigue actually felt by the occupants, it is possible to select appropriate awakening measures and adjust the degree of awakening treatment, thereby reducing the risk of accidents caused by fatigue.

[0094] <Modification> For example, in the above embodiment, a configuration is shown in which a first judgment unit 30, 30C, a second judgment unit 40, 40B, and a control unit 50, 50A, 50B, 50C are provided within the safety driving support device 1, 1A, 1B, 1C, but the information from the imaging unit 10 and the information from the information acquisition unit 20 may be transferred to a server connected to the vehicle, and the judgment processing of the first judgment unit 30, 30C, the judgment processing of the second judgment unit 40, 40B, and the control processing of the control units 50, 50A, 50B, 50C may be executed on the server. In this way, it is possible to quickly process a large amount of information, select an appropriate awakening method, and adjust the level of awakening treatment, thereby reducing the risk of accidents caused by fatigue.

[0095] 4 shows an example of biological information acquired by the information acquisition unit 20 in the above embodiment, but the invention is not limited to this. For example, information such as brain waves, sweat rate, skin temperature, and blood sugar level may be acquired, and fatigue may be determined by combining various types of biological information.

[0096] The safe driving support device of the present invention can be realized by recording the processing of the control units 50, 50A, 50B, and 50C on a computer-readable recording medium, and having the control units 50, 50A, 50B, and 50C read and execute the program recorded on the recording medium. The computer system here includes hardware such as an OS and peripheral devices.

[0097] Furthermore, if a WWW (World Wide Web) system is used, the "computer system" also includes the homepage provision environment (or display environment). The above program may be transmitted from a computer system that stores the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) like the Internet or a communication line (communication line) like a telephone line.

[0098] The program may also be for realizing part of the above-mentioned functions. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

[0099] The above describes an embodiment of the present invention in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0100] 1. Safe driving support device 1A: Safe driving support device 1B: Safe driving support device 1C: Safe driving support device 10. Imaging unit 20; Information acquisition department 30: First judgment section 30C: First judgment section 40: Second Judgment Section 40B: Second Judgment Section 50; control section 50A; Control section 50B: Control section 50C: Control section 60;Visual stimulation device 70;Auditory stimulation device 80;Tactile stimulation device 90;Olfactory stimulation device 100;Third Judging Section 200; Driving assistance device

Claims

1. an imaging unit that images an occupant in a vehicle; an information acquisition unit that acquires information including biometric information of the occupant inside and outside the vehicle; a first determination unit that determines a physical fatigue level of the occupant in the vehicle based on an image of the occupant in the vehicle captured by the imaging unit or information including biological information of the occupant inside and outside the vehicle acquired by the information acquisition unit; and a second determination unit that determines a mental fatigue level of the occupant in the vehicle based on an image of the occupant in the vehicle captured by the imaging unit or information including biological information of the occupant inside and outside the vehicle acquired by the information acquisition unit; and a third determination unit that determines a resting state of the occupant in the vehicle from an image of the occupant in the vehicle taking a rest, the image being captured by the imaging unit; a control unit that controls a vehicle device based on at least one of the determination information of the first determination unit and the determination information of the second determination unit and the determination information of the third determination unit; A safe driving support device comprising:

2. 2. The safe driving support device according to claim 1, wherein the control unit controls the vehicle device capable of providing visual, auditory, tactile, and olfactory stimulation to the occupants of the vehicle.

3. The safety driving support device according to claim 1, characterized in that the control unit controls the vehicle device when it determines, based on the judgment information from the second judgment unit and the judgment information from the third judgment unit, that the mental fatigue level of the occupant in the vehicle is at or above a predetermined level and that the occupant has not been able to get a predetermined amount of rest.

4. The vehicle device includes a driving assistance device that assists a driver in driving the vehicle, The safety driving support device according to claim 1, characterized in that the control unit outputs a control signal to the driving support device based on at least one of the judgment information in the first judgment unit or the judgment information in the second judgment unit and the judgment information in the third judgment unit.

5. 5. The safety driving support device according to claim 1, wherein the control unit outputs a control signal based on the judgment information to the vehicle device when it determines, based on the judgment information from the first judgment unit and the judgment information from the third judgment unit, that the occupant's physical fatigue level is at or above a predetermined level and that the occupant has not been able to get a predetermined amount of rest.

Images