DEVICE FOR MONITORING A DRIVER, SYSTEM WITH THE SAME AND METHOD OF IT

Abstract

Device (100) for monitoring a driver in a vehicle, the device comprising: a processor (140) configured to monitor a driver's condition based on acquisition data and driver inattention monitoring logic during vehicle operation, and a display device (130) configured to display a message indicating a start of the driver inattention monitoring logic, wherein the processor (140) is further configured to: perform the driver inattention monitoring logic when, based on the acquisition data during vehicle operation, a dangerous driving situation occurs or the driver does not look ahead of the vehicle, and determine the dangerous driving situation at a preset unit time interval.

Description

Cross-reference to related application

[0001] This application claims priority to and the benefit of Korean Patent Application No. 10-2019-0121819, filed on October 1, 2019, the entire contents of which are incorporated herein by reference for all purposes. Area

[0002] The present disclosure relates to an apparatus for monitoring a driver (hereinafter, "driver" is used as a representative for driver), a system having the same, and a method for the same, and more particularly relates to a technology for monitoring a driver in a self-driving vehicle. background

[0003] The statements in this section merely provide background information related to the present invention and do not necessarily represent prior art.

[0004] Recently, studies and research have been actively conducted for an autonomous vehicle that is capable of performing independent or autonomous driving in a state in which operation by a driver is partially or completely excluded.

[0005] Such an autonomous vehicle detects and determines the surrounding situation of the vehicle and performs driving control, but it cannot fully ensure driving safety.

[0006] Accordingly, when it is difficult to ensure driving safety due to the occurrence of a dangerous situation and the error or defect of software or hardware for autonomous driving, control to perform driving control is immediately transferred to the driver even if the autonomous driving is carried out.

[0007] However, it has been found that a serious accident can occur if the driver fails to recognize requests for transfer of control while not looking forward. Consequently, it is desirable to monitor whether the driver is looking forward or (inattentively) to the side while driving autonomously. Explanation of the invention

[0008] The present disclosure or invention (hereinafter referred to as disclosure) has been made to solve the above-mentioned problems encountered in the prior art while keeping intact the advantages achieved by the prior art.

[0009] One aspect of the present disclosure provides an apparatus for monitoring a driver capable of improving reliability thereof by monitoring the driver based on sensing data and driver inattention checking logic during vehicle travel, a system having the same, and a method for the same.

[0010] The technical problems to be solved by the present disclosure are not limited to the problems disclosed in this specification, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure belongs from the following description.

[0011] According to one aspect of the present disclosure, an apparatus for monitoring a driver in a vehicle (e.g., motor vehicle, in particular, a passenger vehicle) may comprise: a processor for monitoring a driver condition based on detection data and driver inattention check logic during vehicle travel, and a display device for displaying a message announcing (e.g., reporting, notifying) the start of the driver inattention check logic. The processor may perform the driver inattention check logic when it is determined that a dangerous driving situation is occurring or that the driver is not looking ahead of the vehicle based on the detection data during vehicle travel.

[0012] According to one embodiment, the processor can determine whether the dangerous driving situation occurs by using at least one of the detection data, navigation information or weather information during the vehicle drive.

[0013] According to one embodiment, the dangerous driving situation may include at least one of a situation of entering an intersection, a situation of entering a school zone, a situation of entering an accident blackspot (e.g. danger zone with frequent and / or serious accidents), a situation of a change in vehicle traffic flow, a situation of sudden braking of the vehicle (e.g. due to a vehicle cutting in), a situation of bad weather, a situation of the beginning of a sharp curve or a situation of approaching a motorway and / or federal highway.

[0014] According to one embodiment, the processor can determine the dangerous driving situation at a preset unit time interval.

[0015] According to a further embodiment, the processor can adapt the unit time interval depending on a traveled road (e.g., the type of traveled road) or a traffic situation.

[0016] According to one embodiment, the processor may increase the unit time interval when the vehicle is traveling on a highway and decrease the unit time interval when the vehicle is traveling in a city or in a traffic congestion area.

[0017] According to another embodiment, the processor may announce the initiation of the driver inattention check logic using at least one of a haptic signal, an audible signal, or a visual signal.

[0018] According to a further embodiment, the processor can change at least one of a size of a warning symbol displayed on the display device, a movement change of the warning symbol, a color change of the warning symbol, a color change of a mood lamp, a type of an audible warning tone, a volume of the audible warning tone, or a ringing duration (e.g., ringing duration) of the audible warning tone based on the dangerous driving situation and a general driving situation.

[0019] According to one embodiment, the processor can output on the display device an element for executing a check of the inattention of a driver (driver inattention check execution element, e.g. also driver inattention check performing element) in order to perform the driver inattention check logic.

[0020] According to a further embodiment, the display device may include a head-up display device or a front display device (eg, front display), and the display device may determine whether the driver inattention check execution item is performed in a state that the driver is looking forward.

[0021] According to a further embodiment, the driver inattention check execution element may comprise an action that is performable (e.g., by the driver) while the driver is holding a steering wheel.

[0022] According to yet another embodiment, when the driver inattention check logic is repeated in a state in which the vehicle is not in the dangerous driving situation, the processor may prompt the driver to perform an action for a driver inattention check execution item that is different from the driver inattention check execution item that was previously performed.

[0023] According to one embodiment, if the dangerous driving situation persists after the driver inattention check logic is performed in the dangerous driving situation, the processor may prompt the driver to perform a driver inattention check execution element using a voice.

[0024] According to one embodiment, the processor may determine whether the driver inattention check execution element is performed by the driver based on the detection data during the vehicle travel.

[0025] According to one embodiment, the processor may request a transfer of control (e.g., control over driving the vehicle, in particular acceleration, deceleration, steering, etc.) from a system to the driver if the driver inattention check execution element is not performed by the driver.

[0026] According to one embodiment, if the driver does not obtain control (e.g., does not take over), the processor may stop a self-driving function and command a deceleration stop in a road or a stop on a secondary road and / or side of the road.

[0027] According to one embodiment, the processor may provide feedback to the driver by operating a sunroof or a side window after confirming operation of a steering wheel when the driver is not looking forward.

[0028] According to another aspect of the present disclosure, a vehicle system may include a detection device for detecting whether a driver of a vehicle is looking ahead of the vehicle, and a vehicle monitoring device for detecting a driver condition based on the detection data by the detection device and on driver inattention check logic during vehicle travel. The driver monitoring device may execute the driver inattention check logic when, based on the detection data, a dangerous driving situation occurs during vehicle travel or the driver is not looking ahead of the vehicle.

[0029] According to one embodiment, the detection device may comprise a photographing device for photographing a face of the driver, a first sensor for detecting that the driver is holding a steering wheel, and a second sensor for detecting an eye of the driver.

[0030] According to another aspect of the present disclosure, a method for monitoring a driver may include: determining, by a processor, whether a driving situation is a dangerous driving situation or a normal driving situation based on detection data during vehicle travel, determining, by the processor, whether the driver is looking forward based on the detection data during vehicle travel in the normal driving situation, and performing, by the processor, driver inattention check logic when the driver is not looking forward or when the driving situation is the dangerous driving situation.

[0031] According to one embodiment, determining whether the driving situation is the dangerous driving situation or the normal driving situation may comprise: determining whether the driving situation is the dangerous driving situation using at least one of the detection data, navigation information, or weather information during the vehicle driving.

[0032] According to a further embodiment, performing the driver inattention check logic may include issuing a driver inattention check execution element to execute the driver inattention check logic, determining whether the driver inattention check execution element is performed by the driver based on the sensing data during vehicle travel, requesting a transfer of control from a system to the driver if the driver inattention check execution element is not performed, and, if the driver does not receive control, terminating a self-driving function (e.g., autonomous driving function) and controlling a deceleration stop in a road or a stop on a side road and / or side of the road.

[0033] Further areas of applicability will become apparent from the description provided herein. It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present invention. List of characters

[0034] In order that the invention may be thoroughly understood, various embodiments thereof will now be described, given by way of example, with reference to the accompanying drawings, in which: Fig. 1 is a block diagram illustrating components of a vehicle system having a device for monitoring a driver (hereinafter referred to as a “driver monitoring device”), Fig. 2 is a view showing images divided according to levels of autonomous driving, Fig. 3 is a view showing an image for monitoring a driver based on a camera image, Fig. 4 is a view showing an image for monitoring a driver based on a thermal sensor, Fig. 5 is a view showing an image for monitoring a driver based on an infrared sensor, Fig. 6 is a flowchart illustrating a method for monitoring a driver during autonomous driving, and Fig. 7 is a view illustrating a data processing system.

[0035] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present invention in any way. Detailed description

[0036] The following description is merely exemplary in nature and is not intended to limit the disclosure, application, or use. It is to be understood that corresponding reference characters indicate similar or corresponding parts and features throughout the drawings.

[0037] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. It should be noted that when reference numerals are added to the components of each drawing, the same or equivalent components will be designated by the same reference numeral even if they are shown in different drawings. Furthermore, in describing the embodiments of the present disclosure, a detailed description of well-known features or functions may be omitted in order not to unnecessarily obscure the gist of the present disclosure.

[0038] When describing elements of exemplary embodiments of the present invention, the terms 1st, 2nd, first, second, A, B, (a), (b), and the like may be used herein. These terms are used merely to distinguish one element from another, but do not limit the corresponding elements or indicate an order or priority between the corresponding elements. Unless otherwise defined, all terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention / disclosure belongs.Such terms as those present in a commonly used dictionary should be interpreted as having meanings equivalent to the context-dependent meanings in the relevant technical field and should not be interpreted as having ideal or overly formal meanings unless clearly defined in the present application as having such meanings.

[0039] The present disclosure discloses a technology of improving the reliability of the monitoring result for the driver's inattention state by determining whether the driver is looking forward during vehicle travel using a sensor and logic for detecting (checking) the driver's inattention.

[0040] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the Fig. 1 to Fig. 7 described.

[0041] Fig. 1 is a block diagram illustrating components of a vehicle system including a device for monitoring a driver (hereinafter referred to as a “driver monitoring device”) according to an embodiment of the present disclosure.

[0042] Referring to Fig. 1 the vehicle system may include the driver monitoring device 100 and a detection device 200 have.

[0043] The driver monitoring device 100 can monitor a driver condition based on detection data or logic (driver inattention check logic) for checking a driver's inattention during self-driving or autonomous driving. In other words, the driver monitoring device can 100Execute the driver inattention verification logic if it is determined based on the detection data that a dangerous driving situation is occurring or if it is determined that the driver is not looking ahead of the vehicle.

[0044] For this purpose, the driver monitoring device 100 a communication device 110 , a storage device 120 , a display device 130 and a processor 140 have.

[0045] The communication device 110 is a hardware device implemented with various electronic circuits to send and receive a signal via a wireless or wired connection. According to the present disclosure, the communication device may 110In-vehicle communication can be carried out via CAN communication (where CAN stands for “Controller Area Network”) or via LIN communication (where LIN stands for “Local Interconnect Network”), Ethernet communication.

[0046] The storage device 120 a detection result of the detection device 200 and a result of the investigation, which is carried out by the processor 140 obtained from a driving situation. The storage device 120can be implemented with at least one storage medium from a memory of a flash memory type, a hard disk type, a micro type, a card type (e.g. an SD card (SD = Secure Digital) or an XD card (XD = eXtreme Digital)), a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), a programmable ROM (PROM), an electrically erasable and programmable ROM (EEPROM), a magnetic RAM (MRAM), a magnetic disk type memory and an optical disk type memory.

[0047] The display device may display a self-driving situation, a screen for notifying the start of the driver inattention check logic, or a driver inattention check execution element for executing the driver inattention check logic, so that the driver recognizes the self-driving situation, the start of the driver inattention check logic, or the driver inattention check execution element. The display device 130 can be implemented by means of a head-up display (HUD), an instrument cluster (e.g., a partially or fully digital instrument cluster), an audio / video / navigation device (AVN device), or a human-machine interface (HMI). The display device 130may further comprise at least one of a liquid crystal display (LCD), a thin-film transistor liquid crystal display (TFT-LCD), a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, an active matrix OLED (AMOLED), a flexible display, a curved display, or a three-dimensional (3D) display. Among these, some display devices may be implemented by means of transparent displays configured by a transparent type or see-through type so that the displays are viewed from the outside. In addition, the display device 130 for example, with a touch-sensitive screen having a touch panel, so that it can be used as an input device in addition to an output device.

[0048] The processor 140 can be electrically connected to the communication device 110 , the storage device 120and the display device 130 connected, can electrically control any component and can be an electrical circuit that executes software instructions. Accordingly, the processor 140 perform various data processing and calculations, which are described below.

[0049] The processor 140 can monitor the driver condition based on the detection data or the driver inattention check logic while the vehicle is driving. In other words, the processor can 140 Execute the driver inattention verification logic if, based on the detection data during autonomous driving, it is determined that the dangerous driving situation occurs or it is determined that the driver is not looking ahead.

[0050] The processor 140can determine that the dangerous driving situation is occurring using at least one of detection data, navigation information, or weather information while driving autonomously. In this case, the dangerous driving situation can include at least one of an intersection entry situation, a school zone entry situation, a high-accident area entry situation (e.g., an area known or marked for frequent and / or serious accidents), a vehicle traffic flow change situation, a sudden vehicle braking situation due to a vehicle cutting in, a bad weather situation (e.g., rain showers, stormy weather, snowfall, freezing rain, hailstorms, etc.), a sharp turn start situation (e.g., cornering through a sharp or tight curve), or a motorway and / or federal highway approach situation.

[0051] The processor 140 can determine the dangerous driving situation at a preset unit time interval (e.g. regularly / periodically after a uniform time interval) and can adapt the unit time interval depending on the road being traveled (e.g. a roadway and / or type of road being traveled) or the traffic situation. In other words, the processor can 140 set the unit time interval to be longer when the vehicle is traveling on a highway, and can set the unit time interval to be shorter when the vehicle is traveling in a city (or performing a city trip) or in a traffic congestion state. Even if the vehicle is traveling on a highway, the processor can 140 For example, set the unit time interval so that it is shorter than during city driving when the vehicle is in the congestion area.

[0052] The processor 140may announce (e.g., notify the user) the start or beginning of the driver inattention monitoring logic using at least one of a haptic signal, an audible signal, or a visual signal. The processor 140 may contain at least one of the size of a 130 displayed warning symbol, change in movement of the warning symbol, change in color of the warning symbol, change in color of a mood lamp, type of audible warning tone, volume of the audible alarm tone or ringing duration (e.g. ringing duration) of the audible alarm tone depending on a dangerous driving situation and a general driving situation.

[0053] The processor 140a driver inattention check execution element (e.g. driver inattention check performing element, e.g. driver executable element, object, point, etc.) for executing the driver inattention check logic can be sent to the display device 130 In this case, the display device 130 a HUD or a front display device (e.g., front display, such as an instrument cluster) located at the front of the vehicle. The processor 140 can therefore determine whether the driver inattention check execution item is performed (e.g., performed by the driver) in the state that the driver is looking forward. In this case, the driver inattention check execution item may include an action that is performable (e.g., by the driver) while the driver is holding a steering wheel.

[0054] If the driver inattention check logic is repeated in the state where the vehicle is in the dangerous driving situation, the processor 140 prompt the driver to perform an action for a driver inattention check execution item that is different from the driver inattention check execution item that was previously performed.

[0055] The processor 140 may prompt the driver to perform a driver inattention verification execution element using a voice if the dangerous driving situation persists after the driver inattention verification logic is performed in the dangerous driving situation.

[0056] The processor 140can determine whether the driver inattention check execution item is performed by the driver based on the detection data during the vehicle driving.

[0057] The processor 140 may request transfer of control from a system to the driver if the driver inattention verification execution element is not performed by the driver.

[0058] The processor 140 can stop the self-driving function and can command a deceleration stop in a street or stopping on a side street and / or side of the road if the driver does not obtain control (e.g. does not take over).

[0059] The processor 140 can provide feedback to the driver by operating a sunroof and / or a side window after confirming a steering wheel operation, thus awakening the driver.

[0060] The detection device200 may include a photography device, such as a camera, for photographing the driver's face, a first sensor for detecting that the driver is holding the steering wheel, and a second sensor for detecting the driver's eyes.

[0061] The first sensor may include a thermal sensor for detecting heat generated by the steering wheel, a capacitive sensor for detecting the capacitance of the steering wheel, or a torque sensor for detecting changes in torque. The first sensor may include any of a variety of sensors for detecting whether the driver is holding the steering wheel with the driver's hand.

[0062] The second sensor, which has an infrared sensor or an electric field sensor for detecting eye movement, may include various sensors for detecting the eyes.

[0063] In addition, the detection device 200have a rain sensor or a brightness sensor to detect the weather.

[0064] As described above, although a system for monitoring the driver using an existing sensor requires additional sensor costs and has problems in terms of sensor output and operational reliability, the present disclosure can improve the reliability of the driver monitoring result without requiring additional material costs, and can expand a target to be feedback-controlled (e.g., a selection of such targets). For example, if the fact that the driver is not looking forward is determined to be related to the driver's fatigue, the sunroof or side window can be operated after confirming the steering wheel operation, thus awakening the driver.

[0065] Fig. 2 is a view illustrating images divided according to stages of autonomous driving according to an embodiment of the present disclosure.

[0066] The levels of autonomous driving can be defined as Level 0, Level 1, Level 2, Level 3, Level 4, and Level 5. Level 0 is a non-autonomization level, which represents a mode in which the driver always drives (independently). Level 1 is a driver assistance level, and the system supports steering or deceleration and acceleration control. In this case, the driver must maintain a hands-on state at Level 0 and Level 1. Level 2 is a partial automation level, in which the system performs steering and deceleration and acceleration control, and the driver's hands-off state may be possible.

[0067] Level 3 is a conditional automation level, where the driver is only involved in the dangerous driving situation and must look ahead. Level 4 is a high automation level, where the driver's involvement is not necessary, and Level 5 is a full automation level, where driver involvement is not required.

[0068] The present disclosure provides a technology for monitoring whether a driver is looking ahead in the condition automation stage, which is Level 3, wherein monitoring is performed primarily based on a sensor using a camera, and driver inattention check logic is performed secondarily when a dangerous driving situation occurs or when the driver is not detected looking ahead, thereby performing driver monitoring twice. Accordingly, the reliability of the monitoring result can be increased.

[0069] Fig. 3 is a view illustrating an image for monitoring a driver based on a camera image according to an embodiment of the present disclosure, Fig. 4 is a view illustrating an image for monitoring a driver based on a thermal sensor according to an embodiment of the present disclosure, and Fig. 5 is a view illustrating an image for monitoring a driver based on an infrared sensor according to an embodiment of the present disclosure.

[0070] The driver monitoring device 100 can determine whether the driver is looking forward by detecting the angle of the face and the direction of the pupil of the eye when the driver’s face is detected from the camera image, as in Fig. 3. However, the provision of multiple cameras is limited in terms of cost and installation space.

[0071] The driver monitoring device 100 can be determined using the heat sensors 402 and 403 which are integrated into the steering wheel 401installed (e.g. embedded in it), determine whether the driver has turned the steering wheel 401 holds, as in Fig. 4, and can determine that the driver is looking forward when the driver turns the steering wheel 401 However, detection errors may be caused because drivers have different stopping positions and habits, and an abnormal situation may be caused that a hands-off self-driving system is detected as being in a hands-on state.

[0072] The driver monitoring device 100 can determine whether the driver is looking forward by detecting the driver’s eyes using the infrared sensor, as shown in Fig. 5. Reference numerals 501 is an example of detecting the diameter of an eye, and reference numerals 502An example is determining whether the driver is looking forward by detecting the contour of the eye. However, if the driver wears sunglasses or glasses, detection errors may occur depending on the light and refractive index, and the eye detection rate may be altered because drivers have different eye shapes.

[0073] Therefore, according to the present disclosure, whether the driver is looking forward is monitored first or primarily through cameras, thermal sensors, or infrared sensors. If it is detected that the dangerous driving situation is occurring or that the driver is looking forward after the primary monitoring is performed, the driver inattention check logic (or driver attention check logic) is performed secondarily, thereby performing dual monitoring of the driver. Accordingly, the reliability of the monitoring result can be improved.

[0074] A method for monitoring a driver during autonomous driving is described below with reference to Fig. 6 according to an embodiment of the present disclosure. Fig. Figure 6 is a flowchart illustrating the method for monitoring the driver during autonomous driving.

[0075] It is assumed below that the driver monitoring device 100 from Fig. 1 the procedure of Fig. 6. It should also be understood that in the following, by reference to Fig. 6, the process described as being carried out by the driver monitoring device 100 described, carried out by the processor 140 the driver monitoring device 100 is controlled.

[0076] Referring to Fig. 6 carries out the driver monitoring device 100 performs driver monitoring based on a camera image to monitor the driver ( S101 ).

[0077] Consequently, the driver monitoring device determines 100 whether a current situation is a normal driving situation (S102) If the current situation is not the normal driving situation, the driver monitoring device determines100 the dangerous driving situation or the existence of the dangerous driving situation ( S105 ).

[0078] In this case, the normal driving situation is not the dangerous driving situation, and the driver monitoring device 100 can determine whether the current situation is the dangerous driving situation by using the detection device 200, such as a brightness sensor, a vehicle speed sensor or a rain sensor, navigation information or infotainment weather information. In this case, the dangerous driving situation can be at least one of a situation of entering an intersection, a situation of entering a school zone, a situation of entering an accident black spot (e.g.an area known or marked for frequent and / or serious accidents), a situation of a change in vehicle traffic flow, for example a situation of deceleration from 100 km / h to 60 km / h when driving on a motorway, a situation of sudden deceleration of the vehicle due to another vehicle merging, a situation of detection of a sudden change in cloudy weather by a rain sensor and a brightness sensor, a situation of receiving information about bad weather, a situation before the vehicle starts to enter a sharp bend based on navigation information or a situation of approaching a motorway and / or federal road.

[0079] In this case, the driver monitoring device 100determine at a preset unit time interval whether the current situation is the normal driving situation. The driver monitoring device 100 may also set a period (unit time interval) for determining whether the vehicle is in the normal driving situation to be longer when the vehicle is driving on the highway, and may set the period (unit time interval) to be shorter when driving in an urban area. If the vehicle is capable of performing a higher-level self-driving function (e.g., autonomous driving with a higher level of autonomy), the driver monitoring device may 100 adjust the period to determine whether the current situation is the normal driving situation, so that it is longer, so that it is suitable / appropriate for the level of independent or autonomous driving of the vehicle.

[0080] If in step S102the current situation is the normal driving situation, then the driver monitoring device determines 100 based on image data obtained from the camera, whether the driver is looking forward ( S103 ).

[0081] In the state that the driver is looking forward, the driver monitoring device 100 continuously monitor the driver using the camera.

[0082] However, if it is not determined based on the camera image data that the driver is looking forward, the driver monitoring device 100 perform the driver inattention check (driver distraction check) logic ( S104 ).

[0083] In other words, when the abnormal inattention of the driver is detected when monitoring the driver based on the image from the camera, then the driver inattention check (driver distraction check) logic is performed to monitor the driver with the two levels.

[0084] If the current situation is the normal driving situation, but the driver is not looking forward, or if the current situation is the dangerous driving situation, the driver monitoring device 100 perform the driver inattention check (driver distraction check) logic ( S104 ).

[0085] The driver monitoring device 100 announces (e.g. to the driver) the start or beginning of the driver inattention check (driver distraction check) logic ( S106 ) and outputs the driver inattention check execution element for checking driver inattention ( S107). In this case, the driver monitoring device 100 output the driver inattention check execution item to the HUD at the front portion of the vehicle or the front display device, thereby determining whether the driver normally performs the driver inattention check execution item in the state that the driver is looking forward.

[0086] In this case, the driver monitoring device notifies 100 the driver about the start of the driver inattention check (driver distraction check) logic, so that the driver has time to prepare for the driver inattention check (driver distraction check) logic. The driver monitoring device 100 can initiate the driver inattention check (driver distraction check) logic in the form of a displayed message (e.g. pop-up message) on the display device 130or announce in audible or tactile form (e.g., communicate, report). The driver monitoring device 100 may also announce the start of the driver inattention check (driver distraction check) logic using at least one of a haptic signal (seat or steering wheel vibration), an audible signal, or a visual signal (an output by AVN device, HUD, or a mood lamp).

[0087] Furthermore, the driver monitoring device 100 provide information to the driver to distinguish between the normal driving situation and the dangerous driving situation so that the driver recognizes the current situation (e.g. the driver is notified of the current situation).

[0088] The driver monitoring device 100 For example, the size of an image on the display device 130displayed warning symbol, a change in the movement of the warning symbol, a color differentiation with regard to the color of the warning symbol, a color change (green → red) with regard to the color of a mood lamp, the type of an audible warning tone, the volume of the audible warning tone or the ringing duration (e.g. ringing duration) of the audible warning tone, so that the driver intuitively recognizes the current driving situation.

[0089] The driver monitoring device 100 can be displayed on the display 130display the driver inattention check execution element. The driver inattention check execution element may include an action to be performed by the driver while holding the steering wheel. Furthermore, the driver inattention check execution element may include an action performed when the driver's hand is positioned on the steering wheel and when the driver's eyes are directed at the HUD. For example, the action performed when the driver's hand is positioned on the steering wheel and when the driver's eyes are directed at the HUD may include a function of recognizing voice input information by receiving a wiper operation (e.g., a corresponding voice command), audio volume control (e.g., a corresponding voice command), or a current speed through a microphone (e.g.,a voice input to state a current speed through a microphone), a function performed by a button on the steering wheel, such as a light-on function (also e.g. a wiper operation and / or audio volume control through e.g. levers / buttons on the steering wheel), or a function performed by a multifunction button.

[0090] Alternatively, if the driver inattention check logic is repeated secondarily and periodically in the normal driving situation, then the driver monitoring device requests 100 an action which is followed by an action which first appears on the display device 130 output, so that the driver can concentrate on driving. In addition, the driver monitoring device can 100 change the color of a mood light and the audible warning tone to improve attention.

[0091] Furthermore, if secondary driver inattention verification logic is first executed in the dangerous driving situation and then an additional determination is desired because it is determined that the dangerous driving situation persists, then further secondary driver inattention verification logic can be executed using only a speech result to separate the logic from a portion required for driving (e.g., separate the actions required for the driver inattention verification logic from the actions required for driving the vehicle) so that driving is not disrupted and / or interrupted. For example, the driver inattention verification execution element can be specified as "Make this sound three times" or "Say the term 'front' three times."

[0092] Consequently, the driver monitoring device 100determine whether the driver inattention check execution element is performed normally ( S108 ). If the driver inattention check execution item is not performed normally, the driver monitoring device requests 100 the system to transfer control to the driver ( S109 ). In this case, the driver monitoring device 100 Using the image from the camera, determine whether the driver inattention verification execution item is performed normally within a preset time.

[0093] For example, if the “wiper operation” is required as the driver inattention check execution element, the driver monitoring device 100determine that the driver inattention check execution item is performed normally when the driver performs the wiper operation normally (e.g., the driver turns the windshield wiper on and / or off). If the wiper operation is not performed normally, the driver monitoring device determines 100 that the driver inattention check execution item is not performed normally.

[0094] If control is then normally transferred from the system to the driver, the independent or autonomous driving is terminated. Consequently, the driver monitoring device 100 stop monitoring the driver.

[0095] If, however, the control is not normally transferred from the system to the driver within the preset time, the driver monitoring device detects 100that the driver is not in a normal state, and may therefore determine that activation of the self-driving function is impossible, so that a control operation for driving with minimizing danger (a deceleration stop within a road or stopping on a secondary road and / or side of the road) is performed. For example, according to the result obtained by executing the driver inattention check execution item using at least two paths, an audible alarm is sounded for 15 seconds and a control transfer request (system → driver) is initiated if the driver's action / action is not detected for 180 seconds or more.

[0096] As described above, according to the present disclosure, the driver inattention check execution item is output to the driver without adding a physical sensor, and it is checked whether the driver normally performs the driver inattention check execution item, thereby monitoring the driver with high reliability.

[0097] According to the present disclosure, an actuation target, such as a sunroof and the steering wheel, may also be extended for feedback when the driver is not looking forward.

[0098] Fig. 7 illustrates a data processing system according to an embodiment of the present disclosure.

[0099] A data processing system (e.g., computing system) 1000 may, with reference to Fig. 7 at least one processor 1100 , a memory 1300 , a user interface input device 1400, a user interface output device 1500 , a (data) storage device 1600 and a network interface 1700 , which is accessible via a bus 1200 are connected to each other.

[0100] The processor 1100 may be a central processing unit (CPU) or a semiconductor device for processing instructions stored in the memory 1300 and / or the storage device 1600 are stored. The memory 1300 and the storage device 1600 can each contain numerous types of volatile or non-volatile storage media. For example, the memory 1300 have a read-only memory (ROM) and a random access memory (RAM).

[0101] The operations / processes of the methods or algorithms described in connection with the embodiments disclosed in the present disclosure can therefore be implemented directly by means of a hardware module, a software module or combinations thereof, which can be implemented by the processor 1100 The software module may be located in a storage medium (ie the memory 1300 and / or the storage device 1600 ), such as a RAM, a flash memory, a ROM, an erasable and programmable ROM (EPROM), an electrical EPROM (EEPROM), a register, a hard disk, a removable disk or a compact disc-ROM (CD-ROM).

[0102] The exemplary storage medium can be connected to the processor 1100 connected. The processor 1100can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can be connected to the processor 1100 be integrated. The processor and storage medium can be located in an application-specific integrated circuit (ASIC). The ASIC can be located in a driver terminal. Alternatively, the processor and storage medium can be located as separate components within the driver terminal.

[0103] Although the present disclosure has been described above with reference to exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto, but can be modified and altered in various ways by those skilled in the art to which the present disclosure belongs, without departing from the spirit and scope of the present disclosure as claimed in the following claims.

[0104] The exemplary embodiments of the present disclosure are therefore provided to explain the spirit and scope of the present disclosure, but not to limit it, so that the spirit and scope of the present disclosure is not limited by the embodiments. The scope of the present disclosure / invention should be interpreted based on the appended claims, and all technical ideas within the scope equivalent to the claims are intended to be included within the scope of the present disclosure.

[0105] As described above, according to the present technology, the reliability of the driver monitoring result can be improved by monitoring the driver based on the detection data and the driver inattention checking logic during the vehicle driving.

[0106] Furthermore, a variety of effects can be provided which are understood directly or indirectly from the disclosure.

[0107] Although the present disclosure has been described above with reference to exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto, but can be modified and altered in various ways by those skilled in the art to which the present disclosure belongs, without departing from the spirit and scope of the present disclosure as claimed in the following claims. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] KR 1020190121819

[0001]

Claims

[1] Device (100) for monitoring a driver in a vehicle, the device comprising: a processor (140) configured to monitor a driver condition based on detection data and driver inattention check logic during vehicle travel, and a display device (130) arranged to display a message indicating a start of the driver inattention check logic, wherein the processor (140) is further configured to: to perform the driver inattention verification logic if, based on the detection data, a dangerous driving situation occurs while the vehicle is moving or the driver is not looking ahead of the vehicle. [2] The apparatus of claim 1, wherein the processor (140) is configured to: to determine whether the dangerous driving situation occurs using at least one of the detection data, navigation information or weather information while the vehicle is driving. [3] Device according to claim 1 or 2, wherein the dangerous driving situation comprises: at least one of a situation of entering an intersection, a situation of entering a school zone, a situation of entering an accident black spot, a situation of a change in vehicle traffic flow, a situation of sudden braking of the vehicle, a situation of bad weather, a situation of the beginning of a sharp curve or a situation of approaching a motorway and / or federal highway. [4] Apparatus according to any one of claims 1 to 3, wherein the processor (140) is arranged to: to determine the dangerous driving situation at a preset unit time interval. [5] The apparatus of claim 4, wherein the processor (140) is configured to: to adjust the preset unit time interval depending on a busy road or traffic situation. [6] The apparatus of claim 5, wherein the processor (140) is configured to: to increase the preset unit time interval when the vehicle is driving on a highway, and to reduce the preset unit time interval when the vehicle is driving in a city or in a traffic congestion area. [7] Apparatus according to any one of claims 1 to 6, wherein the processor (140) is arranged to: announce the initiation of the driver inattention verification logic using at least one of a haptic signal, an audible signal, or a visual signal. [8] Apparatus according to any one of claims 1 to 7, wherein the processor (140) is arranged to: to change at least one of a size of a warning symbol displayed on the display device (130), a movement change of the warning symbol, a color change of the warning symbol, a color change of a mood lamp, a type of an audible warning tone, a volume of the audible warning tone, or a ringing duration of the audible warning tone based on the dangerous driving situation and a general driving situation. [9] Apparatus according to any one of claims 1 to 8, wherein the processor (140) is arranged to: to output on the display device (130) a driver inattention check execution element for performing the driver inattention check logic. [10] The device according to claim 9, wherein the display device (130) comprises a head-up display device or a front display device, and the processor (140) is configured to: to determine whether the driver inattention check execution item is performed in a state that the driver is looking forward. [11] The apparatus according to claim 9 or 10, wherein the driver inattention check execution element comprises an action which is performable while the driver is holding a steering wheel. [12] The apparatus of claim 11, wherein the processor (140) is configured to: when the driver inattention check logic is repeated in a state in which the vehicle is not in the dangerous driving situation, prompt the driver to perform an action for a driver inattention check execution item that is different from the driver inattention check execution item that was previously performed. [13] Apparatus according to claim 11 or 12, wherein the processor (140) is arranged to: if the dangerous driving situation persists after the driver inattention check logic is executed in the dangerous driving situation, prompt the driver to perform a driver inattention check execution element using a voice. [14] Apparatus according to any one of claims 11 to 13, wherein the processor (140) is arranged to: to determine whether the driver inattention check execution item is performed by the driver based on the detection data during the vehicle drive. [15] Apparatus according to any one of claims 11 to 14, wherein the processor (140) is arranged to: request a transfer of control from a system to the driver if the driver inattention verification execution element is not performed by the driver. [16] The apparatus of claim 15, wherein the processor (140) is configured to: if the driver does not gain control, to stop a self-driving function and to control a deceleration stop in a road or a stop on a secondary road and / or side of the road. [17] Apparatus according to any one of claims 1 to 16, wherein the processor (140) is arranged to: to provide feedback to the driver by operating a sunroof or a side window after confirming a steering wheel operation when the driver is not looking forward. [18] Vehicle system, comprising: a detection device (200) which is arranged to detect whether a driver of a vehicle is looking in front of the vehicle, and a vehicle monitoring device (100) configured to detect a driver condition based on the detection data by means of the detection device (200) and on a driver inattention check logic during vehicle travel, wherein the driver monitoring device (100) is configured to perform the driver inattention check logic when, based on the detection data, a dangerous driving situation occurs during vehicle travel or the driver is not looking ahead. [19] The vehicle system of claim 18, wherein the detection device (200) comprises: a photographic device configured to photograph a face of the driver, a first sensor configured to detect that the driver is holding a steering wheel, and a second sensor which is designed to detect one of the driver's eyes. [20] A method for monitoring a driver of a vehicle, the method comprising: Determining (S102) by a processor (140) whether a driving situation is a dangerous driving situation or a normal driving situation based on detection data during a vehicle journey, Determining (S103) by the processor (140) whether the driver is looking forward based on the detection data during vehicle travel in the normal driving situation, and Performing (S104), by means of the processor (140), a driver inattention check logic when the driver is not looking forward or when the driving situation is the dangerous driving situation. [21] The method according to claim 20, wherein determining (S102) whether the driving situation is the dangerous driving situation or the normal driving situation comprises: Determine whether the driving situation is the dangerous driving situation using at least one of the detection data, navigation information or weather information during the vehicle drive. [22] The method of claim 20 or 21, wherein performing (S104) the driver inattention check logic comprises: Outputting (S107) a driver inattention check execution element to execute the driver inattention check logic, Determining (S108) whether the driver inattention check execution item is performed by the driver based on the detection data during the vehicle travel, Requesting (S109) a transfer of control from a system to the driver if the driver inattention check execution element is not performed, and if the driver does not gain control, terminate an autonomous driving function and control a deceleration stop in a road or a stop on a secondary road and / or side of the road.

Images