Fatigue driving management device, system including the device and method thereof

Abstract

The present disclosure relates to a fatigue driving management device, a system including the device, and a method thereof. The fatigue driving management device includes: a processor configured to determine whether a user's slow eye movement occurs and whether a steering torque does not change, and when the user's slow eye movement occurs and when the steering torque does not change within a predetermined time period, determine that the user is driving fatigued; and a memory storing information indicating whether the slow eye movement occurs and a result of determining whether the steering torque does not change.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of Korean Patent Application No. 10-2019-0039170 filed on April 3, 2019, in the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a fatigue driving management device, a system including the fatigue driving management device, and a method thereof, and more particularly to a technology for early and accurate determination of fatigue driving based on slow eye movement and vehicle signals. Background Art

[0004] The fatigue driving warning function has been expanded and applied, and the fatigue driving warning function is used to determine whether the user is driving fatigued in conjunction with vehicle services and issue a fatigue driving warning to the user.

[0005] This fatigue driving warning function has the accuracy of determining fatigue driving and the time when the warning occurs as important performance indicators. Specifically, the time when the warning occurs has high availability only when the user falls asleep (at the beginning of sleep), but the current commercial technology cannot meet this requirement.

[0006] The fatigue driving warning functions currently mass-produced are roughly divided into two methods, including an indirect method that analyzes driving patterns and determines fatigue driving, and a direct method that determines fatigue driving based on the driver's open / closed eye image.

[0007] Because the indirect method based on the driving pattern has low accuracy and a high difference between the warning time and the onset of sleep, the usability is reduced. On the other hand, in the technology of determining fatigue driving based on the driver's open / closed eye image as a direct method, since a warning is generated by detecting a long period of closed eyes, it has high accuracy, but when the driver closes his eyes for a relatively long time on the rear road, the danger level may be high. Therefore, it is necessary to accurately determine whether the driver is driving fatigued at the onset of fatigue and warn the driver of fatigue driving. Summary of the invention

[0008] The present disclosure has been made to solve the above-mentioned problems occurring in the prior art while completely maintaining the advantages achieved by the prior art.

[0009] One aspect of the present disclosure provides a fatigue driving management device for early and accurate detection of a user's fatigue driving based on slow eye movement and an interval in which a steering torque is not changed by a vehicle signal, and also provides a system of the fatigue driving management device and a method thereof.

[0010] The technical problems to be solved by the present inventive concept are not limited to the above-mentioned problems, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure belongs from the following description.

[0011] According to an exemplary embodiment of the present disclosure, a fatigue driving management device may include: a processor configured to determine whether a user's slow eye movement occurs and whether the steering torque does not change, so as to determine that the user is driving fatigued when the user's slow eye movement occurs and when the steering torque does not change within a predetermined time period; and a memory storing information indicating whether the slow eye movement occurs and the result of determining whether the steering torque does not change.

[0012] When there is no user manipulation in a state where a slow eye movement of the user occurs and the steering torque does not change within a predetermined period of time, the processor may determine that the driver is driving in fatigue.

[0013] The processor may calculate eye movement information relative to the vehicle based on information about the user's fixation and information about the user's head posture based on the image data, and may calculate eye movement information relative to the external environment based on the eye movement information relative to the vehicle.

[0014] The information about fixation may include a gaze yaw angle and a gaze pitch angle. The information about head posture may include a head posture yaw angle and a head posture pitch angle.

[0015] The processor may calculate an eye movement yaw angle relative to the vehicle in the eye movement information by subtracting the head pose yaw angle from the gaze yaw angle, and may calculate an eye movement pitch angle relative to the vehicle in the eye movement information by subtracting the head pose pitch angle from the gaze pitch angle.

[0016] The processor may calculate the eye movement yaw angle relative to the external environment in the eye movement information relative to the external environment using the eye movement yaw angle relative to the vehicle and the yaw rate of the vehicle, and may calculate the eye movement pitch angle relative to the external environment in the eye movement information relative to the external environment using the eye movement pitch angle relative to the vehicle and the pitch angle of the vehicle.

[0017] The processor may calculate the eye movement distance within a predetermined time period using a value obtained by adding the eye movement yaw angle relative to the external environment and the eye movement pitch angle relative to the external environment, and determine that slow eye movement has occurred when the eye movement distance is less than a predetermined reference value.

[0018] When the output value of the steering torque sensor is less than a predetermined reference value, the processor may determine that the steering torque has not changed.

[0019] According to another exemplary embodiment of the present disclosure, a vehicle system may include: a fatigue driving management device, configured to: determine whether slow eye movement of the user occurs and whether the steering torque does not change, and when slow eye movement of the user occurs and the steering torque does not change within a predetermined time period, determine that the user is driving in fatigue; and a warning device, which outputs a warning to the user when it is determined that the user is driving in fatigue.

[0020] The warning device may provide at least one or more of a visual warning, a tactile warning, and an audible warning.

[0021] The vehicle system may also include: a camera that detects information about the user's gaze and information about the user's head posture; a yaw pitch sensor that senses the vehicle's yaw rate and the vehicle's pitch angle; a steering torque sensor that senses changes in the vehicle's steering torque; and a decelerator / accelerator pedal sensor that senses changes in the decelerator / accelerator pedal value.

[0022] The fatigue driving management device may include: a processor, which determines whether slow eye movement of the user occurs and whether the steering torque does not change, and determines that the user is driving fatigued when slow eye movement of the user occurs and when the steering torque does not change within a predetermined time period; and a memory, which stores information indicating whether slow eye movement occurs and a result of determining whether the steering torque does not change, the information and result being obtained by the processor.

[0023] When there is no user manipulation in a state where a slow eye movement of the user occurs and the steering torque does not change within a predetermined period of time, the processor may determine that the driver is driving in fatigue.

[0024] According to another exemplary embodiment of the present disclosure, a fatigue driving management method may include: determining whether slow eye movement of the user occurs and whether the steering torque does not change; and when slow eye movement of the user occurs and when the steering torque does not change within a predetermined time period, determining that the user is driving fatigued.

[0025] Determining that the user is driving fatigued may include determining that the driver is driving fatigued when there is no user manipulation in a state where slow eye movement of the user occurs and the steering torque does not change within a predetermined period of time.

[0026] Determining whether slow eye movement of the user occurs and whether the steering torque does not change may include: calculating eye movement information relative to the vehicle based on information about the user's fixation and information about the user's head posture based on image data; and calculating eye movement information relative to the external environment based on the eye movement information relative to the vehicle.

[0027] The information about fixation may include a gaze yaw angle and a gaze pitch angle. The information about head posture may include a head posture yaw angle and a head posture pitch angle.

[0028] Determining whether slow eye movement of the user occurs and whether the steering torque does not change may include calculating the eye movement yaw angle relative to the vehicle in the eye movement information relative to the vehicle by subtracting the head posture yaw angle from the gaze yaw angle, and calculating the eye movement pitch angle relative to the vehicle in the eye movement information relative to the vehicle by subtracting the head posture pitch angle from the gaze pitch angle.

[0029] Determining whether slow eye movement of the user occurs and whether the steering torque does not change may include: calculating the eye movement yaw angle relative to the external environment in the eye movement information relative to the external environment using the eye movement yaw angle relative to the vehicle and the yaw rate of the vehicle, and calculating the eye movement pitch angle relative to the external environment in the eye movement information relative to the external environment using the eye movement pitch angle relative to the vehicle and the pitch angle of the vehicle.

[0030] Determining whether a user's slow eye movement occurs and whether the steering torque does not change may include: using a value obtained by adding an eye movement yaw angle relative to an external environment and an eye movement pitch angle relative to the external environment, calculating an eye movement distance within a predetermined time period, and when the eye movement distance is less than a predetermined reference value, determining that slow eye movement occurs. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the present disclosure will become more apparent through the following detailed description in conjunction with the accompanying drawings:

[0032] Figure 1 is a block diagram showing a configuration of a vehicle system including a fatigue driving management device according to an exemplary embodiment of the present disclosure;

[0033] Figure 2 is a graph showing gaze yaw and vehicle yaw according to an exemplary embodiment of the present disclosure;

[0034] Figure 3 is a graph illustrating exemplary operations of fixation and panning according to an exemplary embodiment of the present disclosure;

[0035] Figure 4 is a graph showing gaze yaw indicated by fixation, saccade, and slow eye movements according to an exemplary embodiment of the present disclosure;

[0036] Figure 5 is a flowchart showing a fatigue driving management method according to an exemplary embodiment of the present disclosure;

[0037] Figure 6is a detailed flow chart showing an example of calculating eye movements according to an exemplary embodiment of the present disclosure; and

[0038] Figure 7 is a block diagram illustrating a computing system according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. When adding reference numerals to the components of each drawing, it should be noted that even if the same or equivalent components are shown on other drawings, they are represented by the same reference numerals. In addition, when describing the embodiments of the present disclosure, detailed descriptions of well-known features or functions will be excluded so as not to unnecessarily make the subject matter of the present disclosure unclear.

[0040] When describing components according to embodiments of the present disclosure, terms such as first, second, "A", "B", (a), (b), etc. may be used. These terms are intended only to distinguish one component from another, and these terms do not limit the nature, sequence, or order of the constituent components. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by those skilled in the art to which the present disclosure belongs. Terms defined in general dictionaries should be interpreted as having the same meaning as the contextual meaning associated with the relevant field, and should not be interpreted as having an ideal or overly formal meaning unless explicitly defined as having such a meaning in this application.

[0041] Because fatigue relaxes muscles, it is difficult to take purposeful actions, such as driving. Taking the eyes as an example, when fatigued, the movement of the pupils is not expressed as a saccade, and the fixation of the eyes moves at a slow speed. Taking the hands as an example, in the case of fatigue, clear actions cannot be performed due to muscle relaxation, and the grip strength of the hands is reduced. Therefore, an exemplary embodiment of the present disclosure may disclose a technology that determines fatigue driving at the beginning of fatigue based on the user's slow eye movements and vehicle signals (e.g., steering torque and information indicating whether various switches have been manipulated) and warns the user to avoid accidents due to fatigue driving.

[0042] In the following, reference will be made to Figures 1 to 7 Embodiments of the present disclosure are described in detail.

[0043] Figure 1 is a block diagram showing a configuration of a vehicle system including a fatigue driving management device according to an exemplary embodiment of the present disclosure.

[0044] Reference Figure 1 , a vehicle system according to an exemplary embodiment of the present disclosure may include a fatigue driving management device 100 , a sensing module 200 , and a warning device 300 .

[0045] The fatigue driving management device 100 may determine whether the user's slow eye movement occurs and whether the steering torque does not change. When the user's slow eye movement occurs and the steering torque does not change within a predetermined period of time, the fatigue driving management device 100 may determine that the user is fatigue driving.

[0046] In addition, when there is no user manipulation in a state where the user's slow eye movement occurs and the steering torque does not change within a predetermined period of time, the fatigue driving management device 100 may determine that the user is driving fatigued.

[0047] The fatigue driving management device 100 may include a communicator 110 , a memory 120 , and a processor 130 .

[0048] The communicator 110 may be a hardware device implemented with various electronic circuits to send and receive signals through wireless or wired connections. In an embodiment of the present disclosure, the communicator 110 may perform inter-vehicle communication through a controller area network (CAN) communication, a local interconnect network (LIN) communication, etc., and may communicate with the sensing module 200 and the warning device 300.

[0049] The memory 120 may store the sensing result of the sensing module 200 obtained by the processor 130 and information indicating whether a slow eye movement occurs, a result of determining whether there is no change in the steering torque, etc. The memory 120 may include at least one type of storage medium, such as a flash memory type memory, a hard disk type memory, a micro memory type, a card type memory (e.g., a secure digital (SD) card or an extreme digital (XD) card), a random access memory (RAM), a static RAM (SRAM), a read-only memory (ROM), a programmable ROM (PROM), an electrically erasable PROM (EEPROM), a magnetic RAM (MRAM), a magnetic disk, and an optical disk.

[0050] The processor 130 may be electrically connected to the communicator 110, the memory 120, the warning device 300, etc., and may electrically control each component. The processor 130 may be a circuit that executes software instructions and may perform various data processing and calculations described below.

[0051] The processor 130 may determine whether the user's slow eye movement occurs and whether the steering torque does not change. When the user's slow eye movement occurs and the steering torque does not change during a predetermined time, the processor 130 may determine that the user is driving fatigued.

[0052] When there is no user manipulation in a state where the user's slow eye movement occurs and the steering torque does not change within a predetermined period of time, the processor 130 may determine that the user is driving fatigued.

[0053] Processor 130 may calculate eye movement information relative to the vehicle based on information about the user's fixation and information about the user's head posture based on image data, and may calculate eye movement information relative to the external environment based on the eye movement information relative to the vehicle.

[0054] The processor 130 may subtract the head posture yaw angle from the gaze yaw angle to calculate the eye movement yaw angle relative to the vehicle in the eye movement information relative to the vehicle. The processor 130 may subtract the head posture pitch angle from the gaze pitch angle to calculate the eye movement pitch angle relative to the vehicle in the eye movement information relative to the vehicle.

[0055] The processor 130 may use the eye movement yaw angle relative to the vehicle and the vehicle yaw rate to calculate the eye movement yaw angle relative to the external environment in the eye movement information relative to the external environment. The processor 130 may use the eye movement pitch angle relative to the vehicle and the vehicle pitch angle to calculate the eye movement pitch angle relative to the external environment in the eye movement information relative to the external environment.

[0056] The processor 130 may calculate the eye movement distance within a predetermined period of time using a value obtained by adding the eye movement yaw angle relative to the external environment to the eye movement pitch angle relative to the external environment. When the eye movement distance is less than a predetermined reference value, the processor 130 may determine that slow eye movement occurs.

[0057] When the output value of the steering torque sensor 230 is less than a predetermined reference value, the processor 130 may determine that there is no change in the steering torque.

[0058] The sensing module 200 may sense information about the user's fixation, about the user's head posture (head movement), a vehicle yaw rate, a vehicle pitch angle, a change in a steering torque of the vehicle, a decelerator / accelerator pedal operation, and the like.

[0059] To this end, the sensing module 200 may include a camera 210 , a yaw / pitch sensor 220 , a steering torque sensor 230 , and a decelerator / accelerator pedal sensor 240 .

[0060] The camera 210 may be a driver state monitor (DSM), and may detect information about a user's gaze and information about a user's head posture.

[0061] The yaw / pitch sensor 220 may sense the vehicle yaw rate and the vehicle pitch angle, and may transmit the sensed information to the fatigue driving management device 100 .

[0062] The steering torque sensor 230 may sense a change in the steering torque of the vehicle, and may transmit the sensed information to the fatigue driving management device 100 .

[0063] The decelerator / accelerator pedal sensor 240 may sense a changed value of the decelerator / accelerator pedal and may transfer the sensed information to the fatigue driving management device 100 .

[0064] Although not in Figure 1 As shown in , the vehicle system may also include various switches that can be manipulated by the user.

[0065] When it is determined by the fatigue driving management device 100 that the user is driving fatigued, the warning device 300 can be controlled by the fatigue driving management device 100 to output at least one or more of a visual warning, a tactile warning, and an auditory warning to the user, and guide the user to get rid of fatigue, so as to display a screen that advises the user to take a break, advises the user to operate a driving assistance function, or guides the user to a rest area or a sleeping habitat, or outputs a sound to the user. The warning device 300 can be implemented as a head-up display (HUD), a cluster, an audio video navigation (AVN), etc. In addition, the warning device 300 may include at least one of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), a light emitting diode (LED) display, an organic LED (OLED) display, an active matrix OLED (AMOLED) display, a flexible display, a curved display, and a three-dimensional (3D) display. Some of them can be implemented as a transparent display, which is configured as a transparent type or a translucent type to view the outside. In addition, the warning device 300 can be implemented as a touch screen including a touch panel, which will be used as an input device in addition to an output device.

[0066] In this way, embodiments of the present invention can determine the user's fatigue driving with high reliability when fatigue driving begins based on the intervals in which slow eye movement and steering torque do not change. In addition, embodiments of the present disclosure can determine that the driver is driving fatigued when there is no input from the user for manipulating various switches (such as pedal operation) and there are no intervals in which slow eye movement and steering torque do not change.

[0067] Figure 2 is a graph showing gaze yaw and vehicle yaw according to an exemplary embodiment of the present disclosure. Figure 2 , when the vehicle moves while the user is looking at a point outside the vehicle, it may be falsely detected as slow eye movement because it is shown moving slowly relative to a fixed point inside the vehicle.

[0068] Figure 3 is a graph illustrating exemplary operations of fixation and panning according to an exemplary embodiment of the present disclosure. Figure 4 is a graph showing gaze yaw indicated by fixation, saccade, and slow eye movements according to an exemplary embodiment of the present disclosure.

[0069] Reference Figure 3, when a user in a normal state rather than a fatigued state wants to look at one place while looking at another place, he or she may perform an eye movement in the form of quickly moving his or her fixation and looking at another point. This eye movement is called a saccadic movement. When eye movements are measured using a gaze tracking camera, it is Figure 3 On the other hand, when the user is tired or cognitively careless, slow eye movement is displayed instead of saccade. In other words, slow eye movement refers to a phenomenon in which fixation flows between fixation and fixation without saccade, and it is Figure 4 Shown in.

[0070] refer to Figure 4 , when the user looks at a fixed position and moves to another fixed position, the glance is displayed in a straight line. When the user is in a state of fatigue, a slow eye movement state can be seen, which is in a chaotic state in a straight form.

[0071] In the following, reference will be made to Figure 5 A fatigue driving management method according to an embodiment of the present disclosure is described in detail. Figure 5 is a flowchart illustrating a fatigue driving management method according to an embodiment of the present disclosure.

[0072] In the following, it is assumed Figure 1 The fatigue driving management device 100 performs Figure 5 In addition, Figure 5 In the description, the operations described as being performed by the device may be understood as being controlled by the processor 130 of the fatigue driving management device 100.

[0073] Reference Figure 5 In S101, the fatigue driving management device 100 may be based on Figure 1 The image data of the camera 210 is used to detect the fixation as the point at which the user is looking. In this case, because the fixation moves by adding eye movement and head movement, the fatigue driving management device 100 can subtract the fixation caused by the head movement from the fixation, thereby outputting only the fixation caused by the eye movement.

[0074] In S102, the fatigue driving management device 100 can use fixed and Figure 1The value output by the yaw / pitch sensor 220 converts the relative fixation relative to the vehicle into an absolute fixation relative to the external environment. In other words, because the camera 210 is installed on the vehicle, the fixation measured using the camera 210 may be a relative position relative to the vehicle. However, the fixation may be different from the general slow eye movement in a moving vehicle. For example, when the vehicle moves while the user is looking at a point outside the vehicle, the fixation relative to the interior of the vehicle may be mistakenly detected as a slow eye movement because the fixation is shown to move slowly. In order to prevent this phenomenon, the fatigue driving management device 100 may calculate the fixation compensation movement of the vehicle to detect the slow eye movement. The fatigue driving management device 100 may convert the relative fixation relative to the vehicle into an absolute fixation relative to the external environment.

[0075] In S103, the fatigue driving management device 100 may verify slow eye movement based on fixation. When a person in a normal state wants to look at another place while looking at one place, he or she performs eye movement in the form of fast moving fixation and looking at another point. This eye movement is called saccadic movement. When eye movement is measured using a gaze tracking camera, it is Figure 3 On the other hand, the user feels tired or is cognitively careless, showing slow eye movement. Slow eye movement refers to the phenomenon that its fixation flows between fixation and fixation without scanning, and it Figure 4 Shown in.

[0076] In S104, the fatigue driving management device 100 can use Figure 1 The fatigue driving management device 100 can determine whether there is no user manipulation of the decelerator / accelerator pedal and other buttons at moments such as slow eye movement and intervals without steering torque changes to improve detection reliability.

[0077] In S105, the fatigue driving management device 100 may be based on Figure 1 The output values ​​of the decelerator / accelerator pedal sensor 240 and the switch (not shown) are applied to verify the interval without user manipulation.

[0078] In S106 , the fatigue driving management device 100 may determine whether there is a slow eye movement, an interval in which the steering torque does not change, and an interval in which there is no user manipulation at the same time, to determine whether the user is driving fatigued.

[0079] When it is determined in S107 that the user is driving fatigued, the fatigue driving management device 100 can Figure 1 The warning device 300 warns the user of fatigue.

[0080] Figure 6is a detailed flowchart illustrating an example of calculating eye movements according to an exemplary embodiment of the present disclosure.

[0081] Reference Figure 6 , in S201, Figure 1 The fatigue driving management device 100 can be obtained from Figure 1 The information about the user's fixation is detected from the image data of the camera 210. In S202, the fatigue driving management device 100 may detect information about the user's head posture.

[0082] In this case, the information about the user's fixation may include the gaze yaw angle w gaze and the gaze pitch angle v gaze The information about the head posture may include the head posture yaw angle w head and head posture pitch angle v head .

[0083] In S203 , the fatigue driving management device 100 may calculate eye movement information with respect to the vehicle using information about the fixation of the user and information about the head posture of the user.

[0084] In this case, the eye movement information relative to the vehicle may include the eye movement yaw angle w' relative to the vehicle eye and the eye movement pitch angle v' relative to the vehicle eye .

[0085] In this case, the eye movement yaw angle w' relative to the vehicle eye and the eye movement pitch angle v' relative to the vehicle eye It can be calculated by the following equation 1.

[0086] [Equation 1]

[0087] w′ eye =w gaze -w head

[0088] v′ eye =v gaze -v head

[0089] In other words, the fatigue driving management device 100 can be used to monitor the yaw angle w gaze Subtract the head posture yaw angle w from head , to calculate the eye movement yaw angle w' relative to the vehicle eye The fatigue driving management device 100 can be used to monitor the pitch angle v gaze Subtract the head pose pitch angle v head , to calculate the eye movement pitch angle v' relative to the vehicle eye .

[0090] In the gaze yaw angle w due to head posture gaze and the gaze pitch angle v gaze After the movement amount is greater than or equal to a certain ratio (e.g., 30%), the fatigue driving management device 100 may not determine fatigue driving. That is, when the user moves his or her sight while turning his or her face (head), this is not fatigue driving.

[0091] In S204, the fatigue driving management device 100 may Figure 1 The yaw / pitch sensor 220 receives the vehicle yaw rate y car and the vehicle pitch angle v car .

[0092] In S205, the fatigue driving management device 100 may use the eye movement information relative to the vehicle, the vehicle yaw rate y car and the vehicle pitch angle v car To calculate the eye movement information relative to the external environment, as shown in the following equation 2, to convert the relative fixation relative to the vehicle and the absolute fixation relative to the external environment. In this case, the eye movement information relative to the external environment may include the eye movement yaw angle w relative to the external environment eye and the eye movement pitch angle v relative to the external environment eye .

[0093] [Equation 2]

[0094] w eye =w' eye -Jy car

[0095] v eye =v eye -v car

[0096] In other words, when the yaw angle w' is determined according to the eye movement relative to the vehicle eye Evaluate the vehicle yaw rate y car When the fatigue driving management device 100 is used, the integral value may be subtracted within a time period (e.g., 2 seconds) to calculate the eye movement yaw angle w relative to the external environment. eye The fatigue driving management device 100 can be obtained from the eye movement pitch angle v' relative to the vehicle. eye Subtract the vehicle pitch angle v car , to calculate the eye movement pitch angle v relative to the external environment eye .

[0097] In S206 , the fatigue driving management device 100 may determine the slow eye movement based on the total eye movement distance.

[0098] In other words, the fatigue driving management device 100 may determine the eye movement yaw angle w relative to the external environment. eye and the eye movement pitch angle v relative to the external environment eye Is the absolute value of the value obtained by adding equal to the eye movement yaw angle w relative to the external environment to which the cosine function is applied? eye and the eye movement pitch angle v relative to the external environment eye A value obtained by applying the sum of to the arccosine function. When the absolute value is the same as this value, the fatigue driving management device 100 may determine that a slow eye movement has occurred.

[0099] [Equation 3]

[0100] |w eye +v eye |=cos -1 (cos(w eye )+cos(v eye ))

[0101] For example, when the eye movement distance in 2 seconds is greater than or equal to 10 degrees, and there is no eye movement at a speed greater than or equal to 300 degrees / second within 2 seconds, the fatigue driving management device 100 can determine that slow eye movement occurs. When fixation moves at a speed greater than or equal to 300 degrees / second through eye movement, the fatigue driving management device 100 can determine that a saccade occurs.

[0102] In S207, the fatigue driving management device 100 may determine whether there is understeering based on the maximum and minimum values ​​of the steering torque. In other words, the fatigue driving management device 100 may determine whether there is understeering. Figure 1 The output value of the steering torque sensor 230 is kept small to detect whether there is an interval in which the steering torque does not change due to the reduction in the grip force of the user. For example, when the value obtained by subtracting the minimum value of the steering torque from the maximum value of the steering torque within 2 seconds is less than 0.2Nm, and when the maximum value of the absolute value of the steering torque within 2 seconds is less than 0.4Nm, the fatigue driving management device 100 can determine the current state as an understeering state (interval without steering torque).

[0103] In S208, the fatigue driving management device 100 may determine whether the turn signal is working, whether the decelerator / accelerator pedal is changed, or whether various switches are working. For example, when the accelerator pedal compression changes by more than or equal to 5% and when the decelerator pedal compression changes by more than or equal to 10 bar, the fatigue driving management device 100 may determine that there is a user change switch operation.

[0104] When slow eye movement occurs, in an understeering state, and there is no operation of the turn signal, when the decelerator / accelerator pedal does not change, and when there is no operation of various switches, in S209, the fatigue driving management device 100 can determine that the user is driving fatigued.

[0105] In this way, an exemplary embodiment of the present disclosure can determine the user's fatigue driving based on slow eye movements and muscle relaxation as initial fatigue biomarkers with higher reliability than long-term eye opening and closing as existing fatigue driving detection methods, and can provide warnings at a fast and accurate time.

[0106] In addition, when slow eye movements and glances are determined using a camera monitoring the user's face, embodiments of the present disclosure can compensate for the yaw and pitch of the vehicle to prevent erroneous detection of similar slow eye movements caused by vehicle motion, thereby more accurately detecting fatigue driving.

[0107] Figure 7 is a block diagram illustrating a computing system according to an exemplary embodiment of the present disclosure.

[0108] Reference Figure 7 , the computing system 1000 may include at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , a storage 1600 , and a network interface 1700 connected to each other via a bus 1200 .

[0109] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and / or the memory 1600. The memory 1300 and the memory 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) 1310 and a RAM (Random Access Memory) 1320.

[0110] Therefore, the operations of the methods or algorithms described in conjunction with the embodiments disclosed herein may be directly embodied in a hardware or software module executed by the processor 1100, or in a combination thereof. The software module may reside on a storage medium such as RAM memory, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, and CD-ROM (i.e., memory 1300 and / or memory 1600).

[0111] An exemplary storage medium may be coupled to the processor 1100, and the processor 1100 may read information from the storage medium and may record information in the storage medium. Alternatively, the storage medium may be integrated with the processor 1100. The processor 1100 and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside in a user terminal. In another case, the processor 1100 and the storage medium may reside in a user terminal as separate components.

[0112] The present technology can detect the user's fatigue driving early and accurately based on slow eye movements and an interval in which the steering torque caused by a vehicle signal does not change, thereby preventing accidents caused by fatigue driving.

[0113] In addition, various effects directly or indirectly determined by the present disclosure can be provided.

[0114] Although the present disclosure has been described above with reference to exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto but may be variously modified and altered by those skilled in the art without departing from the spirit and scope of the present disclosure as claimed in the appended claims.

[0115] Therefore, the exemplary embodiments of the present disclosure are provided to explain the spirit and scope of the present disclosure, but not to limit them, so that the spirit and scope of the present disclosure are not limited by the embodiments. The scope of the present disclosure should be interpreted based on the attached claims, and all technical ideas within the scope equivalent to the claims should be included in the scope of the present disclosure.

Claims

1. A fatigue driving management device, comprising: processor, configured as Determine if slow eye movement of the user occurs and if there is no change in steering torque, and When the slow eye movement of the user occurs and when the steering torque does not change within a predetermined period of time, determining that the user is driving in fatigue; as well as a memory storing information indicating whether the slow eye movement occurs and a result of determining whether the steering torque does not change, Wherein, the processor is configured to: calculating first eye movement information relative to the vehicle based on information about fixation of the user and information about a head posture of the user based on the image data; and Based on the first eye movement information, calculating second eye movement information relative to the external environment, The information about the fixation includes a gaze yaw angle and a gaze pitch angle, and the information about the head posture includes a head posture yaw angle and a head posture pitch angle.

2. The fatigue driving management device according to claim 1, wherein: The processor is configured to determine that the user is driving in fatigue when there is no user manipulation in a state where the slow eye movement of the user occurs and the steering torque does not change within the predetermined period of time.

3. The fatigue driving management device according to claim 1, wherein: The processor is configured to: calculating a first eye movement yaw angle relative to the vehicle in the first eye movement information by subtracting the head posture yaw angle from the gaze yaw angle, and By subtracting the head posture pitch angle from the gaze pitch angle, a first eye movement pitch angle relative to the vehicle in the first eye movement information is calculated.

4. The fatigue driving management device according to claim 3, wherein: The processor is configured to: calculating a second eye movement yaw angle relative to the external environment in the second eye movement information using the first eye movement yaw angle and the vehicle yaw rate, and Using the first eye movement pitch angle and the vehicle pitch angle, a second eye movement pitch angle relative to the external environment in the second eye movement information is calculated.

5. The fatigue driving management device according to claim 4, wherein: The processor is configured to: calculating the eye movement distance within the predetermined time period using a value obtained by adding the second eye movement yaw angle to the second eye movement pitch angle; and When the eye movement distance is less than a predetermined reference value, it is determined that the slow eye movement occurs.

6. The fatigue driving management device according to claim 1, wherein: The processor is configured to determine that the steering torque has not changed when an output value of the steering torque sensor is less than a predetermined reference value.

7. A vehicle system comprising: The fatigue driving management device is configured to: Determine if slow eye movement of the user occurs and if there is no change in steering torque, and When the slow eye movement of the user occurs and when the steering torque does not change within a predetermined period of time, determining that the user is driving in fatigue; as well as a warning device configured to output a warning to the user when it is determined that the user is driving fatigued, Wherein, the fatigue driving management device is configured as follows: calculating first eye movement information relative to the vehicle based on information about fixation of the user and information about a head posture of the user based on the image data; and Based on the first eye movement information, calculating second eye movement information relative to the external environment, The information about the fixation includes a gaze yaw angle and a gaze pitch angle, and the information about the head posture includes a head posture yaw angle and a head posture pitch angle.

8. The vehicle system according to claim 7, wherein: The warning device is configured to provide at least one or more of a visual warning, a tactile warning, and an audible warning.

9. The vehicle system according to claim 7, further comprising: a camera configured to detect information about a gaze of the user and information about a head posture of the user; a yaw pitch sensor configured to sense a vehicle yaw rate and a vehicle pitch angle; a steering torque sensor configured to sense a change in a steering torque of the vehicle; and The decelerator / accelerator pedal sensor is configured to sense a change in a decelerator / accelerator pedal value.

10. The vehicle system according to claim 7, wherein: The fatigue driving management device comprises: The processor is configured as: determining whether a slow eye movement of the user occurs and whether the steering torque does not change, and When the slow eye movement of the user occurs and when the steering torque does not change within a predetermined period of time, determining that the user is driving in fatigue; and A memory stores information indicating whether the slow eye movement occurs and a result of determining whether the steering torque does not change, the information and the result being obtained by the processor.

11. The vehicle system according to claim 10, wherein: The processor is configured to determine that the user is driving in fatigue when there is no user manipulation in a state where the slow eye movement of the user occurs and the steering torque does not change within the predetermined period of time.

12. A method for managing fatigue driving, comprising: Determine if slow eye movement of the user occurs and if there is no change in steering torque, and When the slow eye movement of the user occurs and when the steering torque does not change within a predetermined period of time, it is determined that the user is driving in fatigue, Among them, determining whether the user's slow eye movement occurs and whether the steering torque does not change includes: calculating first eye movement information relative to the vehicle based on information about fixation of the user and information about a head posture of the user based on the image data; and Based on the first eye movement information, calculating second eye movement information relative to the external environment, The information about the fixation includes a gaze yaw angle and a gaze pitch angle, and the information about the head posture includes a head posture yaw angle and a head posture pitch angle.

13. The fatigue driving management method according to claim 12, wherein: Determining that the user is driving in fatigue includes: determining that the user is driving in fatigue when there is no user manipulation in a state where the slow eye movement of the user occurs and the steering torque does not change within the predetermined time period.

14. The method for managing fatigue driving according to claim 12, wherein: Determining whether a user's slow eye movement has occurred and whether there has been no change in steering torque includes: calculating a first eye movement yaw angle relative to the vehicle in the first eye movement information by subtracting the head posture yaw angle from the gaze yaw angle, and By subtracting the head posture pitch angle from the gaze pitch angle, a first eye movement pitch angle relative to the vehicle in the first eye movement information is calculated.

15. The method for managing fatigue driving according to claim 14, wherein: Determining whether a user's slow eye movement has occurred and whether there has been no change in steering torque includes: Calculating a second eye movement yaw angle relative to the external environment in the second eye movement information using the first eye movement yaw angle and the vehicle yaw rate; and Using the first eye movement pitch angle and the vehicle pitch angle, a second eye movement pitch angle relative to the external environment in the second eye movement information is calculated.

16. The method for managing fatigue driving according to claim 15, wherein: Determining whether a user's slow eye movement has occurred and whether there has been no change in steering torque includes: calculating the eye movement distance within the predetermined time period using a value obtained by adding the second eye movement yaw angle to the second eye movement pitch angle; and When the eye movement distance is less than a predetermined reference value, it is determined that the slow eye movement occurs.

Images