Method for measuring driver gaze movement in vehicle using head-up display, and electronic device therefor

Abstract

The present invention provides a method for measuring driver gaze movement in a vehicle using a head-up display, the method comprising the steps of: capturing an image of the face of the driver by using at least one camera; detecting the gaze of the driver by analyzing the captured image of the face of the driver; determining a region of interest of the driver on the basis of the gaze of the driver; if the gaze of the driver deviates from the region of interest of the driver within a preset time interval, measuring the duration for which the gaze of the driver deviated from the region of interest of the driver and the frequency with which the gaze of the driver deviated from the region of interest of the driver; and acquiring side-glance data related to the duration and direction of a side-glance of the driver on the basis of the duration for which the gaze of the driver deviated from the region of interest of the driver and the frequency with which the gaze of the driver deviated from the region of interest of the driver.

Description

Method for measuring driver's eye movement in a vehicle equipped with a head-up display and electronic device therefor

[0001] The present invention provides a method for measuring driver gaze movement in a vehicle equipped with a head-up display and an electronic device therefor.

[0002] Recently, traffic accidents caused by drivers using portable devices like smartphones while driving have been increasing year after year. This is because drivers attempting to use these devices while driving often divert their gaze from the vehicle's forward direction to the devices, delaying their ability to respond to situations ahead.

[0003] In this regard, technology is being considered to provide a warning for forward gaze deviation by utilizing an in-vehicle eye-recognition (Eye Tracker, Eye Detecting, Eye-recognition) device or wearable device.

[0004] Specifically, it is a technology that performs eye recognition (Eye Tracker, Eye Detecting, Eye-recognition) using a camera in a vehicle or a camera in a wearable device, compares the gaze direction recognized as a result of the eye recognition with the driving direction, and if the gaze direction is inconsistent with the driving direction and the frequency is high, provides a warning to the driver about distraction through a warning sound or warning vibration, thereby encouraging the driver to focus on driving.

[0005] In other words, if it is determined that the driver's line of sight is distracted compared to the driving direction, a warning is provided for safe driving.

[0006] However, in this case, when using a portable device such as a smartphone on the steering wheel, it is difficult to recognize it as a deviation from the driving direction. In addition, there is an ambiguity in setting the range of direction discrepancy when comparing the direction of gaze and the driving direction. In addition, there is a limitation in that only passive methods such as a simple warning sound or warning vibration are provided to the user.

[0007] The present invention provides a method and an electronic device capable of accurately detecting whether a driver's attention ability is present by detecting the driver's sidelong glance by measuring the eye movement of the driver driving a vehicle equipped with a head-up display.

[0008] The present invention provides a method for measuring driver gaze movement in a vehicle equipped with a head-up display, the method including the steps of: capturing a driver's face using at least one camera; detecting the driver's gaze by analyzing an image of the captured driver's face; determining a region of interest of the driver based on the driver's gaze; measuring, when the driver's gaze deviates from the region of interest within a preset time interval, the time for which the driver's gaze deviates from the region of interest and the frequency with which the driver's gaze deviates from the region of interest; and obtaining sidelong glance data related to the driver's sidelong glance maintenance time and the driver's sidelong glance direction based on the time for which the driver's gaze deviates from the region of interest or the frequency with which the driver's gaze deviates from the region of interest.

[0009] According to one embodiment, the step of obtaining the sidelong glance data may include a step of determining that the driver sidelong glanced and determining the time the driver sidelong glanced away from the driver's area of ​​interest as the driver's sidelong glance maintenance time if the time the driver's gaze was outside the driver's area of ​​interest exceeds a first preset reference time, and a step of determining that the driver sidelong glanced and determining the time the driver sidelong glanced away from the driver's area of ​​interest as the driver's sidelong glance maintenance time if the frequency of the driver's gaze being outside the driver's area of ​​interest exceeds a first preset reference number of times.

[0010] According to one embodiment, the first reference time corresponding to a driver riding in a vehicle equipped with a head-up display may be longer than the first reference time corresponding to a driver riding in a vehicle not equipped with the head-up display.

[0011] According to one embodiment, the step of obtaining the sideways glance data may include a step of determining that the driver sideways glanced and determining information about a direction in which the driver sideways glanced away from the area of ​​interest of the driver as the direction of the sideways glance of the driver if the time for which the driver's gaze departs from the area of ​​interest of the driver exceeds the first reference time, and a step of determining that the driver sideways glanced and determining information about a direction in which the driver sideways glanced away from the area of ​​interest of the driver as the direction of the sideways glance of the driver if the frequency with which the driver's gaze departs from the area of ​​interest of the driver exceeds the first reference number of times.

[0012] According to one embodiment, when the head-up display is installed in a vehicle in which the driver is riding, the step of excluding information about a direction outside the driver's area of ​​interest that matches the direction in which the head-up display is installed from the information included in the information about a direction outside the driver's area of ​​interest from the information for determining the driver's sideways glance direction may be further included.

[0013] According to one embodiment, the method for measuring driver gaze movement in a vehicle equipped with the head-up display may further include a step of providing an alarm to the driver when the driver's sidelong glance maintenance time exceeds a preset second reference time or the driver's sidelong glance direction matches a preset danger direction through the interpretation of the sidelong glance data.

[0014] According to one embodiment, the driver's area of ​​interest includes an area corresponding to the cluster of the vehicle, the forward direction of the driver, a side mirror of the vehicle, and a room mirror of the vehicle, and if the vehicle in which the driver is riding is equipped with a head-up display, the driver's area of ​​interest may additionally include an area where the head-up display is displayed.

[0015] The present invention provides an electronic device including at least one camera for capturing a driver's face and a processor for detecting the driver's gaze through analysis of an image of the captured driver's face, determining a region of interest of the driver based on the driver's gaze, and, when the driver's gaze deviates from the region of interest of the driver within a preset time interval, measuring the time for which the driver's gaze deviates from the region of interest of the driver and the frequency with which the driver's gaze deviates from the region of interest of the driver, and obtaining sidelong glance data related to the driver's sidelong glance maintenance time and the driver's sidelong glance direction based on the time for which the driver's gaze deviates from the region of interest of the driver or the frequency with which the driver's gaze deviates from the region of interest of the driver.

[0016] According to one embodiment, if the time for which the driver's gaze is out of the driver's area of ​​interest exceeds a first reference time, the processor determines that the driver has glanced sideways and determines the time for which the driver has glanced sideways as the driver's glance retention time, and if the frequency with which the driver's gaze is out of the driver's area of ​​interest exceeds a first reference number of times, the processor determines that the driver has glanced sideways and determines the time for which the driver has glanced sideways as the driver's glance retention time.

[0017] According to one embodiment, if the time for which the driver's gaze is out of the driver's area of ​​interest exceeds the first reference time, the processor determines that the driver has glanced sideways and determines information about the direction in which the driver has glanced sideways, and if the frequency with which the driver's gaze has glanced sideways exceeds the first reference number of times, the processor determines that the driver has glanced sideways and determines information about the direction in which the driver has glanced sideways, ...

[0018] According to one embodiment, when the head-up display is installed in a vehicle in which the driver is riding, the processor may exclude information about a direction outside the driver's area of ​​interest that matches the direction in which the head-up display is installed from information about a direction outside the driver's area of ​​interest, among information included in the information about a direction outside the driver's area of ​​interest, from information for determining the driver's sideways glance direction.

[0019] According to one embodiment, the processor may provide an alarm to the driver when the driver's sidelong glance maintenance time exceeds a preset second reference time or the driver's sidelong glance direction matches a preset dangerous direction through the interpretation of the sidelong glance data.

[0020] According to one embodiment disclosed in the present invention, the accuracy of driver inattention detection based on detection of the driver's gaze can be improved by preventing the driver's gaze from being judged as a sidelong glance while looking at the head-up display.

[0021] FIG. 1 is a flowchart of a method for measuring driver gaze movement in a vehicle equipped with a head-up display according to one embodiment of the present invention.

[0022] Figure 2 is a flowchart of a method for obtaining side-eye data according to a first embodiment of the present invention.

[0023] Figure 3 is a flowchart of a method for obtaining side-eye data according to a second embodiment of the present invention.

[0024] Figure 4 is a flowchart of a method for providing a driver alarm according to one embodiment of the present invention.

[0025] FIG. 5 is a drawing for explaining a region of interest according to one embodiment of the present invention.

[0026] Figure 6 is a block diagram of an electronic device according to one embodiment of the present invention.

[0027] The present invention provides a method for measuring driver gaze movement in a vehicle equipped with a head-up display, the method including the steps of: capturing a driver's face using at least one camera; detecting the driver's gaze by analyzing an image of the captured driver's face; determining a region of interest of the driver based on the driver's gaze; measuring, when the driver's gaze deviates from the region of interest within a preset time interval, the time for which the driver's gaze deviates from the region of interest and the frequency with which the driver's gaze deviates from the region of interest; and obtaining sidelong glance data related to the driver's sidelong glance maintenance time and the driver's sidelong glance direction based on the time for which the driver's gaze deviates from the region of interest or the frequency with which the driver's gaze deviates from the region of interest.

[0028] The present invention provides an electronic device including at least one camera for capturing a driver's face and a processor for detecting the driver's gaze through analysis of an image of the captured driver's face, determining a region of interest of the driver based on the driver's gaze, and, when the driver's gaze deviates from the region of interest of the driver within a preset time interval, measuring the time for which the driver's gaze deviates from the region of interest of the driver and the frequency with which the driver's gaze deviates from the region of interest of the driver, and obtaining sidelong glance data related to the driver's sidelong glance maintenance time and the driver's sidelong glance direction based on the time for which the driver's gaze deviates from the region of interest of the driver or the frequency with which the driver's gaze deviates from the region of interest of the driver.

[0029] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0030] Terms such as first, second, A, and B may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0031] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0032] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0033] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0034]

[0035] FIG. 1 is a flowchart illustrating a method for measuring driver gaze movements in a vehicle equipped with a head-up display according to one embodiment of the present invention. The flowchart illustrated in FIG. 1 can be performed by the electronic device illustrated in FIG. 6.

[0036] In one embodiment, step S110 may be used to capture the driver's face using at least one camera. In various embodiments, the driver's face may be captured using a camera capturing the driver's eyes, an infrared camera positioned inside the vehicle, a camera capturing the front of the vehicle, or the like.

[0037] According to one embodiment, the driver's gaze can be detected by analyzing the image of the driver's face captured through step S120. According to various embodiments, the driver's gaze can be detected by extracting the driver's pupil movement from the captured image of the driver's face. A method for detecting the driver's gaze by extracting the driver's pupil movement

[0038] In one embodiment, step S130 determines the driver's area of ​​interest based on the driver's gaze. In various embodiments, the area where the driver's gaze remains for a prolonged period of time may be the area of ​​interest. For example, the area of ​​interest may include the front of the vehicle, the vehicle's side mirrors, or the navigation display. A detailed description of the area of ​​interest will be provided later in connection with the description of FIG. 5.

[0039] In one embodiment, step S140 may determine whether the driver's gaze has departed from the driver's area of ​​interest within a preset time interval. In various embodiments, step S140 may be performed by comparing whether the driver's area of ​​interest determined in step S130 is included in an area matching the driver's gaze detected in step S120.

[0040] In one embodiment, when it is determined that the driver's gaze has left the driver's area of ​​interest, the time for which the driver's gaze has left the driver's area of ​​interest and the frequency with which the driver's gaze has left the driver's area of ​​interest may be measured in step S150. In various embodiments, the time for which the driver's gaze has left the driver's area of ​​interest may be the time interval between the time for which the driver's gaze has left the area of ​​interest and the time for which the driver's gaze has re-entered the area of ​​interest, and the frequency with which the driver's gaze has left the area of ​​interest may be the frequency with which the driver's gaze has left the area of ​​interest within a specific time interval (e.g., 5 minutes).

[0041] In one embodiment, even if it is determined that the driver's gaze has not left the driver's area of ​​interest, it is possible to continuously monitor the driver's gaze and the driver's area of ​​interest to determine whether the driver's gaze has left the driver's area of ​​interest.

[0042] In one embodiment, step S160 may be used to acquire side-glance data related to the driver's side-glance duration and side-glance direction based on the time the driver's gaze leaves the driver's area of ​​interest or the frequency with which the driver's gaze leaves the driver's area of ​​interest. A specific method for acquiring side-glance data will be described later with reference to FIGS. 2 and 3 .

[0043]

[0044] FIG. 2 is a flowchart illustrating a method for acquiring side-eye data according to a first embodiment of the present invention. The flowchart illustrated in FIG. 2 can be performed by the electronic device illustrated in FIG. 6.

[0045] In one embodiment, step S210 determines whether the time the driver's gaze leaves the driver's area of ​​interest exceeds a preset first reference time. In various embodiments, the first reference time may be the minimum time required for the driver to be deemed to have lost attention. For example, the first reference time may be 0.01 seconds.

[0046] In one embodiment, the first reference time corresponding to a driver in a vehicle equipped with a head-up display may be longer than the first reference time corresponding to a driver in a vehicle not equipped with a head-up display.

[0047] In the case of a vehicle equipped with a head-up display, the driver's act of looking at the head-up display or the driver's act of moving their gaze to look at the head-up display may be included as a time outside the driver's area of ​​interest. However, since it is difficult to determine that the driver has lost attention when looking at the head-up display, the first reference time corresponding to a driver in a vehicle equipped with a head-up display may be set longer than the first reference time corresponding to a driver in a vehicle without a head-up display to minimize the error of determining that the driver has lost attention when looking at the head-up display. For example, the first reference time corresponding to a driver in a vehicle equipped with a head-up display may be 0.03 seconds, and the first reference time corresponding to a driver in a vehicle without a head-up display may be 0.01 seconds.

[0048] According to one embodiment, if the time during which the driver's gaze is out of the driver's area of ​​interest exceeds the first reference time, the time during which the driver's gaze is out of the driver's area of ​​interest may be determined as the driver's sidelong glance retention time through step S220. The time during which the driver's gaze is out of the driver's area of ​​interest may be regarded as a time period during which the driver loses attention, and thus, the time period during which the driver's gaze is out of the driver's area of ​​interest during the period during which the driver's gaze is monitored may be determined as the driver's sidelong glance retention time. According to various embodiments, there may be multiple sidelong glance retention times during a specific time period, and information about the sidelong glance retention times (the time during which the driver's gaze is out of the area of ​​interest, the time during which the driver's gaze returns to the area of ​​interest, etc.) may be recorded.

[0049] In one embodiment, if the time the driver's gaze leaves the driver's area of ​​interest is less than or equal to the first reference time, step S230 determines whether the frequency of the driver's gaze leaving the driver's area of ​​interest exceeds a preset first reference number. In various embodiments, the first reference number may be the minimum number of times the driver can be determined to have lost attention. For example, the first reference number may be three.

[0050] According to one embodiment, if the frequency with which the driver's gaze deviates from the driver's area of ​​interest exceeds the first reference number of times, the time for which the driver's gaze deviates from the area of ​​interest can be determined as the driver's sideways glance maintenance time through step S220. A specific description of step S220 will be replaced with the previous description. According to various embodiments, if the frequency with which the driver's gaze deviates from the area of ​​interest is less than or equal to the first reference number of times, the driver's gaze can be periodically monitored through step S240 to determine whether the time or frequency with which the driver's gaze deviates from the area of ​​interest exceeds the first reference time or the first reference number of times.

[0051] Meanwhile, although FIG. 2 illustrates that step S230 is performed after step S210, steps S210 and S230 may be performed as separate steps, or step S210 may be performed after step S230 is performed.

[0052]

[0053] FIG. 3 is a flowchart illustrating a method for acquiring side-eye data according to a second embodiment of the present invention. The flowchart illustrated in FIG. 3 can be performed by the electronic device illustrated in FIG. 6.

[0054] In one embodiment, step S310 determines whether the time the driver's gaze is outside the driver's area of ​​interest exceeds a preset first reference time. A detailed description of step S310 is provided in the description of step S210 of FIG. 2.

[0055] According to one embodiment, if the time for which the driver's gaze is outside the driver's area of ​​interest exceeds the first reference time, it is determined through step S320 that the driver has glanced sideways, and information about the direction in which the driver's gaze has departed from the driver's area of ​​interest can be determined as the direction of the driver's sideways glance. According to various embodiments, the information about the direction in which the driver's gaze has departed from the driver's area of ​​interest can be coordinate information related to the driver's gaze that has departed from the area of ​​interest. For example, the coordinate information about the location of the driver's gaze in a coordinate system centered on a camera capturing the driver's face (e.g., a Cartesian coordinate system) can be determined as the direction of the driver's sideways glance.

[0056] According to one embodiment, if the time for which the driver's gaze leaves the region of interest is less than or equal to a first reference time, it may be determined through step S330 whether the frequency with which the driver's gaze leaves the region of interest exceeds the first reference number of times. A specific description of step S330 may be replaced with the description of step S230 of FIG. 2. According to various embodiments, if the frequency with which the driver's gaze leaves the region of interest is less than or equal to the first reference number of times in step S230, the driver's gaze may be continuously monitored through step S340. A specific description of step S340 may be replaced with the description of step S240 of FIG. 2.

[0057] In one embodiment, after step S320, it may be determined through step S350 whether a head-up display is installed in the vehicle in which the driver is riding. In various embodiments, information about directions outside the driver's area of ​​interest may be modified based on whether a head-up display is installed.

[0058] According to one embodiment, if a head-up display is installed in a vehicle in which a driver is riding, information about a direction outside the driver's area of ​​interest that matches the direction in which the head-up display is installed among the information about the direction in which the driver leaves the area of ​​interest through step S360 may be excluded from information for determining the direction in which the driver looks sideways. For example, if information about a direction in which the driver leaves the area of ​​interest includes information about a direction in which the driver moved -200 mm in the x-axis, 50 mm in the y-axis, and 200 mm in the z-axis from the cluster, and if the area in which the driver moved -200 mm in the x-axis, 50 mm in the y-axis, and 200 mm in the z-axis from the cluster is an area matching the head-up display, then the driver's sidelong glance related to the direction in which the driver moved -200 mm in the x-axis, 50 mm in the y-axis, and 200 mm in the z-axis from the cluster may be regarded as a sidelong glance to look at the head-up display and may be excluded from information for determining the direction in which the driver looks sidelong glance.

[0059] According to one embodiment, the accuracy of driver side-eye detection in a vehicle equipped with a head-up display can be improved by excluding the driver's gaze looking at the head-up display from the side-eye data through step S360. Meanwhile, although step S330 is illustrated as being performed after step S310 in FIG. 3 , steps S310 and S330 may be performed separately, or step S310 may be performed after step S330.

[0060]

[0061] FIG. 4 is a flowchart illustrating a method for providing a driver alarm according to one embodiment of the present invention. The flowchart illustrated in FIG. 4 can be performed by the electronic device illustrated in FIG. 6.

[0062] In one embodiment, step S410 may be used to interpret the driver's sidelong glance duration and the driver's sidelong glance direction contained in the sidelong glance data. In various embodiments, step S410 may be used to extract the total time the driver sidelong glanced for a preset time interval, the average time of the sidelong glance interval, coordinate information of the driver's sidelong glance based on the center of the driver's face as the reference coordinate, and the like.

[0063] In one embodiment, step S420 determines whether the sidelong glance duration exceeds a preset second reference time. In various embodiments, the second reference time serves as a reference for determining whether to provide an alarm to the driver, and thus may be longer than the first reference time mentioned above. For example, if the first reference time is 0.01 seconds, the second reference time may be 0.5 seconds.

[0064] In one embodiment, if the driver's sidelong glance duration exceeds the second reference time, an alarm may be provided to the driver through step S430. In various embodiments, if the sidelong glance duration exceeds a specific time, this indicates that the driver is not paying close attention to the road ahead, and thus a visual, auditory, or tactile alarm may be provided to alert the driver.

[0065] In one embodiment, if the driver's sidelong glance duration is less than or equal to the second reference time, it may be determined through step S440 whether the driver's sidelong glance direction matches a preset danger direction. In various embodiments, if the driver's sidelong glance direction is in an area where a head-up display is placed or an area for operating a warning light, it is highly likely that the driver is in a state where the vehicle can be controlled, and therefore, it may be determined that the driver's sidelong glance direction in such a case does not match a danger direction.

[0066] In one embodiment, if the driver's sideways glance direction matches an area matching the ceiling or floor of the vehicle interior, it is highly likely that the driver is unable to control the vehicle, and thus, the driver's sideways glance direction in such a case may be determined to match a dangerous direction. In various embodiments, the area inside the vehicle matching the preset dangerous direction may be an area where there are no components (buttons, displays, etc.) related to vehicle operation, such as the vehicle ceiling, vehicle floor, or the lower left side of the vehicle.

[0067] In one embodiment, if the sideways glance direction matches a dangerous direction, an alarm may be provided to the driver through step S430. Meanwhile, although step S440 is illustrated as being performed after step S420 in FIG. 4, steps S420 and S440 may be performed as separate steps, or step S420 may be performed after step S440.

[0068]

[0069] FIG. 5 is a diagram illustrating a region of interest according to one embodiment of the present invention. According to one embodiment, the cluster (510), windshield (520), rearview mirror (530), side mirror (540), and vehicle infotainment display unit (550), which are areas that a driver must pay attention to while driving a vehicle, may be set as regions of interest. According to various embodiments, in a vehicle equipped with a head-up display, the area where the head-up display is positioned may also be included in the region of interest.

[0070]

[0071] Figure 6 is a block diagram of an electronic device according to one embodiment of the present invention.

[0072] According to one embodiment, the electronic device (600) may include at least one camera (610) for capturing a driver's face, and a processor (620) for detecting the driver's gaze through analysis of an image of the captured driver's face, determining a region of interest of the driver based on the driver's gaze, and, when the driver's gaze deviates from the region of interest of the driver within a preset time period, measuring the time for which the driver's gaze deviates from the region of interest of the driver and the frequency with which the driver's gaze deviates from the region of interest of the driver, and obtaining sidelong glance data related to the driver's sidelong glance maintenance time and the driver's sidelong glance direction based on the time for which the driver's gaze deviates from the region of interest of the driver or the frequency with which the driver's gaze deviates from the region of interest of the driver.

[0073] According to one embodiment, if the time for which the driver's gaze is out of the driver's area of ​​interest exceeds a first reference time, the processor (620) determines that the driver has glanced sideways and determines the time for which the driver has glanced sideways outside the area of ​​interest as the driver's sideways glance maintenance time, and if the frequency with which the driver's gaze is out of the driver's area of ​​interest exceeds a first reference number of times, the processor may determine that the driver has glanced sideways and determine the time for which the driver has glanced sideways outside the area of ​​interest as the driver's sideways glance maintenance time. According to various embodiments, if the time for which the driver's gaze is out of the driver's area of ​​interest exceeds the first reference time, the processor (620) determines that the driver has glanced sideways and determines information about the direction in which the driver has glanced sideways outside the area of ​​interest as the driver's sideways glance direction, and if the frequency with which the driver's gaze is out of the driver's area of ​​interest exceeds the first reference number of times, the processor may determine that the driver has glanced sideways and determine information about the direction in which the driver has glanced sideways outside the area of ​​interest as the driver's sideways glance direction.

[0074] According to one embodiment, when the head-up display is installed in a vehicle in which the driver is riding, the processor (620) may exclude from the information for determining the driver's sideways glance direction the information about the direction in which the driver has left the area of ​​interest that matches the direction in which the head-up display is installed among the information included in the information about the direction in which the driver has left the area of ​​interest. According to various embodiments, the processor (620) may provide an alarm to the driver when, through the analysis of the sideways glance data, the driver's sideways glance maintenance time exceeds a preset second reference time or the driver's sideways glance direction matches a preset dangerous direction.

[0075]

[0076] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. A step of photographing the driver's face using at least one camera; A step of detecting the driver's gaze by analyzing the captured image of the driver's face; A step of determining the driver's area of ​​interest based on the driver's line of sight; A step of measuring the time for which the driver's gaze is out of the driver's area of ​​interest and the frequency with which the driver's gaze is out of the driver's area of ​​interest within a preset time interval; and A step of obtaining sideways glance data related to the driver's sideways glance duration and the driver's sideways glance direction based on the time the driver's gaze is out of the driver's area of ​​interest or the frequency with which the driver's gaze is out of the driver's area of ​​interest, A method for measuring driver eye movements in a vehicle equipped with a head-up display.

2. In paragraph 1, The steps for obtaining the above side-eye data are If the time for which the driver's gaze is out of the driver's area of ​​interest exceeds a preset first reference time, a step for determining that the driver has glanced sideways and determining the time for which the driver's gaze is out of the driver's area of ​​interest as the driver's sideways glance maintenance time; and A method characterized by comprising a step of determining that the driver has glanced sideways when the frequency of the driver's gaze leaving the driver's area of ​​interest exceeds a preset first reference number of times, and determining the time for which the driver has glanced sideways as the driver's sideways glance maintenance time. A method for measuring driver eye movements in a vehicle equipped with a head-up display.

3. In paragraph 2, The first reference time corresponding to a driver riding in a vehicle with a head-up display installed is characterized in that it is longer than the first reference time corresponding to a driver riding in a vehicle without the head-up display installed. A method for measuring driver eye movements in a vehicle equipped with a head-up display.

4. In paragraph 1, The steps for obtaining the above side-eye data are If the time for which the driver's gaze is out of the driver's area of ​​interest exceeds the first reference time, a step for determining that the driver has glanced sideways and determining information about the direction in which the driver's area of ​​interest has been out of the driver's area of ​​interest as the direction of the driver's sideways glance; and A method characterized by comprising a step of determining that the driver has glanced sideways when the frequency of the driver's gaze leaving the driver's area of ​​interest exceeds the first reference number of times, and determining information about the direction in which the driver has glanced sideways as the direction in which the driver has glanced sideways. A method for measuring driver eye movements in a vehicle equipped with a head-up display.

5. In paragraph 4, In a case where the head-up display is installed in the vehicle in which the driver is riding, the method further comprises a step of excluding information about a direction outside the driver's area of ​​interest that matches the direction in which the head-up display is installed from the information for determining the driver's sideways glance direction, among the information included in the information about a direction outside the driver's area of ​​interest. A method for measuring driver eye movements in a vehicle equipped with a head-up display.

6. In paragraph 1, Further comprising a step of providing an alarm to the driver when the driver's sidelong glance maintenance time exceeds a preset second reference time or the driver's sidelong glance direction matches a preset dangerous direction through the above sidelong glance data interpretation. A method for measuring driver eye movements in a vehicle equipped with a head-up display.

7. In paragraph 1, The driver's area of ​​interest includes an area corresponding to the cluster of the vehicle, the driver's forward direction, the side mirror of the vehicle, and the room mirror of the vehicle, and, if the vehicle in which the driver is riding is equipped with a head-up display, the area where the head-up display is displayed is additionally included in the driver's area of ​​interest. A method for measuring driver eye movements in a vehicle equipped with a head-up display.

8. At least one camera for photographing the driver's face; and A processor comprising: a processor configured to detect the driver's gaze through image analysis of the captured driver's face, determine the driver's area of ​​interest based on the driver's gaze, measure the time for which the driver's gaze was out of the driver's area of ​​interest and the frequency with which the driver's gaze was out of the driver's area of ​​interest when the driver's gaze is out of the driver's area of ​​interest within a preset time period, and obtain sidelong glance data related to the driver's sidelong glance maintenance time and the driver's sidelong glance direction based on the time for which the driver's gaze was out of the driver's area of ​​interest or the frequency with which the driver's gaze was out of the driver's area of ​​interest. Electronic devices.

Images