A driver distraction driving recognition method, device and storage medium

Abstract

The application relates to a driver distraction driving recognition method and device and a storage medium, which are applied to the technical field of automobile auxiliary driving and comprise the following steps: collecting eye movement data of a driver in a driving process, obtaining a visual point angle of the driver through the eye movement data, judging whether the line of sight of the driver is in a main visual area through the visual point angle, judging whether the driver is visually distracted through calculation of the time when the line of sight of the driver leaves the main visual area, obtaining an average blinking time mean value of the driver through the eye movement data of the driver, and judging whether the driver is fatigued and distracted through the average blinking time mean value. The application compares the visual angle and the blinking time obtained through the eye movement data with a standard visual angle threshold interval and a blinking time under normal conditions to determine whether the driver is in a distracted driving condition. The eye movement line-of-sight tracking technology of the driver can identify whether the driver is visually distracted and fatigued and distracted from the state of the eyes, and has a significant progress in precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobile auxiliary driving, in particular to a driver distraction driving recognition method and device and a storage medium. BACKGROUND

[0002] Driver distraction driving includes visual distraction (driver's line of sight deviates from the main viewing area) and fatigue distraction (driver's brain thinks about tasks unrelated to driving), according to the data of the National Highway Traffic Safety Administration (NHTSA), 25%-30% of traffic accidents are caused by driver distraction every year, so it is necessary to recognize and warn driver distraction in advance to effectively reduce safety accidents caused by driver distraction.

[0003] The existing driver state monitoring system based on video picture monitoring needs to collect the picture of the driver into the terminal, and judge whether the driver produces fatigue distraction and visual distraction according to the algorithm, but requires that the yawning action is > 2s, the eye closing time is > 1s, and the driver's head deflection angle is greater than 45° to identify the corresponding distraction driving, but only through the yawning action and eye closing time cannot accurately judge the distraction state of the driver, resulting in low recognition accuracy of the existing driver distraction driving. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a driver distraction driving recognition method, device and storage medium to solve the problem of low accuracy of driver distraction recognition in the prior art by monitoring the driver's state through video pictures and judging whether the driver produces fatigue distraction and visual distraction according to the algorithm.

[0005] According to a first aspect of an embodiment of the present application, a driver distraction driving recognition method is provided, comprising:

[0006] In the driving process, the eye movement data of the driver is obtained by an eye tracker, and the left and right rotation angles and the up and down rotation angles of the driver's eyes are obtained according to the eye movement data of the driver;

[0007] When the left and right rotation angles of the driver's eyes do not satisfy the 0.95 probability interval value of the preset left and right rotation angles, or the up and down rotation angles of the driver's eyes do not satisfy the 0.95 probability interval value of the preset up and down rotation angles, it is judged that the driver's line of sight deviates from the main viewing area;

[0008] The duration of the driver's line of sight deviating from the main viewing area is counted, and if the duration of the line of sight deviating from the main viewing area exceeds the preset 0.95 quantile, the driver is determined to be visually distracted;

[0009] According to the eye movement data of the driver obtained by the eye tracker, the total blinking time and the blinking times of the driver within a preset time period are obtained;

[0010] The average blinking time is obtained by dividing the total blinking time by the blinking times, and the average blinking time is compared with a preset 0.95 confidence interval threshold value. If the average blinking time exceeds the preset 0.95 confidence interval threshold value, it is determined that the driver is fatigued and distracted.

[0011] Preferably,

[0012] The preset 0.95 probability interval value of the left and right rotation angle and the preset 0.95 probability interval value of the up and down rotation angle include:

[0013] Obtain the test eye movement data of the driver within a period of time on the test vehicle, and determine the main viewing area of the driver;

[0014] According to the test eye movement data of the driver, the angle data of the left and right rotation of the eyes and the angle data of the up and down rotation of the eyes of the driver when the line of sight stays in the main viewing area within a preset time period are determined;

[0015] The angle data of the left and right rotation of the eyes is fitted according to the normal distribution to obtain the 0.95 probability interval value of the left and right rotation angle, and the angle data of the up and down rotation of the eyes is fitted according to the logarithmic normal distribution to obtain the 0.95 probability interval value of the up and down rotation angle.

[0016] Preferably,

[0017] The preset 0.95 quantile includes:

[0018] When the left and right rotation angles and the up and down rotation angles of the eyes of the driver meet the 0.95 probability interval value respectively, it is judged that the line of sight of the driver is in the main viewing area, otherwise, the line of sight of the driver is not in the main viewing area;

[0019] According to the test eye movement data of the driver within a preset period of time, a plurality of time length data of the line of sight of the user not in the main viewing area are obtained, and the 0.95 quantile is obtained by fitting the distribution statistics of the plurality of time length data.

[0020] Preferably,

[0021] The determination of the main viewing area of the driver includes:

[0022] The windshield of the driver of the test vehicle is divided into a plurality of sub-viewing areas, and the windshield sub-viewing area is divided into upper left area, lower left area, middle lower area, middle upper area, lower right area and upper right area according to the average size;

[0023] The proportion of the driver's view point falling in a certain windshield sub-viewing area within a preset time period is determined by the test eye movement data of the driver.

[0024] If there is a windshield sub-region such that the time length of the driver's view point falling in the windshield sub-region within the preset time period exceeds 80% of the preset time period, the windshield sub-region is the main view region;

[0025] If not, the sub-regions of the windshield are re-divided until the time length of the driver's view point falling in a windshield sub-region within the preset time period exceeds 80% of the preset time period.

[0026] Preferably,

[0027] The obtaining of the 0.95 probability interval value of the preset left-right rotation angle or the 0.95 probability interval value of the preset up-down rotation angle further comprises:

[0028] Obtaining the 0.95 probability interval value of the left-right rotation angle or the 0.95 probability interval value of the up-down rotation angle under different road environments;

[0029] Traversing the 0.95 probability interval value of the left-right rotation angle or the 0.95 probability interval value of the up-down rotation angle under all different road environments until the difference value of the 0.95 probability interval value of the left-right rotation angle or the 0.95 probability interval value of the up-down rotation angle under any two different road environments is calculated, and the average value of the difference value is calculated;

[0030] If the average value of the difference value is less than a preset first average threshold, the average value of the 0.95 probability interval value of the left-right rotation angle or the 0.95 probability interval value of the up-down rotation angle under different road environments is taken as the 0.95 probability interval value of the left-right rotation angle or the 0.95 probability interval value of the up-down rotation angle under all road environments, and if the average value of the difference value is greater than or equal to the preset first average threshold, the 0.95 probability interval value of the left-right rotation angle or the 0.95 probability interval value of the up-down rotation angle under different road environments is output respectively.

[0031] Preferably,

[0032] The obtaining of the preset 0.95 quantile further comprises:

[0033] Obtaining the 0.95 quantile under different road environments, and traversing all the 0.95 quantiles under different road environments until the difference value of the 0.95 quantile under any two different road environments is calculated;

[0034] Calculate the average of the difference values, if the average of the difference values is less than a preset second average threshold, take the average of the 0.95 quantile of different road environments as the 0.95 quantile of all road environments, if the average of the difference values is greater than or equal to the preset second average threshold, output the 0.95 quantile of different road environments respectively.

[0035] According to a second aspect of the embodiments of the present application, a driver distraction driving recognition device is provided, comprising:

[0036] An angle data acquisition module is configured to acquire eye movement data of the driver through an eye tracker during driving, and acquire left and right rotation angles and up and down rotation angles of the eyes of the driver according to the eye movement data of the driver.

[0037] A main visual field judgment module is configured to judge that the line of sight of the driver is out of the main visual field when the left and right rotation angles of the eyes of the driver do not satisfy a preset 0.95 probability interval value of the left and right rotation angles, or the up and down rotation angles of the eyes of the driver do not satisfy a preset 0.95 probability interval value of the up and down rotation angles.

[0038] A visual distraction judgment module is configured to count a time length during which the line of sight of the driver is out of the main visual field, and determine that the driver is visually distracted when the time length during which the line of sight of the driver is out of the main visual field exceeds a preset 0.95 quantile.

[0039] An eye blinking data acquisition module is configured to acquire total eye blinking time and eye blinking frequency of the driver within a preset time period according to the eye movement data of the driver acquired by the eye tracker.

[0040] A fatigue distraction judgment module is configured to obtain an average eye blinking time by dividing the total eye blinking time by the eye blinking frequency, compare the average eye blinking time with a preset 0.95 confidence interval threshold, and determine that the driver is fatigued and distracted when the average eye blinking time exceeds the preset 0.95 confidence interval threshold.

[0041] According to a third aspect of the embodiments of the present application, a storage medium is provided, the storage medium stores a computer program, and the computer program is executed by a host controller to implement each step of the above method.

[0042] The technical solutions provided by the embodiments of the present application can have the following beneficial effects:

[0043] The application can determine whether the driver's line of sight is in the main viewing area by comparing the point of view angle with the preset 0.95 probability interval value, by calculating the time when the driver's line of sight leaves the main viewing area, and comparing it with the preset 0.95 quantile, when greater than the preset 0.95 quantile, it is determined that the driver is visually distracted, similarly, by the driver's eye movement data to obtain the average blinking time mean of the driver, compare the average blinking time mean with the preset 0.95 confidence interval threshold, can determine whether the driver exists fatigue distraction, in the application, the line of sight angle and the blinking time are obtained by the eye movement data, and compared with the standard line of sight angle threshold interval and the blinking time under normal circumstances, to determine whether the driver exists distracted driving, the eye movement line of sight tracking technology of the driver can collect eye movement state with up to 60Hz video, can judge the blinking time of ms level, smaller head rotation angle, can identify whether the driver produces visual distraction and fatigue distraction from the state of the eye, compared with the video judgment in the prior art, requiring yawning action > 2s, eye closing time > 1s, driver head deflection angle greater than 45°, there is a significant progress in precision.

[0044] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0045] The drawings incorporated into the specification and forming part of the specification, show embodiments consistent with the application, and together with the specification, serve to explain the principles of the application.

[0046] Figure 1 It is a flowchart of a driver distraction driving recognition method according to an exemplary embodiment;

[0047] Figure 2 It is a flowchart of the acquisition of the 0.95 probability interval value of the left and right rotation angle and the preset 0.95 probability interval value of the up and down rotation angle according to an exemplary embodiment;

[0048] Figure 3 It is a flowchart of the acquisition of the 0.95 quantile according to another exemplary embodiment;

[0049] Figure 4 It is a flowchart of the determination of the main viewing area according to another exemplary embodiment;

[0050] Figure 5 It is a schematic diagram of the 0.95 confidence interval threshold of the blinking time mean of different fatigue states according to another exemplary embodiment;

[0051] Figure 6 is a schematic diagram of the principle of the closed-eye backup rate PERCLOS 80 according to another exemplary embodiment;

[0052] Figure 7 is a schematic diagram of the determination of the main visual field according to the statistical analysis of the driver's eye point position according to another exemplary embodiment;

[0053] Figure 8 is a schematic diagram of the statistical fitting of the GazeHeading and GazePitch test data distribution and the calculation of the 0.95 probability interval according to another exemplary embodiment;

[0054] Figure 9 is a schematic diagram of the statistical fitting of the test data distribution of the duration of the driver's line of sight leaving the main visual field and the calculation of the 0.95 quantile according to another exemplary embodiment;

[0055] Figure 10 is a schematic diagram of the significant difference test of the distribution of the GazeHeading, GazePitch line of sight angle and the duration of the line of sight leaving the main visual field in different driving environments according to another exemplary embodiment;

[0056] Figure 11 is a schematic diagram of a driver distraction driving recognition device according to another exemplary embodiment;

[0057] In the drawings: 1 - angle data acquisition module, 2 - main visual field judgment module, 3 - visual distraction judgment module, 4 - blink data acquisition module, 5 - fatigue distraction judgment module. DETAILED DESCRIPTION

[0058] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers refer to the same or similar elements throughout the drawings. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0059] Embodiment One

[0060] Figure 1 is a schematic diagram of a driver distraction driving recognition method according to an exemplary embodiment, as shown in Figure 1 , the method comprises:

[0061] S1, in the driving process, the eye tracker is used to acquire the eye movement data of the driver, and the left and right rotation angles and the up and down rotation angles of the driver's eyes are acquired according to the eye movement data of the driver;

[0062] S2, when the left and right rotation angles of the driver's eyes do not satisfy the preset 0.95 probability interval value of the left and right rotation angles, or the up and down rotation angles of the driver's eyes do not satisfy the preset 0.95 probability interval value of the up and down rotation angles, it is judged that the driver's line of sight leaves the main viewing area;

[0063] S3, the duration of the driver's line of sight leaving the main viewing area is counted, and if the duration of the line of sight leaving the main viewing area exceeds the preset 0.95 quantile, it is determined that the driver's visual distraction;

[0064] S4, according to the eye movement data of the driver obtained by the eye tracker, the total blinking time and the blinking times of the driver in a preset time period are obtained;

[0065] S5, the average blinking time is obtained by the total blinking time and the blinking times, and the average blinking time is compared with the preset 0.95 confidence interval threshold value, if the average blinking time exceeds the preset 0.95 confidence interval threshold value, it is determined that the driver is fatigued and distracted;

[0066] It can be understood that, by collecting the eye movement data of the driver during driving, the left and right rotation angles and the up and down rotation angles of the driver are obtained by the eye movement data, it is worth noting that: GazeHeading is the angle of the left and right rotation of the driver's eyes, the left rotation is positive and the right rotation is negative, GazePitch is the angle of the up and down rotation of the driver's eyes, the down rotation is positive and the up rotation is negative; by comparing the left and right rotation angles and the up and down rotation angles with the preset 0.95 probability interval value of the left and right rotation angles and the 0.95 probability interval value of the up and down rotation angles, it can be judged whether the driver's line of sight is in the main viewing area, by calculating the time of the driver's line of sight leaving the main viewing area, and comparing with the preset 0.95 quantile, when greater than the preset 0.95 quantile, it is determined that the driver's visual distraction, similarly, by the eye movement data of the driver, the average blinking time of the driver is obtained, and the average blinking time is compared with the preset 0.95 confidence interval threshold value, in the present application, the 0.95 confidence interval threshold value directly uses the research results of Tongji University, such as the attached Figure 5As shown, it can be determined whether the driver is in a fatigue distraction situation, in this application, the gaze angle and blink time are obtained through the eye movement data, and compared with the standard gaze angle threshold interval and the blink time in the normal case, to determine whether the driver is in a distraction driving situation, the eye movement gaze tracking technology of the driver can collect eye movement state up to 60Hz video, can judge the blink time of ms level, the smaller head rotation angle, can identify whether the driver produces visual distraction and fatigue distraction from the state of the eye, compared with the video judgment in the prior art, requiring yawning action > 2s, eye closing time > 1s, driver head deflection angle greater than 45°, there is a significant progress in accuracy;

[0067] It is worth emphasizing that the determination of the fatigue state of the driver can adopt the eye closure percentage PERCLOS80, the pupil diameter coefficient of variation and the average blink time, the above process of the application adopts the average blink time, and the average blink time B ave The total blink time t b in the analysis window is divided by the number of blinks, according to the characteristics of the collected data, the total number of non-zero values n of the blink flag in the time window is statistically analyzed, the maximum value of the blink flag is b max , and the minimum value is b min . Generally, the sampling frame rate of the eye tracker is 60Hz, so the average blink time is calculated as follows:

[0068] Blink ave =n / [60(b max -b min )]

[0069] In addition, the application also discloses an eye closure percentage PERCLOS80, as shown in the accompanying drawings, Figure 6 The application also discloses a pupil diameter coefficient of variation, and its expression formula is as follows:

[0070]

[0071] In the formula, sigma is the standard deviation of the pupil diameter, The average value of the pupil diameter, the above data can also be obtained from the eye movement data.

[0072] Preferably,

[0073] The acquisition of the 0.95 probability interval value of the preset left and right rotation angle and the 0.95 probability interval value of the preset up and down rotation angle comprises:

[0074] S201, obtaining a test eye movement data of a driver in a period of time on a test vehicle, and determining a main visual area of the driver;

[0075] S202, according to the driver test eye movement data, the angle data of the left and right rotation of the eyes and the angle data of the up and down rotation of the eyes when the driver's line of sight stays in the main viewing area for a preset time period are determined;

[0076] S203, the angle data of the left and right rotation of the eyes are subjected to normal distribution fitting to obtain the 0.95 probability interval value of the left and right rotation angle, and the angle data of the up and down rotation of the eyes are subjected to logarithmic normal distribution fitting to obtain the 0.95 probability interval value of the up and down rotation angle;

[0077] It can be understood that, as shown in the accompanying Figure 2 The application also discloses a process for obtaining the 0.95 probability interval value of the left and right rotation angle and the preset 0.95 probability interval value of the up and down rotation angle, and the process is as follows: test eye movement data of a driver for a period of time is obtained on a test vehicle, a main viewing area of the driver is determined, according to the driver test eye movement data, the angle data of the left and right rotation of the eyes and the angle data of the up and down rotation of the eyes when the driver's line of sight stays in the main viewing area for a preset time period are determined, and the specific calculation principle is as shown in the accompanying Figure 8 The angle data of the left and right rotation of the eyes are subjected to normal distribution fitting to obtain the 0.95 probability interval value of the left and right rotation angle, and the angle data of the up and down rotation of the eyes are subjected to logarithmic normal distribution fitting to obtain the 0.95 probability interval value of the up and down rotation angle.

[0078] An eye movement test of the driver's line of sight tracking under different road driving environments is performed, and after the test is completed, the position points of the driver's line of sight under different road environments are output and it is judged whether the line of sight is located in the main viewing area determined in the previous step; if the line of sight is located in the main viewing area, the left and right swing angles GazeHeading and the up and down swing angles GazePitch of the driver's line of sight are output, as shown in the following table:

[0079]

[0080] The data in the table are subjected to statistical distribution probability density function fitting.

[0081] Preferably,

[0082] The preset 0.95 quantile is obtained by:

[0083] S301, when the left and right rotation angles and the up and down rotation angles of the driver's eyes respectively meet the 0.95 probability interval values, it is judged that the driver's line of sight is in the main viewing area, otherwise, the driver's line of sight is not in the main viewing area;

[0084] S302, according to the driver test eye movement data for a preset period of time, a plurality of time length data of the user's line of sight not being in the main viewing area are obtained, and the plurality of time length data are subjected to distribution statistical fitting to obtain the 0.95 quantile;

[0085] It can be understood that, as shown in the accompanying Figure 3 The application also discloses a process for obtaining the 0.95 quantile, and the process is specifically as follows: when the left and right rotation angles and the up and down rotation angles of the eyes of the driver respectively meet the 0.95 probability interval value, it is determined that the line of sight of the driver is in the main viewing area, otherwise, the line of sight of the driver is not in the main viewing area, and a plurality of time length segment data of the line of sight of the driver not in the main viewing area is obtained according to driver test eye movement data in a preset period of time, as shown in the accompanying Figure 9 The 0.95 quantile is obtained by performing distribution statistical fitting on the plurality of time length segment data, 18000 frames of eye tracker monitoring data are selected as the analysis time window, the test data is selected for 300s, and the frame rate of the eye tracker is 60Hz.

[0086] Preferably,

[0087] The determining the main viewing area of the driver comprises:

[0088] S401, a windshield of a driver of a test vehicle is divided into a plurality of sub-viewing areas, and the windshield sub-viewing areas are divided into an upper left area, a lower left area, a middle lower area, a middle upper area, a lower right area and an upper right area according to average sizes;

[0089] S402, a proportion of the driver's view point falling in a certain windshield sub-viewing area in a preset period of time is determined through driver test eye movement data;

[0090] S403, if there is a certain windshield sub-viewing area, and the time length of the driver's view point falling in the windshield sub-viewing area in the preset period of time exceeds 80% of the preset period of time, the windshield sub-viewing area is determined as the main viewing area;

[0091] S404, if not, the sub-viewing areas of the windshield are re-divided until the time length of the driver's view point falling in a certain windshield sub-viewing area in the preset period of time exceeds 80% of the preset period of time;

[0092] It can be understood that, as shown in the accompanying Figure 4 The application also discloses a process for determining the main viewing area, and the process is specifically as follows: as shown in the accompanying Figure 7 The viewing area of the driver of the test vehicle is first divided into sub-viewing areas, generally including left and right rearview mirrors, an inner rearview mirror, a front row left glass, a right glass, an instrument panel area, an electrical button area and a gear shifting area, the windshield is divided into an upper left area, a lower left area, a middle lower area, a middle upper area, a lower right area and an upper right area, the windshield sub-viewing areas are first divided according to average sizes, statistical analysis is performed on the test data, the proportion of the driver's view point falling in a certain windshield sub-viewing area in 300s exceeds 80%, and the viewing area is the main viewing area, otherwise, the sub-viewing areas of the windshield need to be re-divided until the proportion of a certain sub-viewing area exceeds 80%.

[0093] Preferably,

[0094] The preset left and right rotation angle 0.95 probability interval value and the preset up and down rotation angle 0.95 probability interval value are obtained by:

[0095] The left and right rotation angle 0.95 probability interval value or the up and down rotation angle 0.95 probability interval value under different road environments is obtained.

[0096] All the left and right rotation angle 0.95 probability interval values or the up and down rotation angle 0.95 probability interval values under different road environments are traversed until the difference of the left and right rotation angle 0.95 probability interval values or the up and down rotation angle 0.95 probability interval values under any two different road environments is calculated, and the average value of the difference is calculated.

[0097] If the average value of the difference is less than a preset first average threshold, the average value of the left and right rotation angle 0.95 probability interval values or the up and down rotation angle 0.95 probability interval values under different road environments is taken as the left and right rotation angle 0.95 probability interval value or the up and down rotation angle 0.95 probability interval value under all road environments, and if the average value of the difference is greater than or equal to the preset first average threshold, the left and right rotation angle 0.95 probability interval values or the up and down rotation angle 0.95 probability interval values under different road environments are output respectively.

[0098] It can be understood that the driver's eye movement monitoring road test environment generally considers high-speed, urban roads and national roads, and the influence of light environment needs to be considered. After the left and right rotation angle 0.95 probability interval values and the up and down rotation angle 0.95 probability interval values under different road environments are obtained, the difference of the 0.95 probability interval values under different road environments needs to be judged. Two difference judgment methods are given in this embodiment: (1) all the 0.95 probability interval values under different road environments are traversed until the difference of the 0.95 probability interval values under any two different road environments is calculated, and the average value of the difference is calculated. If the average value of the difference is less than a preset first average threshold, it is considered that the influence of different road environments on the 0.95 probability interval value is not great, and the average value of the 0.95 probability interval values under all road environments is taken as the 0.95 probability interval value common to all road environments. If the average value of the difference is greater than or equal to the preset first average threshold, it is considered that the influence of road environment on the 0.95 probability interval value is great. In this case, the 0.95 probability interval value needs to be determined considering the influence of the environment, and the 0.95 probability interval values calculated under each road environment are taken as the 0.95 probability interval values under the respective environment respectively. (2) as shown in FIG. 6, the 0.95 probability interval values under different road environments are compared, and the 0.95 probability interval value under the road environment with the maximum difference is taken as the 0.95 probability interval value under the road environment with the maximum influence. Figure 10As shown, the difference significance P value under each road environment is directly calculated, the P value is compared with the preset difference significance criterion a, if P is less than a, it is considered that the sample data has no obvious difference, and the average value of the 0.95 probability interval value of all road environments is taken as the 0.95 probability interval value shared by all road environments, if P is greater than a, it is considered that the sample data has obvious difference, and the 0.95 probability interval value calculated under each road environment is taken as the 0.95 probability interval value under the respective environment.

[0099] Preferably,

[0100] The preset 0.95 quantile is obtained by:

[0101] The 0.95 quantile under different road environments is obtained, and the 0.95 quantiles under all different road environments are traversed until the difference value of the 0.95 quantiles under any two different road environments is calculated.

[0102] The average value of the difference value is calculated, if the average value of the difference value is less than the preset second average threshold, the average value of the 0.95 quantiles under different road environments is taken as the 0.95 quantile under all road environments, if the average value of the difference value is greater than or equal to the preset second average threshold, the 0.95 quantiles under different road environments are respectively outputted.

[0103] It can be understood that, similarly, the 0.95 quantile also needs to consider the influence of the road environment, and the difference is also determined by the above two methods, which will not be described in detail here.

[0104] Embodiment two

[0105] Figure 11 It is a system schematic diagram of a driver distraction driving recognition device according to another exemplary embodiment, comprising:

[0106] The angle data acquisition module 1 is used to acquire the eye movement data of the driver through the eye tracker during driving, and the left and right rotation angles and the up and down rotation angles of the eyes of the driver are acquired according to the eye movement data of the driver.

[0107] The main visual field judgment module 2 is used to judge that the line of sight of the driver leaves the main visual field when the left and right rotation angles of the eyes of the driver do not satisfy the preset 0.95 probability interval value of the left and right rotation angles, or the up and down rotation angles of the eyes of the driver do not satisfy the preset 0.95 probability interval value of the up and down rotation angles.

[0108] The visual distraction judgment module 3 is used to count the time length when the line of sight of the driver leaves the main visual field, and if the time length when the line of sight of the driver leaves the main visual field exceeds the preset 0.95 quantile, it is determined that the driver is visually distracted.

[0109] The blink data acquisition module 4 is configured to acquire the total blink time and the blink times of the driver in a preset time period according to the eye movement data of the driver acquired by the eye tracker;

[0110] The fatigue distraction judgment module 5 is configured to obtain the average blink time by dividing the total blink time by the blink times, compare the average blink time with the preset 0.95 confidence interval threshold, and determine that the driver is fatigued and distracted if the average blink time exceeds the preset 0.95 confidence interval threshold.

[0111] It can be understood that the application also discloses a system diagram of a driver distraction driving recognition device. In the driving process, the angle data acquisition module 1 acquires the eye movement data of the driver by the eye tracker, and acquires the left and right rotation angles and the up and down rotation angles of the eyes of the driver according to the eye movement data of the driver. The main visual field judgment module 2 judges that the line of sight of the driver deviates from the main visual field when the left and right rotation angles of the eyes of the driver do not satisfy the 0.95 probability interval value of the preset left and right rotation angles, or the up and down rotation angles of the eyes of the driver do not satisfy the 0.95 probability interval value of the preset up and down rotation angles. The visual distraction judgment module 3 counts the time length when the line of sight of the driver deviates from the main visual field, and determines that the driver is visually distracted if the time length exceeds the preset 0.95 quantile. The blink data acquisition module 4 acquires the total blink time and the blink times of the driver in a preset time period according to the eye movement data of the driver acquired by the eye tracker. The fatigue distraction judgment module 5 obtains the average blink time by dividing the total blink time by the blink times, compares the average blink time with the preset 0.95 confidence interval threshold, and determines that the driver is fatigued and distracted if the average blink time exceeds the preset 0.95 confidence interval threshold. The application acquires the line of sight angle and the blink time through the eye movement data, and compares them with the standard line of sight angle threshold interval and the blink time in the normal case, to determine whether the driver is in the distraction driving condition. The eye movement line of sight tracking technology of the driver can collect the eye movement state at a video rate of up to 60Hz, can judge the blink time of ms level, the smaller head rotation angle, and can identify whether the driver is visually distracted and fatigued from the eye state. Compared with the video judgment in the prior art, the yawning action is required to be >2s, the eye closing time is required to be >1s, and the head deflection angle of the driver is required to be greater than 45°. The precision has been significantly improved.

[0112] Embodiment three

[0113] The embodiment provides a storage medium storing a computer program, and the computer program is executed by a host computer to implement each step in the above method.

[0114] It can be understood that the storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk.

[0115] It can be understood that the same or similar parts in the above embodiments can be mutually referenced, and the content not described in detail in some embodiments can refer to the same or similar content in other embodiments.

[0116] It should be noted that in the description of the present application, the terms "first", "second", etc. are only for the purpose of description, and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is at least two.

[0117] Any process or method descriptions in flow charts or otherwise described herein represents an example of embodiments of the present application that can be embodied in code means that include one or more steps for accomplishing a specified logical function or process. The scope of embodiments of the present application encompass hardware and software implementations of the process or method described in flow charts or otherwise described herein, and the preferred embodiments of the present application include additional implementation in which the functions described in the flow charts or otherwise described herein are performed in an order different from that shown or discussed, including substantially simultaneously, and in reverse order, as will be understood by those skilled in the art of the embodiments of the present application.

[0118] It should be understood that parts of the present application can be realized in hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized in hardware, and as in another embodiment, it can be realized by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logical functions on data signals, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0119] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, which includes one or a combination of steps of the method embodiments when executed.

[0120] In addition, each functional unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically separately, or two or more units can be integrated in one module. The above integrated module can be realized in the form of hardware or in the form of software functional module. The integrated module, if realized in the form of software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0121] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0122] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in an appropriate manner.

[0123] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary, and are not to be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

Claims

1. A method of identifying driver inattentive driving, characterized by, The method comprises: During driving, the eye movement data of the driver is acquired by the eye tracker, and the left and right rotation angles and the up and down rotation angles of the eyes of the driver are acquired according to the eye movement data of the driver; When the left and right rotation angles of the eyes of the driver do not satisfy the 0.95 probability interval value of the preset left and right rotation angles, or the up and down rotation angles of the eyes of the driver do not satisfy the 0.95 probability interval value of the preset up and down rotation angles, it is judged that the line of sight of the driver leaves the main viewing area; The acquisition of the 0.95 probability interval value of the preset left and right rotation angles and the 0.95 probability interval value of the preset up and down rotation angles comprises: The test eye movement data of the driver in a period of time is acquired on the test vehicle, and the main viewing area of the driver is determined; According to the test eye movement data of the driver, the angle data of the left and right rotation of the eyes of the driver and the angle data of the up and down rotation of the eyes of the driver when the line of sight of the driver stays in the main viewing area in a preset time period are determined; The angle data of the left and right rotation of the eyes is subjected to normal distribution fitting to obtain the 0.95 probability interval value of the left and right rotation angles, and the angle data of the up and down rotation of the eyes is subjected to logarithmic normal distribution fitting to obtain the 0.95 probability interval value of the up and down rotation angles; The duration of the line of sight of the driver leaving the main viewing area is counted, and if the duration of the line of sight leaving the main viewing area exceeds the preset 0.95 quantile, it is determined that the driver is visually distracted; The acquisition of the preset 0.95 quantile comprises: When the left and right rotation angles and the up and down rotation angles of the eyes of the driver respectively satisfy the 0.95 probability interval value, it is judged that the line of sight of the driver is in the main viewing area, otherwise, the line of sight of the driver is not in the main viewing area; According to the test eye movement data of the driver in a preset period of time, a plurality of duration data of the line of sight of the user not being in the main viewing area is acquired, and the 0.95 quantile is obtained by distribution statistical fitting of the plurality of duration data; According to the eye movement data of the driver acquired by the eye tracker, the total blinking time and the blinking frequency of the driver in a preset time period are acquired; The average blinking time is obtained by the total blinking time and the blinking frequency, and the average blinking time is compared with the preset 0.95 confidence interval threshold value, and if the average blinking time exceeds the preset 0.95 confidence interval threshold value, it is determined that the driver is fatigued and distracted.

2. The method of claim 1, wherein The determination of the main viewing area of the driver comprises: The windshield of the test vehicle is divided into a plurality of sub-viewing areas, and the windshield sub-viewing areas are divided into upper left, lower left, middle lower, middle upper, lower right and upper right areas according to the average size; The proportion of the driver's visual point falling in a certain windshield sub-viewing area in a preset time period is determined according to the test eye movement data of the driver, and if there is a certain windshield sub-viewing area such that the duration of the driver's visual point falling in the windshield sub-viewing area in the preset time period exceeds 80% of the preset time period, the windshield sub-viewing area is the main viewing area, and if not, the sub-viewing areas of the windshield are re-divided until the duration of the driver's visual point falling in a certain windshield sub-viewing area in the preset time period exceeds 80% of the preset time period.

3. The method of claim 2, wherein The obtaining of the preset 0.95 probability interval value of the left and right rotation angle and the preset 0.95 probability interval value of the up and down rotation angle further includes: obtaining the 0.95 probability interval value of the left and right rotation angle or the 0.95 probability interval value of the up and down rotation angle under different road environments; traversing the 0.95 probability interval value of the left and right rotation angle or the 0.95 probability interval value of the up and down rotation angle under all different road environments until the difference value of the 0.95 probability interval value of the left and right rotation angle or the 0.95 probability interval value of the up and down rotation angle under any two different road environments is calculated, and the average value of the difference value is calculated; if the average value of the difference value is less than a preset first average threshold, the average value of the 0.95 probability interval value of the left and right rotation angle or the 0.95 probability interval value of the up and down rotation angle under different road environments is taken as the 0.95 probability interval value of the left and right rotation angle or the 0.95 probability interval value of the up and down rotation angle under all road environments, and if the average value of the difference value is greater than or equal to the preset first average threshold, the 0.95 probability interval value of the left and right rotation angle or the 0.95 probability interval value of the up and down rotation angle under different road environments is output respectively.

4. The method of claim 3, wherein the obtaining of the preset 0.95 quantile further includes: obtaining the 0.95 quantile under different road environments, traversing the 0.95 quantile under all different road environments until the difference value of the 0.95 quantile under any two different road environments is calculated; calculating the average value of the difference value, if the average value of the difference value is less than a preset second average threshold, the average value of the 0.95 quantile under different road environments is taken as the 0.95 quantile under all road environments, and if the average value of the difference value is greater than or equal to the preset second average threshold, the 0.95 quantile under different road environments is output respectively. The device includes:

5. A driver distraction recognition apparatus characterized by comprising: an angle data acquisition module configured to acquire eye movement data of a driver through an eye tracker during driving, and acquire a left and right rotation angle and an up and down rotation angle of an eye of the driver according to the eye movement data of the driver; a main viewing area judgment module configured to judge that a line of sight of the driver is out of a main viewing area when the left and right rotation angle of the eye of the driver does not satisfy a preset 0.95 probability interval value of the left and right rotation angle or the up and down rotation angle of the eye of the driver does not satisfy a preset 0.95 probability interval value of the up and down rotation angle; the obtaining of the preset 0.95 probability interval value of the left and right rotation angle and the preset 0.95 probability interval value of the up and down rotation angle includes: acquiring test eye movement data of the driver for a period of time on a test vehicle, and determining a main viewing area of the driver; determining angle data of left and right rotation of the eye of the driver and angle data of up and down rotation of the eye of the driver when the line of sight of the driver stays in the main viewing area within a preset time period according to the test eye movement data of the driver; ​ The angle data of left and right rotation of the eyes is fitted with a normal distribution to obtain a 0.95 probability interval value of the left and right rotation angle, and the angle data of up and down rotation of the eyes is fitted with a logarithmic normal distribution to obtain a 0.95 probability interval value of the up and down rotation angle; The visual distraction judgment module is configured to count the time length of the driver's line of sight leaving the main view area, and determine that the driver is visually distracted if the time length of the line of sight leaving the main view area exceeds a preset 0.95 quantile; The preset 0.95 quantile is obtained by: When the left and right rotation angles and the up and down rotation angles of the driver's eyes respectively satisfy the 0.95 probability interval value, it is determined that the driver's line of sight is in the main view area, otherwise, the driver's line of sight is not in the main view area; The 0.95 quantile is obtained by fitting the distribution statistics of a plurality of time length data of the driver's line of sight not being in the main view area according to the driver's test eye movement data in a preset period of time; The blink data acquisition module is configured to acquire the total blink time and the blink frequency of the driver in a preset period of time according to the eye movement data of the driver acquired by the eye tracker; The fatigue distraction judgment module is configured to obtain a blink time average value by comparing the total blink time with the blink frequency, compare the blink time average value with a preset 0.95 confidence interval threshold, and determine that the driver is fatigued and distracted if the blink time average value exceeds the preset 0.95 confidence interval threshold.

6. A storage medium, characterized by The storage medium stores a computer program, and the computer program is executed by the host controller to implement each step of the driver distraction driving recognition method according to any one of claims 1-4.

Images