A driver gaze direction implicit calibration method and device based on a gaze pattern

Abstract

The application provides a driver gaze direction implicit calibration method and device based on a gaze pattern. The method comprises: acquiring serialized driver gaze feature data; performing global clustering on the head posture data to estimate a gaze area; selecting a movement trajectory segment of the driver's head posture between the gaze areas to obtain gaze pattern data; projecting a gaze direction point of a real trajectory onto a virtual trajectory; establishing a gaze estimation model between the gaze feature data and the gaze direction point pair, and mapping the gaze feature data to the calibrated gaze direction and gaze point based on mapping parameters. The application can implicitly extract driver gaze pattern data in a short normal driving process without any cooperation of the driver, unsupervised online train a gaze direction estimation model, automatically adjust model parameters, and enable the gaze estimation model to quickly and accurately estimate the driver's gaze direction after the environment changes.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of driving safety and driver monitoring, in particular to the field of driver gaze region estimation, and more particularly to a driver gaze direction implicit calibration method and device based on gaze patterns. BACKGROUND

[0002] Cars have brought convenience to our daily life, and with the development of car intelligence and electrification, future cars will inevitably play a more important role in our life. However, with the increasing number of cars, traffic accidents occur frequently, and the overall harm to society also becomes greater and greater. In the driving scene, most accidents are caused by human factors of drivers, and the driver's distracted driving behavior will reduce the driver's alertness to the environment, thereby prolonging the reaction time and affecting the driver's handling of the road conditions and emergency situations. Driver distraction driving is one of the important reasons for traffic accidents.

[0003] Gaze direction is a key indicator of driver attention evaluation. When the gaze direction frequently deviates from the straight-ahead direction of driving, it is easy to cause lane line deviation, rear-end collision, emergency braking and other dangerous situations. Long-term residence of gaze direction is also taboo in safe driving of drivers. Through the gaze direction estimation result, the driver's gaze direction residence time, gaze direction change of environmental reaction time, etc. can be further calculated. These information has direct relevance to driver attention evaluation and safe driving analysis.

[0004] The existing driver gaze direction calibration method usually needs to explicitly calibrate the head posture and line-of-sight direction of a specific driver in a fixed scene in advance. When the camera position deviates, the driver's sitting posture is adjusted or the driver is replaced, the estimation accuracy will be severely reduced, and the adaptability to individual drivers is poor. SUMMARY

[0005] In view of the technical problem of the above-mentioned existing method that the gaze direction calibration precision is seriously dependent on specific scenes and poor adaptability, a driver gaze direction implicit calibration method and device based on gaze patterns are provided. The present application does not require any cooperation of the driver, and can implicitly extract the driver's gaze pattern data in a short normal driving process, unsupervised online train the gaze direction estimation model, and automatically adjust the model parameters, so that the gaze estimation model can still quickly and accurately estimate the driver's gaze direction after the environment changes.

[0006] The technical means adopted by the present application are as follows:

[0007] A driver gaze direction implicit calibration method based on gaze patterns, comprising the following steps:

[0008] acquire a driving real-time video stream collected by a camera, and perform face recognition on the real-time video stream to acquire sequenced driver gaze feature data, the driver gaze feature data including head posture data, head position data, and human eye rotation angle data with time labels;

[0009] perform global clustering on the head posture data, and perform gaze region estimation according to complementarity constraints and boundary constraints, including front gaze region estimation and rearview mirror gaze region estimation;

[0010] select a motion trajectory segment of the driver head posture between gaze regions, the gaze points at both ends of the trajectory being located at the center anchor points of the corresponding gaze regions, and the included angle between the connection line of the gaze points in the trajectory and the connection line of the gaze points at both ends satisfying a preset condition, and taking the selected motion trajectory segment as gaze pattern data;

[0011] calculate the head rotation momentum relationship of the start and end points of the gaze pattern data, project the gaze pattern data of the real trajectory onto a virtual trajectory, the virtual trajectory being a straight line connecting the center points of the gaze regions, and for a region with dense real trajectory data, obtain a gaze direction point pair by using a linear relationship constraint, the gaze direction point pair including the gaze pattern data and a gaze point corresponding to the gaze pattern data, the gaze point being the position of the gaze pattern data projected onto the virtual trajectory;

[0012] construct and train a gaze estimation model by a machine learning method, the gaze estimation model being used to learn the mapping relationship between the gaze feature data and the gaze points, and map the to-be-processed gaze feature data to the calibrated gaze points based on the mapping parameters obtained by training, and obtain a gaze direction according to the gaze feature data and the position of the calibrated gaze points.

[0013] Further, the step of estimating the front gaze region includes:

[0014] acquire driver head motion momentum data from the driver gaze feature data, perform region density clustering on the points of the driver head motion momentum data satisfying a preset requirement, and take the global dense center points as the center anchor points of the front gaze region.

[0015] Further, the step of estimating the rearview mirror gaze region includes:

[0016] acquire time sequence data with a duration of more than 10 frames as candidate data of gaze behavior, and take the data point farthest from the center anchor point of the front gaze region as a rearview mirror gaze candidate point;

[0017] According to cosine similarity between rearview mirror gaze candidate points, the rearview mirror gaze candidate points are put into corresponding clustering areas, and different clustering areas are clustered by using Gaussian models respectively to obtain clustering center points as center anchor points of corresponding gaze areas.

[0018] According to the topological structure between the gaze areas, labels with center position information of the gaze areas are assigned to different gaze areas, and the labels include a left rearview mirror area, a right rearview mirror area and a central rearview mirror area.

[0019] Further, after the gaze mode data is obtained, the gaze mode data is further screened, and the screening step includes:

[0020] Points with positive acceleration in the gaze mode data are obtained.

[0021] Trajectories with a number of occurrences not satisfying a preset requirement are removed.

[0022] Trajectories with a difference from other motion trajectories not satisfying a preset requirement are removed.

[0023] The application further discloses a driver gaze direction implicit calibration device based on a gaze mode, which comprises:

[0024] A gaze feature data acquisition unit is configured to acquire a real-time driving video stream collected by a camera, and perform face recognition on the real-time video stream to obtain serialized driver gaze feature data, wherein the driver gaze feature data includes head posture data, head position data and eye rotation angle data with time labels.

[0025] A gaze area estimation unit is configured to perform global clustering on the head posture data, and perform gaze area estimation according to complementary constraints and boundary constraints, including front gaze area estimation and rearview mirror gaze area estimation.

[0026] A gaze mode data acquisition unit is configured to select a motion trajectory segment of a driver head posture between gaze areas, wherein gaze points at both ends of the trajectory are located at center anchor points of corresponding gaze areas, and an included angle between a connection line of the gaze points in the trajectory and a connection line of the gaze points at both ends is less than 5 degrees, and the selected motion trajectory segment is taken as gaze mode data.

[0027] A gaze direction point pair extraction unit is configured to calculate a head rotation momentum relationship of trajectory start and end points in the gaze mode data, project real trajectory gaze mode data onto a virtual trajectory, and obtain gaze direction point pairs by using linear relationship constraints for a region with dense real trajectory data, wherein the gaze direction point pairs include gaze mode data and a gaze point corresponding to the gaze mode data, and the gaze point is a position of the gaze mode data projected onto the virtual trajectory.

[0028] A calibration unit is configured to construct and train a gaze estimation model by a machine learning method, the gaze estimation model is configured to learn a mapping relationship between gaze feature data and a gaze point, and map the to-be-processed gaze feature data to a calibrated gaze point based on the mapping parameters obtained by training, and obtain a gaze direction according to the gaze feature data and the position of the calibrated gaze point.

[0029] Compared with the prior art, the present application has the following advantages:

[0030] The present application provides a driver gaze direction implicit calibration method based on gaze pattern. Compared with the prior art, the present application has the following advantages: strong generalization ability, no dependence on vehicle, camera position, camera angle, etc., no need for gaze calibration of specific driver in fixed scene in advance, automatic completion of driver gaze estimation recalibration for calibrated or uncalibrated system during or after system running, no need for other manual intervention, flexible online application, and effective monitoring of driver attention state. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A flow chart of a driver gaze direction implicit calibration method based on gaze pattern.

[0033] Figure 2 A specific process for gaze direction calibration in the embodiment. DETAILED DESCRIPTION

[0034] In order to make the person skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0035] A driver gaze direction implicit calibration method based on gaze pattern, the main steps include the following steps:

[0036] S1, acquiring a driving real-time video stream collected by a camera, and performing face recognition on the real-time video stream to obtain sequenced driver gaze feature data, the driver gaze feature data including head posture data, head position data and human eye rotation angle data with time labels.

[0037] Further, the method further comprises: S11, tracking and calculating motion parameters according to the sequenced driver gaze feature data by using a prediction tracking algorithm, and filtering the driver gaze features. Preferably, the calculated motion parameters at least include motion momentum of the driver's head.

[0038] S2, performing global clustering on the head posture data, and performing gaze region estimation according to complementary constraints and boundary constraints, including front gaze region estimation and rearview mirror gaze region estimation.

[0039] In the present application, the complementary constraint refers to the complementary relationship between the head posture data and the gaze point position data, which can be solved by a simple linear function. The boundary constraint refers to that for a two-dimensional gaze plane, the position of the gaze point should be located inside the display boundary of the plane, and preferably, in the present application, the gaze point position coordinates in the horizontal direction and the vertical direction are greater than zero and less than the maximum value of the boundary.

[0040] Specifically, the step of estimating the front gaze region includes: clustering points with small head motion momentum in region density, and taking the global dense center point as the center anchor point of the front gaze region. Further, the head motion momentum calculation method is: subtracting the head posture estimated in the current image frame from the head posture estimated in the previous three image frames, calculating the weighted head motion difference, and averaging. Whether the head motion is small enough is judged, including: calculating the average cumulative motion of the head motion momentum, setting the judgment threshold as half of the average cumulative motion, and the data smaller than the threshold has a small enough head motion momentum. Considering the jitter noise of the real scene in the vehicle, the Euler angle value is generally less than 5 degrees.

[0041] The step of estimating the rearview mirror gaze region includes: acquiring time sequence data with a duration of more than 10 frames as candidate data of gaze behavior, wherein the data point farthest from the center anchor point of the front gaze region is the rearview mirror gaze candidate point; according to the cosine similarity between the candidate points, the candidate points are put into the corresponding clustering region, and different clustering regions are clustered by using a Gaussian model respectively to obtain the clustering center point as the center anchor point of the corresponding gaze region; according to the topological structure between the gaze regions, different gaze regions are assigned with labels, the labels include left rearview mirror region, right rearview mirror region and central rearview mirror region.

[0042] Specifically, first, continuous time series data is acquired, for data greater than 10 frames of time series data, the Euclidean distance between all data and the center point of the front gaze area is calculated, and the maximum value or the last data in all data is the gaze candidate point. Secondly, the candidate point with the maximum distance between the front gaze area center is the left rearview mirror gaze candidate point or the right rearview mirror gaze candidate point, according to the left and right directions, the data belonging to the left rearview mirror gaze area or the right rearview mirror gaze area can be distinguished.

[0043] In the present application, the region to be clustered includes the left rearview mirror area, the right rearview mirror area, the central rearview mirror area, the center console area, and the instrument panel area. Here, the region to be clustered data and the front gaze area data are collected in advance, and the cosine similarity threshold between the regions is set. Then, according to the similarity of the candidate point with the previously preset data, the probability of the candidate point belonging to the region to be clustered is judged, and the data with a probability greater than 0.9 is put into the corresponding region to be clustered for subsequent operation.

[0044] S3, selecting the motion trajectory segment of the driver's head posture between the gaze areas, the gaze points at both ends of the trajectory should be as close as possible to the center anchor point of the corresponding gaze area, and the angle between the connection line of the gaze points in the trajectory and the connection line of the two end gaze points is less than 5 degrees, and the selected motion trajectory segment is taken as the gaze mode data.

[0045] In the present application, in order to realize that the gaze points at both ends of the trajectory should be as close as possible to the center anchor point of the corresponding gaze area, the sum of the distances of the gaze points at both ends of the trajectory and the center anchor point of the corresponding gaze area is calculated, the sum of the distances is arranged from small to large, and the first ten percent of the data is selected as the selected motion trajectory segment.

[0046] Further, the method further comprises: S31, screening the gaze mode data to obtain points with positive acceleration in the motion momentum of the gaze mode data; removing trajectories with less occurrence frequency; and removing trajectories with large differences from other motion trajectories. Specifically, when the local density of the trajectory is less than 10, it can be considered that the occurrence frequency is less. The cosine similarity between the trajectories is calculated, and the similarity less than 0.6 can be considered as a large difference. S4, calculating the head rotation momentum relationship of the trajectory start and end points in the gaze mode data projects the gaze mode data of the real trajectory onto a virtual trajectory, the virtual trajectory is a straight line connecting the center points of the gaze areas, for the area with dense real trajectory data, the gaze direction point pair is obtained by using the linear relationship constraint, the gaze direction point pair includes the gaze mode data and the gaze point corresponding to the gaze mode data, the gaze point is the position of the gaze mode data projected onto the virtual trajectory.

[0047] Specifically, the local density of single data and the local density of trajectory data are calculated in the following manner:

[0048] The local density calculation method of single data: neighborhood point number ρ = ∑ i′ sgn(d ii′ -d c ), minimum adjacent distance δ = min i'∈Ω-i d ii' , wherein sgn(·) is a sign function, sgn(·) = 1 when the value is negative, otherwise sgn(·) = 0, d ii′ represents the cluster between the point i and i ′ , d c represents the truncation distance of the local neighborhood radius, and Ω represents all points in the local neighborhood. In the dense area, the neighborhood point number should be larger, and the minimum adjacent distance should be smaller.

[0049] The local density calculation method of trajectory data: set the local neighborhood as the center point of the gaze area, calculate the local density of all points on the trajectory, and obtain the local density of the trajectory data by using the Gaussian weighted average, including the neighborhood point number less than the truncation distance and the minimum adjacent distance. In the driving scene, the local density threshold of the trajectory data between the gaze areas is set to 30 to increase the stability of the method. S5, an eye gaze estimation model is constructed and trained by a machine learning method, the eye gaze estimation model is used to learn the mapping relationship between the eye gaze feature data and the eye gaze point, and the eye gaze feature data to be processed is mapped to the calibrated eye gaze point based on the mapping parameters obtained by training, and the eye gaze direction is obtained according to the eye gaze feature data and the position of the calibrated eye gaze point.

[0050] The application preferably uses head pose data to establish an initial eye gaze estimation model, which can be least square regression, support vector machine regression, Gaussian process regression, deep convolutional network, etc. The model realizes the conversion between the head pose space and the eye gaze point space, that is, the eye gaze feature data is input, and the eye gaze point data is output. The initial eye gaze estimation model outputs the initial eye gaze point data. The eye gaze point data satisfies the complementary constraint and the boundary constraint. Preferably, the eye gaze feature data includes: face feature point position data (128 dimensions), head pose data (2 dimensions), and eye line of sight angle data (2 dimensions), which can be calculated by a trained face feature point positioning model and algorithm. The eye gaze direction is an angle value, the landing point of the eye gaze direction on the gaze plane is the eye gaze point, and the eye gaze point is the value of the eye gaze direction on the plane position.

[0051] The scheme and effect of the application will be further described below through a specific application example:

[0052] For example Figure 2As shown, a driver gaze direction implicit calibration method based on gaze pattern is provided in the embodiment. The specific technical solutions are as follows.

[0053] 1. Driver data initialization

[0054] Real-time video stream of the driving process is acquired through a camera placed in front of the driver, and driver gaze feature data in sequence is extracted from the video stream by using face recognition analysis technology, specifically head posture, head position, and eye rotation angle data with time label index within a period of time.

[0055] 2. Driver state tracking

[0056] In order to eliminate the noise interference of the complexity and uncertainty of driving behavior on the driver gaze feature, it is assumed that the driver's head is a rigid body, and a prediction tracking algorithm is used to track and calculate the motion parameters according to the driver gaze feature data in sequence (head position, head rotation, and eye rotation), filter the extracted driver gaze feature, realize the optimal estimation of the driver gaze state, and remove the image noise interference.

[0057] 3. Front gaze area estimation

[0058] During driving, most of the time, the driver's line of sight is long-term focused on the road, that is, the gaze area in front. In the gaze samples within a period of time, most of the driver's gaze points will fall within the front gaze area, and the head movement amplitude is small. Therefore, according to this feature, the front gaze area estimation is realized, the points with small head movement momentum are clustered in the area density, and the global dense center point is taken as the center anchor point of the forward gaze area, that is, the zero point of the head posture.

[0059] 4. Rearview mirror gaze area estimation

[0060] Although the driver's attention is mainly focused on the forward gaze area, it is necessary to confirm the safety of the surrounding environment by constantly checking the rearview mirror, which will be accompanied by a large head posture change. Among them, the driver checks the rearview mirror, including the left rearview mirror, the right rearview mirror and the central rearview mirror. When the driver looks at the rearview mirror, the momentum of the head movement is large, and a discrete feature point appears on the time series data, which leaves the center anchor point of the forward gaze area, reaches the center of the rearview mirror feature, and returns to the center anchor point of the forward gaze area. Select time series data with a duration of more than 10 frames as candidate data for gaze behavior, and the data point farthest from the center anchor point of the forward gaze area is the rearview mirror gaze candidate point. According to the cosine similarity between the candidate points, the candidate points are put into the corresponding clustering area. For different clustering areas, respectively use Gaussian model for clustering, and different clustering centers are the center anchor points of the corresponding gaze area. According to the topological structure between the gaze areas, different gaze areas are assigned labels with gaze area center position information.

[0061] 5. Gaze mode data acquisition

[0062] The driver's own behavior of looking at each gaze area is the driver's gaze mode. In order to find effective gaze mode data, the driver's head posture motion trajectory segment between gaze areas is selected, and the gaze points at both ends of the trajectory should be as close as possible to the center anchor points of the corresponding gaze area, and the angle between the connection line of the gaze points in the trajectory and the connection line of the two end gaze points should be less than 5 degrees. According to the head motion trajectory, only the transfer trajectory from the forward gaze area to other gaze areas is selected at present, and the motion trajectory is closer to straight line motion at this time.

[0063] 6. Gaze mode data screening

[0064] According to the motion relationship, the motion between the forward gaze area and other gaze areas should be from fast to slow, and when approaching other gaze areas, eye movement dominates, and head movement slows down. Adjust the eye and head movement to gaze at the corresponding target. Therefore, in order to ensure the accuracy of the trajectory, only the points with positive acceleration are selected as the motion points in the trajectory. According to the number of times the driver's gaze mode occurs, remove the trajectories with fewer occurrences. At the same time, judge the similarity between the trajectories and remove the trajectories with large differences from other motion trajectories.

[0065] 7. Gaze mode data mapping

[0066] For a certain specific gaze pattern data, its true motion trajectory satisfies the complementarity constraint of head posture, the center prior constraint of gaze point space and the display boundary constraint. Since the true trajectory has great distortion, the head rotation momentum relationship of the start and end point trajectory is calculated, and the gaze direction point of the true trajectory is projected onto the virtual trajectory (a straight line connecting the center points of the gaze area). For the area with dense trajectory data, the gaze direction point pair is obtained by using linear relationship constraint. The label of the gaze direction point pair is the position on the virtual trajectory. The gaze point position is updated periodically.

[0067] 8. Gaze direction calibration

[0068] Important gaze features are extracted from the serialized driver gaze feature data, a gaze estimation model is established between the gaze features and the gaze direction point pairs, the model is trained by machine learning method, and the mapping parameters between the gaze feature mapping and the gaze point are learned periodically. Mapping the gaze feature data to the correct gaze position and direction, i.e. realizing the gaze direction calibration.

[0069] The application also discloses a driver gaze direction implicit calibration device based on gaze pattern, comprising:

[0070] A gaze feature data acquisition unit is used to acquire the real-time driving video stream collected by a camera, and perform face recognition on the real-time video stream to obtain serialized driver gaze feature data, the driver gaze feature data including head posture data, head position data and eye rotation angle data with time labels;

[0071] A gaze area estimation unit is used to perform global clustering on the head posture data, and estimate the gaze area according to the complementarity constraint and the boundary constraint, including front gaze area estimation and rearview mirror gaze area estimation;

[0072] A gaze pattern data acquisition unit is used to select the motion trajectory segments of the driver head posture between the gaze areas, the gaze points at both ends of the trajectory are located at the center anchor points of the corresponding gaze areas, and the included angle between the connecting line of the gaze points in the trajectory and the connecting line of the two end gaze points is less than 5 degrees, and the selected motion trajectory segments are used as the gaze pattern data;

[0073] A gaze direction point pair extraction unit is used to calculate the head rotation momentum relationship of the start and end points of the gaze pattern data, and project the gaze pattern data of the true trajectory onto the virtual trajectory, the virtual trajectory being a straight line connecting the center points of the gaze areas, for the area with dense true trajectory data, the gaze direction point pair is obtained by using linear relationship constraint, the gaze direction point pair including the gaze pattern data and the gaze point corresponding to the gaze pattern data, the gaze point being the position of the gaze pattern data projected onto the virtual trajectory;

[0074] a calibration unit configured to construct and train a gaze estimation model by a machine learning method, the gaze estimation model being configured to learn a mapping relationship between gaze feature data and a gaze point, and to map the gaze feature data to be processed to a calibrated gaze point based on the mapping parameters obtained by the training, and to obtain a gaze direction according to the gaze feature data and the position of the calibrated gaze point.

[0075] For the embodiment of the driver gaze direction implicit calibration device based on gaze pattern, since it corresponds to the above embodiment of the driver gaze direction implicit calibration method based on gaze pattern, the description is relatively simple, and the relevant similar parts can be seen from the above embodiment of the driver gaze direction implicit calibration method based on gaze pattern, and will not be described in detail here.

[0076] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for implicit calibration of driver gaze direction based on gaze pattern, characterized in that, The method comprises the following steps: acquiring a driving real-time video stream collected by a camera, and performing face recognition on the real-time video stream to obtain sequenced driver gaze feature data, the driver gaze feature data comprising head posture data, head position data and human eye rotation angle data with time labels; globally clustering the head posture data, and estimating gaze regions according to complementarity constraints and boundary constraints, including front gaze region estimation and rearview mirror gaze region estimation; the complementarity constraints refer to the complementary relationship between the head posture data and the gaze point position data; the boundary constraints refer to the gaze point position coordinates in the horizontal direction and the vertical direction of the two-dimensional gaze plane being greater than zero and less than the maximum boundary value; selecting a motion trajectory segment of the driver's head posture between the gaze regions, the gaze points at both ends of the trajectory being located at the center anchor points of the corresponding gaze regions, and the angle between the connecting line of the gaze points in the trajectory and the connecting line of the gaze points at both ends satisfying a preset condition, and taking the selected motion trajectory segment as gaze mode data; wherein the way of selecting the motion trajectory segment of the driver's head posture between the gaze regions comprises: calculating the distance sum of the gaze points at both ends of the trajectory and the center anchor points of the corresponding gaze regions, arranging the distance sum values from small to large, and selecting the top ten percent of the data as the selected motion trajectory segment; screening the gaze mode data, the screening steps comprising: acquiring points with positive acceleration in the motion momentum in the gaze mode data; removing trajectories with a number of occurrences that do not meet a preset requirement; removing trajectories with a difference from other motion trajectories that does not meet a preset requirement; calculating the head rotation momentum relationship of the start and end points of the trajectory in the gaze mode data, projecting the real trajectory gaze mode data onto a virtual trajectory, the virtual trajectory being a straight line connecting the center points of the gaze regions, and for regions with dense real trajectory data, using linear relationship constraints to obtain gaze direction point pairs, the gaze direction point pairs comprising gaze mode data and gaze points corresponding to the gaze mode data, the gaze points being the positions of the gaze mode data projected onto the virtual trajectory; local density calculation of the trajectory data comprises: setting the local neighborhood as the connection between the center points of the gaze regions, calculating the local density of all points on the trajectory, and using Gaussian weighted average to obtain the local density of the trajectory data, and setting the local density threshold of the trajectory data between the gaze regions to 30; constructing and training a gaze estimation model through a machine learning method, the gaze estimation model being used to learn the mapping relationship between the gaze feature data and the gaze points, and mapping the to-be-processed gaze feature data to the calibrated gaze points based on the mapping parameters obtained through the training, and obtaining the gaze direction according to the gaze feature data and the position of the calibrated gaze points.

2. The method according to claim 1, wherein, The step of estimating the front gaze region comprises: obtaining driver head motion momentum data from the driver gaze feature data, clustering the driver head motion momentum data points that meet a preset requirement in terms of region density, and taking the global dense center points as the center anchor points of the front gaze region.

3. The method according to claim 2, wherein, The step of estimating the rearview mirror gaze region comprises: Obtain time series data with each duration greater than 10 frames as candidate data of gaze behavior, and the data point farthest from the center anchor point of the front gaze region as the rearview mirror gaze candidate point; According to the cosine similarity between the rearview mirror gaze candidate points, the rearview mirror gaze candidate points are put into the corresponding clustering region, and the Gaussian model is used for clustering in different clustering regions to obtain the clustering center point as the center anchor point of the corresponding gaze region; According to the topological structure between the gaze regions, different gaze regions are assigned with labels with gaze region center position information, and the labels include left rearview mirror region, right rearview mirror region and central rearview mirror region.

4. A gaze pattern based driver gaze direction implicit calibration apparatus for implementing a gaze pattern based driver gaze direction implicit calibration method as claimed in any one of claims 1 to 3, characterized by It comprises: A gaze feature data acquisition unit is configured to acquire a real-time video stream of driving collected by a camera, and perform face recognition on the real-time video stream to obtain serialized driver gaze feature data, which includes head posture data, head position data and eye rotation angle data with time labels; A gaze region estimation unit is configured to perform global clustering on the head posture data, and perform gaze region estimation according to complementary constraints and boundary constraints, including front gaze region estimation and rearview mirror gaze region estimation; A gaze mode data acquisition unit is configured to select a motion trajectory segment of the driver's head posture between gaze regions, with the gaze points at both ends of the trajectory located at the center anchor points of the corresponding gaze regions, and the angle between the connecting line of the gaze points in the trajectory and the connecting line of the two end gaze points being less than 5 degrees. The selected motion trajectory segment is used as gaze mode data; A gaze direction point pair extraction unit is configured to calculate the head rotation momentum relationship of the start and end points of the gaze mode data, project the real trajectory gaze mode data onto a virtual trajectory, and obtain gaze direction point pairs using linear relationship constraints for regions with dense real trajectory data. The gaze direction point pair includes gaze mode data and a gaze point corresponding to the gaze mode data, and the gaze point is the position of the gaze mode data projected onto the virtual trajectory; A calibration unit is configured to construct and train a gaze estimation model by a machine learning method, the gaze estimation model is used to learn the mapping relationship between the gaze feature data and the gaze point, and map the to-be-processed gaze feature data to the calibrated gaze point based on the mapping parameters obtained by training, and obtain the gaze direction according to the gaze feature data and the position of the calibrated gaze point.

Images