Gaze vector detection method, apparatus and electronic device

Abstract

The application provides a gaze vector detection method and device and electronic equipment, in which the pupil rotation center is taken as a fixed point in space, the problems that the gaze vector and the line of sight direction are not clear and coupled caused by the drift of the fixed point are avoided, and then the gaze vector determined according to the pupil center coordinates and the pupil rotation center coordinates can accurately reflect the line of sight direction. In addition, when the pupil center coordinates are determined, the pupil center distribution probability of the target sub-region to which the detected pupil center coordinates belong is checked while the pupil center detection is performed on the global of the preprocessed current frame human eye image, so that the accuracy of the pupil center coordinates is ensured. The accuracy of the gaze vector of the finally determined pupil center coordinates and the pupil rotation center coordinates is good, and the line of sight direction can be accurately reflected.

Description

Technical Field

[0001] This invention relates to the field of gaze tracking technology, and in particular to a method, apparatus and electronic device for detecting gaze vectors. Background Technology

[0002] Eye-tracking technology is a technique used to record and measure eye movements (i.e., changes in gaze direction), and it has a wide range of applications. For example, in the medical field, eye tracking can be used for disease analysis and to identify and analyze an individual's visual attention patterns when performing a task (such as reading, searching, browsing an image, or driving), thereby quantifying visual attention and exploring human behavior. In VR and gaming, using the eyes to operate VR scenes or games can reduce user fatigue. Furthermore, by capturing the user's gaze point, image enhancement can be applied to the gaze area to present high-quality images, while weakening the image of non-gaze areas—a technique known as foveated rendering—which can improve data processing speed, reduce the hardware requirements of VR devices or applications, and better align with the characteristics of the human eye, providing a better user experience. In neuromarketing, eye tracking allows businesses to improve packaging design, store layout, and point-of-sale displays based on what users pay attention to or ignore. In human-computer interaction, eye tracking can identify user intentions by detecting eye movements, thereby helping users interact with their environment. Therefore, eye tracking plays a vital role in all aspects of daily life.

[0003] Currently, the most commonly used eye-tracking technology is video analytics (VOG), which captures images of the patient's eyes and face using a head-mounted or fixed camera and matches eye movements with gaze points in the scene in a non-contact manner. It generally includes two steps: gaze vector detection and calibration.

[0004] Based on the structure of the human eye, the angle λ between the pupil axis and the line of sight for a specific individual (which is a specific individual for each person) does not change significantly over a relatively long period of time. Therefore, the direction of the pupil axis reflects the direction of the line of sight. The pupil axis is a straight line passing through the center of the pupil and perpendicular to the cornea. Since both the pupil and the cornea follow the rotation of the eyeball, their relative positions remain unchanged. Based on this, the angle between the pupil axis and the pupil normal remains fixed, and there is a definite mapping relationship between the direction of the pupil normal and the direction of the line of sight. Since the center of rotation of the pupil is not in the pupillary plane, the coordinates of the pupil center correspond one-to-one with the pupil normal (for every pupil center coordinate, there is a corresponding pupil normal). Consequently, the coordinates of the pupil center also correspond one-to-one with the direction of the line of sight. Therefore, as long as a point with a fixed position in space is found, the line connecting that point and the coordinates of the pupil center has a definite one-to-one correspondence with the direction of the line of sight. Since there is a definite mapping relationship between the line of sight and the coordinates of the gaze point in the scene, the vector (the line connecting the fixed point and the coordinates of the pupil center, which can be called the gaze vector used to reflect the direction of the line of sight) also has a definite mapping relationship with the coordinates of the gaze point.

[0005] Currently, common methods in VOG (Voice of Gait) technology include the PCCR (Posterior Chorbital Reflection) method and the pupil center-corneal reflex (P-CR) method. In the PCCR method, the eye is illuminated with visible or infrared light, and the gaze direction is reflected by the pupil center-corneal reflex point (P-CR) vector. In this method, the eyeball and cornea are considered standard spheres; therefore, theoretically, the position of the corneal reflex point does not change with eye movement, meaning it is a fixed point (i.e., the fixed endpoint of the gaze vector). However, since the eyeball is not a standard sphere, and the curvature of the cornea varies at different locations, the position of the corneal reflex point changes with eye movement (i.e., the corneal reflex point is not actually a fixed point in space), and the range of change is large. Therefore, when using the corneal reflex point as the fixed endpoint of the gaze vector for gaze vector detection, this gaze vector cannot accurately reflect the gaze direction; that is, the gaze vector represented by the pupil center-corneal reflex point has a large error when reflecting the gaze direction. In the pupil center-corner-of-the-eye method, the position of the corner of the eye is identified, and the gaze vector (pupil center - corner of the eye) is used to reflect the direction of the gaze, where the corner of the eye is considered a fixed point. However, the position of the corner of the eye changes during different facial expressions, which can easily lead to significant errors. Furthermore, the irregular shape of the corner of the eye and significant individual differences increase the difficulty of recognition and introduce more errors.

[0006] In summary, in both of the above schemes, the points considered to be fixed in position within the gaze vector reflecting the gaze direction are actually not fixed in physical space. Therefore, the relationship between the ultimately detected gaze vector reflecting the gaze direction and the gaze direction itself is unclear (i.e., the detected gaze vector cannot accurately reflect the gaze direction). Although calibration and fitting can eliminate some of the errors, the fitted mapping relationship is inaccurate. Furthermore, due to the drift of the fixed points, gaze vectors corresponding to different gaze directions may exhibit linear correlation, leading to the mapping problem of multiple different gaze directions corresponding to the same gaze vector.

[0007] Furthermore, pupil detection is easily affected by ambient light and camera angle. For example, dim lighting or uneven illumination can cause the dark areas at the corners of the eyes in the image to be misidentified as pupils during binarization (human eye images are generally RGB or grayscale images; when processing human eye images, they need to be converted to binary images. If a dark area should be white, it will become black because it is dark, just like the pupil). This leads to incorrect pupil center localization and ultimately tracking failure. Currently, to reduce interference from other areas and thus reduce pupil misidentification, existing solutions redefine the Region of Interest (ROI) for each captured human eye image. Specifically, a specific area around the pupil center, defined by a formula, is considered the ROI for the next pupil center detection. For example, T. Santini et al. defined an ROI whose width is equal to twice the major axis of the pupil. S. Lee et al. defined a circular ROI whose radius and center are adjusted by the coordinates of the pupil center. Defining a dynamic ROI reduces interference from other areas; however, the shape and size of the ROI are based on empirical definitions and may differ from the user's actual situation. Therefore, the ROI may contain interfering regions or exclude the true pupil region, and then the darkest area in the ROI is identified as the pupil center, resulting in continuous errors.

[0008] In summary, existing gaze vectors cannot accurately reflect the direction of gaze, and existing gaze vector detection methods often suffer from misidentification of the pupil center due to the influence of ambient light and camera angle, resulting in poor accuracy of the detected gaze vectors. Summary of the Invention

[0009] In view of this, the purpose of the present invention is to provide a method, apparatus and electronic device for detecting gaze vectors, so as to alleviate the technical problems that existing gaze vectors cannot accurately reflect the direction of gaze and that the accuracy of the detected gaze vectors is poor due to misidentification of the pupil center.

[0010] In a first aspect, embodiments of the present invention provide a method for detecting gaze vectors, comprising:

[0011] The preprocessed current frame human eye image is obtained, and the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is determined, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions.

[0012] Perform pupil center detection on the preprocessed current frame human eye image to obtain the pupil center coordinates in the preprocessed current frame human eye image, and determine the target sub-region to which the pupil center coordinates belong;

[0013] Determine whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold.

[0014] If the result is achieved, then the pupil center coordinate detection is confirmed to be correct;

[0015] The pupil rotation center coordinates are calculated based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the pupil rotation center coordinates are the projection coordinates of the pupil rotation center in the camera imaging plane.

[0016] The gaze vector is determined based on the pupil center coordinates and the pupil rotation center coordinates.

[0017] Further, the preprocessed current frame human eye image is acquired, and the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is determined, including:

[0018] The current frame human eye image is acquired, and the current frame human eye image is preprocessed to obtain the preprocessed current frame human eye image;

[0019] The original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is determined based on the pupil center distribution probability of each sub-region in the previous frame human eye image and the preset state transition matrix.

[0020] Wherein, when the previous frame human eye image is the first frame human eye image, the pupil center distribution probability of each sub-region in the previous frame human eye image is determined by the sub-region to which the pupil center belongs after manually marking the pupil center in the previous frame human eye image. The element in the i-th row and j-th column of the preset state transition matrix represents the probability that the pupil center is in the i-th sub-region in the previous frame human eye image and the pupil center is in the j-th sub-region in the current frame human eye image.

[0021] The original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is normalized to obtain the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image.

[0022] Furthermore, the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is determined based on the pupil center distribution probability of each sub-region in the previous frame human eye image and the preset state transition matrix, including:

[0023] According to the first formula Calculate the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, where, This represents the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image. This represents the preset state transition matrix. The preset state transition matrix represents the probability distribution of pupil centers in each sub-region of the previous frame of human eye image. The preset state transition matrix is ​​calculated by statistical methods from pre-collected human eye test data. The human eye test data is the sub-region to which the pupil center belongs after manually marking the pupil center of the collected human eye image.

[0024] Furthermore, the method also includes:

[0025] If the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image does not reach the preset probability threshold, then the pupil center coordinate detection error is determined, and the sub-regions in the preprocessed current frame human eye image where the pupil center distribution probability reaches the preset probability threshold are taken as the pupil center detection regions.

[0026] The pupil center detection area is subjected to pupil center detection to obtain the correct pupil center coordinates.

[0027] Furthermore, the coordinates of the pupil rotation center are calculated based on the line containing the minor axis of the pupil projection ellipse of the previous frame's human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame's human eye image, including:

[0028] Calculate the intersection point of the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, and determine the coordinates of the intersection point;

[0029] The coordinates of the intersection point are used as the coordinates of the pupil rotation center.

[0030] Furthermore, determining the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates includes:

[0031] According to the second formula Calculate the gaze vector, where, (u t+1 v t+1 ) represents the gaze vector, (x t+1 y t+1 (x) represents the coordinates of the pupil center. s y s ) represents the coordinates of the pupil rotation center.

[0032] Furthermore, the method also includes:

[0033] The corresponding gaze direction is determined based on the gaze vector;

[0034] The coordinates of the gaze point in the scene image coordinate system are determined based on the mapping relationship between the gaze vector, the human eye camera coordinate system, and the scene camera coordinate system.

[0035] Secondly, embodiments of the present invention also provide a gaze vector detection device, comprising:

[0036] The acquisition and determination unit is used to acquire the preprocessed current frame human eye image and determine the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions.

[0037] The pupil center detection unit is used to perform pupil center detection on the preprocessed current frame human eye image, obtain the pupil center coordinates in the preprocessed current frame human eye image, and determine the target sub-region to which the pupil center coordinates belong.

[0038] The judgment unit is used to determine whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold.

[0039] The first determining unit is used to determine that the pupil center coordinate detection is correct if the condition is met.

[0040] The calculation unit is used to calculate the coordinates of the pupil rotation center based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the coordinates of the pupil rotation center are the projection coordinates of the pupil rotation center in the camera imaging plane.

[0041] The second determining unit is used to determine the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates.

[0042] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0043] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when invoked and executed by a processor, cause the processor to perform the method described in any of the first aspects above.

[0044] In this embodiment of the invention, a method for detecting gaze vectors is provided, comprising: acquiring a preprocessed current frame human eye image and determining the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions; performing pupil center detection on the preprocessed current frame human eye image to obtain the pupil center coordinates in the preprocessed current frame human eye image and determining the target sub-region to which the pupil center coordinates belong; determining whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold; if it does, determining that the pupil center coordinate detection is correct; calculating the pupil rotation center coordinates based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the pupil rotation center coordinates are the projection coordinates of the pupil rotation center in the camera imaging plane; and determining the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates. As described above, the gaze vector detection method of the present invention uses the pupil rotation center as a fixed point in space, avoiding the problems of unclear gaze vector and gaze direction caused by fixed point drift. Therefore, the gaze vector determined based on the pupil center coordinates and pupil rotation center coordinates can accurately reflect the gaze direction. Furthermore, when determining the pupil center coordinates, while performing global pupil center detection on the preprocessed current frame human eye image, the probability distribution of the pupil center in the target sub-region to which the detected pupil center coordinates belong is also checked, ensuring the accuracy of the pupil center coordinates. The final gaze vector determined by the pupil center coordinates and pupil rotation center coordinates has good accuracy and can accurately reflect the gaze direction, alleviating the technical problems of existing gaze vectors failing to accurately reflect the gaze direction and poor accuracy of the detected gaze vector caused by misidentification of the pupil center. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 A flowchart of a gaze vector detection method provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the sphere formed by rotating the pupil center, the pupil, and the camera position provided in an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of the projection theorem of a circle provided in an embodiment of the present invention;

[0049] Figure 4 A schematic diagram of the sphere O and the pupil circular surface O1 of the imaging perspective provided in an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the cross-section of sphere O after being cut by plane S2 according to an embodiment of the present invention;

[0051] Figure 6 A schematic diagram of a gaze vector detection device provided in an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0053] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Traditional gaze vectors cannot accurately reflect the direction of gaze, and in traditional gaze vector detection methods, the pupil center is often misidentified due to the influence of ambient light and camera angle, resulting in poor accuracy of the detected gaze vectors.

[0055] Based on this, the gaze vector detection method of the present invention uses the pupil rotation center as a fixed point in space, avoiding the problems of unclear gaze vector and gaze direction caused by fixed point drift. Therefore, the gaze vector determined based on the pupil center coordinates and the pupil rotation center coordinates can accurately reflect the gaze direction. In addition, when determining the pupil center coordinates, while performing pupil center detection globally on the preprocessed current frame human eye image, the probability distribution of pupil centers in the target sub-region to which the detected pupil center coordinates belong is also checked, ensuring the accuracy of the pupil center coordinates. The gaze vector determined by the pupil center coordinates and the pupil rotation center coordinates has good accuracy and can accurately reflect the gaze direction.

[0056] To facilitate understanding of this embodiment, a method for detecting gaze vectors disclosed in this embodiment of the invention will first be described in detail.

[0057] Example 1:

[0058] According to an embodiment of the present invention, an embodiment of a gaze vector detection method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0059] Figure 1 This is a flowchart of a gaze vector detection method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0060] Step S102: Obtain the preprocessed current frame human eye image and determine the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image. Each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions.

[0061] Step S104: Perform pupil center detection on the preprocessed current frame human eye image to obtain the pupil center coordinates in the preprocessed current frame human eye image, and determine the target sub-region to which the pupil center coordinates belong.

[0062] The above-mentioned pupil center detection process is a traditional technique. Specifically, image processing techniques can be used to fit the pupil ellipse to obtain the pupil center coordinates.

[0063] Step S106: Determine whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold;

[0064] If step S108 is achieved, then the pupil center coordinate detection is confirmed to be correct.

[0065] To overcome the problem of misidentifying dark areas at the corner of the eye as pupils in dim environments or under uneven lighting conditions, a method for establishing a temporal pupil center position prediction model based on Markov chains in steps S102 to S108 was explored. Eye movement is a continuous process; pupil center positions captured in two consecutive scans are close. If other areas are incorrectly detected, a distance jump occurs. Based on this, a temporal prediction method based on Markov chains was designed, which reduces the error rate by comparing the pupil center distribution probability of the target sub-region to which the detected pupil center position belongs with a preset probability threshold.

[0066] Step S110: Calculate the pupil rotation center coordinates based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image. The pupil rotation center coordinates are the projection coordinates of the pupil rotation center in the camera imaging plane.

[0067] The following is a theoretical demonstration of the fixed position of the pupil rotation center in space and its calculation method:

[0068] The center of the pupil rotates around its center of rotation, forming a hemisphere. The line connecting the center of rotation and the center of the pupil is perpendicular to the pupillary surface. Therefore, the pupil is a tangent circle of this hemisphere, with the point of tangency being the center of the pupil. For example... Figure 2 As shown, let the sphere be O, with its center at point O, the center of pupil rotation, and the pupil be the tangent circle O1 of sphere O, with its center at point O1, the center of pupil.

[0069] The image formed by a camera is equivalent to the projection of the pupil onto the camera's imaging plane. According to the projection theorem of a circle, the projection of the pupil onto the camera's imaging plane must be either a circle or an ellipse. In the ellipse, the minor axis is perpendicular to the intersection of the plane containing the pupil and the projection plane, and it must pass through the projection of the pupil's center. For example... Figure 3 As shown.

[0070] Looking from the camera's direction, as Figure 4 As shown, the images of sphere O and pupil circle O1 are the same as their projections onto a plane parallel to the camera's imaging plane.

[0071] Assuming the plane containing the pupil circle O1 is S1, the sphere O is cut by a plane S2 parallel to the camera's imaging plane and passing through point O1, with the cross-section being a circular surface O2, and the center of the circle being point O2. Figure 5 As shown.

[0072] Project the sphere O and the circular surface O1 onto plane S2. This projection is the same as the image formed by the camera. The projection of the center O of the sphere coincides with point O2. Since S2 passes through point O1, the projection of the center O1 is itself. To find the projection of the circular surface O1 onto plane S2, we must first find the intersection line l of plane S1 and plane S2 containing the circular surface O1.

[0073] Since the center O1 lies in both plane S1 and plane S2, line l must pass through O1. Plane S1 is tangent to sphere O; therefore, plane S1 intersects sphere O at only one point O1, and consequently, plane S1 also intersects circle O2 at only one point O1. Since line l lies in plane S1, it intersects circle O2 at only one point O1. As mentioned earlier, both line l and circle O2 lie in plane S2; therefore, line l is a tangent to circle O2, with the point of tangency being point O1. By the properties of tangents, line O1O2 is perpendicular to line l. Since the minor axis of the ellipse projected from circle O1 onto plane S2 must pass through O1 and be perpendicular to line l, the minor axis must lie on line O1O2.

[0074] Therefore, we can conclude that the minor axes of the ellipses (i.e., pupil projection ellipses) obtained by projecting different pupil circles onto the camera's imaging plane intersect at a single point (point O2). This point is the projection of the pupil rotation center onto the camera's imaging plane. Since the position of the pupil rotation center in space is fixed, it can serve as the fixed end of the gaze vector, overcoming the problems of unclear gaze vector and line-of-sight direction and coupling caused by fixed-point drift. Furthermore, the coordinates of the pupil rotation center can be solved by finding the intersection of the minor axes of the ellipses obtained by projecting different pupil circles onto the camera's imaging plane.

[0075] Step S112: Determine the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates.

[0076] In this embodiment of the invention, a method for detecting gaze vectors is provided, comprising: acquiring a preprocessed current frame human eye image and determining the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions; performing pupil center detection on the preprocessed current frame human eye image to obtain the pupil center coordinates in the preprocessed current frame human eye image and determining the target sub-region to which the pupil center coordinates belong; determining whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold; if it does, determining that the pupil center coordinate detection is correct; calculating the pupil rotation center coordinates based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the pupil rotation center coordinates are the projection coordinates of the pupil rotation center in the camera imaging plane; and determining the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates. As described above, the gaze vector detection method of the present invention uses the pupil rotation center as a fixed point in space, avoiding the problems of unclear gaze vector and gaze direction caused by fixed point drift. Therefore, the gaze vector determined based on the pupil center coordinates and pupil rotation center coordinates can accurately reflect the gaze direction. Furthermore, when determining the pupil center coordinates, while performing global pupil center detection on the preprocessed current frame human eye image, the probability distribution of the pupil center in the target sub-region to which the detected pupil center coordinates belong is also checked, ensuring the accuracy of the pupil center coordinates. The final gaze vector determined by the pupil center coordinates and pupil rotation center coordinates has good accuracy and can accurately reflect the gaze direction, alleviating the technical problems of existing gaze vectors failing to accurately reflect the gaze direction and poor accuracy of the detected gaze vector caused by misidentification of the pupil center.

[0077] The above provides a brief overview of the gaze vector detection method of the present invention. The specific details involved are described in detail below.

[0078] In an optional embodiment of the present invention, obtaining the preprocessed current frame human eye image and determining the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image specifically includes the following steps:

[0079] (1) Obtain the current frame human eye image and preprocess the current frame human eye image to obtain the preprocessed current frame human eye image;

[0080] Specifically, human eye images can be captured using a head-mounted or fixed camera to obtain the current frame human eye image. The current frame human eye image can then be preprocessed to obtain the preprocessed current frame human eye image.

[0081] The preprocessing steps include: 1) defining the region of interest (i.e. the pupil center detection region) to reduce computation and eliminate interference; 2) converting the RGB image to grayscale; 3) smoothing high-frequency noise using Gaussian filtering; and 4) highlighting image features using morphological methods.

[0082] (2) Determine the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image based on the pupil center distribution probability of each sub-region in the previous frame human eye image and the preset state transition matrix.

[0083] Wherein, when the previous frame of human eye image is the first frame of human eye image, the pupil center distribution probability of each sub-region in the previous frame of human eye image is determined by the sub-region to which the pupil center belongs after manually marking the pupil center in the previous frame of human eye image. The element in the i-th row and j-th column of the preset state transition matrix represents the probability that the pupil center is in the i-th sub-region in the previous frame of human eye image and the pupil center is in the j-th sub-region in the current frame of human eye image.

[0084] Specifically, according to the first formula Calculate the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, where, This represents the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image. This represents the preset state transition matrix. This represents the probability distribution of pupil centers in each sub-region of the previous frame of the human eye image. The preset state transition matrix is ​​calculated by statistical methods from the pre-collected human eye test data. The human eye test data is the sub-region to which the pupil center belongs after the pupil center is manually marked on the collected human eye image.

[0085] The process is explained in detail below:

[0086] The current frame's human eye image and the previous frame's human eye image are evenly divided into 5×9 sub-regions (this invention does not impose a specific limit on the number of sub-regions), denoted as A1, A2, A3, ..., A 45 The coordinates of the pupil center in the previous frame of the human eye image (i.e., the human eye image at time t) are (x... t y t That is, the coordinates of the pupil center at time t are (x... t y t The coordinates of the pupil center in the current frame of the human eye image (i.e., the human eye image at time t+1) are (x... t+1 y t+1 That is, the coordinates of the pupil center at time t+1 are (x t+1 y t+1 (This can be done via Q)t+1 =P·Q t Calculate the original pupil center distribution probability of each sub-region in the current frame's human eye image (the original pupil center distribution probability of each sub-region at time t+1), where Q t+1 and Q t Q represents the probability distribution of pupil centers in each sub-region of the current frame's human eye image and the probability distribution of pupil centers in each sub-region of the previous frame's human eye image, i.e., the probability matrix of pupil center distribution in each sub-region at time t and time t+1. t+1 =[q 1，t+1 q 2，t+1 , ..., q 45，t+1 ] T Q t =[q 1，t q 2，t , ..., q 45，t ] T .

[0087] q i，t+1 and q i，t (i = 1, 2, ..., 45) represents the location of the pupil center in subregion A. i The probability, since (x t y t Subregion A to which ) belongs j Given (i.e., the sub-region to which the pupil center coordinates belong in the previous frame of the human eye image are known; when the previous frame of the human eye image is the first frame of the human eye image, the sub-region to which the pupil center coordinates belong in the previous frame of the human eye image is the sub-region to which the pupil center belongs after the pupil center is manually marked in the previous frame of the human eye image), therefore, This indicates that the center of the pupil in the previous frame of the human eye image is located in sub-region A. j The probability is 1, meaning the pupil center in the previous frame of the human eye image is not in sub-region A. j The probability is 0.

[0088] P represents the preset state transition matrix. Where, p i，j This represents the probability that the pupil center is located in the i-th sub-region of the previous frame's eye image and in the j-th sub-region of the current frame's eye image, i.e., the probability that the pupil center is in region A at time t. i And at time t+1, it is in position A j The probability, p ij =P{(x t+1 y t+1 )∈A j |(x t y t )∈A i Using the collected 5-minute human eye test data, P was calculated using statistical methods.

[0089] Simplifying the above formula, we get:

[0090]

[0091] Then, based on the first formula mentioned above, the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is calculated.

[0092] (3) Normalize the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image to obtain the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image.

[0093] Specifically, in order to standardize the measurement, normalization is required. The process is as follows: The probability distribution of pupil centers in each sub-region of the preprocessed current frame human eye image is obtained.

[0094] In an optional embodiment of the present invention, the method further includes:

[0095] (1) If the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image does not reach the preset probability threshold, then the pupil center coordinate detection error is determined, and the sub-regions in the preprocessed current frame human eye image where the pupil center distribution probability reaches the preset probability threshold are taken as the pupil center detection area.

[0096] Specifically, assuming the pupil center coordinates (x t+1 y t+1 (Belongs to target sub-region A) m (m = 1, 2, ..., 45), if (q represents a preset probability threshold), then determine the pupil center coordinates (x) t+1 y t+1 Error detected. All sub-regions are defined as the pupil center detection region.

[0097] (2) Perform pupil center detection on the pupil center detection area to obtain the correct pupil center coordinates.

[0098] In an optional embodiment of the present invention, the pupil rotation center coordinates are calculated based on the line containing the minor axis of the pupil projection ellipse of the previous frame of the human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame of the human eye image. This specifically includes the following steps:

[0099] (1) Calculate the intersection of the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, and determine the coordinates of the intersection point.

[0100] Specifically, assume that the line containing the minor axis of the pupil projection ellipse at time t+1 (the current frame of the human eye image) is m. t+1 At time t (of the previous frame of the human eye image), the line containing the minor axis of the pupil projection ellipse is m. t Find the intersection point (x) of the two lines. s y s ).

[0101] (2) Use the coordinates of the intersection point as the coordinates of the pupil rotation center.

[0102] In an optional embodiment of the present invention, determining the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates specifically includes the following steps:

[0103] According to the second formula Calculate the gaze vector, where, (u t+1 v t+1 ) represents the gaze vector, (x t+1 y t+1 (x) represents the coordinates of the pupil center. s y s ) represents the coordinates of the pupil rotation center.

[0104] In an optional embodiment of the present invention, the method further includes:

[0105] (1) Determine the corresponding gaze direction based on the gaze vector;

[0106] Specifically, the corresponding gaze direction is determined based on the conversion relationship between the gaze vector and the gaze direction.

[0107] (2) Determine the coordinates of the gaze point in the scene image coordinate system based on the mapping relationship between the gaze vector, the human eye camera coordinate system and the scene camera coordinate system, and then determine the user's gaze point in the scene image.

[0108] Specifically, (u, v) represents the gaze vector (i.e., the pupil center-pupil rotation center vector in the human eye image coordinate system), (s, t) represents the coordinates of the gaze point in the scene image coordinate system, and a i (i = 0, 1, 2, 3) and b i (i = 0, 1, 2, 3, 4) represents the constants in the mapping relationship, and the values ​​of these constants are determined using the 9-point calibration method.

[0109] During calibration, the user fixates on nine points at a time while keeping their head still. The corresponding pupil center-pupil rotation center vector (i.e., fixation vector) is recorded, and then the constants in the mapping relationship are calculated through regression analysis.

[0110] To demonstrate the feasibility of the method, subjects moved their eyes for 3 minutes, and pupil centers were detected using both the proposed method and a PCCR algorithm without Markov chains. The processing results were saved frame-by-frame into two groups, each containing 5400 images. In the group without Markov chains, pupil centers were correctly detected in 4826 images, while in the other group, pupil centers were correctly detected in 5219 images. This demonstrates that the proposed method improves accuracy by 7.3%. Compared to PCCR, which uses corneal reflective points as fixed points, this method uses fixed points with fixed positions in physical space, avoiding the problems of unclear gaze vectors and gaze direction coupling caused by fixed point drift. Compared to dynamic ROI methods, this algorithm calculates distribution probabilities using statistical methods, resulting in higher reliability (this invention does not extract only a small region of interest (i.e., the pupil center detection region, defined as the area near the pupil center), meaning it searches the entire region, not just the pupil center). Furthermore, by combining global detection (i.e., pupil center detection on the preprocessed current frame of the human eye image) and local detection (i.e., pupil center detection on the pupil center detection region), pupil features can be preserved while eliminating interference areas. Therefore, this method has more accurate and reliable performance.

[0111] This invention presents a novel eye-tracking method based on two-dimensional images. It constructs a pupil rotation center-pupil center vector to reflect the gaze direction, calculates the pupil rotation center coordinates as a fixed point, and performs real-time calibration. This avoids the problems of unclear gaze vector and gaze direction caused by fixed-point drift, thus improving the accuracy of eye tracking. Furthermore, it studies the sources and characteristics of interference under imperfect lighting conditions, overcoming the bottleneck of poor pupil center detection accuracy under imperfect lighting conditions in the field of eye tracking. A temporal pupil position prediction model based on Markov chains is established, and the influence of uneven light is overcome by narrowing the detection range, achieving accurate detection of the pupil center.

[0112] Example 2:

[0113] This invention also provides a gaze vector detection device, which is mainly used to execute the gaze vector detection method provided in Embodiment 1 of this invention. The gaze vector detection device provided in this invention will be described in detail below.

[0114] Figure 6 This is a schematic diagram of a gaze vector detection device according to an embodiment of the present invention, as shown below. Figure 6 As shown, the device mainly includes: an acquisition and determination unit 10, a pupil center detection unit 20, a judgment unit 30, a first determination unit 40, a calculation unit 50, and a second determination unit 60, wherein:

[0115] The acquisition and determination unit is used to acquire the preprocessed current frame human eye image and determine the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions.

[0116] The pupil center detection unit is used to perform pupil center detection on the preprocessed current frame human eye image, obtain the pupil center coordinates in the preprocessed current frame human eye image, and determine the target sub-region to which the pupil center coordinates belong.

[0117] The judgment unit is used to determine whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold.

[0118] The first determining unit is used to determine if the pupil center coordinate detection is correct if the target is reached.

[0119] The calculation unit is used to calculate the coordinates of the pupil rotation center based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the coordinates of the pupil rotation center are the projection coordinates of the pupil rotation center in the camera imaging plane.

[0120] The second determining unit is used to determine the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates.

[0121] In this embodiment of the invention, a gaze vector detection device is provided, comprising: acquiring a preprocessed current frame human eye image and determining the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions; performing pupil center detection on the preprocessed current frame human eye image to obtain the pupil center coordinates in the preprocessed current frame human eye image and determining the target sub-region to which the pupil center coordinates belong; determining whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold; if it does, determining that the pupil center coordinate detection is correct; calculating the pupil rotation center coordinates based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the pupil rotation center coordinates are the projection coordinates of the pupil rotation center in the camera imaging plane; and determining the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates. As described above, the gaze vector detection device of the present invention uses the pupil rotation center as a fixed point in space, avoiding the problems of unclear gaze vector and gaze direction caused by fixed point drift. Therefore, the gaze vector determined based on the pupil center coordinates and the pupil rotation center coordinates can accurately reflect the gaze direction. Furthermore, when determining the pupil center coordinates, while performing global pupil center detection on the preprocessed current frame of the human eye image, the probability distribution of the pupil center in the target sub-region to which the detected pupil center coordinates belong is also checked, ensuring the accuracy of the pupil center coordinates. The gaze vector determined by the pupil center coordinates and the pupil rotation center coordinates has good accuracy and can accurately reflect the gaze direction, alleviating the technical problems of existing gaze vectors failing to accurately reflect the gaze direction and poor accuracy of the detected gaze vector caused by misidentification of the pupil center.

[0122] Optionally, the acquisition and determination unit is further configured to: acquire the current frame human eye image, and preprocess the current frame human eye image to obtain the preprocessed current frame human eye image; determine the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image based on the pupil center distribution probability of each sub-region in the previous frame human eye image and a preset state transition matrix; wherein, when the previous frame human eye image is the first frame human eye image, the pupil center distribution probability of each sub-region in the previous frame human eye image is determined based on the sub-region to which the pupil center belongs after manually marking the pupil center in the previous frame human eye image, and the element in the i-th row and j-th column of the preset state transition matrix represents the probability that the pupil center is in the i-th sub-region in the previous frame human eye image and the pupil center is in the j-th sub-region in the current frame human eye image; normalize the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image to obtain the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image.

[0123] Optionally, the acquisition and determination unit is also used to: according to the first formula Calculate the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, where, This represents the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image. This represents the preset state transition matrix. This represents the probability distribution of pupil centers in each sub-region of the previous frame of the human eye image. The preset state transition matrix is ​​calculated by statistical methods from the pre-collected human eye test data. The human eye test data is the sub-region to which the pupil center belongs after the pupil center is manually marked on the collected human eye image.

[0124] Optionally, the device is further configured to: if the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image does not reach a preset probability threshold, determine that the pupil center coordinate detection is incorrect, and take the sub-regions in the preprocessed current frame human eye image where the pupil center distribution probability reaches the preset probability threshold as the pupil center detection region; perform pupil center detection on the pupil center detection region to obtain the correct pupil center coordinates.

[0125] Optionally, the calculation unit is also used to: calculate the intersection point of the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, and determine the coordinates of the intersection point; and use the coordinates of the intersection point as the coordinates of the pupil rotation center.

[0126] Optionally, the second determining unit is further configured to: determine the formula according to the second formula. Calculate the gaze vector, where, (u t+1 v t+1) represents the gaze vector, (x t+1 y t+1 (x) represents the coordinates of the pupil center. s y s ) represents the coordinates of the pupil rotation center.

[0127] Optionally, the device is also used to: determine the corresponding gaze direction based on the gaze vector; and determine the coordinates of the gaze point in the scene image coordinate system according to the mapping relationship between the gaze vector, the human eye camera coordinate system, and the scene camera coordinate system.

[0128] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0129] like Figure 7 As shown in the embodiment of this application, an electronic device 600 includes a processor 601, a memory 602, and a bus. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 via the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the gaze vector detection method described above.

[0130] Specifically, the memory 602 and processor 601 mentioned above can be general-purpose memory and processor, without any specific limitations. When the processor 601 runs the computer program stored in the memory 602, it can execute the above-mentioned gaze vector detection method.

[0131] The processor 601 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 601 or by instructions in software form. The processor 601 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 602, and processor 601 reads the information from memory 602 and, in conjunction with its hardware, completes the steps of the above method.

[0132] Corresponding to the above-described gaze vector detection method, this application embodiment also provides a computer-readable storage medium storing machine-executable instructions. When the machine-executable instructions are invoked and executed by a processor, the machine-executable instructions cause the processor to perform the steps of the above-described gaze vector detection method.

[0133] The gaze vector detection device provided in this application embodiment can be specific hardware on a device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0134] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0135] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0138] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the vehicle marking method described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0139] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0140] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for detecting gaze vectors, characterized in that, include: The preprocessed current frame human eye image is obtained, and the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is determined, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions. Perform pupil center detection on the preprocessed current frame human eye image to obtain the pupil center coordinates in the preprocessed current frame human eye image, and determine the target sub-region to which the pupil center coordinates belong; Determine whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold. If the result is achieved, then the pupil center coordinate detection is confirmed to be correct; The pupil rotation center coordinates are calculated based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the pupil rotation center coordinates are the projection coordinates of the pupil rotation center in the camera imaging plane. The gaze vector is determined based on the pupil center coordinates and the pupil rotation center coordinates; The calculation of the pupil rotation center coordinates, based on the line containing the minor axis of the pupil projection ellipse of the previous frame and the line containing the minor axis of the pupil projection ellipse of the current frame, includes: Calculate the intersection point of the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, and determine the coordinates of the intersection point; The coordinates of the intersection point are used as the coordinates of the pupil rotation center. Determining the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates includes: According to the second formula Calculate the gaze vector, where, Represents the gaze vector, Indicates the coordinates of the pupil center. This indicates the coordinates of the pupil rotation center.

2. The method according to claim 1, characterized in that, Acquire the preprocessed current frame human eye image and determine the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, including: The current frame human eye image is acquired, and the current frame human eye image is preprocessed to obtain the preprocessed current frame human eye image; The original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is determined based on the pupil center distribution probability of each sub-region in the previous frame human eye image and the preset state transition matrix. Wherein, when the previous frame human eye image is the first frame human eye image, the pupil center distribution probability of each sub-region in the previous frame human eye image is determined by the sub-region to which the pupil center belongs after manually marking the pupil center in the previous frame human eye image. The element in the i-th row and j-th column of the preset state transition matrix represents the probability that the pupil center is in the i-th sub-region in the previous frame human eye image and the pupil center is in the j-th sub-region in the current frame human eye image. The original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is normalized to obtain the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image.

3. The method according to claim 2, characterized in that, The original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image is determined based on the pupil center distribution probability of each sub-region in the previous frame human eye image and a preset state transition matrix, including: According to the first formula Calculate the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, where, This represents the original pupil center distribution probability of each sub-region in the preprocessed current frame human eye image. This indicates that the pupil center is in the sub-region at time t+1. The probability, , This represents the preset state transition matrix. This indicates that the pupil center is in a sub-region at time t. And at time t+1, it is in a sub-region The probability, This represents the probability distribution of pupil centers in each sub-region of the previous frame of the human eye image. This indicates that the pupil center is in a sub-region at time t. The probability of the preset state transition matrix is ​​calculated by statistical methods from the pre-collected human eye test data. The human eye test data is the sub-region to which the pupil center belongs after the pupil center is marked manually on the collected human eye images.

4. The method according to claim 1, characterized in that, The method further includes: If the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image does not reach the preset probability threshold, then the pupil center coordinate detection error is determined, and the sub-regions in the preprocessed current frame human eye image where the pupil center distribution probability reaches the preset probability threshold are taken as the pupil center detection regions. The pupil center detection area is subjected to pupil center detection to obtain the correct pupil center coordinates.

5. The method according to claim 1, characterized in that, The method further includes: The corresponding gaze direction is determined based on the gaze vector; The coordinates of the gaze point in the scene image coordinate system are determined based on the mapping relationship between the gaze vector, the human eye camera coordinate system, and the scene camera coordinate system.

6. A gaze vector detection device, characterized in that, include: The acquisition and determination unit is used to acquire the preprocessed current frame human eye image and determine the pupil center distribution probability of each sub-region in the preprocessed current frame human eye image, wherein each sub-region in the preprocessed current frame human eye image is obtained by uniformly dividing the preprocessed current frame human eye image into multiple regions. The pupil center detection unit is used to perform pupil center detection on the preprocessed current frame human eye image, obtain the pupil center coordinates in the preprocessed current frame human eye image, and determine the target sub-region to which the pupil center coordinates belong. The judgment unit is used to determine whether the pupil center distribution probability of the target sub-region in the preprocessed current frame human eye image reaches a preset probability threshold. The first determining unit is used to determine that the pupil center coordinate detection is correct if the condition is met. The calculation unit is used to calculate the coordinates of the pupil rotation center based on the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, wherein the coordinates of the pupil rotation center are the projection coordinates of the pupil rotation center in the camera imaging plane. The second determining unit is used to determine the gaze vector based on the pupil center coordinates and the pupil rotation center coordinates; The calculation unit is further configured to: calculate the intersection point of the line containing the minor axis of the pupil projection ellipse of the previous frame human eye image and the line containing the minor axis of the pupil projection ellipse of the current frame human eye image, and determine the coordinates of the intersection point; and use the coordinates of the intersection point as the coordinates of the pupil rotation center. The second determining unit is further configured to: determine the formula according to the second formula. Calculate the gaze vector, where, Represents the gaze vector, Indicates the coordinates of the pupil center. This indicates the coordinates of the pupil rotation center.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 5.

Images