Gaze information determination method and device, equipment and storage medium

Abstract

The invention discloses a fixation information determination method, apparatus and device, and a storage medium. The method comprises the steps of obtaining an eye electric signal and eyeball center position information of a user; based on the signal intensity in the eye electric signal, determining eye data corresponding to the eye electric signal, the eye data including pupil information; and determining gaze information based on the pupil information and the eyeball center position information. According to the method, the signal intensity of the eye electric signal is analyzed to obtain the eye data, and the fixation information is determined based on the eyeball center position information and the pupil information included in the eye data, so that the fixation information is determined based on the pupil information, and the problem of inaccurate fixation point determination in the prior art is solved; when the fixation information is determined, the eye electric signal instead of the eye image is used, and the corresponding eye data is determined based on the signal intensity in the eye electric signal, so that the technical defects of high power consumption and large data transmission quantity of an eye movement tracking method using a camera are solved.

Description

Technical Field

[0001] The present invention relates to the field of eye tracking technology, and in particular to a method, device, equipment and storage medium for determining gaze information. Background Art

[0002] With the rapid development of computer vision, artificial intelligence technology and digital technology, eye tracking technology has become a hot research field and has been widely used in the field of human-computer interaction. Eye tracking, also known as gaze tracking, is a technology that estimates eye gaze information by measuring eye movement. Gaze information includes line of sight and / or gaze point.

[0003] The existing technology uses pre-deployed optical signal transmitters and optical signal receivers at multiple locations to confirm whether corneal reflection occurs in the corresponding direction, determine the approximate direction of eye movement based on its orientation, and then obtain the required line of sight information. The above scheme for determining line of sight information can only obtain low-precision eye tracking results, because its eye tracking accuracy is directly restricted by the number of optical signal receivers, and if too many optical signal receivers are deployed, the overall signal refresh rate will be affected. Summary of the invention

[0004] The present invention provides a gaze information determination method, device, equipment and storage medium to solve the problem of inaccurate gaze point calculation in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a method for determining gaze information, including:

[0006] Obtain eye electrical signals and user's eye center position information;

[0007] Determining eye data corresponding to the eye electrical signal based on the signal strength in the eye electrical signal, wherein the eye data includes pupil information;

[0008] Based on the pupil information and the eyeball center position information, gaze information is determined.

[0009] Optionally, based on the pupil information and the eyeball center position information, determining the gaze information includes:

[0010] Based on the pupil information, determining pupil contour information;

[0011] Based on the pupil contour information and the eyeball center position information, gaze information is determined.

[0012] Optionally, the pupil contour information includes left eye pupil contour information and right eye pupil contour information, and the eyeball center position information includes left eye eyeball center position information and right eye eyeball center position information;

[0013] The determining of gaze information based on the pupil contour information and the eyeball center position information includes:

[0014] Determine the sight direction of the left eye based on the left eye pupil contour information and the left eye eyeball center position information;

[0015] Determine the sight direction of the right eye based on the right eye pupil contour information and the right eyeball center position information;

[0016] Based on the left eye sight line direction and the right eye sight line direction, gaze information is determined.

[0017] Optionally, the gaze information includes a gaze point, and determining the gaze information based on the left eye sight direction and the right eye sight direction includes:

[0018] Determine the shortest target line segment between the left eye straight line where the left eye sight direction is located and the right eye straight line where the right eye sight direction is located, wherein the target line segment is perpendicular to the left eye straight line and the right eye straight line respectively;

[0019] The midpoint of the target line segment is determined as the fixation point.

[0020] Optionally, determining gaze information based on the pupil information and the eyeball center position information includes:

[0021] Determining first gaze information based on pupil information and eyeball center position information included in the eye data;

[0022] determining second gaze information based on the eye electrical signal;

[0023] Perform a weighted summation on the first gaze information and the second gaze information to obtain gaze information.

[0024] Optionally, the weight coefficients of the first gaze information and the second gaze information are determined based on an angle formed by a line of sight direction and a direction of the exact center of the field of view.

[0025] Optionally, when the angle formed by the sight line direction and the direction of the center of the field of view is greater than a set threshold, the weight coefficient of the first gaze information is less than the weight coefficient of the second gaze information.

[0026] Optionally, the method further includes:

[0027] Constructing a database including a plurality of data sets, wherein the data sets include calibration points, calibration eye electrical signals corresponding to the calibration points, and calibration eye data corresponding to the calibration points;

[0028] Based on the multiple data sets, the user's eye center position information is determined.

[0029] Optionally, determining the user's eye center position information based on the multiple data sets includes:

[0030] Determine the pupil orientation corresponding to the calibration point in each data set;

[0031] Based on the pupil orientation corresponding to each of the data sets, the eyeball center position information is determined.

[0032] In a second aspect, an embodiment of the present invention provides a gaze information determination device, including:

[0033] An acquisition module, used to acquire eye electrical signals and the user's eye center position information;

[0034] A first determination module, configured to determine eye data corresponding to the eye electrical signal based on the signal strength in the eye electrical signal, wherein the eye data includes pupil information;

[0035] The second determination module is used to determine the gaze information based on the pupil information and the eyeball center position information.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0037] An optical transmitter, an optical receiver and a scanner, wherein the optical transmitter emits a light beam toward the scanner, the scanner reflects the light beam toward the user's eyes, the optical receiver acquires a light signal reflected from the user's eyes, and the optical receiver converts the light signal into an eye electrical signal;

[0038] The electronic device further comprises at least one processor coupled to the optical receiver; and

[0039] a memory communicatively connected to the at least one processor; wherein,

[0040] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the gaze information determination method according to any embodiment of the present invention.

[0041] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the gaze information determination method described in any embodiment of the present invention when executed.

[0042] The technical solution of the embodiment of the present invention obtains eye electrical signals and the user's eye center position information; determines eye data corresponding to the eye electrical signals based on the signal strength in the eye electrical signals, and the eye data includes pupil information; determines gaze information based on the pupil information and the eye center position information. The present invention analyzes the signal strength of the eye electrical signals to obtain eye data, and then determines the gaze information based on the eye center position information and the pupil information included in the eye data, thereby achieving determination of the gaze information based on the pupil information, and solving the problem of inaccurate determination of the gaze point in the prior art. In addition, the present invention uses eye electrical signals instead of eye images when determining the gaze information, and determines the corresponding eye data based on the signal strength in the eye electrical signals, thereby solving the technical defects of high power consumption and large data transmission volume in the eye tracking method using a camera.

[0043] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A flowchart of a method for determining gaze information provided in Embodiment 1 of the present invention;

[0046] Figure 2 A flowchart of a method for determining gaze information provided in Embodiment 2 of the present invention;

[0047] Figure 3 A coordinate diagram of cone translation provided in the second embodiment of the present invention;

[0048] Figure 4 A coordinate diagram of a cone that is rotated after translation provided in the second embodiment of the present invention;

[0049] Figure 5 A schematic diagram of the structure of a device for determining gaze information provided in Embodiment 3 of the present invention;

[0050] Figure 6 A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] Embodiment 1

[0054] Figure 1 This is a flowchart of a method for determining gaze information provided in the first embodiment of the present invention. This embodiment is applicable to the case of eye tracking. The method can be executed by a gaze information determination device. The gaze information determination device can be implemented in the form of hardware and / or software. The gaze information determination device can be configured in an electronic device, such as a head-mounted device. The head-mounted device can be considered as a device worn on the user's head. The head-mounted device can be analytical glasses. The electronic device of the present application can also be a VR / AR accessory. The VR / AR accessory can be considered as an accessory of the VR / AR device, which can realize the determination of gaze information. Figure 1 As shown, the method includes:

[0055] S110, obtaining eye electrical signals and information on the center position of the user's eyeball.

[0056] Among them, the eye electrical signal can be an eye electrical signal; the present invention can determine the eye data by analyzing the eye electrical signal. The eyeball center position information can be information characterizing the eyeball center position, such as the coordinates of the eyeball 3D center position. The present invention can determine the eyeball center position information through a pre-established human eye 3D center model, wherein the eyeball center position can be the eyeball 3D center position in the human eye 3D center model. The human eye 3D center model can be a model of the human eye 3D center, which is used to determine the human eye center position information.

[0057] It should be noted that the acquisition, storage, use and processing of data in the technical solution of the present invention are in compliance with the relevant provisions of national laws and regulations and are authorized by the user. The user can be the owner of the data. The data may include information on the center position of the eyeball and eye electrical signals.

[0058] In this embodiment, the specific technical means for obtaining eye electrical signals are not limited here. For example, the eye electrical signals transmitted by the electrical signal acquirer can be obtained by the processor of the electronic device. The electrical signal acquirer is not limited here, and it can be an optical sensor. The means for the optical sensor to obtain eye electrical signals can be as follows: the infrared diode on the electronic device emits infrared light, and the infrared light is emitted to the plane mirror of the scanner. At the same time, the scanner will control the infrared light to be directed to the user's eyes, and then reflect from the corneal surface to the optical sensor, and the optical sensor converts the light signal into an electrical signal (i.e., eye electrical signal).

[0059] The present invention determines gaze information based on eye electrical signals and does not use a camera to capture user eye images. Instead, an infrared diode is used to emit infrared light to obtain user eye electrical signals, thereby ensuring the accuracy of the acquired data while reducing power consumption and data transmission volume.

[0060] In this embodiment, there is no limitation on obtaining the eyeball center position information. For example, this step can determine the user's eyeball center position information based on a pre-established human eye 3D center model, or this step can directly obtain a pre-stored eyeball center position.

[0061] S120. Determine eye data corresponding to the eye electrical signal based on signal strength in the eye electrical signal, where the eye data includes pupil information.

[0062] Among them, signal strength may refer to the strength of the signal, and eye data may refer to various data of the eye, for example, information such as pupil information, corneal data, iris data, eyelid data and / or eye corner data of the user. Pupil information may be considered as information reflecting the pupil, and pupil information may reflect the gaze information of the user's eyes. Pupil information may include information such as pupil contour, pupil size, pupil diameter, pupil position and / or pupil shape of the eye. Corneal data may be data reflecting the cornea, such as corneal position and / or corneal shape, etc. Iris data may be data reflecting the iris, such as iris position and / or iris shape, etc.

[0063] In this embodiment, the pupil area and the non-pupil area can feedback different signal intensities for the same stimulus, and the electrical signal in the pupil area will be significantly lower than the electrical signal in the non-pupil area; therefore, this step can identify the pupil area and the non-pupil area by analyzing the strength of the eye electrical signal. For example, the reflection intensity of the infrared light injected into the pupil is significantly lower than the reflection intensity injected into other parts, which is reflected in the scan image as a low-intensity signal, thereby obtaining the eye pupil data (pupil size and pupil contour, etc.). Specifically, the eye electrical signal is identified, and when the value of the electrical signal is lower than the set value, it can be considered to be the pupil area, and then the pupil contour and pupil size are determined based on the determined pupil area.

[0064] S130. Determine gaze information based on the pupil information and the eyeball center position information.

[0065] The gaze information may be information related to the gaze. The gaze information may be used to track the user's line of sight. For example, the gaze information may be the user's line of sight direction and / or gaze point. The line of sight may be understood as a three-dimensional vector. The gaze point may be understood as a two-dimensional coordinate projected onto a plane by a three-dimensional vector.

[0066] In this step, the gaze information is determined by the pupil information and the eyeball center position information. When determining the gaze information, the pupil information may be analyzed first, and then the gaze information is determined based on the analysis result combined with the eyeball center position information. The content included in the analysis result is not limited here, and may be information that can determine the gaze information after being combined with the eyeball center position information, such as information associated with the pupil contour.

[0067] In one embodiment, after determining the eye data, this step can determine the pupil contour information through the pupil information in the eye data, and calculate and determine the user's final line of sight direction or gaze point based on the pupil contour information and the coordinates of the center position of the eyeball.

[0068] In this embodiment, by acquiring eye electrical signals and the user's eye center position information; based on the signal strength in the eye electrical signals, determining the eye data corresponding to the eye electrical signals, the eye data including pupil information; based on the pupil information and the eye center position information, determining the gaze information, the present invention analyzes the signal strength of the eye electrical signals to obtain the eye data, and then determines the gaze information based on the eye center position information and the pupil information included in the eye data, thereby achieving determination of the gaze information based on the pupil information, solving the problem of inaccurate determination of the gaze point in the prior art. In addition, when determining the gaze information, the present invention uses eye electrical signals instead of eye images, and determines the corresponding eye data based on the signal strength in the eye electrical signals, thereby solving the technical defects of high power consumption and large data transmission volume of the eye tracking method using a camera.

[0069] Optionally, determining gaze information based on the pupil information and the eyeball center position information includes:

[0070] Based on the pupil information, determining pupil contour information;

[0071] Based on the pupil contour information and the eyeball center position information, gaze information is determined.

[0072] The pupil contour information may refer to information indicating the coordinates of the pupil edge contour.

[0073] In this embodiment, pupil information can be analyzed to obtain pupil contour information. The analysis means are not limited here, such as fitting the pupil based on the pupil information, such as fitting to obtain a quasi-circular or quasi-elliptical contour. Then the fitted graphic contour is determined as the pupil contour information.

[0074] After determining the pupil contour information, the position of the center of the fitted figure can be determined as the pupil center position, that is, the pupil center coordinate point. In this embodiment, the gaze information can be determined based on the pupil center coordinate point combined with the eyeball center position information. Specifically, the straight line formed by connecting the pupil center coordinate point and the eyeball 3D center position coordinate point can be considered as the line of sight direction. The gaze information can be the gaze information represented by a single eye, or it can be the gaze information represented after binocular fusion, which is not limited here. In addition, the specific method for determining the gaze information is not limited.

[0075] In one embodiment, the pupil includes a left eye pupil and a right eye pupil; the eyeball also includes a left eyeball and a right eyeball; therefore, this embodiment can fuse the pupil contour information of the left and right eyes and the eyeball center position information to determine the user's line of sight direction or gaze point.

[0076] In this embodiment, pupil contour information is obtained by real-time calculation of pupil information, and a fusion calculation is performed based on the 3D center position coordinates of the user's left and right eyes and the user's pupil contour information calculated in real time to determine the user's line of sight direction or gaze point, thereby improving the accuracy of gaze point determination.

[0077] Optionally, the pupil contour information includes left eye pupil contour information and right eye pupil contour information, and the eyeball center position information includes left eye eyeball center position information and right eye eyeball center position information;

[0078] The determining of gaze information based on the pupil contour information and the eyeball center position information includes:

[0079] Determine the sight direction of the left eye based on the left eye pupil contour information and the left eye eyeball center position information;

[0080] Determine the sight direction of the right eye based on the right eye pupil contour information and the right eyeball center position information;

[0081] Based on the left eye sight line direction and the right eye sight line direction, gaze information is determined.

[0082] Among them, the left eye pupil contour information may refer to the user's left eye pupil contour coordinates, and the right eye pupil contour information may refer to the user's right eye pupil contour coordinates; the pupil center coordinates of the left and right eyes may be calculated based on the pupil contour coordinates of the left and right eyes; the left eye eyeball center position information may refer to the user's left eye eyeball 3D center position coordinates, and the right eye eyeball center position information may refer to the user's right eye eyeball 3D center position coordinates; the left eye sight direction may refer to the direction of the user's left eye sight, and the right eye sight direction may refer to the direction of the user's right eye sight.

[0083] The line connecting the left eye pupil center coordinates and the user's left eye eyeball 3D center position coordinates is the left eye sight direction. Therefore, this embodiment can determine the line connecting the left eye pupil center and the left eyeball center based on the left eye pupil contour information and the left eyeball center position information, and determine the direction corresponding to the line as the left eye sight direction. Similarly, the line connecting the right eye pupil center coordinates and the user's right eyeball 3D center position coordinates is the right eye sight direction. The method for determining the right eye sight direction is similar and will not be repeated here.

[0084] In this embodiment, a fusion calculation can be performed based on the calculated left eye sight direction and right eye sight direction to determine the user's sight direction or gaze point, and the left and right eye data can be used to ensure that the determined gaze information is more accurate.

[0085] In one embodiment, when the left eye sight direction and the right eye sight direction are merged, the intersection of the straight line where the left eye sight direction is located and the straight line where the right eye sight direction is located may be determined as the gaze point.

[0086] In one embodiment, when the left eye sight direction and the right eye sight direction are merged, the gaze information can be determined based on the intersection of the left eye sight direction and the right eye sight direction on the display screen. For example, the circle formed by the two intersections is determined as the gaze area; another example is the midpoint between the two intersections is used as the gaze point; another example is the gaze information is determined based on the shortest line segment from the left eye straight line where the left eye sight direction is located and the right eye straight line where the right eye sight direction is located.

[0087] Optionally, the gaze information includes a gaze point, and determining the gaze information based on the left eye sight direction and the right eye sight direction includes:

[0088] Determine the shortest target line segment between the left eye straight line where the left eye sight direction is located and the right eye straight line where the right eye sight direction is located, wherein the target line segment is perpendicular to the left eye straight line and the right eye straight line respectively;

[0089] The midpoint of the target line segment is determined as the fixation point.

[0090] Among them, the gaze point can be a certain point of the object that the line of sight is aimed at during the visual perception process. In the present embodiment, it can be simply understood as the common landing point of the left and right eye lines of the user in space; the left eye straight line can be a straight line extending in the direction of the left eye line of sight, and the right eye straight line can be a straight line extending in the direction of the right eye line of sight; the shortest target line segment can be understood as the shortest line segment between the left eye straight line and the right eye straight line in space, and the line segment is perpendicular to the left eye straight line and the right eye straight line respectively; and the midpoint of the line segment is the gaze point after the left and right eye line of sight directions are integrated.

[0091] It is understandable that, in theory, the sight lines of the left eye and the right eye should converge on a point in space, but due to calculation errors, the sight lines of the left eye and the right eye do not intersect at a point. Therefore, in this embodiment, the shortest target line segment between the left and right eyes is determined by the straight lines of the left and right eyes, and the midpoint of the target line segment is determined as the gaze point. Even in the presence of calculation errors, the gaze point can be determined, thereby improving the accuracy of the gaze point determination.

[0092] Embodiment 2

[0093] Figure 2 This is a flow chart of a method for determining gaze information provided by Embodiment 2 of the present invention. The technical solution of this embodiment is further refined on the basis of the above embodiment. Figure 2 As shown, the method includes:

[0094] S210: Obtain eye electrical signals and information on the center position of the user's eyeball.

[0095] S220. Determine eye data corresponding to the eye electrical signal based on the signal strength in the eye electrical signal, where the eye data includes pupil information.

[0096] S230, determining first gaze information based on pupil information and eyeball center position information included in the eye data;

[0097] S240, determining second gaze information based on the eye electrical signal;

[0098] S250: Perform a weighted summation on the first gaze information and the second gaze information to obtain gaze information.

[0099] Among them, the first gaze information can be the gaze information determined by the pupil information and the eye center position information included in the eye data; specifically, after determining the eye data, the left and right eye sight directions are determined according to the position coordinates of the pupil information and the eye center position information in the left and right eye eye data, and then the shortest target line segment between the straight lines where the left and right eye sight directions are located is determined, and finally the midpoint of the target line segment is determined as the first gaze information.

[0100] The second gaze information may be gaze information determined by eye electrical signals; specifically, infrared light may be emitted by an infrared diode, and after being emitted, the infrared light is directed toward the plane mirror of the scanner. At the same time, the scanner will control the infrared light to be directed toward the user's eyes, and then reflected from the corneal surface to the optical sensor. The optical sensor converts the light signal into an eye electrical signal. Based on the eye electrical signal, it can be confirmed whether corneal reflection occurs in the corresponding direction, and the direction of eyeball rotation is determined based on its orientation, thereby obtaining the second gaze information.

[0101] Due to the advantages and disadvantages of the two solutions S230 and S240, the first gaze information or the second gaze information may be inaccurate in different situations. In order to improve the accuracy of the gaze point determination, the first gaze information and the second gaze information may be fused according to a certain weight ratio to determine the gaze information. The weight ratio may be fixed or may vary with the deflection angle, which is not limited in this embodiment.

[0102] In this embodiment, the first gaze information and the second gaze information may be determined through different schemes, and the first gaze information and the second gaze information may be weightedly summed according to a weight ratio to obtain the gaze information, thereby further improving the accuracy of gaze point determination.

[0103] Optionally, the weight coefficients of the first gaze information and the second gaze information are determined based on an angle formed by a line of sight direction and a direction of the exact center of the field of view.

[0104] Among them, the line of sight direction can be the line of sight direction of the left and right eyes, including the line of sight direction of the left eye and the line of sight direction of the right eye; the field of view can be the maximum range that the user can look at by turning his eyeballs, and the center direction of the field of view can be the center direction of the up, down, left and right included in the entire field of view; the angle formed by the line of sight direction and the center direction of the field of view can be understood as the angle between the line of sight direction of the left eye and the center direction of the field of view and / or the angle between the line of sight direction of the right eye and the center direction of the field of view.

[0105] Specifically, when the angle between the eye's line of sight and the direction of the center of the field of view is too large, there may be inaccuracy in determining the gaze point through the pupil contour; and the accuracy of the first gaze information calculated by determining the position of the gaze point near the front through the pupil contour will be higher than the second gaze information. Therefore, the weight coefficients of the first gaze information and the second gaze information are determined based on the angle formed by the line of sight and the direction of the center of the field of view. It can be understood that when the angle formed by the line of sight and the direction of the center of the field of view is too large, the weight coefficient of the first gaze information can be reduced and the weight coefficient of the second gaze information can be increased; when the angle formed by the line of sight and the direction of the center of the field of view is too small, the weight coefficient of the first gaze information can be increased and the weight coefficient of the second gaze information can be reduced.

[0106] In this embodiment, the weight coefficients of the first gaze information and the second gaze information are determined by the angle formed by the line of sight and the center direction of the field of view. The weight coefficients are determined based on the angle difference, which improves the accuracy of determining the gaze point at large angles.

[0107] Optionally, when the angle formed by the line of sight direction and the direction of the center of the field of view is greater than a set threshold, the weight coefficient of the first gaze information is less than the weight coefficient of the second gaze information.

[0108] The set threshold may be a preset angle threshold. In this embodiment, the set threshold is not limited and may be set by the user or according to actual calculation conditions. If the angle formed by the sight line direction and the direction of the center of the field of view is greater than the set angle threshold, it can be determined that the eye gaze direction is in a large angle area. If the angle formed by the sight line direction and the direction of the center of the field of view is less than the set angle threshold, it can be determined that the eye gaze direction is in a small angle area. Different weight coefficients of the first gaze information and the second gaze information can be determined according to different angle areas. Specifically, when the angle formed by the sight line direction and the direction of the center of the field of view is greater than the set threshold, the weight coefficient of the first gaze information may be less than the weight coefficient of the second gaze information. For example, the angle threshold is set to 45°. When the eye gaze direction is greater than 45 degrees, the weight coefficient of the first gaze information may be set to 0, and the weight coefficient of the second gaze information may be set to 1, that is, the inaccurate gaze point calculation is stopped. Only the scheme with high accuracy is used to calculate the gaze point. The weight coefficient of the first gaze information may also be set to 0.1, and the weight coefficient of the second gaze information may be set to 0.9.

[0109] In this embodiment, when the angle formed by the line of sight direction and the direction of the center of the field of view is greater than a set threshold, the weight coefficient of the first gaze information is less than the weight coefficient of the second gaze information. At large angles, the weight coefficient for determining the gaze point based on the pupil contour is reduced. If it is set to 0, the accuracy of determining the gaze point at large angles can be improved.

[0110] Optionally, the method further includes:

[0111] Constructing a database including a plurality of data sets, wherein the data sets include calibration points, calibration eye electrical signals corresponding to the calibration points, and calibration eye data corresponding to the calibration points;

[0112] Based on the multiple data sets, the user's eye center position information is determined.

[0113] Among them, a data set may refer to a collection of data; a database may refer to a warehouse that organizes, stores and manages data according to a data structure; a calibration point may refer to a pre-stimulated point for a user to watch; a calibration eye electrical signal may refer to an eye electrical signal generated when a user watches the calibration point; and a calibration eye data may refer to eye data generated when a user watches the calibration point, and the calibration eye data includes pupil information, which may be information such as pupil contour, pupil size, and pupil diameter.

[0114] Exemplarily, when performing eye tracking, a first stimulus is presented to a user wearing a head-mounted device, and the head-mounted device includes an infrared light source, a MEMS micro-electromechanical system, an optical sensor, etc. When the user responds to the first stimulus or expects to respond to the first stimulus, the head-mounted device obtains the user's left eye electrical signal and left eye eyeball data, and right eye electrical signal and right eye eyeball data according to the optical sensor, and generates a first data set including a first left eye electrical signal and left eye data, a first right eye electrical signal and right eye data, and a first stimulus point (calibration point) position coordinate; the first data set is related to the first stimulus, a data link is established between the head-mounted device and the database, and the first data set is added to the database. Similarly, a second, third...multiple calibration point calibrations are performed, and the second, third...multiple data sets are added to the database, and then a database of multiple data sets is formed.

[0115] Among them, the calibration eye electrical signal may include a first left eye eye electrical signal, a first right eye eye electrical signal, a second left eye eye electrical signal, a second right eye eye electrical signal, etc. The calibration eye data may include left eye eye data corresponding to the first left eye eye electrical signal, right eye eye data corresponding to the first right eye eye electrical signal, left eye eye data corresponding to the second left eye eye electrical signal, right eye eye data corresponding to the second right eye eye electrical signal, etc. The eye data includes left eye eye data and right eye eye data. The eye data includes eyeball data.

[0116] According to the multiple eye data in the database, the 3D center position of the left eye and the right eye of the user can be determined. In this embodiment, multiple eye data can be modeled to obtain the 3D center position of the eye based on the modeling result. When modeling, only the eye can be modeled, and the modeling method is not limited here, such as modeling based on the geometric relationship of the control.

[0117] In this embodiment, a plurality of eye data are obtained by pre-stimulation in the database, so as to determine the center positions of the left and right eyes of the user, thereby further improving the accuracy of determining the eye center position information.

[0118] Optionally, determining the user's eye center position information based on the multiple data sets includes:

[0119] Determine the pupil orientation corresponding to the calibration point in each data set;

[0120] Based on the pupil orientation corresponding to each of the data sets, the eyeball center position information is determined.

[0121] Among them, the data set includes eye data for calibration, and the eye data for calibration includes pupil information, and the pupil information may be information such as pupil contour, pupil size, and pupil diameter; then, according to the pupil contour, based on the spatial geometric relationship (back projection), a pupil orientation in space can be obtained, and when the user looks at different positions (different calibration points), multiple pupil orientations will be formed, and multiple pupil orientation information can establish a 3D center model of the human eye; based on the established 3D center model of the human eye, the final 3D center position coordinates of the user's left and right eyes are determined, that is, the eye center position information.

[0122] Exemplarily, the specific calculation method of the 3D center position of the eyeball may be:

[0123] First, the equation of the conic surface (2) is obtained based on the plane ellipse equation (1) combined with information such as the camera focal length.

[0124] The equation of the plane ellipse is:

[0125] F(x, y) = a′z 2 +2h′xy+b′y 2 +2g′x+2f′y+d′=0 (1)

[0126] The plane is the camera sensor plane. The ellipse contour extracted based on the image sensor can obtain its corresponding coefficient parameters (ie, a', h', b', g', etc.) through ellipse fitting.

[0127] Assume that the coordinates of the vertex of the cone (i.e., the center of the camera lens in the elliptical coordinate system) are (α, β, γ), where the values ​​of α, β, and γ are usually 0, and γ is the focal length of the camera.

[0128] The general form of the conic surface equation (2) is as follows:

[0129] ax 2 +by 2 +cz 2+2fyz+2gzx+2hry+2uz+2vy+2wz+d=0 (2)

[0130] Where: a = γ 2 a';

[0131] b=γ 2 b';

[0132] c=a'α 2 +2h'αβ+b'β 2 +2g'α+2f'β+d';

[0133] d = γ2d';

[0134] f = -γ(b'β+h'α+f');

[0135] g = -γ(h'β+a'α+g');

[0136] h = γ2h′;

[0137] u=γ2g';

[0138] v = γ2f';

[0139] w=-γ(f'β+g'α+d');

[0140] Translation matrix TO It is the z of the coordinate origin plus a number, from the center of the cone to the optical center of the camera.

[0141] Solving the cubic system of equations for λ

[0142] Here, we can directly solve the three λ solutions according to the root-finding formula, and then we need to sort them by size.

[0143] t is an intermediate variable used to solve l, m, and n.

[0144] The solution formula for t is

[0145] Solve for m, l, and n based on t.

[0146] Based on l, m, n, we get the rotation matrix T1

[0147] Based on the conic section parameters and λ as well as l, m, n, the translation matrix T2 is obtained

[0148]

[0149] Through the three matrices T0, T1 and T2, the cone can be translated, rotated and then translated again. Figure 3A coordinate diagram of cone translation provided in the second embodiment of the present invention, Figure 4 A coordinate diagram of a cone after translation and rotation provided in the second embodiment of the present invention, such as Figure 3 to Figure 4 As shown, it should be noted that so far, l, m, and n all have subscripts (such as l1 l2 l3). In the next step, we will use l, m, and n without subscripts, and define them as follows: a plane equation.

[0150] lX+mY+nZ=p (Formula 10);

[0151] Among them, l, m, and n without subscripts are calculated by λ.

[0152]

[0153] m=0

[0154]

[0155] It should be noted that the vectors l, m, and n are rotated and translated by T1 to obtain the sight direction. At this point, half of the solution is solved, and the next step is to obtain the gaze point. From an easy-to-understand perspective, calculating a matrix can rotate an ellipse, and the rotation angle is the sight angle in the camera coordinate system, that is, the so-called pupil direction. There are two angles for rotation, and the right angle can be selected according to the camera placement, that is, one of the solutions is filtered out according to the preset camera placement.

[0156] Note that l has two cases, positive and negative. For these two cases, the following rotation matrix T3 is calculated respectively:

[0157]

[0158] This is the inverse matrix of the rotation matrix T3, which is determined by l, m, and n. It means the general conic section to the standard conic section. The T3 we actually want to use means the standard conic section to the general conic section.

[0159] At this point, T0, T1, T2, and T3 are all calculated, and the overall idea is as follows:

[0160]

[0161] That is, the center of the circle with z=k can be found in the standard cone form by translation and rotation. Figure 4 As shown, the circle is a circle instead of an ellipse, so its center can be obtained by the following method: and its ellipse equation with Z'=p can be written as follows:

[0162]

[0163] Right now Figure 4The intersecting curved ellipses with circles in the middle can be written in this form. Its center is (X', Y', Z')

[0164] For this ellipse with Z'=k, it is usually additionally noted as:

[0165]

[0166] B≡(λ1l1n1+λ2l2n2+λ3l3n3)

[0167] C≡(λ1m1n1+λ2m2n2+λ3m3n3)

[0168]

[0169] Since it is a perfect circle, the coefficients of its standard form must satisfy the following rules:

[0170] Ax 2 +Bxy+Cy 2 +Dx+Ey+F=0

[0171] A=C,B=0.

[0172] From this we can get:

[0173]

[0174] After the above equation is simplified to an equation about P,

[0175]

[0176] Note that r is introduced here, which means the scaling of the system. This embodiment only needs to obtain the ray direction, so r can be simply set to 1.

[0177] Then the coordinates of the center of the circle Xo'Yo'Zo' can be simply written out.

[0178]

[0179]

[0180]

[0181] After obtaining the coordinates of the center of the circle Xo'Yo'Zo', it is transformed continuously through T3 T2 T1 T0 to obtain the normalized position of the center in the camera coordinate system, that is, the three-dimensional coordinates of the center of the eyeball relative to the camera coordinate system.

[0182] In this embodiment, pupil contour information can be determined based on the pupil information included in the eye data in the data set, and then the pupil center can be determined. The line connecting the pupil center and the calibration point corresponding to the eye data can be used as the pupil direction, which can be used to characterize the line of sight.

[0183] The data set may include eye data corresponding to multiple calibration points. For each eye data, the corresponding pupil contour may be determined, and then in combination with the calibration point, the pupil orientation corresponding to the calibration point may be determined. When the pupil orientations corresponding to the multiple calibration points are obtained, the intersection of the straight lines of the pupil orientations may be used as the eyeball center position information.

[0184] In the gaze tracking stage, the gaze point, the pupil center and the eyeball center can be located on a straight line, which can be considered as the sight line. In the calibration stage, the calibration point, the pupil center and the eyeball center can be located on a straight line, which can be considered as the sight line.

[0185] In this embodiment, a 3D center model of the human eye is established through the pupil orientation corresponding to each of the data sets, the eyeball center position information is determined, and the accuracy of determining the eyeball center position information is improved.

[0186] A method for determining gaze information provided by an embodiment of the present invention determines first gaze information and second gaze information through different schemes, performs weighted summation of the first gaze information and the second gaze information according to a weight ratio to obtain gaze information, thereby improving the accuracy of determining a gaze point; determines weight coefficients of the first gaze information and the second gaze information through an angle formed by a line of sight direction and a direction of the exact center of the field of view, and determines the weight coefficient based on an angle difference; when the angle formed by the line of sight direction and the direction of the exact center of the field of view is greater than a set threshold, the weight coefficient of the first gaze information is less than the weight coefficient of the second gaze information; at a large angle, the weight coefficient for determining a gaze point based on a pupil contour is reduced, such as being set to 0, thereby improving the accuracy of determining a gaze point at a large angle; determines the final eyeball center position of the left eye and the right eye of the user through multiple eye data in a database, thereby improving the accuracy of determining the eyeball center position information; establishes a 3D center model of the human eye through the pupil directions corresponding to each of the data sets, determines the eyeball center position information, and further improves the accuracy of determining the eyeball center position information.

[0187] Embodiment 3

[0188] Figure 5 FIG. 1 is a schematic diagram of a structure of a gaze information determination device provided in Embodiment 3 of the present invention. Figure 5 As shown, the device comprises:

[0189] An acquisition module 310 is used to acquire eye electrical signals and the user's eyeball center position information;

[0190] A first determination module 320, configured to determine eye data corresponding to the eye electrical signal based on the signal strength in the eye electrical signal, wherein the eye data includes pupil information;

[0191] The second determination module 330 is used to determine gaze information based on the pupil information and the eye center position information.

[0192] Optionally, the second determining module 330 includes:

[0193] A first determining unit, configured to determine pupil contour information based on the pupil information;

[0194] The second determining unit is used to determine the gaze information based on the pupil contour information and the eyeball center position information.

[0195] Optionally, the pupil contour information includes left eye pupil contour information and right eye pupil contour information, and the eyeball center position information includes left eye eyeball center position information and right eye eyeball center position information;

[0196] The second determining unit includes:

[0197] A first determining subunit is used to determine the sight direction of the left eye based on the left eye pupil contour information and the right eye eyeball center position information;

[0198] A second determining subunit is used to determine the sight direction of the right eye based on the right eye pupil contour information and the right eyeball center position information;

[0199] The third determining subunit is used to determine the gaze information based on the left eye sight direction and the right eye sight direction.

[0200] Optionally, the gaze information includes a gaze point, and the third determining subunit is specifically configured to:

[0201] Determine the shortest target line segment between the left eye straight line where the left eye sight direction is located and the right eye straight line where the right eye sight direction is located, wherein the target line segment is perpendicular to the left eye straight line and the right eye straight line respectively;

[0202] The midpoint of the target line segment is determined as the fixation point.

[0203] Optionally, the second determining module 330 is specifically configured to:

[0204] Determining first gaze information based on pupil information and eyeball center position information included in the eye data;

[0205] determining second gaze information based on the eye electrical signal;

[0206] Perform a weighted summation on the first gaze information and the second gaze information to obtain gaze information.

[0207] Optionally, the weight coefficients of the first gaze information and the second gaze information are determined based on an angle formed by a line of sight direction and a direction of the exact center of the field of view.

[0208] Optionally, when the angle formed by the line of sight direction and the direction of the center of the field of view is greater than a set threshold, the weight coefficient of the first gaze information is less than the weight coefficient of the second gaze information.

[0209] Optionally, the gaze information determining device further includes a third determining module, including:

[0210] A construction unit, configured to construct a database including a plurality of data sets, wherein the data sets include calibration points, calibration eye electrical signals corresponding to the calibration points, and calibration eye data corresponding to the calibration points;

[0211] A determination unit is used to determine the user's eye center position information based on the multiple data sets.

[0212] Optionally, determine the unit, specifically for:

[0213] Determine the pupil orientation corresponding to the calibration point in each data set;

[0214] Based on the pupil orientation corresponding to each of the data sets, the eyeball center position information is determined.

[0215] The gaze information determination device provided in the embodiment of the present invention can execute the gaze information determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0216] Embodiment 4

[0217] Figure 6 A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device 10 is intended to represent various forms of digital computers that can have a gaze information determination function. Such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 10 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0218] The electronic device 10 includes an optical transmitter (not shown in the figure), an optical receiver (not shown in the figure) and a scanner (not shown in the figure), wherein the optical transmitter emits a light beam (such as infrared light) directed to the scanner, the scanner reflects the light beam to the user's eye, the optical receiver obtains the light signal reflected from the user's eye, and the optical receiver converts the light signal into an eye electrical signal; the electronic device 10 also includes at least one processor 11 connected to the optical receiver, and a memory connected to the at least one processor 11 in communication, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program executable by at least one processor 11, and the computer program is executed by the at least one processor 11 so that the at least one processor 11 can execute the method provided by the present invention. The optical receiver can be an optical sensor. The optical transmitter can be an infrared light source.

[0219] The processor 11 can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 12 or a computer program loaded from a storage unit 18 into a random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0220] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0221] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the gaze information determination method.

[0222] In some embodiments, the gaze information determination method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of a gaze information determination method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a gaze information determination method in any other appropriate manner (e.g., by means of firmware).

[0223] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0224] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0225] In the context of the present invention, a computer-readable storage medium stores computer instructions, which are used to implement the gaze information determination method provided by the present invention when the processor executes the instructions. The computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0226] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD monitor)) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user may provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0227] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication (e.g., a communication network) in any form or medium. Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN), blockchain network, and the Internet.

[0228] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and virtual private servers (VPS) services.

[0229] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0230] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining gaze information, characterized in that: include: Obtain eye electrical signals and user's eye center position information; Determining eye data corresponding to the eye electrical signal based on the signal strength in the eye electrical signal, wherein the eye data includes pupil information; Based on the pupil information and the eyeball center position information, gaze information is determined.

2. The method according to claim 1, characterized in that Determining gaze information based on the pupil information and the eyeball center position information includes: Based on the pupil information, determining pupil contour information; Based on the pupil contour information and the eyeball center position information, gaze information is determined.

3. The method according to claim 2, characterized in that The pupil contour information includes left eye pupil contour information and right eye pupil contour information, and the eyeball center position information includes left eye eyeball center position information and right eye eyeball center position information; The determining of gaze information based on the pupil contour information and the eyeball center position information includes: Determine the sight direction of the left eye based on the left eye pupil contour information and the left eye eyeball center position information; Determine the sight direction of the right eye based on the right eye pupil contour information and the right eyeball center position information; Based on the left eye sight line direction and the right eye sight line direction, gaze information is determined.

4. The method according to claim 3, characterized in that The gaze information includes a gaze point, and determining the gaze information based on the left eye sight direction and the right eye sight direction includes: Determine the shortest target line segment between the left eye straight line where the left eye sight direction is located and the right eye straight line where the right eye sight direction is located, wherein the target line segment is perpendicular to the left eye straight line and the right eye straight line respectively; The midpoint of the target line segment is determined as the fixation point.

5. The method according to claim 1, characterized in that The determining of gaze information based on the pupil information and the eyeball center position information includes: Determining first gaze information based on pupil information and eyeball center position information included in the eye data; determining second gaze information based on the eye electrical signal; Perform a weighted summation on the first gaze information and the second gaze information to obtain gaze information.

6. The method according to claim 5, characterized in that The weight coefficients of the first gaze information and the second gaze information are determined based on an angle formed by a line of sight direction and a direction of a center of the field of view.

7. The method according to claim 6, characterized in that When the angle formed by the sight line direction and the center direction of the field of view is greater than a set threshold, the weight coefficient of the first gaze information is less than the weight coefficient of the second gaze information.

8. The method according to claim 1, characterized in that Also includes: Constructing a database including a plurality of data sets, wherein the data sets include calibration points, calibration eye electrical signals corresponding to the calibration points, and calibration eye data corresponding to the calibration points; Based on the multiple data sets, the user's eye center position information is determined.

9. The method according to claim 8, characterized in that The determining the eye center position information of the user based on the multiple data sets includes: Determine the pupil orientation corresponding to the calibration point in each data set; Based on the pupil orientation corresponding to each of the data sets, the eyeball center position information is determined.

10. A gaze information determination device, characterized in that: include: An acquisition module, used to acquire eye electrical signals and the user's eye center position information; A first determination module, configured to determine eye data corresponding to the eye electrical signal based on the signal strength in the eye electrical signal, wherein the eye data includes pupil information; The second determination module is used to determine the gaze information based on the pupil information and the eyeball center position information.

11. An electronic device, characterized in that: The electronic device comprises: an optical transmitter, an optical receiver and a scanner, wherein the optical transmitter emits a light beam toward the scanner, the scanner reflects the light beam to the user's eyes, the optical receiver acquires a light signal reflected from the user's eyes, and the optical receiver converts the light signal into an eye electrical signal; The electronic device further comprises at least one processor coupled to the optical receiver; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the gaze information determination method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the gaze information determination method according to any one of claims 1 to 9 when executed.

Images