Eye movement tracking method and device for near-eye display equipment, equipment and storage medium

Abstract

The invention relates to the technical field of eye movement tracking, and provides an eye movement tracking method, device and equipment of near-eye display equipment and a storage medium, and the near-eye display equipment is provided with a shooting module and a preset light source; the method comprises the following steps: acquiring a geometric eyeball model corresponding to a user wearing the near-to-eye display equipment; the geometric eyeball model is used for indicating geometric dimension characteristics of the eyes of the user; obtaining a target image obtained by shooting the eyes of the user by a shooting module; the target image comprises a target light spot formed by reflecting light emitted by the preset light source by the eyes of the user and a target pupil area corresponding to pupils in the eyes of the user on the target image; and obtaining eye movement information of the user according to the geometric dimension feature, the target light spot and the target pupil region, so as to improve the eye movement tracking accuracy of the near-eye display equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of eye movement tracking, and particularly relates to an eye movement tracking method and device of a near-eye display device, a near-eye display device and a storage medium. BACKGROUND

[0002] In the related art, when a near-eye display device tracks the eye movement of a user, a pupil center-corneal reflection (PCCR) scheme or a single light flood eye movement tracking scheme is generally used. However, when the PCCR scheme is used to track the eye movement of the user, if only a single light source is arranged on the near-eye display device, the accuracy of tracking the eye movement of the user is poor. Accordingly, when the single light flood eye movement tracking scheme is used, because there are individual differences in the eyes of different users, the accuracy of tracking the eye movement of the user is poor. Therefore, it is urgent to improve the accuracy of tracking the eye movement of the near-eye display device. SUMMARY

[0003] The main purpose of the present application is to provide an eye movement tracking method and device of a near-eye display device, a near-eye display device and a storage medium, which aims to solve the technical problem of poor accuracy of tracking the eye movement of the near-eye display device due to only a single light source arranged on the near-eye display device or individual differences in the eyes of different users.

[0004] In a first aspect, the present application provides an eye movement tracking method of a near-eye display device, wherein a shooting module and a preset light source are arranged on the near-eye display device.

[0005] The eye movement tracking method comprises the following steps.

[0006] Obtaining a geometric eyeball model corresponding to a user wearing the near-eye display device, wherein the geometric eyeball model is used to indicate the geometric size features of the user's eyes.

[0007] Obtaining a target image of the user's eyes obtained by the shooting module, wherein the target image comprises a target light spot formed by the user's eyes reflecting the light emitted by the preset light source, and a target pupil region corresponding to the pupil in the target image.

[0008] Obtaining the eye movement information of the user according to the geometric size features, the target light spot and the target pupil region.

[0009] In a second aspect, the present application provides an eye movement tracking device of a near-eye display device, wherein the eye movement tracking device comprises:

[0010] The model obtaining module is configured to obtain a geometric eyeball model corresponding to a user wearing the near-eye display device, wherein the geometric eyeball model is used to indicate geometric size features of the user's eye.

[0011] The target image obtaining module is configured to obtain a target image obtained by a photographing module of the near-eye display device photographing the user's eye, wherein the target image includes a target light spot formed by the user's eye reflecting light emitted by the preset light source of the near-eye display device, and a target pupil region corresponding to a pupil in the user's eye on the target image.

[0012] The eye movement tracking module is configured to obtain eye movement information of the user according to the geometric size features, the target light spot, and the target pupil region.

[0013] In a third aspect, the present application provides a near-eye display device, wherein the near-eye display device is provided with a photographing module and a preset light source, and the near-eye display device comprises a memory and a processor.

[0014] The memory is configured to store a computer program.

[0015] The processor is configured to execute the computer program and realize the steps of the eye movement tracking method of the near-eye display device when the computer program is executed.

[0016] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps of the eye movement tracking method of the near-eye display device.

[0017] The present application provides an eye movement tracking method, device, equipment and storage medium of a near-eye display device, wherein the near-eye display device is provided with a photographing module and a preset light source, and the eye movement tracking method comprises the following steps: obtaining a geometric eyeball model corresponding to a user wearing the near-eye display device; the geometric eyeball model is used to indicate geometric size features of the user's eye; obtaining a target image obtained by a photographing module photographing the user's eye; the target image includes a target light spot formed by the user's eye reflecting light emitted by the preset light source, and a target pupil region corresponding to a pupil in the user's eye on the target image; and obtaining eye movement information of the user according to the geometric size features, the target light spot, and the target pupil region.

[0018] In a case where the geometric eyeball model corresponding to the user is acquired, since the geometric eyeball model can be used to indicate the geometric size feature of the eye of the user, the near-eye display device can perform eye movement tracking on the user in combination with the geometric size feature indicated by the geometric eyeball model, the target light spot included in the target image, and the target pupil region, to obtain the eye movement information of the user. In a case where the eye movement tracking is performed on the user in combination with the geometric size feature of the eye of the user, if the eyes of different users have different geometric size features, the near-eye display device can determine the eye movement information of different users respectively by using the geometric eyeball models corresponding to the different users respectively, and thus the eye movement tracking accuracy of the near-eye display device can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is a flow diagram of a near-eye display device eye movement tracking method provided by an embodiment of the present application;

[0021] Figure 2 is a structural diagram of a geometric eyeball model related to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of the geometric position relationship between the shooting module and the preset light source of a near-eye display device and the eye of a user related to an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a first pupil circle or a second pupil circle related to an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of a pupil center position corresponding to a target image or a pupil center position corresponding to a preset image related to an embodiment of the present application;

[0025] Figure 6 is a partial diagram of a geometric eyeball model related to an embodiment of the present application;

[0026] Figure 7 is a schematic diagram of a normal vector projection line corresponding to a preset pupil circle related to an embodiment of the present application;

[0027] Figure 8 is a schematic block diagram of a near-eye display device eye movement tracking device provided by an embodiment of the present application;

[0028] Figure 9is a schematic block diagram of a near-eye display device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0030] The flowchart shown in the drawings is only an example and does not necessarily include all the contents and operations / steps, nor does it necessarily be executed in the order described. For example, some operations / steps can be further decomposed, combined or partially merged, so the actual execution order may be changed according to the actual situation.

[0031] The embodiments of the present application provide an eye movement tracking method, device and equipment of a near-eye display device and a storage medium. The eye movement tracking method of the near-eye display device can be applied to the near-eye display device. The near-eye display device can include an augmented reality (AR) glasses, virtual reality (VR) glasses, mixed reality (MR) glasses, AR helmet, VR helmet, MR helmet, etc., which are not limited herein. The eye movement tracking method of the near-eye display device can also be applied to a server, which can be a separate server, or a cloud server providing cloud services, cloud database, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0032] Some embodiments of the present application will be described in detail below with reference to the drawings. The following embodiments and features in the embodiments can be combined with each other without conflict.

[0033] Please refer to Figure 1 , Figure 1 is a flowchart of an eye movement tracking method of a near-eye display device provided by an embodiment of the present application. It should be noted that the eye movement tracking method of the near-eye display device provided by the embodiments of the present application can be used in the near-eye display device, or can be used in the server, which is not limited herein.

[0034] Exemplarily, the near-eye display device is provided with a photographing module and a preset light source. The photographing module can be used to photograph the eye of a user wearing the near-eye display device, i.e., to photograph the user's eye. The preset light source can be used to emit light to the user's eye for eye movement tracking of the user.

[0035] As shown in Figure 1 The display method of the near-eye display device includes steps S101 to S103.

[0036] S101, obtaining a geometric eyeball model corresponding to a user wearing a near-eye display device; the geometric eyeball model is used to indicate the geometric size feature of the user's eye.

[0037] In some embodiments, the geometric eyeball model can be obtained by fitting the geometric shape of the user's eye.

[0038] The geometric eyeball model can include an eyeball sphere and a cornea sphere. The eyeball sphere can be used to indicate the main body of the user's eye. The cornea sphere can be used to indicate the cornea located in the front part of the user's eye, such as the transparent cornea region in the user's eye. The eyeball sphere can have a corresponding eyeball radius to simulate the radius of the main body of the user's eye.

[0039] As shown in Figure 2 The cross section of the geometric eyeball model in the preset direction can include an eyeball sphere corresponding to the eyeball sphere and a cornea sphere corresponding to the cornea sphere. The eyeball center of the eyeball sphere can be represented as eyeball center E. The cornea center of the cornea sphere can be represented as cornea center C. The eyeball radius of the eyeball sphere can be represented as eyeball radius R. In the geometric eyeball model, the iris and pupil of the user's eye can be fitted as two concentric circles included in the geometric eyeball model, i.e., an iris circle corresponding to the iris and a pupil circle corresponding to the pupil. The iris circle and the pupil circle are located on the same plane, and the normal direction of each of the iris circle and the pupil circle is parallel to the connecting line between the eyeball center E and the cornea center C. The common center of the iris circle and the pupil circle can be regarded as the geometric center of the pupil. The geometric center of the pupil can be represented as pupil center P. The pupil center P is located on the surface of the eyeball sphere. The pupil center P can be the center point at which the pupil surface is tangent to the eyeball sphere.

[0040] The geometric eyeball model can be used to simulate the movement of the user's eye. For example, when the geometric eyeball model is used to simulate the rotation of the user's eye, the rotation process of the user's eye can be represented as the rotation process of the eyeball sphere around the eyeball center E in the geometric eyeball model. During the movement of the user's eye, if the position of the pupil in the user's eye changes, the position of the pupil center P in the geometric eyeball model will also change accordingly. During the movement of the user's eye, if the position of the eyeball in the user's eye does not change, the position of the eyeball center E and the eyeball radius R in the geometric eyeball model will also not change. The position of the eyeball center E can also be referred to as the eyeball center position. The position of the pupil center P can also be referred to as the pupil center position. Based on this, the geometric eyeball model can be used to indicate the geometric size features of the user's eye. The geometric size features can include the eyeball center position and the eyeball radius of the user's eye. Accordingly, the eyeball center position of the geometric eyeball model is equivalent to the eyeball center position of the user's eye. The eyeball radius of the geometric eyeball model is equivalent to the eyeball radius of the user's eye. Of course, it is not limited to this, and the geometric size features can also include the distance between the corneal center and the eyeball center of the user's eye. The distance between the corneal center and the eyeball center in the geometric eyeball model is equivalent to the distance between the corneal center and the eyeball center of the user's eye, which is not limited here. It can be understood that the geometric size features of different user's eyes can be different.

[0041] Since the pupil center P can be the center point of the tangent point of the pupil surface and the eyeball surface, the normal vector corresponding to the pupil center P in the geometric eyeball model can be used by the near-eye display device to estimate the user's line of sight, and then determine the eye movement information of the user. Based on this, the geometric eyeball model corresponding to the user can be used by the near-eye display device to track the eye movement of the user's eye.

[0042] In some embodiments, the geometric eyeball model can be pre-constructed. For example, the near-eye display device can obtain a plurality of preset images obtained by the shooting module shooting the user's eye. The plurality of preset images can be obtained during the rotation of the user's eye, and the near-eye display device can use the plurality of preset images to pre-construct the geometric eyeball model corresponding to the user. For example, the near-eye display device can fit the geometric shape of the user's eye according to the plurality of preset images to obtain the geometric eyeball model. The geometric eyeball model can be used to indicate the geometric size features of the user's eye, such as the eyeball center position, the eyeball radius, and the distance between the corneal center and the eyeball center. Of course, the geometric size features of the user's eye are not limited to this, which is not limited here.

[0043] In a case where the near-eye display device pre-establishes a geometric eyeball model, the near-eye display device can establish an association between the geometric eyeball model corresponding to the user and the user identification information. For example, the user identification information can include voiceprint information, fingerprint information, iris information, and the like of the user, which can be used to indicate the identity of the user, without limitation.

[0044] Of course, the geometric eyeball model can also be established in real time. For example, the near-eye display device can establish the geometric eyeball model corresponding to the user in response to a received preset modeling instruction. In an example embodiment, the near-eye display device can detect the preset modeling instruction in a case where the user starts the near-eye display device for the first time. Accordingly, the near-eye display device can acquire a plurality of preset images obtained by the imaging module imaging the user's eyes in response to the preset modeling instruction, to establish the geometric eyeball model corresponding to the user. In another example embodiment, the preset modeling instruction can also be determined according to a preset operation of the user on the near-eye display device. The preset operation can include a preset touch operation, a preset gesture, a preset voice, and the like, without limitation.

[0045] In a case where the geometric eyeball model corresponding to the user wearing the near-eye display device is acquired, the geometric eyeball model can be used by the near-eye display device to track the eye movement of the user. For example, the near-eye display device can determine the eye movement information of the user by using the geometric eyeball model to indicate the geometric size features of the user's eyes, which is beneficial to improve the convenience and accuracy of eye movement tracking of the near-eye display device.

[0046] S102, acquire a target image obtained by an imaging module imaging a user's eyes; the target image includes a target light spot formed by the user's eyes reflecting light emitted by a preset light source, and a target pupil region on the target image corresponding to a pupil in the user's eyes.

[0047] In the process of the user wearing the near-eye display device, a preset light source provided on the near-eye display device can emit light to the user's eyes. The user's eyes can reflect the light emitted by the preset light source. An imaging module of the near-eye display device can image the user's eyes to obtain a corresponding image. The image imaged by the imaging module can include a light spot formed by the user's eyes reflecting the light emitted by the preset light source. Accordingly, since the image is obtained by the imaging module imaging the user's eyes, the image can include a pupil region on the image corresponding to a pupil in the user's eyes.

[0048] In the process of tracking the eye movement of the user, the near-eye display device can determine the image obtained by the imaging module imaging the user's eyes as a target image to acquire the target image. The target image can include a target light spot formed by the user's eyes reflecting the light emitted by the preset light source, and a target pupil region on the target image corresponding to a pupil in the user's eyes.

[0049] Since the positions of the pupil and the cornea in the user's eye are changed in the process that the user rotates the eye, the position of the pupil center P in the geometric eyeball model is changed following the change of the position of the pupil in the user's eye, and the position of the corneal center C in the geometric eyeball model is changed following the change of the position of the cornea in the user's eye, that is, the positions of the pupil center P and the corneal center C in the geometric eyeball model are uncertain. When the target image is obtained, the target image includes the target pupil region and the target light spot, which can be used by the near-eye display device to determine the position of the pupil center P in the geometric eyeball model, and the position of the pupil center P corresponds to the target pupil region and the target light spot in the target image, so the position of the pupil center P can also be referred to as the pupil center position corresponding to the target image in the geometric eyeball model, that is, the pupil center position corresponding to the target image. Correspondingly, when the target image is obtained, the target image includes the target light spot, which can be used by the near-eye display device to determine the position of the corneal center C in the geometric eyeball model, and the position of the corneal center C corresponds to the target light spot in the target image, so the position of the corneal center C can also be referred to as the corneal center position corresponding to the target image in the geometric eyeball model, that is, the corneal center position corresponding to the target image.

[0050] For example, the near-eye display device can calculate the position of the pupil center P corresponding to the target image in the geometric eyeball model according to the target pupil region and the target light spot included in the target image. The near-eye display device can combine the position of the pupil center P corresponding to the target image in the geometric eyeball model and the position of the eyeball center E of the geometric eyeball model to estimate the line of sight of the user's eye and obtain the line of sight information of the user. The near-eye display device can also combine the position of the pupil center P corresponding to the target image in the geometric eyeball model, the position of the eyeball center E of the geometric eyeball model, and the eyeball radius R to calculate the pupil radius of the user's eye corresponding to the target image and obtain the pupil radius of the user's eye. The near-eye display device can obtain the eye movement information of the user according to at least one of the line of sight information of the user and the pupil radius of the user's eye.

[0051] Correspondingly, the near-eye display device can calculate the position of the corneal center of curvature C corresponding to the target image in the geometric eyeball model according to the target spot included in the target image. The near-eye display device can determine the position of the corneal center of curvature corresponding to the target image by using the target spot included in the target image, in combination with the corneal reflection geometry corresponding to the shooting module, the target spot, and the preset light source in the near-eye display device. Since the distance between the corneal center of curvature and the eyeball center of curvature in the user's eye does not change with the rotation of the user's eye, the near-eye display device can update the position of the eyeball center of curvature of the geometric eyeball model in combination with the distance between the corneal center of curvature and the eyeball center of curvature in the geometric eyeball model when the position of the corneal center of curvature corresponding to the target image is determined. For example, in the case where the near-eye display device has a corresponding systematic drift, such as a systematic drift caused by the user's head movement, the near-eye display device can update the position of the eyeball center of curvature of the geometric eyeball model to compensate for the systematic drift of the near-eye display device, thereby ensuring the accuracy of subsequent eye movement tracking of the user by the near-eye display device.

[0052] In the case where the target image obtained by the shooting module shooting the user's eye is obtained, the target image can be used by the near-eye display device to track the eye movement of the user. For example, the near-eye display device can determine the eye movement information of the user by using the geometric size features of the user's eye indicated by the geometric eyeball model, in combination with the target spot and the target pupil region included in the target image, thereby improving the convenience and accuracy of eye movement tracking of the near-eye display device.

[0053] S103, obtaining the eye movement information of the user according to the geometric size features, the target spot, and the target pupil region.

[0054] The near-eye display device is calibrated before it is shipped, so the near-eye display device can determine the setting position of the preset light source on the near-eye display device, and the near-eye display device can also determine the camera parameters of the shooting module. The camera parameters can include camera intrinsic parameters, camera extrinsic parameters, camera distortion, etc., which are not limited herein. Correspondingly, the near-eye display device can determine the reference plane corresponding to the shooting module.

[0055] The near-eye display device can determine the second position of the target spot on the user's eye according to the target spot included in the target image.

[0056] For example, the near-eye display device can determine the depth information of the target spot according to the target spot included in the target image, and determine the second position of the target spot on the user's eye according to the depth information of the target spot.

[0057] In some embodiments, in the case that the photographing module can include at least one of a Time of Flight Camera (TOF) camera, a light field camera, the target image can have corresponding depth information, and then the near-eye display device can obtain the depth information of the target light spot.

[0058] In some other embodiments, since the photographing module has completed calibration, the near-eye display device can determine the depth information of the target light spot in combination with the target image, the camera parameters of the photographing module, and the reference plane corresponding to the photographing module.

[0059] When the light emitted by the preset light source irradiates the user's eye, since the user's eye can be regarded as an eyeball sphere, the depth information of different positions of the user's eye can be different, which is equivalent to the distance between different positions of the user's eye and the photographing module being different. The positions of the light emitted by the preset light source reflected by different positions of the user's eye to the photographing module are also different, and then the target image formed by the light emitted by the preset light source reflected by different positions of the user's eye to the photographing module includes target light spots with different positions. As shown in FIG. 1, under the influence of the geometric positional relationship between the photographing module and the preset light source of the near-eye display device and the user's eye, the second position of the target light spot on the user's eye included in the target image and the position of the target light spot on the reference plane corresponding to the photographing module are offset, that is, there is a parallax. The near-eye display device can determine the parallax corresponding to the target light spot in combination with the target image according to the reference plane corresponding to the photographing module, and then determine the depth information of the target light spot. Figure 3

[0060] The near-eye display device can determine a first offset position of the target light spot on the reference plane corresponding to the photographing module according to the first position of the target light spot on the target image and the camera parameters of the photographing module; determine the parallax corresponding to the target light spot according to the first position of the target light spot and the first offset position of the target light spot; and determine the depth information of the target light spot according to the parallax, the distance between the intersection of the preset light source and the reference plane, the relative horizontal distance between the photographing module and the preset light source, and the focal length of the photographing module. The calculation formula related to the depth information of the target light spot can be represented as:

[0061]

[0062] wherein Depth is used to indicate the depth information of the target light spot; D is used to indicate the distance between the intersection of the preset light source and the reference plane, and the intersection of the preset light source and the reference plane is used to indicate the intersection between the preset light source and the reference plane; d is used to indicate the parallax corresponding to the target light spot; f is used to indicate the focal length of the photographing module; x is used to indicate the relative horizontal distance between the photographing module and the preset light source. x c ​​The relative horizontal distance between the photographing module and the preset light source. In the case that the photographing module has completed the calibration, the near-eye display device can directly obtain the camera parameters of the photographing module, such as the distance D between the preset light source and the intersection of the reference plane, the relative horizontal distance x between the photographing module and the preset light source, the focal length f of the photographing module, and the like, which is equivalent to that the photographing module has completed the determination of the camera parameters in the case of completing the calibration, and is not limited herein. Accordingly, the parallax d corresponding to the target light spot can be determined by using the first position of the target light spot on the target image and the camera parameters of the photographing module. c and the focal length f of the photographing module x and the like, which is equivalent to that the photographing module has completed the determination of the camera parameters in the case of completing the calibration, and is not limited herein. Accordingly, the parallax d corresponding to the target light spot can be determined by using the first position of the target light spot on the target image and the camera parameters of the photographing module.

[0063] In the case of determining the depth information of the target light spot, the near-eye display device can determine the second position of the target light spot on the user's eye in combination with the depth information of the target light spot and the camera parameters of the photographing module. The second position of the target light spot on the user's eye can be used by the near-eye display device to determine the eye movement information of the user in combination with the geometric eyeball model.

[0064] The near-eye display device can determine the position of the pupil center P corresponding to the target image in the geometric eyeball model according to the target pupil region included in the target image. As shown in Figure 4 The near-eye display device can perform perspective projection on the target pupil region to obtain at least two first pupil circles corresponding to the target pupil region. The depth information of the at least two first pupil circles can be a preset depth. The value of the preset depth can be variable. Each same preset depth can correspond to two first pupil circles. In the case of determining the first pupil circle, the position of the center point of the first pupil circle can be used to determine the position of the pupil center corresponding to the target image. For example, if the first connecting line between the center point of the first pupil circle and the camera optical center of the photographing module covers the position of the pupil center corresponding to the target image, the near-eye display device can determine the position of the pupil center corresponding to the target image by using the first connecting line between the center point of the first pupil circle and the camera optical center of the photographing module.

[0065] The normal vector of each of the two first pupil circles at the same preset depth is different. The near-eye display device can determine a target pupil circle from the at least two first pupil circles according to a second position of the target light spot on the user's eye. The second position of the target light spot is equivalent to the position of the target light spot in the spatial coordinate system corresponding to the shooting module, and the second position of the target light spot can include a 3D coordinate. The second position of the target light spot on the user's eye can be used to reflect the position of the reflection point when the user's eye reflects the light emitted by the preset light source. For example, the near-eye display device can determine the first pupil circle whose normal vector meets the corneal reflection geometry of the light emitted by the preset light source as the target pupil circle according to the geometric relationship between the preset light source and the shooting module, and in combination with the second position of the target light spot on the user's eye. For example, in the case where the normal vector of the first pupil circle meets the corneal reflection geometry, the light emitted by the preset light source can be reflected by the user's eye and can enter the shooting range corresponding to the shooting module to form a target light spot included in the target image, and the near-eye display device can determine the first pupil circle whose normal vector meets the corneal reflection geometry as the target pupil circle. In the case where the normal vector of the first pupil circle does not meet the corneal reflection geometry, the normal vector of the first pupil circle can correspond to a user eye orientation, at which the light emitted by the preset light source cannot be reflected by the user's eye to enter the shooting range corresponding to the shooting module to form a target light spot included in the target image, or at which the light emitted by the preset light source can be reflected by the user's eye to enter the shooting range corresponding to the shooting module to form a light spot different from the target light spot included in the target image. Each preset depth can correspond to a target pupil circle. The pupil center position of the target pupil circle can cover the pupil center position corresponding to the target image. For example, the pupil center position of the target pupil circle at a certain preset depth can be used as the pupil center position corresponding to the target image, and the distance between the pupil center position of the target pupil circle and the eyeball center position of the geometric eyeball model is equal to the eyeball radius of the geometric eyeball model.

[0066] For example, the target pupil circle can be used by the near-eye display device to determine the eye movement information of the user. The eye movement information can include at least one of the line of sight information of the user and the pupil radius of the user's eye. Of course, the eye movement information of the user is not limited to this, and is not limited herein.

[0067] For example, the geometric dimension feature can include an eyeball center position of the user's eye. A normal vector of the target pupil circle can be used to estimate the user's line of sight to obtain the user's line of sight information. The geometric dimension feature can include an eyeball center position of the user's eye. For example, the target pupil circles at different preset depths are parallel to each other, and the normal vectors of the target pupil circles at different preset depths are parallel to each other. In the case that the normal vector of the target pupil circle passes through the eyeball center position, the normal vector of the target pupil circle can be used to indicate the direction from the eyeball center position to the pupil center position corresponding to the target image, and the direction indicated by the normal vector of the target pupil circle can be determined as the user's line of sight direction, and then the user's line of sight information is obtained. The user's line of sight information can reflect the user's real line of sight information.

[0068] The geometric dimension feature can include an eyeball radius of the user's eye. As shown in Figure 5 In the case of determining the target pupil circle, the near-eye display device can use the eyeball radius to determine the pupil center position corresponding to the target image from the pupil center positions of the target pupil circles at different preset depths. The distance between the eyeball center position and the pupil center position corresponding to the target image is equal to the eyeball radius. Accordingly, the near-eye display device can determine the pupil radius of the user's eye corresponding to the target image according to the radius of the target pupil circle in which the pupil center position corresponding to the target image is located. The pupil radius of the user's eye corresponding to the target image can reflect the actual pupil radius of the user under the real line of sight information.

[0069] The near-eye display device can determine the eye movement information of the user according to at least one of the user's line of sight information and the pupil radius of the user's eye.

[0070] In the case of tracking the eye movement of the user to obtain the eye movement information of the user according to the geometric dimension feature, the target light spot, and the target pupil region, the near-eye display device can use one or more of the geometric dimension feature, the target light spot, and the target pupil region to determine at least one of the user's line of sight information and the pupil radius of the user's eye corresponding to the target image. The user's line of sight information can reflect the user's real line of sight information, and the pupil radius of the user's eye can reflect the user's real pupil radius, which is beneficial to improve the eye movement tracking accuracy of the near-eye display device.

[0071] For example, the geometric dimension feature includes an eyeball center position and an eyeball radius of the user's eye.

[0072] In some embodiments, a first position of the target glint on the target image is determined; a second position of the target glint on the user's eye is determined according to the first position of the target glint; the target pupil region is perspective projected to obtain at least two first pupil circles corresponding to the target pupil region; a target pupil circle of the target image is determined from the at least two first pupil circles according to the second position of the target glint; and eye movement information of the user is obtained according to the eyeball center position, the eyeball radius, and the target pupil circle.

[0073] For example, in a case where the target image is obtained, the near-eye display device can determine a first position of the target glint on the target image. The near-eye display device can perform image detection processing on the target image to detect the target glint included in the target image, to obtain the first position of the target glint on the target image. In a case where the target image is a two-dimensional (2D) image, the first position of the target glint can include 2D coordinates. For example, the first position of the target glint can be represented as (u, v), where u is used to indicate a horizontal pixel position of the target glint on the target image, and v is used to indicate a vertical pixel position of the target glint on the target image.

[0074] The near-eye display device can determine a second position of the target glint on the user's eye according to the first position of the target glint. For example, the near-eye display device can perform conversion processing on the first position of the target glint by using camera parameters of the photographing module, to convert the target glint from an image coordinate system corresponding to the target image to a spatial coordinate system corresponding to the photographing module, to obtain the second position of the target glint on the user's eye. The second position of the target glint can include three-dimensional (3D) coordinates. For example, the target glint can be represented as glint Pglint. The second position of the glint Pglint can be represented as (X, Y, Z), where X is used to indicate a horizontal transverse position of the glint Pglint relative to the photographing module, Y is used to indicate a vertical position of the glint Pglint relative to the photographing module, and Z is used to indicate a depth distance of the glint Pglint relative to the photographing module.

[0075] For example, in a case where the target image is obtained, the near-eye display device can perform perspective projection on the target pupil region included in the target image. The near-eye display device can perform perspective projection on the target pupil region based on the pinhole imaging principle. For example, the near-eye display device can perform perspective projection on the target pupil region by using the camera parameters of the photographing module. Figure 4As shown, in the process of perspective projection on the target pupil region, a target cone can be constructed with the target pupil region as the bottom surface and the camera optical center / camera focal point of the shooting device as the top vertex. The target pupil region is equivalent to the intersection line between the target cone and the image plane where the target image is located. Based on the constructed target cone, the circular back projection corresponding to the target pupil region can be determined, that is, at least two first pupil circles corresponding to the target pupil region. Among them, the circular back projection corresponding to the target pupil region at a certain specified depth is two first pupil circles with different normal vectors. The two first pupil circles with different normal vectors are symmetric about the major axis of the ellipse.

[0076] Taking the two first pupil circles with different normal vectors corresponding to the target pupil region at a certain specified depth as an example, the two first pupil circles can be equivalent to the double-circle solution of the target pupil region at the specified depth, hereinafter referred to as double-circle solution. If the geometric shape of the target pupil region on the target image is an ellipse, then the double-circle solution is caused by the symmetry of the target pupil region about its major axis, and physically, there is only one correct user eye orientation for the user eye. In the case of determining the second position of the target light spot, the near-eye display device can take the second position of the target light spot as the basis for determining the correct solution from the double-circle solution. Since the near-eye display device has completed the calibration, the position of the preset light source on the near-eye display device is fixed, and the position of the shooting device on the near-eye display device is fixed, so the position distribution of the target light spot relative to the user eye will change systematically due to different user eye orientations. For example, in the double-circle solution, only the normal vector of one solution can satisfy the corneal reflection geometry corresponding to the light emitted by the preset light source: the normal vector of the correct solution in the double-circle solution satisfies the corneal reflection geometry, so that the light emitted by the preset light source can be reflected by the user eye to form a target light spot included in the target image in the corresponding shooting range of the camera module; the normal vector of the incorrect solution in the double-circle solution does not satisfy the corneal reflection geometry, and the normal vector of the incorrect solution may correspond to a user eye orientation, in which the light emitted by the preset light source cannot be reflected by the user eye into the corresponding shooting range of the shooting module, or in which the predicted light spot formed by the light emitted by the preset light source reflected by the user eye into the corresponding shooting range of the shooting module is different from the target light spot included in the target image. For example, the first position of the predicted light spot on the target image is different from the first position of the target light spot on the target image. For another example, the second position of the predicted light spot on the user eye is different from the second position of the target light spot on the target image. Therefore, by the existence of the target light spot in the target image and the position of the target light spot, the near-eye display device can eliminate the incorrect solution from the double-circle solution and obtain the correct solution in the double-circle solution.

[0077] Correspondingly, in a case that the target pupil region is perspective projected to obtain a target cone corresponding to the target pupil region, and at least two first pupil circles corresponding to the target pupil region are determined according to different preset depths, each preset depth can correspond to one target pupil circle. Different target pupil circles are parallel to each other, i.e., normal vectors of different target pupil circles are the same. Based on this, the number of target pupil circles can include at least one.

[0078] In a case that the target pupil circle is determined, the near-eye display device can obtain eye movement information of the user according to the geometric size features of the user's eye indicated by the geometric eyeball model and the target pupil circle, such as eyeball center position, eyeball radius, and target pupil circle.

[0079] As shown in Figure 5 , in a case that the target pupil circle is determined, the normal vector of the target pupil circle can be used to indicate a direction from the eyeball center position to a pupil center position corresponding to the target image, and then the near-eye display device can determine the line of sight information of the user according to the normal vector of the target pupil circle, such as determining the line of sight direction of the user.

[0080] As shown in Figure 5 , in a case that the target pupil circle is determined, the near-eye display device can determine a first connecting line between the pupil center of the target pupil circle and the camera optical center according to the pupil center of the target pupil circle and the camera optical center. The near-eye display device can determine a second connecting line between the eyeball center position and the pupil center position corresponding to the target image according to the normal vector of the target pupil circle. The intersection of the first connecting line and the second connecting line can be determined as the pupil center position corresponding to the target image. The distance between the eyeball center position and the pupil center position corresponding to the target image is equal to the eyeball radius of the geometric eyeball model. Correspondingly, the radius of the target pupil circle in which the pupil center position corresponding to the target image is located can be determined as the pupil radius of the user's eye corresponding to the target image.

[0081] At least one of the line of sight information of the user and the pupil radius of the user's eye can be used to determine the eye movement information of the user, which is beneficial to improve the eye movement tracking convenience of the near-eye display device. Correspondingly, in a case that the near-eye display device determines the eye movement information of the user according to the geometric size features indicated by the geometric eyeball model, the line of sight direction of the user can reflect the real line of sight information of the user, and the pupil radius of the user's eye can reflect the actual pupil radius of the user under the real line of sight information, which is beneficial to improve the eye movement tracking accuracy of the near-eye display device.

[0082] Illustratively, the geometric size features include the eyeball center position of the user's eye and the distance between the corneal center and the eyeball center.

[0083] In some embodiments, the corneal sphere center position corresponding to the target image is determined according to the second position of the target light spot on the user's eye; and the eyeball center position is updated according to the corneal sphere center position corresponding to the target image and the distance between the corneal sphere center and the eyeball center, to obtain an updated eyeball center position.

[0084] In the case of determining the second position of the target light spot on the user's eye, the near-eye display device can determine the corneal sphere center position corresponding to the target image in combination with the corneal reflection geometry corresponding to the shooting module, the target light spot and the preset light source in the near-eye display device. Of course, it is not limited to this. In the process of determining the geometric eyeball model corresponding to the user's eye, the near-eye display device can use the preset light spots included in the plurality of preset images to fit the corneal curvature radius of the corneal sphere in the geometric eyeball model. Therefore, the geometric size features included in the geometric eyeball model can also include the corneal curvature radius. The corneal curvature radius will not change with the rotation of the user's eye or the head movement of the user. Therefore, in the process of determining the corneal sphere center position corresponding to the target image, the near-eye display device can determine the corneal sphere center position corresponding to the target image in combination with the second position of the target light spot on the user's eye and the corneal curvature radius of the geometric eyeball model.

[0085] Since the distance between the corneal sphere center and the eyeball center in the user's eye will not change with the rotation of the user's eye or the head movement of the user, in the case of determining the corneal sphere center position corresponding to the target image, the near-eye display device can calculate the eyeball center position corresponding to the target image in combination with the distance between the corneal sphere center and the eyeball center in the geometric eyeball model. The near-eye display device can determine whether the eyeball center position corresponding to the target image is the same as the eyeball center position of the geometric eyeball model. If not, the near-eye display device can determine that it has systematic drift, and can update the eyeball center position corresponding to the target image to the eyeball center position of the geometric eyeball model, and then compensate for the systematic drift of the near-eye display device, so as to improve the accuracy of the eye movement tracking of the near-eye display device on the user.

[0086] The eye movement information of the user is obtained according to the updated eyeball center position, the eyeball radius and the target pupil circle.

[0087] For example, the process of obtaining the eye movement information of the user according to the updated eyeball center position, the eyeball radius and the target pupil circle can refer to the aforementioned process of obtaining the eye movement information of the user according to the eyeball center position, the eyeball radius and the target pupil circle, which will not be repeated here. In the case of determining the eye movement information of the user by replacing the pre-updated eyeball center position with the updated eyeball center position, the updated eyeball center position can compensate for the systematic drift of the near-eye display device in the process of eye movement tracking of the user, so as to avoid the adverse effects of the systematic drift of the near-eye display device on the eye movement tracking accuracy of the near-eye display device.

[0088] Based on this, in the case that the near-eye display device obtains the target image, the near-eye display device can determine the corneal center position corresponding to the target image in combination with the second position of the target light spot on the user's eye included in the target image, update the eyeball center position in combination with the corneal center position corresponding to the target image and the distance between the corneal center and the eyeball center, obtain the updated eyeball center position, and then compensate for the systematic drift of the near-eye display device in the process of eye movement tracking of the user, so as to avoid the adverse effects of the systematic drift of the near-eye display device on the eye movement tracking accuracy of the near-eye display device, which is beneficial to improving the eye movement tracking accuracy of the near-eye display device for the user.

[0089] In some embodiments, a plurality of preset images obtained by the photographing module photographing the user's eye are obtained; the preset image includes a preset light spot formed by the user's eye reflecting light emitted by a preset light source, and a preset pupil region corresponding to the pupil in the user's eye on the preset image; the positions of the preset pupil regions of the plurality of preset images on the corresponding preset images are different; the user's eye is geometrically modeled according to the preset light spots and the preset pupil regions of the plurality of preset images, to obtain a corresponding geometric eyeball model of the user.

[0090] The near-eye display device can construct the corresponding geometric eyeball model of the user.

[0091] In the process of constructing the corresponding geometric eyeball model of the user, the near-eye display device can determine an image obtained by the photographing module photographing the user's eye as a preset image to obtain the preset image. The preset image can include a preset light spot formed by the user's eye reflecting light emitted by a preset light source, and a preset pupil region corresponding to the pupil in the user's eye on the preset image.

[0092] In the process of constructing the geometric eyeball model corresponding to the user, in order to improve the construction accuracy of the geometric eyeball model, so that the geometric eyeball model can be adapted to the user's eye, the near-eye display device can acquire a plurality of preset images. The plurality of preset images can be obtained by the shooting module in the process that the preset light source continuously emits light to the user's eye in a rotating state. The positions of the preset pupil regions of the plurality of preset images on the corresponding preset images are different. The number of preset images is at least three.

[0093] In the case of acquiring a plurality of preset images, the near-eye display device can perform geometric modeling processing on the user's eye according to the preset light spots and the preset pupil regions of the plurality of preset images, to obtain a geometric eyeball model corresponding to the user. For example, the near-eye display device can determine the pupil center positions corresponding to the plurality of preset images by using the preset light spots and the preset pupil regions included in the plurality of preset images. Accordingly, the near-eye display device can fit the eyeball sphere center position and the eyeball radius of the geometric eyeball model by using the pupil center positions corresponding to the plurality of preset images, to determine the geometric eyeball model.

[0094] In the case that the near-eye display device can perform geometric modeling processing on the user's eye by using the plurality of preset images to obtain a geometric eyeball model corresponding to the user, it is beneficial to improve the geometric modeling convenience and accuracy of the user's eye.

[0095] In some embodiments, the first position of the preset light spot on the preset image is determined, the second position of the preset light spot on the user's eye is determined according to the first position of the preset light spot, the preset pupil region included in the same preset image is perspective projected to obtain at least two second pupil circles corresponding to the preset pupil region, the preset pupil circle of the preset image is determined from the at least two second pupil circles according to the second position of the preset light spot, and the geometric modeling processing on the user's eye is performed according to the preset pupil circles of the plurality of preset images to obtain a geometric eyeball model corresponding to the user.

[0096] For example, in the case of acquiring a preset image, the preset image can be used to determine the first position of the preset light spot on the preset image. The near-eye display device can determine the first position of the preset light spot on the preset image included in the preset image. The step of determining the first position of the preset light spot on the preset image included in the preset image can refer to the aforementioned step of determining the first position of the target light spot on the target image, which will not be described here.

[0097] The near-eye display device can determine the second position of the preset light spot on the user's eye according to the first position of the preset light spot. The step of determining the second position of the preset light spot on the user's eye can refer to the aforementioned step of determining the second position of the target light spot on the user's eye, which will not be repeated here.

[0098] For example, in the case of obtaining a preset image, the preset image can be used to determine at least two second pupil circles corresponding to the preset pupil region. The near-eye display device can perform perspective projection on the preset pupil region included in the same preset image to obtain at least two second pupil circles corresponding to the preset pupil region. The at least two second pupil circles can be as shown in Figure 4 The step of performing perspective projection on the preset pupil region included in the same preset image can refer to the aforementioned step of performing perspective projection on the target pupil region, which will not be repeated here.

[0099] Correspondingly, since the preset image includes the preset light spot and the preset pupil region, in the case of determining at least two pupil circles corresponding to the preset pupil region, the preset pupil circle can be determined from the at least two second pupil circles according to the second position of the preset light spot. The step of determining the preset pupil circle of the preset image from the at least two second pupil circles according to the second position of the preset light spot can refer to the aforementioned step of determining the target pupil circle from the at least two first pupil circles according to the second position of the target light spot, which will not be repeated here.

[0100] For the same preset image, each preset pupil circle at a preset depth can include preset pupil circles at different preset depths. The pupil center position of the preset pupil circle can cover the pupil center position corresponding to the preset image. For example, the pupil center position of the preset pupil circle at a certain preset depth can be used as the pupil center position corresponding to the preset image.

[0101] Each preset image can determine at least one preset pupil circle. Since the preset pupil regions of the plurality of preset images are located at different positions on the corresponding preset images, the normal vectors of the preset pupil circles of the plurality of preset images are also correspondingly different.

[0102] The near-eye display device can perform geometric modeling processing on the user's eye according to the preset pupil circles of the plurality of preset images to obtain a corresponding geometric eyeball model of the user.

[0103] As shown in Figure 4In a case where the preset pupil circles are determined, as shown, the normal vector of the preset pupil circle can be used to indicate a direction from the eyeball sphere center of the geometric eyeball model to the pupil center corresponding to the preset image, and thus the normal vector of the preset pupil circle of each preset image can pass through the eyeball sphere center. Based on this, the near-eye display device can use the preset pupil circles of the preset images to fit the eyeball sphere of the geometric eyeball model to determine the eyeball sphere center position and the eyeball radius, and thus obtain the geometric size features of the user's eye part indicated by the geometric eyeball model, and complete the construction of the geometric eyeball model.

[0104] In a case where the near-eye display device can determine the preset pupil circles of the preset images, and use the preset pupil circles of the preset images to perform geometric modeling processing on the user's eye part to obtain the corresponding geometric eyeball model, the geometric modeling convenience of the user's eye part is improved.

[0105] In some embodiments, the eyeball sphere center position of the geometric eyeball model is obtained by performing eyeball sphere center fitting processing on the geometric eyeball model according to the normal vectors of the preset pupil circles of the preset images; the eyeball radius of the geometric eyeball model is determined according to the preset pupil circles and the eyeball sphere center position of the geometric eyeball model; and the geometric eyeball model is obtained by performing geometric modeling processing on the user's eye part according to the eyeball sphere center position and the eyeball radius.

[0106] For example, as the normal vector of the preset pupil circle of each preset image can pass through the eyeball sphere center position of the geometric eyeball model, the near-eye display device can perform eyeball sphere center fitting processing on the geometric eyeball model according to the normal vectors of the preset pupil circles to obtain the eyeball sphere center position of the geometric eyeball model. The eyeball sphere center position of the geometric eyeball model can be used to indicate the eyeball sphere center position of the user's eye part.

[0107] In a case where the preset pupil circles of the preset images are determined, the pupil center positions of the preset pupil circles under different preset images can be different, and thus the near-eye display device can use the pupil center positions of the preset pupil circles of different preset images to fit the eyeball radius of the geometric eyeball model. The eyeball radius of the geometric eyeball model can be used to indicate the eyeball radius of the user's eye part. Based on this, the geometric eyeball model can be used to indicate the geometric size features of the user's eye part.

[0108] For example, in the process of fitting the eyeball radius of the geometric eyeball model, the near-eye display device can determine the pupil center positions of the preset pupil circles of the preset images on the geometric eyeball model; and the eyeball radius of the geometric eyeball model is determined according to the distances between the pupil center positions corresponding to the preset images and the eyeball sphere center position of the geometric eyeball model.

[0109] like Figure 6 As shown, the pupil center positions of the preset pupil circles in multiple preset images include, for example, positions P1, P2, and P3. The near-eye display device can calculate the distances between each of positions P1, P2, and P3 and the eyeball center E, and determine the eyeball radius of the geometric eyeball model based on these distances. For example, the near-eye display device can determine the eyeball radius of the geometric eyeball model based on one of the following: the average, median, maximum, minimum, or weighted average of the distances between each of positions P1, P2, and P3 and the eyeball center E.

[0110] Given a defined center and radius of the eyeball in the geometric eyeball model, a near-eye display device can construct a corresponding geometric eyeball model for the user. For example, the near-eye display device can set appropriate parameters for the eyeball within a pre-defined geometric eyeball model based on the center and radius, thereby obtaining the user's corresponding geometric eyeball model.

[0111] When the position of the eyeball center and the eyeball radius of the geometric eyeball model can be determined based on the normal vectors of multiple preset pupil circles, the geometric eyeball model can be used to indicate the geometric size characteristics of the user's eye, which helps to improve the convenience of determining the geometric size characteristics of the user's eye, and thus helps to improve the convenience of determining the geometric eyeball model of the user's eye.

[0112] In some implementations, the normal vector of a preset pupil circle in a preset image is projected onto a preset plane to obtain the normal vector projection line corresponding to the preset pupil circle; the position closest to multiple normal vector projection lines is determined as the eye center position of the geometric eyeball model on the preset plane; the eye center position is back-projected to obtain the eyeball center position of the geometric eyeball model.

[0113] For example, the normal vector of the preset pupil circle of each preset image passes through the center of the eyeball in the geometric eyeball model. Based on this, the near-eye display device can use the normal vectors of the preset pupil circles of multiple preset images to determine the center of the eyeball in the geometric eyeball model.

[0114] The pupil of a user will contract in different environments, that is, the pupil radius in a real scene is variable. Accordingly, if the respective pupil radii of the respective preset pupil circles of different preset images are all determined as a same preset pupil radius for the near-eye display device to determine the pupil center position of the geometric eyeball model, the respective pupil center positions determined by the different preset images each have a different scale error. For example, when the pupil radius actually becomes larger, that is, the pupil radius of the preset pupil circle is larger than the preset pupil radius, if the preset pupil radius is still used to calculate the pupil center position of the geometric eyeball model, the calculated pupil center position will be farther away from the original pupil center position of the preset pupil circle. Conversely, when the pupil radius actually becomes smaller, that is, the pupil radius of the preset pupil circle of the preset image is smaller than the preset pupil radius, if the preset pupil radius is still used to calculate the pupil center position of the geometric eyeball model, the calculated pupil center position will be closer to the shooting module than the original pupil center position of the preset pupil circle. Based on this, the preset pupil circle of each preset image will have different degrees of deviation, such as "small and close" or "large and far", due to the deviation of the pupil radius from the preset pupil radius, and these deviations cannot be corrected by a unified global scaling factor.

[0115] To solve the above problem, the near-eye display device can project the normal vector of the preset pupil circle of each preset image to the preset plane to obtain a normal vector projection line corresponding to the preset pupil circle. The normal vector projection line can cover the projection corresponding to the center of the preset pupil circle, which is not limited herein.

[0116] For at least two second pupil circles corresponding to the preset pupil region on the same preset image, the normal vector projection line of each second pupil circle is parallel in the preset plane, as shown in Figure 7 The preset pupil circle of the preset image can be represented as (p i ,n i ,r i ), where p i is used to indicate the center position of the preset pupil circle of the ith preset image, n i is used to indicate the normal vector of the preset pupil circle of the ith preset image, r i is used to indicate the pupil radius of the preset pupil circle of the ith preset image, and i is used to indicate the number of preset images. In the case of projecting the normal vector of the preset pupil circle to the preset plane to obtain the normal vector projection line corresponding to the preset pupil circle, the normal vector projection line can be represented as:

[0117]

[0118] where L i is used to indicate the normal vector projection line corresponding to the ith preset pupil circle. Used to indicate the starting point of the normal vector projection line corresponding to the i-th preset pupil circle; The direction vector corresponding to the i-th preset pupil circle is used to indicate the extension direction of the projection line of the normal vector corresponding to the i-th preset pupil circle; s is a parameter used to indicate the position of the control point on the projection line of the normal vector corresponding to the i-th preset pupil circle. Used to indicate the set of real numbers.

[0119] By performing the above processing on all preset images, the normal vector projection lines corresponding to each preset image can be obtained. The near-eye display device can calculate the point closest to each normal vector projection line to obtain the eye center position of the geometric eye model on the preset plane. The eye center position is a 2D position, and it corresponds to the eyeball center position of the geometric eye model. The near-eye display device can then back-project the eye center position to obtain the eyeball center position of the geometric eye model.

[0120] With the normal vectors of multiple preset pupil circles determined, the near-eye display device can determine the position of the eyeball center of the geometric eyeball model based on the normal vectors of the multiple preset pupil circles. This helps to improve the convenience of determining the position of the eyeball center of the geometric eyeball model, and thus improves the convenience of geometric modeling of the geometric eyeball model.

[0121] In some implementations, a first line is determined between the preset pupil center of the preset pupil circle and the camera light rays of the shooting module; a second line is determined between the eyeball center position of the geometric eyeball model and the pupil center position corresponding to the preset image, based on the normal vector of the preset pupil circle; the preset pupil center position corresponding to the preset image is determined based on the intersection of the first and second lines; and the eyeball radius of the geometric eyeball model is determined based on the distances between the preset pupil center positions corresponding to each of the multiple preset images and the eyeball center position.

[0122] like Figure 5 As shown, given a predetermined pupil circle, if the first line connecting the predetermined pupil center of the predetermined pupil circle and the optical center of the camera in the imaging module can cover the pupil center position corresponding to the predetermined image, then the near-eye display device can determine the first line connecting the predetermined pupil center of the predetermined pupil circle and the optical center of the camera in the imaging module. The predetermined pupil center of the predetermined pupil circle can be determined based on the pupil center of the second pupil circle. Since the normal vector of the predetermined pupil circle can pass through the eyeball center position of the geometric eyeball model, and the normal vector of the predetermined pupil circle can point from the eyeball center position to the pupil center position corresponding to the predetermined image, the near-eye display device can determine the second line connecting the eyeball center position and the pupil center position corresponding to the predetermined image based on the normal vector of the predetermined pupil circle. Accordingly, the near-eye display device can determine the pupil center position corresponding to the predetermined image based on the intersection of the first and second lines.

[0123] Given multiple preset images, each preset image may have a different pupil center position. The near-eye display device can calculate the distance between the pupil center position and the eyeball center position of each preset image based on the pupil center position and the eyeball center position of each preset image. Then, the near-eye display device can determine the eyeball radius of the geometric eyeball model based on the distances of each preset image.

[0124] like Figure 6 As shown, the pupil center positions of the preset pupil circles in multiple preset images include, for example, positions P1, P2, and P3. The near-eye display device can calculate the distances between each of positions P1, P2, and P3 and the position E of the eyeball center, and determine the eyeball radius of the geometric eyeball model based on the distances between each of positions P1, P2, and P3 and the position E of the eyeball center.

[0125] In one exemplary embodiment, the near-eye display device can be calibrated before leaving the factory, such as calibrating the reference plane corresponding to the shooting module, etc., without limitation. During the process of determining the user's corresponding geometric eye model, the near-eye display device can emit light to the user's eye through a preset light source and capture a preset image of the user's eye through its shooting module. The preset image includes a preset light spot formed by the reflection of the light emitted by the preset light source from the user's eye. The near-eye display device can determine a first position of the preset light spot on the preset image and a second position of the preset light spot on the user's eye. Based on the second position of the preset light spot on the user's eye, the normal vector of the plane where the preset pupil center of the preset image is located is calculated, i.e., the normal vector of the preset pupil circle of the preset image is determined. The near-eye display device can use the normal vector of the preset pupil circle of the preset image to determine the position of the pupil center P corresponding to the preset image. A near-eye display device can capture multiple preset images, which can be taken while the user moves their eyes. The device can then determine the position of the pupil center P corresponding to each of the preset images. Using these positions, it can fit the eyeball center position and radius of a geometric eyeball model, thus completing the construction of the geometric eyeball model. The multiple preset images can include at least three preset images.

[0126] In a case where the preset pupil circles of the plurality of preset images and the eyeball center position of the geometric eyeball model are determined, the near-eye display device can determine the pupil center positions corresponding to the preset images by using the preset pupil circles and the eyeball center position, and determine the eyeball radius of the geometric eyeball model by combining the pupil center positions corresponding to the plurality of preset images, thereby facilitating the geometric modeling of the geometric eyeball model.

[0127] In a case where the geometric eyeball model corresponding to the user is determined, the geometric eyeball model can be used to determine the geometric size features of the user's eye when the eye movement of the user is tracked, thereby facilitating the accuracy of the eye movement tracking of the near-eye display device.

[0128] In some embodiments, the corneal center position corresponding to the preset image is determined according to the second position of the preset light spot on the user's eye, and the distance between the corneal center and the eyeball center in the geometric eyeball model is determined according to the corneal center positions corresponding to the plurality of preset images and the eyeball center position of the geometric eyeball model.

[0129] For example, in a case where the second position of the preset light spot on the user's eye is determined, the near-eye display device can determine the corneal center position corresponding to the preset image by using the second position of the preset light spot on the user's eye and the corneal reflection geometric relationship corresponding to the preset light spot, the preset light source and the shooting module in the near-eye display device. Accordingly, the near-eye display device can determine the distance between the corneal center and the eyeball center in the geometric eyeball model by using the corneal center positions corresponding to the plurality of preset images and the eyeball center position of the geometric eyeball model. For example, the near-eye display device can determine the distance between the corneal center and the eyeball center in the geometric eyeball model according to the average value of the differences between the corneal center positions corresponding to the plurality of preset images and the eyeball center position of the geometric eyeball model. Of course, the disclosure is not limited thereto.

[0130] In the process of constructing the geometric eyeball model, the near-eye display device can also fit the corneal curvature radius of the corneal sphere in the geometric eyeball model by using the corneal center positions corresponding to the plurality of preset images and the eyeball center position of the geometric eyeball model, such as iteratively optimizing the corneal curvature radius of the corneal sphere in the geometric eyeball model, thereby obtaining the corneal curvature radius of the corneal sphere in the geometric eyeball model. The disclosure is not limited thereto.

[0131] In a case where the near-eye display device determines the distance between the corneal center and the eyeball center in the geometric eyeball model by using the preset light spots included in the plurality of preset images and the eyeball center position of the geometric eyeball model, the distance between the corneal center and the eyeball center can be used by the near-eye display device to improve the adaptability of the geometric eyeball model to the user's eye, thereby facilitating subsequent eye movement tracking to improve the accuracy of the eye movement tracking of the near-eye display device.

[0132] The display method of the near-eye display device provided in the above embodiment comprises: obtaining a geometric eyeball model corresponding to a user wearing the near-eye display device; the geometric eyeball model is used to indicate geometric size features of the user's eye; obtaining a target image obtained by a shooting module shooting the user's eye; the target image comprises a target light spot formed by the user's eye reflecting light emitted by a preset light source, and a target pupil region corresponding to a pupil in the user's eye on the target image; and obtaining eye movement information of the user according to the geometric size features, the target light spot and the target pupil region.

[0133] In the case of obtaining the geometric eyeball model corresponding to the user, since the geometric eyeball model can be used to indicate the geometric size features of the user's eye, the near-eye display device can track the eye movement of the user in combination with the geometric size features indicated by the geometric eyeball model, the target light spot and the target pupil region included in the target image, to obtain the eye movement information of the user. In the case of tracking the eye movement of the user in combination with the geometric size features of the user's eye, if different users have different geometric size features, the near-eye display device can determine the eye movement information of different users by using the geometric eyeball models corresponding to the different users respectively, thereby facilitating improvement of the eye movement tracking accuracy of the near-eye display device.

[0134] Please refer to Figure 8 , Figure 8 is a schematic block diagram of an eye movement tracking device of a near-eye display device provided in an embodiment of the present application. The eye movement tracking device of the near-eye display device can be configured in the near-eye display device or a server, and is used to execute the eye movement tracking method of the near-eye display device described above. The near-eye display device can comprise AR glasses, VR glasses, MR glasses, AR headgear, VR headgear, MR headgear, etc., without limitation. The server can be a separate server, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content distribution network, and big data and artificial intelligence platform.

[0135] As shown in Figure 8 , the eye movement tracking device of the near-eye display device comprises a model obtaining module 110, a target image obtaining module 120 and an eye movement tracking module 130.

[0136] The model obtaining module is used to obtain a geometric eyeball model corresponding to a user wearing the near-eye display device; the geometric eyeball model is used to indicate geometric size features of the user's eye;

[0137] an object image acquisition module, configured to acquire an object image obtained by a photographing module of the near-eye display device photographing the user's eye; the object image includes a target light spot formed by the user's eye reflecting light emitted by a preset light source of the near-eye display device, and a target pupil region corresponding to a pupil in the user's eye on the object image;

[0138] an eye movement tracking module, configured to obtain eye movement information of the user according to the geometric size feature, the target light spot, and the target pupil region.

[0139] For example, the geometric size feature includes a eyeball center position of the user's eye and an eyeball radius; the eye movement tracking module 130 includes a first position determination sub-module, a second position determination sub-module, a first pupil circle determination sub-module, a target pupil circle determination sub-module, and an eye movement information determination sub-module.

[0140] The first position determination sub-module is configured to determine a first position of the target light spot on the object image.

[0141] The second position determination sub-module is configured to determine a second position of the target light spot on the user's eye according to the first position of the target light spot.

[0142] The first pupil circle determination sub-module is configured to perform perspective projection on the target pupil region to obtain at least two first pupil circles corresponding to the target pupil region.

[0143] The target pupil circle determination sub-module is configured to determine a target pupil circle of the object image from the at least two first pupil circles according to the second position of the target light spot.

[0144] The eye movement information determination sub-module is configured to obtain the eye movement information of the user according to the eyeball center position, the eyeball radius, and the target pupil circle.

[0145] For example, the geometric size feature includes a eyeball center position of the user's eye and a distance between a corneal center and the eyeball center; the eye movement tracking device includes a first corneal center position determination sub-module and an eyeball center position updating sub-module.

[0146] The corneal center position determination sub-module is configured to determine a corneal center position corresponding to the object image according to the second position of the target light spot on the user's eye.

[0147] The eyeball center position updating sub-module is configured to update the eyeball center position according to the corneal center position corresponding to the object image and the distance between the corneal center and the eyeball center to obtain an updated eyeball center position.

[0148] The eye movement information determination sub-module comprises an eye movement information updating sub-module.

[0149] The eye movement information updating sub-module is configured to obtain the eye movement information of the user according to the updated eyeball center position, the eyeball radius, and the target pupil circle.

[0150] The model obtaining module 110 comprises a preset image obtaining sub-module and a modeling sub-module.

[0151] The preset image obtaining sub-module is configured to obtain a plurality of preset images obtained by the photographing module photographing the eye of the user; the preset images comprise preset light spots formed by the eye of the user reflecting light emitted by the preset light source, and preset pupil regions corresponding to the pupil in the eye of the user in the preset images; the positions of the preset pupil regions in the respective preset images are different.

[0152] The modeling sub-module is configured to perform geometric modeling processing on the eye of the user according to the respective preset light spots and the preset pupil regions in the plurality of preset images, to obtain the corresponding geometric eyeball model of the user.

[0153] The modeling sub-module comprises a third position determining sub-module, a fourth position determining sub-module, a second pupil circle determining sub-module, a preset pupil circle determining sub-module, and a geometric modeling sub-module.

[0154] The third position determining sub-module is configured to determine a first position of the preset light spot in the preset image.

[0155] The fourth position determining sub-module is configured to determine a second position of the preset light spot on the eye of the user according to the first position of the preset light spot.

[0156] The second pupil circle determining sub-module is configured to perform perspective projection on the preset pupil region in the same preset image, to obtain at least two second pupil circles corresponding to the preset pupil region.

[0157] The preset pupil circle determining sub-module is configured to determine a preset pupil circle of the preset image from the at least two second pupil circles according to the second position of the preset light spot.

[0158] The geometric modeling sub-module is configured to perform geometric modeling processing on the eye of the user according to the respective preset pupil circles of the plurality of preset images, to obtain the corresponding geometric eyeball model of the user.

[0159] The geometric modeling sub-module comprises an eyeball center position determining sub-module, an eyeball radius determining sub-module, and a modeling sub-module.

[0160] an eyeball center position determination submodule, configured to determine an eyeball center position of the geometric eyeball model according to the normal vector of each preset pupil circle of the plurality of preset images;

[0161] an eyeball radius determination submodule, configured to determine an eyeball radius of the geometric eyeball model according to the plurality of preset pupil circles and the eyeball center position of the geometric eyeball model;

[0162] a modeling submodule, configured to perform geometric modeling on the user's eye according to the eyeball center position and the eyeball radius, to obtain the geometric eyeball model.

[0163] Exemplarily, the eyeball center position determination submodule comprises a normal vector projection submodule, an eye center position determination submodule and a back projection submodule.

[0164] the normal vector projection submodule is configured to project the normal vector of the preset pupil circle of the preset image to a preset plane to obtain a normal vector projection line corresponding to the preset pupil circle;

[0165] the eye center position determination submodule is configured to determine, as the eye center position of the geometric eyeball model in the preset plane, a position closest to the plurality of normal vector projection lines;

[0166] the back projection submodule is configured to back project the eye center position to obtain the eyeball center position of the geometric eyeball model.

[0167] Exemplarily, the eyeball radius determination submodule comprises a first connecting line submodule, a second connecting line submodule, an intersection point determination submodule and a distance determination submodule.

[0168] the first connecting line submodule is configured to determine a first connecting line between a preset pupil center of the preset pupil circle and a camera optical center of the photographing module;

[0169] the second connecting line submodule is configured to determine a second connecting line between the eyeball center position of the geometric eyeball model and a pupil center position corresponding to the preset image according to the normal vector of the preset pupil circle;

[0170] the intersection point determination submodule is configured to determine the pupil center position corresponding to the preset image according to an intersection point of the first connecting line and the second connecting line;

[0171] the distance determination submodule is configured to determine the eyeball radius of the geometric eyeball model according to distances between the eyeball center position and respective pupil center positions corresponding to the plurality of preset images.

[0172] Exemplarily, the eye movement tracking model comprises a corneal sphere center position fitting sub-module and a sphere center distance fitting sub-module.

[0173] The corneal sphere center position fitting sub-module is configured to determine a corneal sphere center position corresponding to the preset image according to the second position of the preset light spot on the eye of the user.

[0174] The sphere center distance fitting sub-module is configured to determine a distance between a corneal sphere center and an eyeball sphere center in the geometric eyeball model according to the respective corneal sphere center positions corresponding to the plurality of preset images and the eyeball sphere center position of the geometric eyeball model.

[0175] It should be noted that, for the convenience and brevity of description, the specific working processes of the above-described device and each module and unit can be clearly understood by those skilled in the art, and the corresponding processes in the foregoing method embodiments can be referred to, which will not be described herein.

[0176] The method of the present application can be used in a variety of general-purpose or special-purpose computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0177] Exemplarily, the above-described method and device can be implemented in the form of a computer program, which can run on a near-eye display device or a server to control the near-eye display device. Exemplarily, the near-eye display device can include AR glasses, VR glasses, MR glasses, AR headgear, VR headgear, MR headgear, etc., without limitation. The server can be a separate server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks, and basic cloud computing services such as big data and artificial intelligence platforms.

[0178] Please refer to Figure 9 , Figure 9 is a structural schematic block diagram of a near-eye display device provided by an embodiment of the present application.

[0179] like Figure 9 As shown, the near-eye display device is equipped with a shooting module and a preset light source; the near-eye display device includes a memory and a processor. The memory and processor can be connected via a system bus, and the memory may include a storage medium and internal memory.

[0180] The storage medium can store the operating system and computer programs. When the computer program is executed, it enables the processor to perform eye-tracking methods for any near-eye display device.

[0181] The processor provides computing and control capabilities to support the operation of the entire near-eye display device.

[0182] Internal memory provides an environment for the execution of computer programs stored in storage media. When executed by a processor, the computer program enables the processor to perform eye-tracking methods for any near-eye display device.

[0183] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the near-eye display device to which the present application is applied. A specific near-eye display device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0184] It should be understood that a processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other convertible logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0185] In one embodiment, the processor is configured to execute a computer program and, when executing the computer program, perform the following steps:

[0186] Obtain a geometric eye model corresponding to the user wearing the near-eye display device; the geometric eye model is used to indicate the geometric size characteristics of the user's eye.

[0187] obtaining a target image of the user's eye obtained by the photographing module; the target image includes a target light spot formed by the user's eye reflecting light emitted by the preset light source, and a target pupil region corresponding to a pupil in the user's eye in the target image;

[0188] obtaining the eye movement information of the user according to the geometric size feature, the target light spot, and the target pupil region.

[0189] It should be noted that, for the convenience and brevity of description, the above description of the specific working process of the eye movement tracking of the near-eye display device can refer to the corresponding process in the foregoing embodiments of the eye movement tracking method of the near-eye display device, and will not be described here.

[0190] The embodiments of the application further provide a computer-readable storage medium, and the computer-readable storage medium stores a computer program. The method implemented by the computer program executed by a processor can refer to each embodiment of the eye movement tracking method of the near-eye display device.

[0191] The computer-readable storage medium can be an internal storage unit of the near-eye display device, for example, a hard disk or a memory of the near-eye display device. The computer-readable storage medium can also be an external storage device of the near-eye display device, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc.

[0192] It should be understood that the terms used herein in the specification and the appended claims are only for the purpose of describing particular embodiments and do not intend to limit the application. As used in the specification and the appended claims of the present application, unless otherwise specified, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0193] It should also be understood that, in the specification and the appended claims, the terms "and / or" is used to mean one or more of the associated listed items, as well as any combination of any of the associated listed items. It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application cover modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

[0194] The above-mentioned embodiment serial numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments. The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of eye movement tracking for a near-eye display device, the method comprising: The near-eye display device is provided with a shooting module and a preset light source; The eye movement tracking method comprises: Obtaining a geometric eyeball model corresponding to a user wearing the near-eye display device; the geometric eyeball model is used to indicate geometric size features of the user's eye; Obtaining a target image obtained by the shooting module shooting the user's eye; the target image comprises a target light spot formed by the user's eye reflecting light emitted by the preset light source, and a target pupil region corresponding to a pupil in the user's eye on the target image; Obtaining eye movement information of the user according to the geometric size features, the target light spot and the target pupil region.

2. The eye-tracking method of claim 1, wherein, The geometric size features comprise an eyeball center position of the user's eye and an eyeball radius; The method for obtaining eye movement information of the user according to the geometric size features, the target light spot and the target pupil region comprises: Determining a first position of the target light spot on the target image; Determining a second position of the target light spot on the user's eye according to the first position of the target light spot; Perspective projecting the target pupil region to obtain at least two first pupil circles corresponding to the target pupil region; Determining a target pupil circle of the target image from the at least two first pupil circles according to the second position of the target light spot; Obtaining eye movement information of the user according to the eyeball center position, the eyeball radius and the target pupil circle.

3. The eye-tracking method of claim 2, wherein, The geometric size features comprise an eyeball center position of the user's eye and a distance between a corneal center and the eyeball center; After the method for determining the second position of the target light spot on the user's eye according to the first position of the target light spot, the eye movement tracking method further comprises: Determining a corneal center position corresponding to the target image according to the second position of the target light spot on the user's eye; Updating the eyeball center position according to the corneal center position corresponding to the target image and the distance between the corneal center and the eyeball center to obtain an updated eyeball center position; The method for obtaining eye movement information of the user according to the eyeball center position, the eyeball radius and the target pupil circle comprises: Obtaining eye movement information of the user according to the updated eyeball center position, the eyeball radius and the target pupil circle.

4. The eye-tracking method according to any one of claims 1 to 3, characterized in that, The method for obtaining a geometric eyeball model corresponding to a user wearing the near-eye display device comprises: Obtaining a plurality of preset images obtained by the shooting module shooting the user's eye; the preset images comprise preset light spots formed by the user's eye reflecting light emitted by the preset light source, and preset pupil regions corresponding to a pupil in the user's eye on the preset images; positions of the preset pupil regions of the plurality of preset images on the corresponding preset images are different; Geometrically modeling the user's eye according to the preset light spots and the preset pupil regions of the plurality of preset images to obtain the geometric eyeball model corresponding to the user.

5. The eye-tracking method of claim 4, wherein, The geometric modeling processing of the user's eye according to the preset spot and the preset pupil region of each of the plurality of preset images comprises: determining a first position of the preset spot on the preset image; determining a second position of the preset spot on the user's eye according to the first position of the preset spot; perspective projection is performed on the preset pupil region included in the same preset image to obtain at least two second pupil circles corresponding to the preset pupil region; determining a preset pupil circle of the preset image from the at least two second pupil circles according to the second position of the preset spot; geometric modeling processing of the user's eye according to the preset pupil circle of each of the plurality of preset images to obtain a corresponding geometric eyeball model of the user.

6. The eye-tracking method of claim 5, wherein, The geometric modeling processing of the user's eye according to the preset pupil circle of each of the plurality of preset images to obtain a corresponding geometric eyeball model of the user comprises: eyeball center fitting processing of the geometric eyeball model of the user according to the normal vector of the preset pupil circle of each of the plurality of preset images to obtain an eyeball center position of the geometric eyeball model; determining an eyeball radius of the geometric eyeball model according to the plurality of preset pupil circles and the eyeball center position of the geometric eyeball model; geometric modeling processing of the user's eye according to the eyeball center position and the eyeball radius to obtain the geometric eyeball model.

7. The eye-tracking method of claim 6, wherein, The eyeball center fitting processing of the geometric eyeball model of the user according to the normal vector of the preset pupil circle of each of the plurality of preset images to obtain an eyeball center position of the geometric eyeball model comprises: projecting the normal vector of the preset pupil circle of the preset image to a preset plane to obtain a normal vector projection line corresponding to the preset pupil circle; determining a position closest to a plurality of normal vector projection lines as an eye center position of the geometric eyeball model in the preset plane; back projection of the eye center position to obtain the eyeball center position of the geometric eyeball model.

8. The eye-tracking method of claim 6, wherein, The determining of the eyeball radius of the geometric eyeball model according to the plurality of preset pupil circles and the eyeball center position of the geometric eyeball model comprises: determining a first connecting line between a preset pupil center of the preset pupil circle and a camera optical center of the shooting module; determining a second connecting line between the eyeball center position of the geometric eyeball model and a pupil center position corresponding to the preset image according to the normal vector of the preset pupil circle; determining the pupil center position corresponding to the preset image according to the intersection of the first connecting line and the second connecting line; determining the eyeball radius of the geometric eyeball model according to the distance between the pupil center position corresponding to each of the plurality of preset images and the eyeball center position, respectively.

9. The eye tracking method of claim 6, wherein, After obtaining the eyeball center position of the geometric eyeball model, the eye movement tracking method further comprises: determining a corneal center position corresponding to the preset image according to the second position of the preset spot on the user's eye; According to the corneal sphere center positions corresponding to the plurality of preset images respectively and the eyeball sphere center position of the geometric eyeball model, a distance between the corneal sphere center and the eyeball sphere center in the geometric eyeball model is determined.

10. An eye movement tracking apparatus of a near-eye display device, characterized by, The eye movement tracking device comprises: a model obtaining module, configured to obtain a geometric eyeball model corresponding to a user wearing the near-eye display device; the geometric eyeball model is used to indicate geometric size features of the user's eye; a target image obtaining module, configured to obtain a target image obtained by a photographing module of the near-eye display device photographing the user's eye; the target image comprises a target light spot formed by the user's eye reflecting light emitted by a preset light source of the near-eye display device, and a target pupil region corresponding to a pupil in the user's eye on the target image; an eye movement tracking module, configured to obtain eye movement information of the user according to the geometric size features, the target light spot and the target pupil region.

11. A near-eye display device, comprising: The near-eye display device is provided with a photographing module and a preset light source; the near-eye display device comprises a memory and a processor; the memory, configured to store a computer program; the processor, configured to execute the computer program and implement steps of the eye movement tracking method of the near-eye display device according to any one of claims 1 to 9 when the computer program is executed.

12. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement steps of the eye movement tracking method of the near-eye display device according to any one of claims 1 to 9.

Images