Optical path generation device based on eyeball, determination method of visual axis equation set and eye movement tracking method

Abstract

The technical scheme of the invention discloses an eyeball-based light path generation device, a determination method of a visual axis equation set and an eye movement tracking method. The eyeball-based light path generation device comprises an image acquisition assembly; a first adjusting assembly to an Nth adjusting assembly; first to Nth reflectors; the camera is used as a watching target; wherein when a user watches the lens of the camera serving as the watching target, the image acquisition assembly is used for acquiring a user eye image in a mirror surface; the first adjusting assembly to the Nth adjusting assembly are respectively used for adjusting the spatial positions of the first reflective mirror to the Nth reflective mirror; the camera serving as a watching target is used for collecting images of eyes of a user; the user eye image in the mirror surface and the user eye image are used for determining a light path model; the light path model is used for determining a visual axis equation set; the visual axis equation set is used for realizing an eye movement tracking method.

Description

Technical Field

[0001] This application relates to the technical field of eye movement tracking, and more specifically, to an optical path generation device based on the eyeball, a method for determining a visual axis equation set, and a method for eye movement tracking. Background Art

[0002] In the prior art, for eye movement tracking, the first method of eye movement tracking is the pupil corneal reflection method, and this method includes:

[0003] (1) Equipment preparation process: Roughly, target points at different spatial positions, a point light source, and camera A are preset in advance. The target points can be used to make the user's eyes fixate on them; the point light source can be used to emit detection light, which can irradiate the user's eyes. Camera A can be used to collect the light diffusely reflected by the pupil of the user's eyes and incident on this camera A, thereby forming an image of the pupil in the obtained eye image. Camera A can also be used to collect the light diffusely reflected by the iris of the user's eyes and incident on this camera, thereby forming an image of the iris in the obtained eye image. Camera A can also be used to collect the detection light diffusely reflected by the corneal specular reflection of the user's eyes and incident on camera A, thereby forming a light spot in the obtained eye image. (2) Data acquisition process: Roughly, taking camera A as the coordinate system for measuring the orientation of the eyeball, when the point light source emits the detection light to the outside world, the user's eyeball is in different orientations. When the user's eyeball is in different orientations, the eyeball fixates on different target points. When the eyeball fixates on different target points, different eye images are collected. (3) Data analysis process: Roughly, by measuring the images of the pupil, the iris, and the light spot included in different eye images, different independent variables corresponding to different eye images are obtained, and by measuring the spatial positions of different target points corresponding to different eye images, different dependent variables corresponding to different eye images are obtained. Different independent variables and different dependent variables corresponding to different eye images form different data nodes corresponding to different eye images. According to different data nodes and through the interpolation method, an interpolation polynomial is obtained, and this interpolation polynomial passes through different data nodes. (4) Usage process: Roughly, when the user fixates on an object in reality, the user's eyeball is irradiated by the point light source, the eye image is collected by camera A, and by measuring the images of the pupil, the iris, and the light spot included in this eye image, the independent variable corresponding to this eye image is obtained, and this independent variable is substituted into the interpolation polynomial to obtain the dependent variable. Through this dependent variable, the landing point of the user's gaze on the object can be solved, thereby realizing the eye movement tracking function. In practical applications, the dependent variables included in the data nodes at most include 6. The 6 independent variables included in the data nodes can be used to measure the relative position between the user's eyeball and camera A (at least 6 independent variables are required to determine a reference to fully describe the spatial position of any object). During the usage process of this method, when extracting these 6 independent variables from the eye image and substituting them into the interpolation polynomial, the landing point of the user's gaze on the object can be solved.

[0004] However, if there are many dependent variables in the data node, a large number of data nodes need to be obtained in the data analysis process of this method, resulting in the need to collect many different eye images during the data collection process of this method. For example, if the data node contains 6 independent variables, usually at least 3 to the power of 6, that is, 729 different eye images need to be collected, which will be a relatively large workload. Moreover, if there are many independent variables in the data node, the obtained interpolation polynomial will be very complex, which will lead to a large amount of computation in the process of substituting the independent variables into the interpolation polynomial to solve the landing point of the user's gaze on the object each time during the use of this method, imposing a heavy computational burden on the computer. If there are few independent variables in the data node, the landing point of the user's gaze on the object can be accurately solved in a few cases, but in most cases, the solved landing point is not accurate enough. For example, when using a common eye tracker, accurate eye movement tracking results can be obtained when the user's head is directly facing the screen, at an appropriate distance from the screen, and without tilting the head. However, when the user's head moves slightly, the obtained results will not be accurate enough, which is caused by few independent variables in the data point.

[0005] In the prior art, for eye movement tracking, the second eye movement tracking method is specifically as follows: (1) Preset camera B; (2) In the three-dimensional space simulated by the computer, build a three-dimensional model of camera B based on camera B, and build a three-dimensional model of the user's eyeball in this simulated three-dimensional space, and the three-dimensional model of the user's eyeball includes a visual axis model represented by a straight line; (3) In actual use, capture the user's eyeball through camera B to obtain eye image ', and calculate the relative positions of the three-dimensional model of camera B and the three-dimensional model of the user's eyeball in this simulated three-dimensional space based on this eye image ', and then determine the relative position between camera B and the visual axis model, and then determine the relative position between camera B and the visual axis of the user's eyeball in reality, and then take the intersection point of the visual axis of the user's eyeball and the object as the landing point of the user's gaze on the object.

[0006] However, the shape of the user's eyeball is very complex, and it is difficult to accurately build a three-dimensional model of the user's eyeball with existing technologies. This will lead to, in actual use, that as long as there is a very small error value between the three-dimensional model of the user's eyeball and the real user's eyeball, usually the calculated fixation point will have an error value 30 times that of the real fixation point. Moreover, there will inevitably be curved surfaces in the three-dimensional model of the user's eyeball, and there will be a large amount of computation in the process of calculating the relative positions of the three-dimensional model of camera B and the three-dimensional model of the user's eyeball in this simulated three-dimensional space based on the eye image ', imposing a heavy computational burden on the computer. Summary of the Invention

[0007] In view of this, the present application provides a method for establishing a gaze direction measurement device, a gaze direction model, and an eye model, and the solution is as follows:

[0008] An eyeball-based optical path generating device, the eyeball-based optical path generating device comprising:

[0009] An image acquisition component; the image acquisition component at least includes a first measurement unit and a second measurement unit; the first measurement unit includes a first camera and a first light-emitting element; the second measurement unit includes a second camera and a second light-emitting element;

[0010] N adjustment components, the N adjustment components are sequentially the first adjustment component to the Nth adjustment component, and the first reflecting mirror to the Nth reflecting mirror sequentially installed on the first adjustment component to the Nth adjustment component; N is an integer greater than or equal to 1;

[0011] A reflecting mirror for reflecting a gaze target or a camera serving as a gaze target; the eyeball-based optical path generating device at least includes any one of the reflecting mirror for reflecting the gaze target and the camera serving as the gaze target; the reflecting mirror for reflecting the gaze target is used to specularly reflect the light diffusely reflected by the user's eyeball into the lens of the first camera so that the first camera can receive the light diffusely reflected by the user's eyeball and incident into the lens of the first camera, and the reflecting mirror for reflecting the gaze target is also used to specularly reflect the light diffusely reflected by the first camera at various angles into the pupil of the user's eyeball, so that the user can see and gaze at the image of the lens of the first camera reflected in the mirror surface of the reflecting mirror for reflecting the gaze target; the camera serving as the gaze target is used to acquire an image of the user's eye when the user's eyeball gazes at the lens of the camera serving as the gaze target, and the image of the user's eye at least includes an image of the pupil and an image of the iris;

[0012] A connecting member; the connecting member is used to install the image acquisition component, the first adjustment component to the Nth adjustment component, the reflecting mirror for reflecting the gaze target or the camera serving as the gaze target;

[0013] Wherein, when the user's eyeball is within a specific spatial region and is gazing at the image of the first camera lens reflected in the mirror surface of the mirror that reflects the gaze target or gazing at the lens of the camera serving as the gaze target, the image acquisition component is configured to collect the light reflected on the mirror surfaces of the mirrors including the first mirror to the Nth mirror and the mirror that reflects the gaze target, thereby forming an image of the user's eye in the mirror surface; the first adjustment component to the Nth adjustment component are respectively configured to adjust the spatial positions of the first mirror to the Nth mirror; the camera serving as the gaze target is configured to collect an image of the user's eye; the image of the user's eye in the mirror surface and the image of the user's eye are used to determine an optical path model; the optical path model is used to determine an optical axis equation set; the optical axis equation set is used to implement a method for eye movement tracking; if the eyeball-based optical path generating device does not include the camera serving as the gaze target, the optical path model is determined only through the image of the user's eye in the mirror surface.

[0014] Preferably, in the above-mentioned eyeball-based optical path generating device, the first adjustment component to the Nth adjustment component respectively include at least a first adjustment module to an Nth adjustment module, and the first adjustment component to the Nth adjustment component may further include a displacement adjustment member; when the first adjustment component to the Nth adjustment component include the displacement adjustment member, the first adjustment component to the Nth adjustment component are mounted on the displacement adjustment member, and the displacement adjustment member is mounted on the connection member; when the first adjustment component to the Nth adjustment component do not include the displacement adjustment member, the first adjustment component to the Nth adjustment component are mounted on the connection member;

[0015] Wherein, the first mirror to the Nth mirror are respectively mounted on the first adjustment module to the Nth adjustment module, and the first adjustment module to the Nth adjustment module are respectively configured to adjust the spatial positions of the first mirror to the Nth mirror.

[0016] Preferably, in the above-mentioned eyeball-based optical path generating device, any one of the adjustment modules including the first adjustment module to the Nth adjustment module includes: G movement guiding components, which are sequentially the 1st movement guiding component to the Gth movement guiding component, and G is an integer greater than or equal to 1;

[0017] The 1st movement guiding component is mounted on the displacement adjustment member or on the connection member, and the (v + 1)th movement guiding component is movably mounted on the vth movement guiding component, where v is a positive integer not greater than G;

[0018] The first reflector to the Nth reflector are respectively installed on the Gth motion guiding component included in the first adjustment module to the Gth motion guiding component included in the Nth adjustment module;

[0019] Wherein, the (v + 1)th motion guiding component can slide or rotate relative to the vth motion guiding component under the action of a driving force.

[0020] Preferably, in the above-mentioned eyeball-based optical path generating device, the first adjustment component to the Nth adjustment component further include: a position sensor, which is used to measure the spatial positions of the first reflector to the Nth reflector to obtain the spatial position information of the first reflector to the Nth reflector; the spatial position information of the first reflector to the Nth reflector is used to determine the optical path model.

[0021] Preferably, in the above-mentioned eyeball-based optical path generating device, the image acquisition component includes:

[0022] A first measurement unit, the first measurement unit includes: a first camera and a first light-emitting element; the first camera is used to acquire a first image; the first light-emitting element at least includes a first light-emitting part, and the first light-emitting part is used to emit first detection light in a scattered manner;

[0023] A second measurement unit, the second measurement unit includes: a second camera and a second light-emitting element; the second camera is used to acquire a second image; the second light-emitting element at least includes a second light-emitting part, and the second light-emitting part is used to emit second detection light in a scattered manner;

[0024] Wherein, the user eye image in the mirror surface includes the first image and the second image; at different time points, the image acquisition component will acquire different user eye images in the mirror surface, and different user eye images in the mirror surface include different first images and second images.

[0025] Preferably, in the above-mentioned eyeball-based optical path generating device, in the first camera and the first light-emitting element included in the first measurement unit, the geometric center of the first light-emitting part included in the first light-emitting element is located within the light convergence area of the first camera; in the second camera and the second light-emitting element included in the second measurement unit, the geometric center of the second light-emitting part included in the second light-emitting element is located within the light convergence area of the second camera;

[0026] Among them, the light-gathering area of any one of the cameras, including the first camera and the second camera, is the spatial area where the straight lines where all the light rays that can enter the camera lens and be received by the photosensitive element included in the camera can intersect. The light-gathering area of any one of the cameras is a spatial area inside the camera.

[0027] Preferably, in the above-mentioned eye-based optical path generating device, the first light-emitting element is used to emit a first detection light ray; the second light-emitting element is used to emit a second detection light ray; the first mirror to the Nth mirror and the mirror for reflecting the fixation target are used to specularly reflect the first detection light ray and the second detection light ray to the cornea of the user's eye, and specularly reflect the first detection light ray and the second detection light ray specularly reflected by the cornea of the user's eye into the lenses of the first camera and the second camera again; the first mirror to the Nth mirror and the mirror for reflecting the fixation target are also used to specularly reflect the light rays diffusely reflected by the pupil and iris of the user's eye into the lens of the first camera; the first camera forms a first image based on all the first detection light rays specularly reflected into the lens of the first camera, all the second detection light rays specularly reflected into the lens of the first camera, and the light rays diffusely reflected by the pupil and iris of the user's eye specularly reflected into the lens of the first camera; the second camera forms a second image based on all the first detection light rays specularly reflected into the lens of the second camera and all the second detection light rays specularly reflected into the lens of the second camera;

[0028] Among them, the first image includes: the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target. Any one of the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target included in the first image includes the image of the pupil, the image of the iris, and the light spot; the second image includes: the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target. All or part of the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target included in the second image include the image of the pupil, the image of the iris, and the light spot.

[0029] Preferably, in the above-mentioned eyeball-based optical path generation device, a computer is further included, and the computer at least includes a controller and an information processor; the controller is used to control the first light-emitting element and the second light-emitting element to emit the first detection light and the second detection light, and control the first camera and the second camera to collect light to obtain the first image and the second image, and the information processor is used to perform data analysis on the first image, the second image, and the spatial position information of the first reflector to the Nth reflector, and determine the optical path model based on the data analysis result; wherein, if the camera as the fixation target is further included, the controller is further used to control the camera as the fixation target to collect light to obtain an image of the user's eyes, and during the process of determining the optical path model, the information processor also needs to perform data analysis on the image of the user's eyes to determine the optical path model.

[0030] The present application also provides a method for determining an optical axis equation set, based on the above-mentioned eyeball-based optical path generation device, the method includes:

[0031] Preset L distance parameter values, M included angle parameter values, and N rotation angle parameter values in advance; the L distance parameter values are successively the first distance parameter value to the Lth distance parameter value, L is an integer greater than 1, the M included angle parameter values are successively the first included angle parameter value to the Mth included angle parameter value, M is an integer greater than 1, and the N rotation angle parameter values are successively the first rotation angle parameter value to the Nth rotation angle parameter value, N is an integer greater than or equal to 1;

[0032] According to the first distance parameter value to the Lth distance parameter value, the first included angle parameter value to the Mth included angle parameter value, and the first rotation angle parameter value to the Nth rotation angle parameter value, determine the first group of preset position arrays to the Lth group of preset position arrays; any group of preset position arrays including the first group of preset position arrays to the Nth group of preset position arrays includes M preset position arrays;

[0033] Determine the optical path model corresponding to each preset position array at different time periods;

[0034] Perform data analysis on all the optical path models, and determine the optical axis equation set according to the data analysis result.

[0035] Preferably, in the above-mentioned method for determining the optical axis equation set, the method for determining the first group of preset position arrays to the Nth group of preset position arrays according to the first distance parameter value to the Lth distance parameter value, the first included angle parameter value to the Mth included angle parameter value, and the first rotation angle parameter value to the Nth rotation angle parameter value includes:

[0036] Determine starting data through the eye-based optical path generation device; the starting data at least includes the first image and the second image, and if the eye-based optical path generation device includes the camera serving as the fixation target, the starting data further includes an image of the user's eyes.

[0037] Perform data analysis on the starting data, and based on the data analysis result, determine a reference model; the reference model at least includes an eye-based optical path generation device model as a reference, a cornea model as a reference, an optic axis model as a reference, a pupil center model as a reference, and an iris feature model as a reference.

[0038] Determine the first set of preset position arrays to the Nth set of preset position arrays according to the reference model.

[0039] Preferably, in the above method for determining the optic axis equation set, the method for determining starting data through the eye-based optical path generation device includes:

[0040] Adjust the spatial positions of the first reflector to the Nth reflector from the first adjustment module to the Nth adjustment module to the starting position; when the spatial positions of the first reflector to the Nth reflector are at the starting position, the light spot reflected by the user's eye cornea will appear on the image of the mirror surface of the reflector included in the first image and the second image collected by the first camera and the second camera, and when the spatial positions of the first reflector to the Nth reflector are at the starting position, the light spots included in the first image and the second image collected by the first camera and the second camera can be used to calculate the shape of the user's eye cornea.

[0041] When the spatial positions of the first reflector to the Nth reflector are at the starting position, and the user's eyes are within the specific spatial region and fixate on the image of the lens of the first camera reflected in the mirror surface of the reflector serving as the fixation target or fixate on the camera serving as the fixation target, use the first image, the second image, and the image of the user's eyes collected by the first camera, the second camera, and the camera serving as the fixation target respectively as the starting data.

[0042] Wherein, if the eye-based optical path generation device includes the reflector serving as the fixation target and does not include the camera serving as the fixation target, the starting data only includes the first image and the second image collected by the second camera; if the eye-based optical path generation device includes the camera serving as the fixation target and does not include the reflector serving as the fixation target, the starting data includes the first image, the second image, and the image of the user's eyes.

[0043] Preferably, in the above method for determining the visual axis equation set, the method for analyzing data based on the starting data and determining the reference model based on the data analysis results includes:

[0044] Determining a reference eyeball-based optical path generation device model based on the spatial positions of the first to Nth reflectors at the starting position; the reference eyeball-based optical path generation device model at least includes a first camera model, and the first camera model at least includes a light convergence area model of the first camera;

[0045] Determining a reference corneal model in the virtual space based on the light spots included in the first image and the second image included in the starting data, and the reference eyeball-based optical path generation device model; the reference corneal model at least includes a curved surface for restoring the shape of the user's eyeball cornea;

[0046] Determining a reference visual axis model in the virtual space based on the image of the pupil included on the mirror surface of the reflector that reflects the fixation target in the first image included in the starting data, or based on the image of the pupil included in the image of the user's eye included in the starting data, through the reference eyeball-based optical path generation device model; if the eyeball-based optical path generation device includes the reflector that reflects the fixation target, the reference visual axis model is determined based on the first image included in the starting data; if the eyeball-based optical path generation device includes the camera as the fixation target, the reference visual axis model is determined based on the image of the user's eye included in the starting data;

[0047] Determining the reference pupil center model based on the reference visual axis model and the reference corneal model;

[0048] Determining a reference iris feature model based on the iris information included on the mirror surface of the reflector that reflects the fixation target in the first image included in the starting data, or based on the iris information included in the image of the user's eye included in the starting data, through the reference eyeball-based optical path generation device model; if the eyeball-based optical path generation device includes the reflector that reflects the fixation target, the reference iris feature model is determined based on the first image included in the starting data; if the eyeball-based optical path generation device includes the camera as the fixation target, the reference iris feature model is determined based on the image of the user's eye included in the starting data.

[0049] Preferably, in the above method for determining the visual axis equations, the method for determining the first set of preset position arrays to the Nth set of preset position arrays according to the reference model includes:

[0050] Establish a dynamic mirror model according to the reference model;

[0051] Determine all possible combinations formed by taking one distance parameter value, one angle parameter value, and one rotation angle parameter value from the L distance parameter values, M angle parameter values, and N rotation angle parameter values respectively; each combination includes one distance parameter value, one angle parameter value, and one rotation angle parameter value;

[0052] Determine the preset position corresponding to each combination through the dynamic mirror model; each preset position corresponds to one distance parameter value, one angle parameter value, and one rotation angle parameter value;

[0053] Form the preset position arrays by combining the preset positions corresponding to the same distance parameter value and the same angle parameter value, and form the first set of preset position arrays to the Lth set of preset position arrays respectively by combining all the preset position arrays corresponding to the first distance parameter value to all the preset position arrays corresponding to the Lth distance parameter value;

[0054] Wherein, any one of the preset position arrays corresponds to one distance parameter value and one angle parameter value, and any one of the preset position arrays includes N preset positions, which are successively the first preset position to the Nth preset position, and any one of the preset position arrays including the first set of preset position arrays to the Lth set of preset position arrays includes M preset position arrays.

[0055] Preferably, in the above method for determining the visual axis equation system, it is characterized in that the method for establishing a dynamic mirror model based on the reference model includes: assuming that the light-gathering area model of the first camera included in the reference model is point A, the point where the pupil center model is located is point B, and the point where the iris feature model serving as the reference is located is point C. Make a point O on the surface included in the reference corneal model, and draw a normal line to the surface where the reference corneal model is located through the point O. Then make a point P on the normal line, and draw a plane through the point P. This plane is the dynamic mirror model. Let the plane where the dynamic mirror model is located be perpendicular to the angular bisector of angle APO. Make a point Q on the plane where the dynamic mirror model is located, and let the angular bisector of angle AQB be perpendicular to the plane where the dynamic mirror model is located. Make a point R on the plane where the dynamic mirror model is located, and let the angular bisector of angle ARC be perpendicular to the plane where the dynamic mirror model is located.

[0056] Preferably, in the above method for determining the visual axis equation system, the method for determining the optical path model corresponding to each of the preset position arrays at different time periods includes:

[0057] Adjust the spatial positions of the first reflector to the Nth reflector respectively through the first adjustment component to the Nth adjustment component, so that the spatial positions of the first reflector to the Nth reflector are respectively at the real-space positions corresponding to the first preset position to the Nth preset position included in each of the preset position arrays during each of the time periods. Through the first camera, the second camera, and the camera serving as the fixation target, collect the first image, the second image, and the image of the user's eye corresponding to each of the preset position arrays during each of the time periods; the first image, the second image, and the image of the user's eye corresponding to each of the preset position arrays are used to determine the optical path model corresponding to each of the preset position arrays. If the camera serving as the fixation target is not included, there is no image of the user's eye, and only determine the optical path model corresponding to each of the preset position arrays through the first image and the second image corresponding to each of the preset position arrays;

[0058] Determine the optical path model corresponding to each of the preset position arrays according to the first image, the second image, and the image of the user's eye corresponding to each of the preset position arrays;

[0059] Among them, each of the optical path models at least includes: a visual axis model, a first to an Nth mirror model, a first to an Nth pupil-based first-segment optical path model, a first to an Nth pupil-based last-segment optical path model, a first to an Nth iris-based first-segment optical path model, a first to an Nth iris-based last-segment optical path model, a first to an Nth cornea-based first-segment optical path model, and a first to an Nth cornea-based last-segment optical path model; if the image of the user's eye is not included, the optical path models corresponding to the respective preset position arrays are determined only by the first image and the second image corresponding to the respective preset position arrays.

[0060] Preferably, in the above method for determining the visual axis equations, the method for determining the optical path models corresponding to the respective preset position arrays according to the first image, the second image, and the image of the user's eye corresponding to the respective preset position arrays includes:

[0061] Based on the spatial positions of the first to the Nth mirrors within each of the time periods, the eyeball-based optical path generation device models corresponding to the respective preset position arrays are determined respectively;

[0062] Based on the light spots in the first image corresponding to the respective preset position arrays and the light spots in the second image corresponding to the respective preset position arrays, and through the device models for collecting eyeball data, the cornea models corresponding to the respective preset position arrays are determined;

[0063] Based on the images of the pupils on the mirror surfaces of the mirrors reflecting the fixation target in the first image corresponding to the respective preset position arrays, and through the device models for collecting eyeball data, the visual axis models corresponding to the respective preset position arrays are determined respectively, or based on the images of the pupils on the images of the user's eyes corresponding to the respective preset position arrays and through the device models for collecting eyeball data, the visual axis models corresponding to the respective preset position arrays are determined respectively; if there is no image of the user's eye because there is no camera as the fixation target, the visual axis model is determined according to the first image, and if there is an image of the user's eye because there is a camera as the fixation target, the visual axis model is determined according to the image of the user's eye;

[0064] An array composed of images of pupils on the first image corresponding to each of the preset position arrays, and determining, through each of the device models for collecting eye data, the first to Nth first optical path models based on pupils and the first to Nth last optical path models based on pupils corresponding to each of the preset position arrays; the array composed of images of pupils is an array composed of images of pupils respectively included on the images of the mirrors of the first to Nth reflectors in the first image;

[0065] An array composed of images of irises on the first image corresponding to each of the preset position arrays, and determining, through each of the device models for collecting eye data, the first to Nth first optical path models based on irises and the first to Nth last optical path models based on irises corresponding to each of the preset position arrays; the array composed of images of irises is an array composed of images of irises respectively included on the images of the mirrors of the first to Nth reflectors in the first image;

[0066] Respectively, an array composed of first light spots on the first image corresponding to each of the preset position arrays, and respectively determining, through each of the device models for collecting eye data, the first to Nth first optical path models based on the cornea and the first to Nth last optical path models based on the cornea corresponding to each of the preset position arrays; the array composed of the first light spots is an array composed of first light spots respectively included on the images of the mirrors of the first to Nth reflectors in the first image, the first light spot is a light spot formed in the first image by the light rays in the first detection light rays emitted by the first light-emitting element that are respectively reflected by the first to Nth reflectors along the normal of the user's eye cornea towards the user's eye cornea and then reflected specularly by the user's eye cornea along the original path and finally enter the lens of the first camera and are received, and there is only one first light spot on the image of the mirror of any one of the reflectors including the images of the mirrors of the first to Nth reflectors in each of the first images;

[0067] Among them, each of the optical path models includes: the optical path generation device model based on the eyeball corresponding to each of the preset position arrays, the corneal model, the visual axis model, the first to the Nth pupil-based first-segment optical path models, the first to the Nth pupil-based last-segment optical path models, the first to the Nth iris-based first-segment optical path models, the first to the Nth iris-based last-segment optical path models, the first to the Nth cornea-based first-segment optical path models, and the first to the Nth cornea-based last-segment optical path models.

[0068] Preferably, in the above method for determining the visual axis equations, the method for determining the visual axis equations according to the data analysis results by performing data analysis on all the optical path models includes:

[0069] By performing data analysis on each of the optical path models, the first to the Nth distance parameter measurement values, the first to the Nth angle parameter measurement values, and the first to the Nth rotation angle parameter measurement values corresponding to each of the optical path models are determined;

[0070] According to the first to the Nth distance parameter measurement values, the first to the Nth angle parameter measurement values, the first to the Nth rotation angle parameter measurement values, and the first to the Nth visual axis parameter A measurement values corresponding to each of the optical path models, the visual axis equation A is determined; the visual axis equation A is used to determine the output value of the visual axis parameter A according to the actual measurement values of the input distance parameter, the angle parameter, and the rotation angle parameter;

[0071] According to the first to the Nth distance parameter measurement values, the first to the Nth angle parameter measurement values, the first to the Nth rotation angle parameter measurement values, and the first to the Nth visual axis parameter B measurement values corresponding to each of the optical path models, the visual axis equation B is determined; the visual axis equation B is used to determine the output value of the visual axis parameter B according to the actual measurement values of the input distance parameter, the angle parameter, and the rotation angle parameter;

[0072] Determine the visual axis equation C based on the measured values of the first to Nth distance parameters, the measured values of the first to Nth angle parameters, the measured values of the first to Nth rotation angle parameters, and the measured values of the first to Nth visual axis parameter C corresponding to each of the optical path models; the visual axis equation C is used to determine the output value of the visual axis parameter C according to the actual measured values of the input distance parameter, the angle parameter, and the rotation angle parameter.

[0073] Determine the visual axis equation D based on the measured values of the first to Nth distance parameters, the measured values of the first to Nth angle parameters, the measured values of the first to Nth rotation angle parameters, and the measured values of the first to Nth visual axis parameter D corresponding to each of the optical path models; the visual axis equation D is used to determine the output value of the visual axis parameter D according to the actual measured values of the input distance parameter, the angle parameter, and the rotation angle parameter.

[0074] Determine the visual axis equation set according to the visual axis equation A, the visual axis equation B, the visual axis equation C, and the visual axis equation D; wherein, the visual axis equation set includes the visual axis equation A, the visual axis equation B, the visual axis equation C, and the visual axis equation D.

[0075] Preferably, in the above method for determining the visual axis equation set, the method for determining the measured values of the first to Nth distance parameters, the measured values of the first to Nth angle parameters, and the measured values of the first to Nth rotation angle parameters corresponding to each of the optical path models by analyzing the data of each of the optical path models includes:

[0076] Determine the measured value of the jth distance parameter corresponding to each of the optical path models by analyzing the jth corneal-based first-segment optical path model and the jth corneal-based last-segment optical path model included in each of the optical path models, or determine the measured value of the jth distance parameter corresponding to each of the optical path models by analyzing the jth pupil-based first-segment optical path model and the jth pupil-based last-segment optical path model included in each of the optical path models, or determine the measured value of the jth distance parameter corresponding to each of the optical path models by analyzing the jth iris-based first-segment optical path model and the jth iris-based last-segment optical path model included in each of the optical path models;

[0077] Determine the measured value of the j-th included angle parameter corresponding to each of the optical path models by analyzing the relative positions between the j-th pupil-based first-segment optical path model and the j-th cornea-based first-segment optical path model included in each of the optical path models, or determine the j-th included angle parameter corresponding to each of the optical path models by analyzing the relative positions between the j-th pupil-based last-segment optical path model and the j-th cornea-based last-segment optical path model included in each of the optical path models; Determine the measured value of the j-th rotation angle parameter corresponding to each of the optical path models by analyzing the relative positions between the j-th pupil-based first-segment optical path model, the j-th cornea-based first-segment optical path model, and the j-th iris-based first-segment optical path model included in each of the optical path models;

[0078] Determine the j-th three-dimensional coordinate system included in each of the optical path models according to the j-th pupil-based first-segment optical path model, the j-th cornea-based first-segment optical path model, and the cornea model included in each of the optical path models, and determine the measured value of the j-th visual axis parameter A, the measured value of the j-th visual axis parameter B, the measured value of the j-th visual axis parameter C, and the measured value of the j-th visual axis parameter D corresponding to each of the optical path models by analyzing the relative positions between the j-th three-dimensional coordinate system and the visual axis model included in each of the optical path models;

[0079] Wherein, j takes any positive integer less than or equal to the total number N of the adjustment components. When j takes 1 up to N respectively, there are the measured value of the first distance parameter, the measured value of the first included angle parameter, the measured value of the first rotation angle parameter, the measured value of the first visual axis parameter A, the measured value of the first visual axis parameter B, the measured value of the first visual axis parameter C, and the measured value of the first visual axis parameter D corresponding to each of the optical path models, up to the measured value of the N-th distance parameter, the measured value of the N-th included angle parameter, the measured value of the N-th rotation angle parameter, the measured value of the N-th visual axis parameter A, the measured value of the N-th visual axis parameter B, the measured value of the N-th visual axis parameter C, and the measured value of the first visual axis parameter D.

[0080] The present application also provides an eye movement tracking method. Based on the visual axis equation set, the method includes:

[0081] Pre-set an eye movement tracking device, which at least includes a third measurement unit, a computer, and a connecting member; the eye movement tracking device may further include a fourth measurement unit or a device capable of measuring the corneal spatial position of the user's eyeball, and the eye movement tracking device can be used to measure the position of the user's eyeball to determine the spatial position information of the user's eyeball; the third measurement unit includes a third camera and a third light-emitting element, and the structure of the third measurement unit is the same as that of the first measurement unit. The third camera included in the third measurement unit is equivalent to the first camera included in the first measurement unit, and the third light-emitting element included in the third measurement unit is equivalent to the first light-emitting element included in the first measurement unit. The third light-emitting element is used to emit a third detection light to the outside; the fourth measurement unit includes a fourth camera and a fourth light-emitting element, and the structure of the fourth measurement unit is the same as that of the first measurement unit. The fourth camera included in the fourth measurement unit is equivalent to the first camera included in the first measurement unit, and the fourth light-emitting element included in the fourth measurement unit is equivalent to the first light-emitting element included in the first measurement unit. The fourth light-emitting element is used to emit a fourth detection light to the outside;

[0082] When the user's eyeball gazes at an object in reality, the third light-emitting element included in the third measurement unit emits the third detection light to the outside, and the third camera included in the third measurement unit collects the light reflected by the user's eyeball and incident into the lens of the third camera to obtain a third image;

[0083] Determine a pupil-based optical path model, an iris-based optical path model, and a cornea-based optical path model in a three-dimensional space simulated by a computer according to the third image;

[0084] In the three-dimensional space simulated by a computer, determine a three-dimensional coordinate system for eye movement tracking according to the pupil-based optical path model, the iris-based optical path model, and the cornea-based optical path model;

[0085] Determine the actual measured value of the distance parameter by analyzing the cornea-based optical path model, or determine the actual measured value of the distance parameter by analyzing the pupil-based optical path model, or determine the actual measured value of the distance parameter by analyzing the cornea-based optical path model; determine the actual measured value of the included angle parameter by analyzing the relative positions between the pupil-based optical path model and the cornea-based optical path model; determine the actual measured value of the rotation angle parameter by analyzing the relative positions between the pupil-based optical path model, the cornea-based optical path model, and the iris-based optical path model;

[0086] Based on the actual measured value of the distance parameter, the actual measured value of the included angle parameter, and the actual measured value of the rotation angle parameter, and according to the visual axis equation set, determine the output value of the visual axis parameter A, the output value of the visual axis parameter B, the output value of the visual axis parameter C, and the output value of the visual axis parameter D;

[0087] Based on the output value of the visual axis parameter A, the output value of the visual axis parameter B, the output value of the visual axis parameter C, the output value of the visual axis parameter D, and the three-dimensional coordinate system for eye movement tracking, determine the visual axis model for eye movement tracking.

[0088] From the above description, it can be seen that the technical solution of this application includes:

[0089] (1) Equipment preparation process: Set the optical path generation device based on the eyeball, including: an image acquisition component; a first adjustment component to an Nth adjustment component, and a first reflecting mirror to an Nth reflecting mirror sequentially installed on the first adjustment component to the Nth adjustment component; a camera serving as a fixation target;

[0090] (2) Data acquisition process: When the user's eyeball is in a specific spatial area and fixates on the lens of the camera serving as the fixation target, the image acquisition component is used to acquire the user's eye image in the mirror; the first adjustment component to the Nth adjustment component are respectively used to adjust the spatial positions of the first reflecting mirror to the Nth reflecting mirror; the camera serving as the fixation target is used to acquire the image of the user's eyes;

[0091] (3) Data analysis process: The user's eye image in the mirror and the image of the user's eyes are used to determine the optical path model; the optical path model is used to determine data nodes, the data nodes are used to determine interpolation polynomials, and the interpolation polynomials are called visual axis equations, and the visual axis equations are used to form a visual axis equation set;

[0092] (4) Usage process: The visual axis equation set is used to implement a method for gaze tracking;

[0093] In the technical solution of the present application, the user eye image in the mirror surface at least includes a first image, and the first image includes: the image of the mirror surface of the first reflector to the image of the mirror surface of the Nth reflector. On the image of the mirror surface of any one of the reflectors included in the first image, from the image of the mirror surface of the first reflector to the image of the mirror surface of the Nth reflector, there are images of the pupil, the iris, and light spots; that is to say, in the technical solution of the present application, the information included in the image of the mirror surface of any one of the reflectors is equivalent to the information included in an eye image in the pupil corneal reflection method in the background art. And any one of the first images includes images of N reflectors' mirror surfaces. That is to say, the amount of information included in the first image in the technical solution of the present application is N times the amount of information included in the eye image in the pupil corneal reflection method in the background art. And in the data acquisition process of the technical solution of the present application, the first camera only needs to collect light once to obtain one first image. While in the data acquisition process of the background art, camera A needs to collect light N times to obtain N eye images. This makes the technical solution provided by the present application greatly improve the data acquisition efficiency in the data acquisition process compared with the background art. The time saved by the improved efficiency can be used to collect more information, so as to obtain more information, and obtaining more information can achieve a more accurate eye movement tracking result.

[0094] In addition, in the data analysis process of the technical solution of the present application, according to the jth pupil-based first-segment optical path model, the jth cornea-based first-segment optical path model, and the cornea model included in each optical path model, the jth three-dimensional coordinate system included in each optical path model is determined, where j takes any positive integer less than or equal to N. Because there is some spatial position information in the jth three-dimensional coordinate system, and the spatial position information existing in the jth three-dimensional coordinate system replaces 3 independent variables, so each data node in the technical solution of the present application only needs to include 3 independent variables to achieve the precision effect that each data node in the pupil corneal reflection method in the prior art needs to include 6 independent variables to achieve. Each data node in the technical solution of the present application only including 3 independent variables can greatly reduce the number of data nodes that need to be collected in the data acquisition process of the technical solution of the present application, and further greatly reduce the number of times of collecting the first image compared with the number of times of collecting the eye image in the prior art. And the interpolation polynomial obtained is much simpler than the interpolation polynomial obtained by using data nodes including 6 independent variables. This will result in much less computational effort required in the process of substituting the independent variables into the interpolation polynomial to solve the landing point of the user's gaze on the object each time in the technical solution of the present application compared with the prior art, thus reducing the computational burden on the computer. Description of the Drawings

[0095] To more clearly illustrate the technical solutions in the embodiments of the present application or in the related art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0096] The structures, proportions, sizes, etc. shown in the drawings of this specification are only used to cooperate with the content disclosed in the specification for those skilled in this technology to understand and read, and are not used to limit the limiting conditions under which the present application can be implemented. Therefore, they do not have a substantial technical meaning. Any modification of the structure, change in the proportional relationship, or adjustment of the size, without affecting the effects that the present application can produce and the purposes that can be achieved, should still fall within the scope that can be covered by the technical content disclosed in the present application.

[0097] Figure 1 Structural schematic diagram of an eyeball-based optical path generation device provided by an embodiment of the present application;

[0098] Figure 1A One provided by an embodiment of the present application Figure 1 Magnified view;

[0099] Figure 2 Structural schematic diagram of an eyeball-based optical path generation device provided by an embodiment of the present application;

[0100] Figure 3 Structural schematic diagram of the first adjustment component to the Nth adjustment component when the first adjustment component to the Nth adjustment component includes the displacement adjustment member provided by an embodiment of the present application;

[0101] Figure 4 Structural schematic diagram of the first adjustment component to the Nth adjustment component when the first adjustment component to the Nth adjustment component does not include the displacement adjustment member provided by an embodiment of the present application;

[0102] Figure 5 For Figure 3 Partial enlarged view of the first adjustment module in

[0103] Figure 6 Cross-sectional schematic diagram of the light collection area of a camera provided by an embodiment of the present application;

[0104] Figure 7 Schematic diagram of a first image provided by an embodiment of the present application;

[0105] Figure 8 Schematic diagram of a second image provided by an embodiment of the present application;

[0106] Fig. 9 Schematic diagram of a method for determining an optical axis equation set provided by an embodiment of the present application;

[0107] Fig.10 Schematic diagram of a method for determining the first group of preset position arrays to the Nth group of preset position arrays provided by an embodiment of the present application;

[0108] Fig.11 Schematic diagram of a method for determining starting data provided by an embodiment of the present application;

[0109] Fig.12 Schematic diagram of a method for determining a reference model provided by an embodiment of the present application;

[0110] Fig.13 Schematic diagram of an eyeball-based optical path generation device model provided by an embodiment of the present application;

[0111] Fig.14 Schematic diagram of the structure of the reference model corresponding to when the eyeball-based optical path generation device includes a reflector for reflecting a fixation target;

[0112] Fig.15 Schematic diagram of an image of a user's eye collected by a camera as a fixation target;

[0113] Fig.16 Schematic diagram of a method for determining the first group of preset position arrays to the Nth group of preset position arrays according to the reference model provided by an embodiment of the present application;

[0114] Fig.17 Schematic diagram of the structure of a dynamic mirror model provided by an embodiment of the present application;

[0115] Fig.18 Schematic diagram of the structure of a preset position array provided by an embodiment of the present application;

[0116] Fig.19 Schematic diagram of a method for determining an optical path model corresponding to each preset position array one by one provided by an embodiment of the present application;

[0117] Fig. 20 Schematic diagram of a first image corresponding to the 10th preset position array provided by an embodiment of the present application;

[0118] Fig.21 Schematic diagram of the structure of the 10th optical path model provided by an embodiment of the present application;

[0119] Fig. 22 provided by an embodiment of the present application Fig. 20Schematic diagram of a simplified diagram of a first image corresponding to a tenth preset position array shown;

[0120] Fig.23 Schematic diagram of the structure of a partial sub - model included in a tenth optical path model provided by an embodiment of the present application;

[0121] Fig.24 Schematic diagram of all the data nodes AⅠ corresponding to all the optical path models provided by an embodiment of the present application in a three - dimensional rectangular coordinate system where the horizontal axis, vertical axis, and height axis are set as the X - axis, Y - axis, and A - axis respectively;

[0122] Fig.25 Schematic diagram of a method of using the visual axis equation set provided by an embodiment of the present application;

[0123] Fig.26 Another schematic diagram of a method of using the visual axis equation set provided by an embodiment of the present application;

[0124] Fig. 27 Schematic diagram of the flow of an eye movement tracking method provided by an embodiment of the present application;

[0125] Fig.28 Schematic diagram of a third image provided by an embodiment of the present application;

[0126] Fig.29 Schematic diagram of a method for determining an optical path model based on the pupil, an optical path model based on the iris, and an optical path model based on the cornea in a computer - simulated three - dimensional space provided by an embodiment of the present application; Detailed implementation manners

[0127] Next, the embodiments in the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0128] To make the above - mentioned objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0129] As Figure 1 and Figure 1 the enlarged views of Figure 1A shown, Figure 1 and Figure 1A are schematic diagrams of a structure of an eyeball - based optical path generating device provided by an embodiment of the present application. The shown eyeball - based optical path generating device includes:

[0130] Image acquisition component 1; the image acquisition component at least includes a first measurement unit 11 and a second measurement unit 12; the first measurement unit 11 includes a first camera 111 and a first light-emitting element 112; the second measurement unit 12 includes a second camera 121 and a second light-emitting element 122.

[0131] N adjustment components, which are the first adjustment component 401 to the Nth adjustment component 410 in sequence, and the first reflector to the Nth reflector installed on the first adjustment component to the Nth adjustment component in sequence; N is an integer greater than or equal to 1; in Figure 1 In the shown embodiment, the number N of the adjustment components is set to be equal to 10. Figure 1 The shown eyeball-based optical path generating device includes the 10 adjustment components which are the first adjustment component 401 to the Nth adjustment component 410 distributed in a circular ring array in sequence, and the 10 reflectors installed on the first adjustment component to the Nth adjustment component in sequence are the first reflector 301 to the Nth reflector 310.

[0132] The reflector 311 for reflecting the fixation target; the reflector 311 for reflecting the fixation target is used to specularly reflect the light diffusely reflected by the user's eyeball 51 into the lens of the first camera 111, so that the first camera 111 can receive the light diffusely reflected by the user's eyeball 51 and incident into the lens of the first camera 111. The reflector 311 for reflecting the fixation target is also used to specularly reflect the light diffusely reflected by the first camera 111 at various angles into the pupil of the user's eyeball 51, so that the user can see and fixate on the image of the lens of the first camera 111 reflected in the mirror surface of the reflector 311 for reflecting the fixation target; the reflector 311 for reflecting the fixation target is one reflector, and this reflector is called the reflector for reflecting the fixation target.

[0133] Connecting member 2; the connecting member 2 is used to install the image acquisition component 1, the first adjustment component 401 to the Nth adjustment component 410, and the reflector 311 for reflecting the fixation target; the eyeball-based optical path generating device and including one connecting member may also include multiple connecting members. In Figure 1 In the shown embodiment, a total of 4 connecting members are set, which are the first connecting member 21, the second connecting member 22, and the third connecting member 23 in sequence. The second connecting member 22 and the third connecting member 23 are fixedly installed on the first connecting member 21; the first measurement unit 11 is installed on the second connecting member 22; the second measurement unit 12 is installed on the third connecting member 23; the first adjustment component 401 to the Nth adjustment component 410 are slidably installed on the first connecting member 21.

[0134] In Figure 1A the embodiment shown, the specific spatial region 518 is specifically a spatial region with reference to the eye-based optical path generating device. When the user's eyeball is within the specific spatial region 518, the first camera and the second camera can clearly capture the images of the user's eyeball reflected in the mirrors from the first mirror to the Nth mirror and the mirror reflecting the gaze target.

[0135] Among them, when the user's eyeball 51 is within the specific spatial region 518 and gazes at the image of the lens of the first camera 111 reflected in the mirror surface of the mirror 311 reflecting the gaze target, the image acquisition component 1 is used to collect the light reflected on the mirror surfaces of the mirrors including the first mirror 301 to the Nth mirror 310 and the mirror 311 reflecting the gaze target, and then form an image of the user's eyes in the mirror surface; the first adjustment component 401 to the Nth adjustment component 410 are respectively used to adjust the spatial positions of the first mirror 301 to the Nth mirror 310; the image of the user's eyes in the mirror surface is used to determine the optical path model; the optical path model is used to determine the visual axis equation set; the visual axis equation set is used to implement a method of eye movement tracking. In Figure 1 the embodiment shown, the eye-based optical path generating device includes the mirror reflecting the gaze target and does not include the camera as the gaze target. If the eye-based optical path generating device does not include the camera as the gaze target, the optical path model is determined only through the image of the user's eyes in the mirror surface.

[0136] In Figure 1In the shown embodiment, when the user's eyeball 51 gazes at the image of the lens of the first camera 111 reflected in the mirror surface of the mirror 311 that reflects the gaze target, the straight line where the visual axis 511 of the user's eyeball is located coincides with the straight line where the light ray 519 is located; wherein, the light ray 519 is: among all the light rays diffusely reflected by the pupil included in the user's eyeball 51, the light ray that can be reflected by the mirror surface of the mirror 311 that reflects the gaze target and then enter the lens of the first camera 111 (the light ray 519 is a section of the light ray after being diffusely reflected by the pupil and before being reflected by the mirror surface of the mirror 311 that reflects the gaze target); because: when the user's eyeball 51 gazes at the image of the lens of the first camera 111 reflected in the mirror surface of the mirror 311 that reflects the gaze target, the straight line where the visual axis 511 of the user's eyeball is located coincides with the straight line where the light ray 114 is located, and the light ray 114 is the light ray that can enter the pupil included in the user's eyeball 51 after being diffusely reflected by the lens of the first camera 111 and then reflected by the mirror surface of the mirror 311 that reflects the gaze target, and according to the reversibility of light, it is known that the straight line where the light ray 519 is located coincides with the straight line where the light ray 114 is located, so the straight line where the visual axis 511 of the user's eyeball is located coincides with the straight line where the light ray 519 is located.

[0137] As Figure 2 shown, Figure 2 This is a schematic structural diagram of an eyeball-based light path generating device provided by an embodiment of the present application. Figure 2 The structure of the shown light path generating device and Figure 1 the structure of the shown light path generating device are similar, the difference is that Figure 2 the shown embodiment includes the camera 191 as the gaze target and does not include Figure 1 the mirror that reflects the gaze target in the shown embodiment; the camera 191 as the gaze target is used to collect an image of the user's eye when the user's eyeball gazes at the lens of the camera shown as the gaze target, and the image of the user's eye at least includes the image of the pupil and the image of the iris.

[0138] Wherein, when the user's eyeball 51 is within a specific spatial region 518 and is gazing at the lens 191 of the camera serving as the gazing target, the image acquisition component 1 is configured to collect the light reflected on the mirror surfaces of the mirrors including the first mirror 301 to the Nth mirror 310, so as to form an image of the user's eye in the mirror; the first adjustment component 401 to the Nth adjustment component 410 are respectively configured to adjust the spatial positions of the first mirror 301 to the Nth mirror 310; the camera 191 serving as the gazing target is configured to collect an image of the user's eye; the image of the user's eye in the mirror and the image of the user's eye are used to determine an optical path model; the optical path model is used to determine an optical axis equation set; and the optical axis equation set is used to implement a method for eye movement tracking.

[0139] In Figure 2 In the illustrated embodiment, when the user's eyeball 51 gazes at the lens of the camera 191 serving as the gazing target, the straight line where the optical axis 511 of the user's eyeball is located coincides with the straight line where the light 519 that can enter the lens of the camera 191 serving as the gazing target among the light diffusely reflected from the pupil of the user's eyeball 51 at various angles is located.

[0140] In specific implementation, the eyeball-based optical path generating device includes a mirror that reflects the gazing target or a camera serving as the gazing target. That is to say, the mirror that reflects the gazing target and the camera serving as the gazing target can replace each other. For example Figure 1 In the illustrated embodiment, the eyeball-based optical path generating device includes the mirror 311 that reflects the gazing target and does not include the camera serving as the gazing target. In the illustrated embodiment of Embodiment 2, the eyeball-based optical path generating device includes the camera 191 serving as the gazing target and does not include the mirror that reflects the gazing target. When the eyeball-based optical path generating device does not include the camera serving as the gazing target, there is no such camera serving as the gazing target, and thus there is no image of the user's eye collected by the camera serving as the gazing target. When there is no image of the user's eye, the optical path model is determined only by the image of the user's eye in the mirror.

[0141] The first adjustment component to the Nth adjustment component at least respectively include a first adjustment module to an Nth adjustment module. The first adjustment component to the Nth adjustment component may further include a displacement adjustment member. When the first adjustment component to the Nth adjustment component includes the displacement adjustment member, the first adjustment component to the Nth adjustment component are mounted on the displacement adjustment member, and the displacement adjustment member is mounted on the connection member. When the first adjustment component to the Nth adjustment component does not include the displacement adjustment member, the first adjustment component to the Nth adjustment component are mounted on the connection member.

[0142] Among them, the first reflector to the Nth reflector are respectively installed on the first adjustment module to the Nth adjustment module, and the first adjustment module to the Nth adjustment module are respectively used to adjust the spatial positions of the first reflector to the Nth reflector. The spatial positions of the first reflector to the Nth reflector can be adjusted respectively through the first adjustment component to the Nth adjustment component, so as to obtain the light reflected by the mirrors of the first reflector to the Nth reflector at different spatial positions, and then obtain the user's eye images in different mirrors.

[0143] Such as Figure 3 shown, Figure 3 As shown in the figure, it is a schematic structural diagram of the first adjustment component to the Nth adjustment component when the first adjustment component to the Nth adjustment component includes the displacement adjustment member. The first adjustment component 401 to the Nth adjustment component 410 respectively include a first adjustment module 4011 to an Nth adjustment module 4101. The first adjustment component to the Nth adjustment component further includes a displacement adjustment member 40. The first adjustment component 401 to the Nth adjustment component 410 are installed on the displacement adjustment member 40, and the displacement adjustment member 40 is installed on the connection member 21. Among them, the first reflector to the 301st Nth reflector 310 are respectively installed on the first adjustment module 4011 to the Nth adjustment module 4101, and the first adjustment module 4011 to the Nth adjustment module 4101 are respectively used to adjust the spatial positions of the first reflector 301 to the Nth reflector 310.

[0144] In Figure 3 the shown embodiment, the displacement adjustment member 40 is installed on the connection member 21. Specifically, 4 first guiding grooves 21b are provided on the first connection member 21, and 4 first linear guide rails 40a are provided on the displacement adjustment member 40. The 4 first linear guide rails 40a are movably installed in the 4 first guiding grooves 21b (only three of the 4 first linear guide rails 40a and three of the 4 first guiding grooves 21b are shown in Figure 3 , and the remaining one first linear guide rail 40a and the first guiding groove 21b are not shown in Figure 3 due to being blocked).

[0145] Such as Figure 4 shown, Figure 4This is a schematic structural diagram of the first adjustment component to the Nth adjustment component provided by the embodiment of the present application when the displacement adjustment member is not included in the first adjustment component to the Nth adjustment component. The first adjustment component 401' to the Nth adjustment component 410' respectively include a first adjustment module 4011' to an Nth adjustment module 4101'; the first adjustment component 401' to the Nth adjustment component 410' are installed on the connecting member 21'; wherein, the first reflecting mirror to the Nth reflecting mirror are respectively installed on the first adjustment module 4011' to the Nth adjustment module 4101', and the first adjustment module 4011' to the Nth adjustment module 4101' are respectively used to adjust the spatial positions of the first reflecting mirror to the Nth reflecting mirror.

[0146] Any one of the adjustment modules including the first adjustment module 4011 to the Nth adjustment module 4101 includes: G movement guiding components, and the G movement guiding components are sequentially the first movement guiding component to the Gth movement guiding component, where G is an integer greater than or equal to 1; Figure 3 In the shown embodiment of the application, the first movement guiding component is installed on the displacement adjustment member 40. Figure 4 In the shown embodiment of the application, the first movement guiding component is installed on the connecting member 2. Figure 3 Or Figure 4 In any one of the shown embodiments of the application, the (v + 1)th movement guiding component is movably installed on the vth movement guiding component, where v is a positive integer not greater than G; the first reflecting mirror to the Nth reflecting mirror are respectively installed on the Gth movement guiding component included in the first adjustment module to the Gth movement guiding component included in the Nth adjustment module; wherein, the (v + 1)th movement guiding component can slide or rotate relative to the vth movement guiding component under the action of a driving force.

[0147] Any one of the first adjustment module to the Nth adjustment module can, through the G movement guiding components with different dimensions included therein, achieve the translation and rotation of the reflecting mirrors including the first reflecting mirror to the Nth reflecting mirror in different dimensions, so as to realize the adjustment of the spatial positions of the first reflecting mirror 301 to the Nth reflecting mirror 310, and thus the light reflected by the mirrors of the first reflecting mirror 301 to the Nth reflecting mirror 310 at different spatial positions can be collected by the image acquisition component to obtain the user's eye images in different mirrors.

[0148] Figure 3 The structures of the first adjustment module 4011 to the Nth adjustment module 4101 included in the shown embodiment are respectively the same as Figure 4The structures of the first adjustment module 4011' to the Nth adjustment module 4101' included in the illustrated embodiment are basically the same. Taking Figure 3 the first adjustment module 4011 to the Nth adjustment module 4101 included in the illustrated embodiment as an example, the structures of any one of the adjustment modules including the first adjustment module 4011 to the Nth adjustment module 4101 are the same. Any one of the adjustment modules includes 4 movement guiding components, namely the first movement guiding component 41, the second movement guiding component 42, the third movement guiding component 43, and the fourth movement guiding component 44 in sequence.

[0149] As Figure 5 shown, Figure 5 for Figure 3 the partial enlarged view of the first adjustment module in Figure 3 and Figure 5 shown, the first movement guiding component 41 included in the first adjustment module 401 is fixedly installed on the displacement adjustment member 40. Specifically, a slot hole 40c is formed on the displacement adjustment member 40, and the first movement guiding component 41 is installed in the slot hole 40c on the displacement adjustment member 40.

[0150] The second adjustment component 42 is movably installed on the first movement guiding component 41. Specifically, the second adjustment assembly 42 includes a second linear guide rail 42a, the first adjustment assembly 41 includes a second guide slot 421b, and the second linear guide rail 42a is slidably installed in the second guide slot 421b.

[0151] The third adjustment component 43 is rotatably installed on the second movement guiding component 42. Specifically, the third adjustment assembly 43 includes a first cylindrical axis center 43a, the second adjustment assembly 42 includes a first cylindrical shaft sleeve 42b, and the first cylindrical axis center 43a is rotatably installed in the first cylindrical shaft sleeve 42b.

[0152] The fourth adjustment component 44 is rotatably installed on the third movement guiding component 43. Specifically, the fourth adjustment assembly 44 includes a second cylindrical shaft sleeve 44b, the third adjustment assembly 43 includes a second cylindrical axis center 43a2, and the second cylindrical axis center 43a2 is rotatably installed in the second cylindrical shaft sleeve 44b.

[0153] The first adjustment module 401 includes a fourth adjustment component 44 for installing the first reflector. Specifically, there is a through hole 44c on the fourth adjustment component 44, and the first reflector can be installed on the fourth adjustment component 44 included in the first adjustment module through screws and through the through hole 44c.

[0154] In Figure 4In the illustrated embodiment, the structures of any one of the adjustment modules from the first adjustment module to the Nth adjustment module are the same. Any one of the adjustment modules includes four motion guiding components, namely, the first motion guiding component 41, the second motion guiding component 42', the third motion guiding component 43, and the fourth motion guiding component 44, in sequence. Figure 4 The illustrated embodiment and Figure 3 the illustrated embodiment are similar. The difference is that the first motion guiding component 41 included in any one of the adjustment modules from the first adjustment module 401 to the tenth adjustment module 410 is fixedly installed on the connecting member 21, rather than on the displacement adjustment member 40.

[0155] It should be noted that the number of motion guiding components and the mechanical structure in any one of the adjustment modules can be set based on requirements to adjust the spatial position of the mirror surface of the mirror installed on the any one of the adjustment modules. The power for driving the mutual sliding or rotation between the motion guiding components included in the adjustment module can be electromagnetic force, or an external force can be applied to the mirror to drive the mutual sliding or rotation between the motion guiding components included in the adjustment module. There are various driving forces for driving the mutual sliding or rotation between the motion guiding components included in the adjustment module. It should be understood that the driving forces for driving the mutual sliding or rotation between the motion guiding components included in the adjustment module are not limited in this article.

[0156] In order to be able to detect the spatial positions of the first mirror to the Nth mirror, the first adjustment component to the Nth adjustment component further include: a position sensor, which is used to measure the spatial positions of the first mirror to the Nth mirror to obtain the spatial position information of the first mirror to the Nth mirror; wherein, the spatial position information of the first mirror to the Nth mirror is used to determine the optical path model. The spatial positions of the first mirror to the Nth mirror are specifically the spatial positions of the first mirror to the Nth mirror relative to the first camera; the spatial position information of the first mirror to the Nth mirror is used to describe the spatial positions of the first mirror to the Nth mirror.

[0157] This application provides two methods for setting the position sensor.

[0158] The first method for setting the position sensor is as follows: Install a position sensor between the (h + 1)-th motion guiding component and the h-th motion guiding component included in all the adjustment modules, and use the position sensor to sense the relative position between the (h + 1)-th motion guiding component and the h-th motion guiding component. When the adjustment assembly further includes the displacement adjustment member, a position sensor also needs to be installed between the displacement adjustment member and the connecting member included in the adjustment assembly, and use the position sensor to sense the relative position between the displacement adjustment member and the connecting member. Among them, the position sensor is divided into two parts and is respectively installed on the (h + 1)-th motion guiding component and the h-th motion guiding component, or respectively installed on the displacement adjustment member and the connecting member. After the relative position between the (h + 1)-th motion guiding component and the h-th motion guiding component changes, or after the relative position between the displacement adjustment member and the connecting member changes, corresponding electrical signals will be generated inside the position sensor, and by receiving and calculating the electrical signals, the relative position between the (h + 1)-th motion guiding component and the h-th motion guiding component can be determined, or the relative position between the displacement adjustment member and the connecting member can be determined. By calculating the relative positions between all the motion guiding components included in any one adjustment assembly, and calculating the relative position between the displacement adjustment member and the connecting member, the relative position information between the first reflector and the N-th reflector and the first camera can be further calculated. Among them, motors capable of sensing the motion state, such as servo motors or stepping motors, should also be regarded as the position sensor. The servo motor or the stepping motor is divided into two parts that can move relative to each other and can be respectively installed between the (h + 1)-th motion guiding component and the h-th motion guiding component, and the servo motor or the stepping motor can be respectively installed between the displacement adjustment member and the connecting member.

[0159] The second method for setting the position sensor is as follows: Use the first camera as the position sensor, or use the second camera as the position sensor, or set a camera with a known relative position to the first camera and capable of photographing the first reflector to the N-th reflector, and call the set camera camera C and use camera C as the position sensor. The specific method includes:

[0160] (1) Make optical marks on the mirror surfaces of the first reflector to the N-th reflector.

[0161] (2) The optical marks on the mirror surfaces of the first mirror to the Nth mirror are collected through the first camera, the second camera, or the camera C to obtain an image, and the relative positions of the optical marks on the mirror surfaces of the first mirror to the Nth mirror and the first camera are determined by an optical positioning method based on the obtained image, so as to determine the relative positions of the mirror surfaces of the first mirror to the Nth mirror and the first camera.

[0162] In Figure 1 and Figure 1 enlarged view, Figure 1A In the application embodiment shown, the image acquisition component includes: a first measurement unit 11, and the first measurement unit 11 includes: a first camera 111 and a first light-emitting element 112; wherein, the first camera 111 is used to acquire a first image; the first light-emitting element 112 at least includes a first light-emitting part, and the first light-emitting part is used to emit first detection light in a scattered manner; when the first light-emitting part emits the first detection light, multiple first detection lights are emitted at different angles centered on the first light-emitting part, and different first detection lights exist at different angles.

[0163] A second measurement unit 12, and the second measurement unit 12 includes: a second camera 121 and a second light-emitting element 122; wherein, the second camera 121 is used to acquire a second image; the second light-emitting element 122 at least includes a second light-emitting part, and the second light-emitting part is used to emit second detection light in a scattered manner; when the second light-emitting part emits the second detection light, multiple second detection lights are emitted at different angles centered on the second light-emitting part, and different second detection lights exist at different angles.

[0164] Wherein, the user eye image in the mirror surface includes the first image and the second image; at different time periods, the image acquisition component will acquire user eye images in different mirror surfaces, and the user eye images in different mirror surfaces include different first images and second images.

[0165] The light-emitting elements including the first light-emitting element and the second light-emitting element include the light-emitting part and also include a conductive part, and the conductive part can transmit current to the light-emitting part so that the light-emitting part can obtain energy to emit the detection light to the outside.

[0166] In Figure 1 and Figure 1 enlarged view, Figure 1AIn the application embodiment shown, among the first camera 111 and the first light-emitting element 112 included in the first measurement unit 11, the geometric center of the first light-emitting portion included in the first light-emitting element 112 is located within the light-gathering area of the first camera 111; among the second camera 121 and the second light-emitting element 122 included in the second measurement unit 12, the geometric center of the second light-emitting portion included in the second light-emitting element 122 is located within the light-gathering area of the second camera 121.

[0167] 111Wherein, the light-gathering area of any one of the cameras including the first camera 111 and the second camera 121 is a spatial area where the straight lines where all the light rays that can enter the camera lens and can be received by the photosensitive element included in the camera can intersect each other. The light-gathering area of any one of the cameras is a spatial area inside the camera. Only when the geometric center of the first light-emitting portion included in the first light-emitting element 112 is located within the light-gathering area of the first camera 111, and the geometric center of the first light-emitting portion included in the second light-emitting element 122 is located within the light-gathering area of the second camera 121, can the first camera 111 obtain an accurate first image, and the second camera 121 can obtain an accurate second image.

[0168] As Figure 6 shown, Figure 6 is a schematic cross-sectional view of the light-gathering area of a camera provided by the application embodiment shown, including a camera 15, a lens 151 of the camera, a photosensitive element 152 of the camera, a point E in space, a point F in space, a point G in space, light rays 1521, light rays 1521, and light rays 1521; the light ray 1521 is a light ray that is diffusely reflected by the point E in space and can enter the lens of the camera 15 and can be received by the photosensitive element 152 included in the camera 15; the light ray 1522 is a light ray that is diffusely reflected by the point F in space and can enter the lens of the camera 15 and can be received by the photosensitive element 152 included in the camera 15; the light ray 1523 is a light ray that is diffusely reflected by the point G in space and can enter the lens of the camera 15 and can be received by the photosensitive element 152 included in the camera 15; in Figure 6 the application embodiment shown, the light ray 1521 is represented by a dotted line, the light ray 1522 is represented by a solid line, and the light ray 1523 is represented by a dashed line.

[0169] In Figure 6 the application embodiment shown, the area where the light rays 1521, the light rays 1522, and the light rays 1523 intersect each other is within the lens 151 of the camera. That is to say, in Figure 6 In the application embodiment shown, the light converging area is within the lens 151 of the camera.

[0170] In Figure 6 In the application embodiment shown, the camera 15 only includes one lens, and the light converging area included in the camera 15 is within the lens 151 of the camera. In actual applications, a camera will include multiple lenses. The light converging area of this camera can be accurately found according to the design drawing of this camera. It is certain that the light converging area included in this camera is within the spatial area formed by the path that all the light that can enter the camera lens and can be received by the photosensitive element included in this camera travels from the first lens included in this camera until it irradiates the photosensitive element included in this camera; wherein, the first lens included in this camera is the lens that all the light that can enter the camera lens and can be received by the photosensitive element included in this camera first passes through when passing through the lens group included in this camera.

[0171] In Figure 1 and Figure 1 the enlarged view of Figure 1A In the application embodiment shown, the first light emitting element 112 is used to emit first detection light; the second light emitting element 122 is used to emit second detection light; the first reflecting mirror 301 to the Nth reflecting mirror 310 and the reflecting mirror 311 for reflecting the fixation target are used to specularly reflect the first detection light and the second detection light to the cornea of the user's eyeball 51, and specularly reflect the first detection light and the second detection light specularly reflected by the cornea of the user's eyeball 51 into the lenses of the first camera 111 and the second camera 121 again; the first reflecting mirror 301 to the Nth reflecting mirror 310 and the reflecting mirror 311 for reflecting the fixation target are also used to specularly reflect the light diffusely reflected by the pupil and iris of the user's eyeball 51 into the lens of the first camera 111; the first camera 111 forms a first image based on all the first detection light specularly reflected into the lens of the first camera 111, all the second detection light specularly reflected into the lens of the first camera, and the light diffusely reflected by the pupil and iris of the user's eyeball 51 specularly reflected into the lens of the first camera 111; the second camera 121 forms a second image based on all the first detection light specularly reflected into the lens of the second camera 121 and all the second detection light specularly reflected into the lens of the second camera 121.

[0172] As Figure 7 shown, Figure 7A schematic diagram of a first image provided by an embodiment of the present application. The first image includes: the image of the mirror surface of the first mirror 301 to the image of the mirror surface of the Nth mirror 310 and the image of the mirror surface of the mirror 311 that reflects the fixation target. Any one of the images of the mirror surfaces of the mirrors, including the image of the mirror surface of the first mirror 301 to the image of the mirror surface of the Nth mirror 310 and the image of the mirror surface of the mirror 311 that reflects the fixation target, contains an image 512 of the user's pupil represented by a circle, an image 513 of the iris represented by a dotted line, and a plurality of light spots 514 represented by solid dots. In Figure 7 In the shown embodiment of the application, the image 512 of the user's pupil is represented by a circle, Figure 7 only 2 are marked out of a total of 11. The image 513 of the iris is represented by a dotted line Figure 7 only 2 are marked out of a total of 11. The light spot 514 is represented by a solid dot, Figure 7 only 3 are marked out of a total of 165. In Figure 7 In the shown embodiment, the image 512 of the pupil and the image 513 of the iris are formed by the first camera receiving the light diffusely reflected by the pupil and iris of the user's eyeball that is reflected by the mirror surface into the lens of the first camera. The light spot 514 is formed by the first camera receiving all the first detection light rays that are reflected by the mirror surface into the lens of the first camera and all the second detection light rays that are reflected by the mirror surface into the lens of the first camera.

[0173] As Figure 8 shown, Figure 8 A schematic diagram of a second image provided by an embodiment of the present application. The second image includes: the image of the mirror surface of the first mirror 301 to the image of the mirror surface of the Nth mirror 310 and the image of the mirror surface of the mirror 311 that reflects the fixation target; all or part of the images of the mirror surfaces of the mirrors, including the image of the mirror surface of the first mirror 301 to the image of the mirror surface of the Nth mirror 310 and the image of the mirror surface of the mirror 311 that reflects the fixation target, contained in the second image contain the image 512 of the pupil, the image 513 of the iris, and the light spot 514. In Figure 8 In the shown embodiment of the application, only the image of the mirror surface of the first mirror 301, the image of the mirror surface of the second mirror 302, the image of the mirror surface of the sixth mirror 306, and the image of the mirror surface of the Nth mirror 310 contain the image 512 of the pupil, the image 513 of the iris, and the light spot 514. In Figure 8 In the shown embodiment of the application, the image 512 of the user's pupil is represented by a circle, Figure 8 only 1 is marked out of a total of 4. The image 513 of the iris is represented by a dotted line Figure 8Only 1 is marked out of a total of 4, and the light spot 514 is represented by a solid dot. Figure 8 Only 3 are marked out of a total of 60. In Figure 8 In the illustrated embodiment, the light spot 514 is formed by the second camera receiving all of the first detection light rays that are specularly reflected into the lens of the second camera and all of the second detection light rays that are specularly reflected into the lens of the second camera.

[0174] Figure 7 and Figure 8 The image 512 of the pupil and the image 513 of the iris shown in are images of the pupil and iris of the same eye reflected on the mirror surfaces of different reflectors.

[0175] If the eye-based optical path generating device does not include the reflector that reflects the fixation target, then the schematic diagram of the first image, Figure 7 and the schematic diagram of the second image, Figure 8 will not include the reflector 311 that reflects the fixation target and the image presented on its mirror surface.

[0176] The eye-based optical path generating device described in the embodiments of the present application further includes a computer, which at least includes a controller and an information processor; the controller is used to control the first light emitting element and the second light emitting element to emit the first detection light rays and the second detection light rays, and control the first camera and the second camera to collect light rays to obtain the first image and the second image, and the information processor is used to perform data analysis on the first image, the second image, and the spatial position information of the first reflector to the Nth reflector, and determine the optical path model based on the data analysis result; by performing data analysis on the optical path model, the visual axis equation set can be obtained, and the axis equation set can be used to implement a method of eye movement tracking. The computer also has an information storage medium for storing the first image, the second image, the image of the user's eye, and the optical path model.

[0177] Wherein, if it further includes the camera serving as the fixation target, the controller is further used to control the camera serving as the fixation target to collect light rays to obtain the image of the user's eye, and during the process of determining the optical path model, the information processor also needs to perform data analysis on the image of the user's eye to determine the optical path model.

[0178] The optical path generation device based on the eyeball described in the embodiments of the present application can be used to determine the optical path model, and the visual axis equation set can be obtained by analyzing the data of the optical path model. The axis equation set can be used to implement an eye movement tracking method, and functions such as advertisement analysis, psychological research, and controlling terminal devices can be realized through this eye movement tracking method.

[0179] Based on the above-mentioned optical path generation device based on the eyeball, another embodiment of the present application further provides a method for determining the visual axis equation set. The method for determining the visual axis equation set is as Fig. 9 shown Fig. 9 It is a schematic diagram of a method for determining the visual axis equation set provided by the embodiment of the present application. The method includes:

[0180] Step S11: Preset L distance parameter values, M included angle parameter values, and N rotation angle parameter values; where, the L distance parameter values are the first distance parameter value to the Lth distance parameter value in sequence, L is an integer greater than 1, the M included angle parameter values are the first included angle parameter value to the Mth included angle parameter value in sequence, M is an integer greater than 1, and the N rotation angle parameter values are the first rotation angle parameter value to the Nth rotation angle parameter value in sequence, N is an integer greater than or equal to 1.

[0181] The total number N of the rotation angle values including the first rotation angle value to the Nth rotation angle value is equal to the total number N of the reflectors including the first reflector to the Nth reflector, and the first rotation angle value to the Nth rotation angle value respectively correspond to the first reflector to the Nth reflector. The preset L distance parameter values, M included angle parameter values, and N rotation angle parameter values are used to determine the preset position array, the preset position array is used to determine the visual axis equation set, and the visual axis equation set is used to implement the eye movement tracking method.

[0182] In the embodiment of the present application, it is set that the total number L of the distance parameter values is equal to 3. The 3 distance parameter values are the first distance parameter value to the third distance parameter value in sequence. And it is set that the first distance parameter value, the second distance parameter value, and the third distance parameter value are equal to 300, 500, and 700 in sequence, and the unit of the distance parameter value is set to millimeters; it is set that the total number M of the included angle parameter values is equal to 5. The 5 included angle parameter values are the first included angle parameter value to the fifth included angle parameter value in sequence. And it is set that the first included angle parameter value, the second included angle parameter, the third included angle parameter, the fourth included angle parameter, and the fifth included angle parameter are equal to 0.1, 0.2, 0.3, 0.4, and 0.5 in sequence, and the unit of the included angle parameter is set to degrees; it is set that the total number N of the rotation angle parameter values is equal to 10. The N rotation angle parameter values are the first rotation angle parameter value to the tenth rotation angle parameter value in sequence. And it is set that the first rotation angle parameter value, the second rotation angle parameter value, the third rotation angle parameter value... the tenth rotation angle parameter value are equal to (1 - 1) * 36 = 0, (2 - 1) * 36 = 36, (3 - 1) * 36 = 72... (10 - 1) * 36 = 324 in sequence, and the unit of the rotation angle parameter value is set to degrees.

[0183] Step S12: Determine the first set of preset position arrays to the L-th set of preset position arrays according to the first distance parameter value to the L-th distance parameter value, the first included angle parameter value to the M-th included angle parameter value, and the first rotation angle parameter value to the N-th rotation angle parameter value; wherein, any set of preset position arrays including the first set of preset position arrays to the N-th set of preset position arrays includes M preset position arrays. Each of the preset position arrays includes L preset positions, and the L preset positions included in each of the preset position arrays are the first preset position to the N-th preset position in sequence.

[0184] Step S13: Determine the optical path model corresponding to each of the preset position arrays at different time periods. The optical path model corresponding to each of the preset position arrays is used to determine the visual axis equations, and the visual axis equations are used to implement the eye movement tracking method.

[0185] Step S14: Perform data analysis on all the optical path models, and determine the visual axis equations according to the data analysis results. The visual axis equations are used to implement the eye movement tracking method.

[0186] In the method for determining the visual axis equation set, in the above step S12, the method for determining the first group of preset position arrays to the Nth group of preset position arrays according to the first distance parameter value to the Lth distance parameter value, the first included angle parameter value to the Mth included angle parameter value, and the first rotation angle parameter value to the Nth rotation angle parameter value is as Fig.10 shown, Fig.10 which is a schematic diagram of a method for determining the first group of preset position arrays to the Nth group of preset position arrays provided by an embodiment of the present application. The method includes:

[0187] Step S21: Determine starting data through the eyeball-based optical path generating device; wherein, the starting data at least includes the first image and the second image. If the eyeball-based optical path generating device includes the camera serving as the fixation target, the starting data further includes the image of the user's eyes.

[0188] Step S22: Perform data analysis on the starting data, and determine a reference model based on the data analysis result; wherein, the reference model at least includes an eyeball-based optical path generating device model as a reference, a cornea model as a reference, a visual axis model as a reference, a pupil center model as a reference, and an iris feature model as a reference.

[0189] Step S23: Determine the first group of preset position arrays to the Nth group of preset position arrays according to the reference model. Determining the first group of preset position arrays to the Nth group of preset position arrays is used to determine the optical path model, the optical path model is used to determine the visual axis equation set, and the visual axis equation set is used to implement the eye movement tracking method.

[0190] In the method for determining the visual axis equation set, in the above step S21, the method for determining starting data through the eyeball-based optical path generating device is as Fig.11 shown, Fig.11 which is a schematic diagram of a method for determining starting data provided by an embodiment of the present application. The method includes:

[0191] Step S31: Adjust the spatial positions of the first reflecting mirror to the Nth reflecting mirror from the first adjusting module to the Nth adjusting module to the starting position.

[0192] The present application provides a method for adjusting the spatial positions of the first reflecting mirror to the Nth reflecting mirror to the starting position. The method includes:

[0193] (1) Store the preset spatial position information of the first to the Nth reflecting mirrors in the information storage medium. The preset spatial position information of the first to the Nth reflecting mirrors is referred to as the initial spatial position information of the first to the Nth reflecting mirrors.

[0194] (2) Continuously adjust the spatial positions of the first to the Nth reflecting mirrors through the first to the Nth adjusting components, and use the position sensor to measure the spatial position information of the first to the Nth reflecting mirrors in real time. When the measured spatial position information of the first to the Nth reflecting mirrors is the same as the stored initial spatial position information of the first to the Nth reflecting mirrors, the spatial positions of the first to the Nth reflecting mirrors are at the initial positions.

[0195] Among them, when the spatial positions of the first to the Nth reflecting mirrors are at the initial positions, the spots of the corneal reflection of the user's eyeball will appear on the images of the mirror surfaces of the reflecting mirrors included in the first image and the second image collected by the first camera and the second camera. And when the spatial positions of the first to the Nth reflecting mirrors are at the initial positions, the spots included in the first image and the second image collected by the first camera and the second camera can be used to calculate the shape of the user's eye cornea. That is to say, when the spatial positions of the first to the Nth reflecting mirrors only need to satisfy that the spots included in the first image and the second image collected by the first camera and the second camera can be used to calculate the shape of the user's eye cornea, the spatial positions of the first to the Nth reflecting mirrors can be used as the initial positions.

[0196] Step S32: When the spatial positions of the first to the Nth reflecting mirrors are at the initial positions, and the user's eyeball is within the specific spatial region and gazes at the image of the lens of the first camera reflected in the mirror surface of the reflecting mirror as the reflection gaze target or gazes at the camera as the gaze target, use the first image, the second image, and the image of the user's eye collected by the first camera, the second camera, and the camera as the gaze target respectively as the initial data.

[0197] Among them, if the eye-based optical path generation device includes the reflector that reflects the fixation target and does not include the camera serving as the fixation target, the starting data only includes the first image and the second image collected by the second camera; if the eye-based optical path generation device includes the camera serving as the fixation target and does not include the reflector that reflects the fixation target, the starting data includes the first image, the second image, and the image of the user's eyes. The starting data is used to determine the reference model, the reference model is used to determine the preset position array, the preset position array is used to determine the optical path model, the optical path model is used to determine the visual axis equation set, and the visual axis equation set is used to implement the eye movement tracking solution.

[0198] In the method for determining the visual axis equation set, in step S22 above, data analysis is performed on the starting data, and based on the data analysis result, the method for determining the reference model is as Fig.12 shown. Fig.12 FIG. is a schematic diagram of a method for determining a reference model provided by an embodiment of the present application. The method includes:

[0199] Step S41: Determine a model of the eye-based optical path generation device serving as a reference according to the spatial positions of the first reflector to the Nth reflector at the starting position; among them, the model of the eye-based optical path generation device serving as a reference at least includes a first camera model, and the first camera model at least includes a light converging area model of the first camera.

[0200] The model of the eye-based optical path generation device is a model of the eye-based optical path generation device created in a computer-simulated virtual space and is used to simulate the eye-based optical path generation device in reality. The method for determining the model of the eye-based optical path generation device is: create a first camera model, a second camera model, a first reflector model to an Nth reflector model in a computer-simulated virtual space respectively to simulate the first camera, the second camera, the first reflector to the Nth reflector in reality; if the eye-based optical path generation device further includes the reflector that reflects the fixation target, a reflector model that reflects the fixation target needs to be created in this virtual space to simulate the reflector that reflects the fixation target in reality; if the eye-based optical path generation device further includes the camera serving as the fixation target, a camera model serving as the fixation target needs to be created in this virtual space to simulate the reflector that reflects the fixation target in reality.

[0201] As Fig.13 shown. Fig.13Schematic diagram of an eyeball-based optical path generation device model provided by an embodiment of the present application. The eyeball-based optical path generation device model shown includes: a second camera model 121M, a camera model 191M serving as a fixation target, a first mirror model 301M to an Nth mirror model 310M, a mirror model 311M that reflects the fixation target, and a first camera model 111M. The first camera model 111M includes a light collection area model A of the first camera and an arrow 111MR.

[0202] In Fig.13 In the illustrated embodiment, the first camera model 111M is represented by a regular square pyramid with a rectangular base, the light collection area model A of the first camera is represented by a point, the vertex of the regular square pyramid representing the first camera model 111M is marked with a "+", and this vertex is used to represent the light collection area model A of the first camera. The arrow 111MR is parallel to the long side of the rectangular base included in the regular square pyramid. The second camera model 12M, the camera model 191M serving as a fixation target, and the first camera model 111M are all the same, being a regular square pyramid.

[0203] Any one of the mirror models, including the first mirror model 301M to the Nth mirror model 3101M and the mirror model 311M that reflects the fixation target, includes at least one bounded surface. The shapes of the bounded surfaces included in the first mirror model 301M to the Nth mirror model 310M are respectively exactly the same as the shapes of the surfaces where the mirrors of the first mirror to the Nth mirror are located in reality, and the shape of the bounded surface included in the mirror simulation that reflects the fixation target is exactly the same as the shape of the mirror surface of the mirror that reflects the fixation target in reality.

[0204] It should be noted that in the virtual space, a three-dimensional rectangular coordinate system Ⅰ is placed with the first camera model 111M included in the eyeball-based optical path generation device model as the reference. In reality, a three-dimensional rectangular coordinate system Ⅱ is placed with the first camera 111 as the reference. The positional relationship between the bounded surfaces where the first mirror model 301M to the Nth mirror model 310M are located and the three-dimensional rectangular coordinate system Ⅰ included in the eyeball-based optical path generation device model used as the reference is equivalent to the positional relationship between the surfaces where the mirrors of the first mirror 301 to the Nth mirror 310 are located and the three-dimensional rectangular coordinate system Ⅱ when collecting the starting data in reality.

[0205] This article provides an implementation method for placing a three-dimensional rectangular coordinate system Ⅰ with the first camera model 111M as the reference in the virtual space: As Fig.13Let the origin of the three-dimensional rectangular coordinate system I coincide with the point where the light-gathering area model A of the first camera is located. Let the Z-axis of the three-dimensional rectangular coordinate system I point to the rectangular base of the regular square pyramid where the first camera model 111M is located and be perpendicular to the plane where the rectangular base of the regular square pyramid is located. Let the X-axis of the three-dimensional rectangular coordinate system I point in the direction indicated by the arrow 111MR and be parallel to the long side of the rectangular base of the regular square pyramid. Among them, the three-dimensional rectangular coordinate system I is a three-dimensional rectangular coordinate system including the X-axis, Y-axis, and Z-axis, and the three-dimensional rectangular coordinate system I is not shown in Fig.13 It is shown in

[0206] This article provides an implementation method for placing the three-dimensional rectangular coordinate system II based on the first camera in reality: Imagine a three-dimensional rectangular coordinate system II, let the origin of the three-dimensional rectangular coordinate system II coincide with the center point of the light-gathering area of the first camera, let the Z-axis of the three-dimensional rectangular coordinate system II point in the direction from the photosensitive element of the first camera to the lens of the first camera and be perpendicular to the plane where the photosensitive element of this camera is located. From the perspective of Figure 1 looking at Figure 1 the first camera 111 and the second camera 121 shown, let the X-axis of the three-dimensional rectangular coordinate system II point in the direction from the first camera 111 to the second camera 121 and be parallel to the long side of the rectangle where the photosensitive element of the first camera is located.

[0207] This article provides a method to ensure that the positional relationship between the bounded surfaces where the first mirror model 301M to the Nth mirror model 310M in the eyeball-based optical path generation device model as the reference and the three-dimensional rectangular coordinate system I is equivalent to the positional relationship between the surfaces where the mirrors of the first mirror 301 to the Nth mirror 310 and the three-dimensional rectangular coordinate system II in reality when collecting the initial data. Specifically, in the virtual space simulated by the computer, let the positional relationship between the bounded surfaces where the first mirror model 301M to the Nth mirror model 310M and the three-dimensional rectangular coordinate system I be equivalent to the positional relationship between the first mirror to the Nth mirror and the three-dimensional rectangular coordinate system II corresponding to the spatial position information of the first mirror to the Nth mirror received by the sensor when collecting the initial data in reality.

[0208] In Fig.13In the application embodiments shown, any one of the camera models from the first camera model to the camera model serving as the fixation target is used to determine a straight line as a coordinate system in the virtual space. The relative position between the determined straight line and the camera model that determines the straight line is used to simulate the positional relationship between the light ray that enters the lens of the corresponding real camera during image acquisition by the real camera and is received to form any point on the image and the position of the real camera. In the embodiments of the present application, the straight line determined based on the camera model is referred to as the image-based light ray model. Generally speaking, the camera model described in the present application is used to determine the image-based light ray model as a coordinate system in the virtual space.

[0209] This article provides a method for determining the image-based light ray model with a camera model as a coordinate system: As Fig.13 shown, Fig.13 As shown, the first camera model 111M is a regular square pyramid. The bottom surface of the regular square pyramid is a rectangle for overlapping the edges of the image captured by the corresponding real camera of the regular square pyramid on the rectangle where the regular square pyramid is located. Any point on the image overlapping the bottom surface of the regular square pyramid is used as a marker point, and the straight line passing through the marker point and the vertex of the regular square pyramid is used as the image-based light ray model; in practical applications, only any point on the image captured by the corresponding real camera of the regular square pyramid needs to be set as the marker point, and the image-based light ray model can be determined according to the marker point through the method for determining the image-based light ray model. The relative positional relationship between the image-based light ray model and the regular square pyramid can be used to simulate the positional relationship between the light ray that enters the lens of the real camera during image acquisition by the real camera and is received to form the point set as the marker point on the image and the position of the real camera. In the following text, the method for the image-based light ray model based on the camera model is simply referred to as: the method for determining the image-based light ray model.

[0210] To simply and clearly illustrate the function of the camera model, in the embodiments provided in this application, a regular square pyramid is used as the camera model. In practical applications, a coordinate system or a three-dimensional design diagram of a camera including a lens, an aperture, and a photosensitive element can also be used as the three-dimensional model of the camera. According to the three-dimensional design diagram of the camera and through computer simulation technologies such as ray simulation, the purpose of using the camera model as a coordinate system to determine the image-based ray model can be achieved. A spatial coordinate system such as a three-dimensional rectangular coordinate system can also be used as the camera model, and a straight line can be mapped through an equation based on the coordinates of points on the image collected by the camera corresponding to the camera model to achieve the purpose of using the camera model as a coordinate system to determine the image-based ray model. It should be understood that this article does not limit the form of the camera model that can achieve the same effect as the camera model described in this article.

[0211] Any one of the first mirror model to the Nth mirror model, and the mirror model that reflects the fixation target is used as a reflecting surface. The reflecting surface is used to regard the straight line where the image-based ray model is located as the incident ray, and make a reflected ray corresponding to the incident ray through the law of reflection of light. Some measurement values can be obtained by measuring the reflection, and the measurement values can determine the visual axis equation set, and the visual axis equation set is used to implement the eye movement tracking scheme.

[0212] Step S42: Determine a reference corneal model in the virtual space according to the spots included in the first image and the second image included in the starting data, and the eyeball-based optical path generation device model used as a reference. The reference corneal model includes at least one curved surface to restore the shape of the user's eyeball's cornea.

[0213] The embodiments of this application provide a method for determining the reference corneal model, and the method includes:

[0214] (1) Regard the spots included in the first image and the second image included in the starting data described in this application as the spots described in step S1 of step 2 in the specific implementation part of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A. In the embodiments of this application, the first image included in the starting data can refer to Figure 7 the spot 514 shown, and the second image included in the starting data can refer to Figure 8 the spot 514 shown.

[0215] (2)Regarding the first measurement unit 11 and the second measurement unit 12 described in this application, they are respectively regarded as the first measurement unit C1 and the second measurement unit C2 shown in the specific implementation part of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A. Figure 1 As shown.

[0216] (3)Regarding the first camera model 111M and the second camera model 121M described in this application, they are regarded as the camera model C1 of the first measurement unit and the camera model C2 of the second measurement unit shown in the specific implementation part of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A. Figure 5 As shown.

[0217] (4)Referring to the description of steps two to three in the specific implementation part of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A, determine the optical path model described in this patent document. According to the description of step Q3 in this patent document: In each optical path model, the relative position between the vertex of the angle formed by two intersecting straight lines and the measurement unit model is equivalent to the relative position between the point on the measured smooth surface and the measurement unit; the angle bisector of the angle is the normal direction of the tangent plane where the point on the smooth surface is located. That is to say, according to the method provided in this patent document, the optical path model described in step Q3 of this patent document can be determined in the virtual space simulated by the computer. According to the optical path model described in step Q3 of this patent document, a curved surface can be modeled to simulate the shape of the measured smooth surface described in this patent document. That is to say, regarding the cornea of the user's eyeball 51 as the smooth surface described in step Q3 of this patent document, a curved surface can be modeled to simulate the shape of the cornea of the user's eyeball 51, and this curved surface is the cornea model used as the reference.

[0218] As for the detailed method of determining the cornea model used as the reference, reference can be made to the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A. Due to space limitations, it will not be elaborated in this article.

[0219] Step S43: According to the image of the pupil included in the mirror surface of the reflecting gaze target included in the first image included in the starting data, or according to the image of the pupil included in the image of the user's eye included in the starting data, determine the reference visual axis model in the virtual space through the reference eyeball-based optical path generating device model.

[0220] As Fig.14 shown. Fig.14 The structural schematic diagram of the reference model corresponding to a light path generation device based on the eyeball when the reference model includes a reflecting mirror that reflects a fixation target. The reference model includes: a first camera model 111M, a reflecting mirror model 311M that reflects the fixation target, a cornea model 51M, an optical axis model 511M as a reference, a light ray model 126 that diffusely reflects the iris feature towards the camera, a pupil center model B, and an iris feature model C.

[0221] In Fig.14 In the illustrated embodiment, the method for determining the optical axis model as a reference when the light path generation device based on the eyeball includes the reflecting mirror that reflects the fixation target includes: based on the image of the pupil included in the mirror surface of the reflecting mirror that reflects the fixation target in the first image included in the starting data, and through the first camera model 111M included in the light path generation device model based on the eyeball as a reference and the reflecting mirror model 311M that reflects the fixation target, determine the optical axis model 511M as a reference; specifically, in the embodiment of the present application, the schematic diagram of the first image included in the starting data can be referred to Figure 7 , take Figure 7 the geometric center of the image of the pupil 512 reflected on the mirror surface of the reflecting mirror 311 that reflects the fixation target shown as the marked point, based on Figure 7 the marked point on the image shown, and based on the first camera model 111M in the embodiment shown in Fig.14 , through the method for determining the light ray model based on the image mentioned above, determine a light ray model based on the image in the reference model, call this light ray model based on the image the light ray model 123M, take the straight line where the light ray model 123M is located as the incident light ray, take the surface included in the reflecting mirror model that reflects the fixation target as the reflecting surface, make a reflected light ray represented by a straight line according to the law of reflection of light, and take the straight line where the reflected light ray is located as the optical axis model 511M as a reference.

[0222] The optical axis model 511M as a reference can be used to simulate the optical axis 511 of the user's eye in the reference model. The specific explanation is as follows. According to the description of the method for determining the light ray model based on the image above, the light ray model 123M can be used to simulate in the reference model the light that actually enters the lens of the first camera 111 and is received to form Figure 7The light ray of the geometric center of the image of the pupil 512 reflected on the mirror surface of the reflector 311 that reflects the fixation target shown. Specifically, this light ray is the light ray that is diffusely reflected by the pupil and then reflected by the mirror surface of the reflector 311 that reflects the fixation target and enters the lens of the first camera 111. Since the light ray model 519M is the reflected light ray of the light ray model 123M made according to the law of light reflection, the light ray model 519M can be used to simulate Figure 1 the light ray 519 described in the embodiment shown in Figure 1 the embodiment shown. It has been proven that the light ray 519 can coincide with the visual axis 511 of the user's eye. Therefore, the visual axis model 511M as the reference can be used to simulate the visual axis 511 of the user's eye in the reference model.

[0223] The method for determining the visual axis model as the reference when the eyeball-based light path generating device includes the camera as the fixation target is:

[0224] Based on the image of the pupil included in the image of the user's eye collected by the camera as the fixation target included in the starting data, determine the visual axis model as the reference through the camera model as the fixation target. Specifically, as Fig.15 shown, Fig.15 is a schematic diagram of the image of the user's eye collected by the camera as the fixation target. The image of the user's eye collected by the camera as the fixation target shown includes: the image 512 of the user's pupil represented by a circle, the image 513 of the iris represented by a dotted line, and the marked point 516 represented by a "+". Fig.15 Set the geometric center of the image 512 of the pupil shown as the marked point. According to Fig.15 the marked point on the image shown, and according to the camera model as the fixation target included in the reference model, through the method for determining the image-based light ray model mentioned above, determine an image-based light ray model through the camera model as the fixation target. This image-based light ray model is called the visual axis model as the reference. The visual axis model as the reference can simulate the visual axis of the user's eyeball when Fig.15 the image shown in the image collected by the camera as the fixation target is obtained.

[0225] Among them, if the eye-based optical path generating device includes the reflector for reflecting the fixation target, the reference visual axis model is determined according to the first image included in the starting data; if the eye-based optical path generating device includes the camera serving as the fixation target, the reference visual axis model is determined according to the image of the user's eyes included in the starting data. The reflector for reflecting the fixation target and the camera serving as the fixation target both serve to determine the reference visual axis model and the reference iris feature model. If the reflector for reflecting the fixation target is included, the camera serving as the fixation target does not need to be included; if the camera serving as the fixation target is included, the reflector for reflecting the fixation target does not need to be included.

[0226] Step S44: Determine the reference pupil center model according to the reference visual axis model and the reference corneal model. As Fig.14 shown, the intersection point of the straight line where the reference visual axis model 511M is located and the curved surface where the reference corneal model 51M is located is used as the reference pupil center model B. The reference pupil center model B can be used to simulate the spatial position of the pupil center of the user's eyeball 51 when the starting data is collected.

[0227] Step S45: Determine the reference iris feature model through the reference eye-based optical path generating device model according to the iris information included in the mirror surface of the reflector for reflecting the fixation target included in the first image included in the starting data, or the iris information included in the image of the user's eyes included in the starting data; among them, if the eye-based optical path generating device includes the reflector for reflecting the fixation target, the reference iris feature model is determined according to the first image included in the starting data; if the eye-based optical path generating device includes the camera serving as the fixation target, the reference iris feature model is determined according to the image of the user's eyes included in the starting data.

[0228] When the eye-based optical path generating device includes the reflector for reflecting the fixation target, the method for determining the iris feature model includes:

[0229] (1) Find a feature on the image of the iris included in the mirror surface of the reflector for reflecting the fixation target included in the first image included in the starting data, and call any feature on the image that is the same as this feature the first feature on the image, and mark the first feature on the image with the marking point.

[0230] (2)Determine the light model of the light diffusely reflected from the iris feature towards the camera based on the marker points determined on the mirror surface of the rearview mirror in the image of the rearview mirror collected by the first camera included in the starting data. Specifically, according to Figure 7 the marker points 516 on the iris reflected on the mirror surface of the rearview mirror 311 that reflects the gaze target shown, and according to Fig.14 the first camera model 111M included in the reference model described in the application embodiment shown, by the method of determining the light model based on the image mentioned above, determine a light model based on the image 125 in the reference model. Take the straight line where the light model based on the image 125 is located as the incident light, take the surface included in the rearview mirror model 311M that reflects the gaze target as the reflecting surface, make the reflected light according to the law of light reflection, and call the straight line where the reflected light is located the light model 126 of the light diffusely reflected from the iris feature towards the camera; the light model 126 of the light diffusely reflected from the iris feature towards the camera can simulate all the light rays that can be reflected by the rearview mirror that reflects the gaze target and then enter the lens of the first camera among the light rays diffusely reflected from the sub-features on the iris of the user's eye in the initial model. The light model of the light diffusely reflected from the iris feature towards the camera is used to determine the iris feature model.

[0231] (3)Determine the iris feature model as the reference based on the light model of the light diffusely reflected from the iris feature towards the camera and the corneal model. Specifically, as Fig.14 shown, take the intersection point of the straight line where the light model 126 of the light diffusely reflected from the iris feature towards the camera is located and the plane of the corneal model 51 as the iris feature model as the reference; the iris feature model as the reference can simulate the first feature on the iris of the real user's eye; among them, the first feature on the iris of the user's eye is a feature on the iris of the real user's eye corresponding to the first feature on the image in the first image included in the starting data; the light diffusely reflected from this feature enters the first camera, forming the first feature on the image in the first image included in the starting data.

[0232] When the eyeball-based light path generating device includes the camera as the gaze target, the method for determining the iris feature model includes:

[0233] (1)Find a feature on the image of the iris included in the image of the user's eye included in the starting data, and call any feature on the image that is the same as this feature the first feature on the image, and mark it with the marker point 516 on the first feature on this image.

[0234] (2) Based on the marker points included in the image of the user's eyes captured by the camera serving as the fixation target in the starting data, determine the light model of the light diffusely reflected from the iris feature towards the camera through the camera model serving as the fixation target. Specifically, according to Fig.15 the marker points 516 shown, and according to the camera model serving as the fixation target included in the reference model, through the method for determining the light model based on the image mentioned above, determine a light model based on the image in the reference model, and refer to this light model based on the image as the light model of the light diffusely reflected from the iris feature towards the camera'; the light model of the light diffusely reflected from the iris feature towards the camera' can simulate, in the reference model, the light among all the light diffusely reflected from the first feature on the iris of the user's eyes that can enter the lens of the first camera, and the light model of the light diffusely reflected from the iris feature towards the camera' is used to determine the iris feature model.

[0235] (3) Determine the iris feature model according to the light model of the light diffusely reflected from the iris feature towards the camera' and the corneal model. Specifically, take the intersection point of the straight line where the light model of the light diffusely reflected from the iris feature towards the camera' is located and the plane where the corneal model 51 is located as the iris feature model; the iris feature model can simulate, in the initial model, the spatial position of the first feature on the iris of the user's eyes.

[0236] The method for determining the marker points according to the image of the iris included on the mirror surface of the reflector that reflects the fixation target in the first image captured by the first camera included in the starting data includes:

[0237] (1) Identify the image of the iris reflected on the mirror surface of the reflector 311 that reflects the fixation target in the first image included in the starting data, and find the first feature on this image of the iris and mark it with the marker point 516 on the first feature on this image.

[0238] (2) Crop the area where the image of the iris reflected on the mirror surface of the reflector 311 that reflects the fixation target in the first image included in the starting data is located into an image, and refer to this image as the iris image; among them, the iris image includes the marker point 516, and the iris image is used to find, in other first images, the image of the iris that is the same as or similar to the image of the iris included in the iris image, and find the first feature on the image of the iris that is the same as or similar, and mark the marker point on the first feature found on this image, and ensure that when the image of the iris that is the same as or similar overlaps with the image of the iris, the marker points made can also overlap or approximately overlap with the marker points included on the image of the iris. In Figure 7In the illustrated embodiment, the image of the iris reflected on the mirror surface of the mirror 311 included in the first image includes the marked point 516 determined according to a certain sub-feature on the image of the iris.

[0239] The method for determining the marked point according to the image of the iris included in the image of the user's eye included in the starting data includes:

[0240] (1) Identify the image of the iris 513 in the image of the user's eye collected by the camera as the fixation target included in the starting data, and find the first feature on the image of the iris 513 and mark the first feature on the image with the marked point 516.

[0241] (2) Crop the area where the image of the iris is identified in the image of the user's eye included in the starting data into an image, and perform mirror processing on the cropped image (only after mirror processing the image can it overlap with the image of the iris feature included in the first image), and this image is called the iris image'; wherein, the iris image' includes the marked point 516. The iris image' has the same function as the iris image. In Fig.15 In the illustrated embodiment, the image of the user's eye includes the marked point 516 determined according to a certain sub-feature on the image of the iris.

[0242] It should be understood that in this example, the iris feature model is represented by points. In practical applications, the light model diffused by the iris feature towards the camera or the straight line where the light model' diffused by the iris feature towards the camera is located can also be used as the iris feature model. It should also be understood that this article does not limit the implementation sequence of steps S42 to S45.

[0243] In the method for determining the visual axis equations, the method for determining the first group of preset position arrays to the Nth group of preset position arrays in step S23 according to the reference model is as Fig.16 shown, Fig.16 This is a schematic diagram of a method for determining the first group of preset position arrays to the Nth group of preset position arrays according to the reference model provided by an embodiment of the present application. The method includes:

[0244] Step S51: Establish a dynamic mirror model according to the reference model.

[0245] As Fig.17 shown, Fig.17 This is a structural schematic diagram of a dynamic mirror model provided by an embodiment of the present application. The shown dynamic mirror model includes: a dynamic mirror model 553M and the reference model. The reference model is Fig.14The reference model described in the application implementation manner shown, the reference model includes the first camera model 111M, the light collection area model A of the first camera, the pupil center model B, the iris feature model C, the corneal model 51M, and the visual axis model 511M.

[0246] In Fig.17 The method for establishing a dynamic mirror model according to the reference model in the implementation manner shown includes: assuming that the light collection area model A of the first camera included in the reference model is point A, the point where the pupil center model is located is point B, the point where the iris feature model used as a reference is located is point C, making a point O on the surface included in the corneal model 51M used as a reference, and passing through the point O to make a normal line 514M of the surface where the corneal model 51M used as a reference is located, and making a point P on the normal line 514M, and passing through the point P to make a plane, this plane is the dynamic mirror model 553M, making the plane where the dynamic mirror model 553M is located perpendicular to the angle bisector of angle APO, and making a point Q on the plane where the dynamic mirror model 553M is located, making the angle bisector of angle AQB perpendicular to the plane where the dynamic mirror model 553M is located, and making a point R on the plane where the dynamic mirror model 553M is located, making the angle bisector of angle ARC perpendicular to the plane where the dynamic mirror model 553M is located. In Fig.17 In the implementation manner shown, the point O is a point represented by "+", and the dynamic mirror model 553M is a plane.

[0247] In Fig.17 In the implementation manner shown, because the shape of the surface where the corneal model 51M used as a reference is located is a quasi-spherical surface, when the point O is at different positions on the surface where the corneal model 51M used as a reference is located, the normal line 514M passing through the point O will rotate to different angles, the dynamic mirror model 553M will be in different spatial positions, and in addition, the sum of the lengths of line segment AP and line segment OP will also determine the spatial position of the dynamic mirror model 553M; wherein, the spatial position of the dynamic mirror model 553M is a spatial position with reference to the first camera 111M or the corneal model 51M used as a reference. The dynamic mirror model 553M is used to determine the preset position, the preset position is used to form the preset position array, the preset position array is used to determine the optical path model, and the optical path model is used to determine the visual axis equation set.

[0248] In practical applications, when the shapes of the mirror surfaces of the first to the Nth reflectors are all flat, the dynamic mirror model is a plane; when the first to the Nth reflectors are exactly the same and the shape of the mirror surface is a curved surface, the shape of the surface where the dynamic mirror model is located is exactly the same as the shape of the mirror surface of any one of the first to the Nth reflectors; when the first to the Nth reflectors are different from each other and the mirror surfaces of one or more reflectors are curved surfaces, N dynamic mirror models need to be established, namely the first to the Nth dynamic mirror models in sequence, and the shapes of the surfaces where the first to the Nth dynamic mirror models are located are exactly the same as the shapes of the mirror surfaces of the first to the Nth reflectors in sequence.

[0249] Step S52: Determine all possible combinations formed by taking one distance parameter value, one angle parameter value, and one rotation angle parameter value from the L distance parameter values, M angle parameter values, and N rotation angle parameter values respectively; wherein, each combination includes one distance parameter value, one angle parameter value, and one rotation angle parameter value.

[0250] In the implementation manner of step S11, it has been set that the total number L of the distance parameter values is equal to 3, the total number M of the angle parameter values is equal to 5, and the total number N of the rotation angle parameter values is equal to 10. The number of combinations that can be formed by taking one distance parameter value, one angle parameter value, and one rotation angle parameter value from 3 distance parameter values, 5 angle parameter values, and 10 rotation angle parameter values respectively is L * M * N = 3 * 5 * 10 = 150.

[0251] Each combination includes a distance parameter value, an angle parameter value, and a rotation angle parameter value. For example, in the embodiment of the present application: the distance parameter value, angle parameter value, and rotation angle parameter value corresponding to the combination including the second distance parameter value, the first angle parameter value, and the second rotation angle parameter value are equal to 500, 0.1, and 36 respectively (according to the settings in the embodiment of step S11, the second distance parameter value is equal to 500, the first angle parameter value is equal to 0.1, and the second rotation angle parameter value is equal to 36). Due to space limitations, only one of all the combinations existing in the embodiment of the present application is exemplified herein, and the remaining combinations are not listed in the embodiment.

[0252] Step S53: Determine the corresponding preset position for each of the combinations through the dynamic mirror model; wherein, each of the preset positions corresponds to a distance parameter value, an included angle parameter value, and a rotation angle parameter value. Since each of the combinations corresponds to a preset position, and each of the combinations corresponds to a distance parameter value, an included angle parameter value, and a rotation angle parameter value, each of the combinations corresponds to a distance parameter value, an included angle parameter value, and a rotation angle parameter value.

[0253] This application provides a method for determining the preset position corresponding to any one of the combinations through a dynamic mirror model. This method is simply referred to as the method for determining the preset position. Specifically, as Fig.17 shown, for any one of the combinations, make the sum of the distance value between point O and point P and the distance value between point P and point A equal to the distance parameter value included in this combination; make any one of the two included angles less than 90° among the four included angles between the straight line where line segment OP is located and the straight line where line segment BQ is located equal to the included angle parameter value included in this combination; make the angle value of the angle that the plane BQO needs to rotate around line segment BQ along the rotation direction indicated by the right-hand rule until it coincides with the plane BQC for the first time equal to the rotation angle parameter value included in this combination (if the rotation angle parameter value is equal to 0, then make the plane BQO coincide with the plane BQC), then the spatial position of the dynamic mirror model 553M can be determined, and moreover, the line segment BQ, the line segment QA, the line segment CR, the line segment RA, the line segment OP, the line segment PA, the plane BQO, and the plane BQC can be determined. The plane where the dynamic mirror model 553M is located at the determined spatial position can be used as the preset position corresponding to this combination one by one, and the determined line segment BQ, the line segment QA, the line segment CR, the line segment RA, the line segment OP, the line segment PA, the plane BQO, and the plane BQC are used as the line segment BQ, the line segment QA, the line segment CR, the line segment RA, the line segment OP, the line segment PA, the plane BQO, and the plane BQC corresponding to this preset position one by one. Multiple preset positions can be used to form the preset position array. Among them, the rotation direction indicated by the right-hand rule is specifically that the right hand holds the line segment BQ, the thumb is straightened and points along the direction from point B to point Q, and the direction in which the four fingers are wound is the rotation direction indicated by the right-hand rule.

[0254] The method for determining the preset position corresponding to any combination one by one through the dynamic mirror model provided by the embodiments of the present application can determine the preset positions corresponding to each of the combinations one by one, and each of the preset positions corresponds one by one to a unique line segment BQ, a unique line segment QA, a unique line segment CR, a unique line segment OP, a unique plane BQO, and a unique plane BQC.

[0255] Step S54: The preset positions corresponding to the same distance parameter value and corresponding to the same angle parameter value are combined to form the preset position array, and all the preset position arrays corresponding to the first distance parameter value to all the preset position arrays corresponding to the L-th distance parameter value are respectively combined to form the first group of preset position arrays to the L-th group of preset position arrays.

[0256] Wherein, any one of the preset position arrays corresponds to a distance parameter value and an angle parameter value, and any one of the preset position arrays contains N preset positions, which are successively the first preset position to the N-th preset position, and any group of preset position arrays including the first group of preset position arrays to the L-th group of preset position arrays contains M preset position arrays.

[0257] Since each combination contains a distance parameter value and an angle parameter value, and each combination corresponds one by one to a preset position, each combination corresponds to a distance parameter value and an angle parameter value.

[0258] Since the preset positions corresponding to the same distance parameter value and corresponding to the same angle parameter value form a preset position array, all the preset positions included in any one of the preset position arrays correspond to the same distance parameter value and the same angle parameter value, so any one of the preset position arrays corresponds to a distance parameter value and an angle parameter value.

[0259] And since the preset positions corresponding to the same distance parameter value and corresponding to the same angle parameter value form a preset position array, any one of the preset position arrays contains N preset positions, and the N preset positions included in any one of the preset position arrays are successively the first preset position to the N-th preset position.

[0260] Since any one of the preset position arrays corresponds to a distance parameter value and an included angle parameter value, and all the preset position arrays corresponding to the first distance parameter value to all the preset position arrays corresponding to the L-th distance parameter value respectively form the first group of preset position arrays to the L-th group of preset position arrays, any group of preset position arrays including the first group of preset position arrays to the L-th group of preset position arrays contains M preset position arrays.

[0261] Each of the M preset position arrays included in any group of preset position arrays corresponds to the first included angle parameter value to the M-th included angle parameter value respectively. In the embodiments of the present application, the preset position arrays corresponding to the first included angle parameter value to the M-th included angle parameter value included in any group of preset position arrays are respectively referred to as the first preset position array to the M-th preset position array included in this group of preset position arrays.

[0262] In the embodiments of the present application, the k-th preset position array included in the i-th group of preset position arrays is referred to as the (i - 1)*M + k-th preset position array, where i takes any positive integer less than or equal to the total number L of groups of preset position arrays, and k takes any positive integer less than or equal to the total number M of the preset position arrays included in each group of preset position arrays. For example, in the embodiments of the present application, the first preset position array to the M-th preset position array included in the first group of preset position arrays are successively referred to as the (1 - 1)*M + 1-th preset position array to the (1 - 1)*M + M-th preset position array. Among them, the (1 - 1)*M + 1-th preset position array is the first preset position array, and the (1 - 1)*M + M-th preset position array is the M-th preset position array; the first preset position array to the M-th preset position array included in the L-th group of preset position arrays are successively referred to as the (L - 1)*M + 1-th preset position array to the (L - 1)*M + M-th preset position array. Among them, the (L - 1)*M + M-th preset position array is the L*M-th preset position array; in the embodiments of the present application, L = 3 and M = 5 are set. Therefore, there are 3 groups of preset position arrays in the embodiments of the present application, namely the first group of preset position arrays to the third group of preset position arrays. The first group of preset position arrays includes the first preset position array to the fifth preset position array, the second group of preset position arrays includes the sixth preset position array to the tenth preset position array, and the third group of preset position arrays includes the eleventh preset position array to the fifteenth preset position array.

[0263] As Fig.18 shown, Fig.18The structural schematic diagram of the preset position array provided by the embodiment of the present application. The shown preset position array includes the 6th preset position array, the 6th preset position array, the 6th preset position array, the 6th preset position array, the 10th preset position array, and the reference model. The reference model is Fig.14 the reference model shown in the application implementation manner shown, at least including the first camera model 111M; the 6th preset position array includes the 1st preset position L2M1N1, the 2nd preset position L2M1N2, up to the Nth preset position L2M1N10 distributed in a circular ring; the 7th preset position array includes the 1st preset position L2M2N1, the 2nd preset position L2M2N2, up to the Nth preset position L2M2N10 distributed in a circular ring; the 8th preset position array includes the 1st preset position L2M3N1, the 2nd preset position L2M3N2, up to the Nth preset position L2M3N10 distributed in a circular ring; the 9th preset position array includes the 1st preset position L2M4N1, the second preset position L2M4N2, up to the Nth preset position L2M4N10 distributed in a circular ring; the 0th preset position array includes the 1st preset position L2M5N1, the 2nd preset position L2M5N2, up to the Nth preset position L2M5N10 distributed in a circular ring and the light convergence area model A of the first camera, that is, the point A; Fig.18 The shown preset position array also includes line segments BQ, QA, CR, RA, OP, and PA corresponding to the preset position L2L3N9 one by one. In Fig.18 the shown embodiment, the line segment BQ is the center line segment that intersects the corneal model 51M and the preset position L2L3N9, the line segment QA is the center line segment that intersects the point A and the preset position L2L3N9, the line segment CR is the long dash line segment that intersects the corneal model 51M and the preset position L2L3N9, the line segment RA is the long dash line segment that intersects the point A and the preset position L2L3N9, the line segment OP is the double dash line segment that intersects the corneal model 51M and the preset position L2L3N9, and the line segment PA is the double dash line segment that intersects the point A and the preset position L2L3N9. Any one of the preset positions includes a unique line segment BQ, QA, CR, RA, OP, and PA. In Fig.18 only the line segments BQ, QA, CR, RA, OP, and PA corresponding to the preset position L2L3N9 one by one are shown, and the line segments BQ, QA, CR, RA, OP, and PA corresponding to the remaining each preset position are not shown.

[0264] In Fig.18In the illustrated embodiment, the name of any one of the preset positions has a suffix, which is divided into three parts, namely the first part, the second part, and the third part. The first part contains the letter L and the Arabic numerals following the letter L. The second part contains the letter M and the Arabic numerals following the letter M. The third part contains the letter N and the Arabic numerals following the letter N. The Arabic numerals contained in the first part represent the serial number of the distance parameter value included in the combination corresponding to the preset position, that is, the serial number of the distance parameter value corresponding to the preset position. The Arabic numerals contained in the second part represent the serial number of the included angle parameter value included in the combination corresponding to the preset position, that is, the serial number of the included angle parameter value corresponding to the preset position. The Arabic numerals contained in the third part represent the serial number of the rotation angle parameter value included in the combination corresponding to the preset position, that is, the serial number of the rotation angle parameter value corresponding to the preset position. Taking the 1st preset position L2M1N1 as an example, the 1st preset position L2M1N1 corresponds to the second distance parameter value, the first included angle parameter value, and the first rotation angle parameter value.

[0265] This application provides an embodiment in which the preset positions corresponding to the combinations containing the same distance parameter values and included angle parameter values are formed into a preset position array. Fig.18 Taking the 6th preset position array in the illustrated embodiment as an example, the 6th preset position array is composed of the 1st preset position L2M1N1, the 2nd preset position L2M1N2, up to the Nth preset position L2M1N10. According to the suffixes of the names of the N preset positions, it can be known that the N preset positions respectively correspond to the same distance parameter value and included angle parameter value, which are the second distance parameter value and the first included angle parameter value respectively. According to the setting in the embodiment provided in step S11, the second distance parameter value is equal to 500 millimeters, and the first included angle parameter value is equal to 0.1 degree. That is to say, the distance parameter values corresponding to the N preset positions are all 500 millimeters, and the included angle parameter values corresponding to the N preset positions are all 0.1 degree. That is to say, the sum of the lengths of the line segment OP and the line segment PA corresponding to any one of the N preset positions is equal to 500 millimeters. Among the four included angles between the straight line where the line segment BQ corresponding to any one of the N preset positions is located and the straight line where the line segment OP is located, the included angle value of any one of the two included angles less than 90° is equal to 0.1 degree.

[0266] There is a regularity among the line segment BQ, the line segment QA, the line segment OP, the plane BQO, and the plane BQC corresponding to any one of the first preset position to the Nth preset position included in any one of the preset position arrays. Any one of the preset position arrays is used to make the spatial positions of the first reflector to the Nth reflector be in the real spatial positions corresponding to the first preset position to the Nth preset position respectively included in the preset position array. When the spatial positions of the first reflector to the Nth reflector are in the real spatial positions corresponding to the first preset position to the Nth preset position respectively included in any one of the preset position arrays, there will be a regularity among the jth pupil-based first optical path, the jth iris-based first optical path, and the jth cornea-based first optical path manufactured by the jth reflector. Then, the visual axis equations can be obtained by analyzing the jth pupil-based first optical path, the jth iris-based first optical path, and the jth cornea-based first optical path manufactured by the jth reflector. Here, j takes any positive integer less than or equal to the total number N of preset positions included in any one of the preset position arrays. If there is no such regularity among the jth pupil-based first optical path, the jth iris-based first optical path, and the jth cornea-based first optical path manufactured by the jth reflector, it will be difficult to obtain accurate visual axis equations by analyzing the jth pupil-based first optical path, the jth iris-based first optical path, and the jth cornea-based first optical path manufactured by the jth reflector. Here, j takes any positive integer less than or equal to N.

[0267] The regularity existing among the line segment BQ, the line segment QA, the line segment CR, the line segment OP, the plane BQO, and the plane BQC corresponding to any one of the preset positions is specifically as follows: Among the line segment BQ, the line segment QA, the line segment CR, the line segment OP, the plane BQO, and the plane BQC corresponding to the same preset position, the sum of the lengths of the line segment OP and the line segment PO is equal to the length parameter value corresponding to the preset position. Among the four included angles between the straight line where the line segment OP is located and the straight line where the line segment BQ is located, the value of any one of the two included angles less than 90° is equal to the included angle parameter value corresponding to the preset position. The angle value that the plane BQO needs to rotate around the line segment BQ along the rotation direction indicated by the right-hand rule until it coincides with the plane BQC for the first time is equal to the rotation angle parameter value corresponding to the preset position.

[0268] In the embodiment of the present application, the first-stage optical path based on the j-th pupil is the path traveled in space by the light rays that can be reflected by the mirror surface of the j-th reflector, enter the first camera lens and be received, among the light rays diffusely reflected from the pupil of the user's eye in various directions, starting from when the light rays are diffusely reflected from the pupil of the user's eye until before being reflected by the mirror surface of the j-th reflector; the last-stage optical path based on the j-th pupil is the path traveled in space by the light rays in the first-stage optical path based on the j-th pupil, starting from when the light rays are reflected by the mirror surface of the j-th reflector until they enter the lens of the first camera; the first-stage optical path based on the j-th iris is the path traveled in space by the light rays that can be reflected by the mirror surface of the j-th reflector, enter the first camera lens and be received, among the light rays diffusely reflected from the first feature on the iris of the user's eye in various directions, starting from when the light rays are diffusely reflected from the first feature until before being reflected by the mirror surface of the j-th reflector respectively; the last-stage optical path based on the j-th iris is the path traveled in space by the light rays in the first-stage optical path based on the j-th iris, starting from when the light rays are reflected by the mirror surface of the j-th reflector until they enter the lens of the first camera; the last-stage optical path based on the j-th cornea is the path traveled in space by the first detection light rays that can be reflected by the mirror surface of the j-th reflector and travel along the normal line of the user's eye cornea to the user's eye cornea, among all the first detection light rays emitted from the first light-emitting element in various directions, starting from when the first detection light rays are emitted from the first light-emitting element until before being reflected by the mirror surface of the j-th reflector; the first-stage optical path based on the j-th cornea is the path traveled in space by the light rays in the last-stage optical path based on the j-th cornea, starting from when the light rays are reflected by the mirror surface of the j-th reflector until they irradiate on the cornea of the user's eye; where j takes any positive integer less than or equal to N. When j takes 1 to N respectively, there will be the first-stage optical path based on the 1st pupil, the last-stage optical path based on the 1st pupil, the first-stage optical path based on the 1st iris, the last-stage optical path based on the 1st iris, the first-stage optical path based on the 1st cornea, the last-stage optical path based on the 1st cornea generated by the first reflector, until the first-stage optical path based on the Nth pupil, the last-stage optical path based on the Nth pupil, the first-stage optical path based on the Nth iris, the last-stage optical path based on the Nth iris, the first-stage optical path based on the Nth cornea, the last-stage optical path based on the Nth cornea generated by the Nth reflector.In addition, the first to the Nth reflectors are in different spatial positions at different times, and the first to the Nth based-pupil first-segment optical paths, the first to the Nth based-pupil last-segment optical paths, the first to the Nth based-iris first-segment optical paths, the first to the Nth based-iris last-segment optical paths, the first to the Nth based-cornea first-segment optical paths, and the first to the Nth based-cornea last-segment optical paths generated by the first to the Nth reflectors in different spatial position systems are all different.

[0269] When the spatial positions of the first to the Nth reflectors are at the real spatial positions corresponding to the first to the Nth preset positions respectively included in any one of the preset position arrays, the specific reason for the regularity among the jth based-pupil first-segment optical path, the jth based-iris first-segment optical path, and the jth based-cornea first-segment optical path manufactured by the jth reflector is that: there is such a regularity among the line segment BQ, the line segment QA, the line segment CR, the line segment OP, the plane BQO, and the plane BQC corresponding to any one of the preset positions. That is to say, there is such a regularity among the line segment BQ, the line segment QA, the line segment CR, the line segment OP, the plane BQO, and the plane BQC corresponding to the jth preset position included in any one of the preset position arrays. When the spatial positions of the first to the Nth reflectors are at the real spatial positions corresponding to the first to the Nth preset positions respectively included in any one of the preset position arrays, the relative positions among the jth based-pupil first-segment optical path, the jth based-pupil last-segment optical path, the jth based-iris first-segment optical path, the jth based-iris last-segment optical path, the jth based-cornea first-segment optical path, and the jth based-cornea last-segment optical path manufactured by the jth reflector are equivalent to the relative positions among the line segment BQ, the line segment QA, the line segment CR, the line segment RA, the line segment OP, and the line segment PA corresponding to the jth preset position included in this preset position array.

[0270] In the method for determining the visual axis equation system, in the above step S13, the method for determining the optical path model corresponding to each preset position array in different time periods is as Fig.19 shown Fig.19 is a schematic diagram of a method for determining the optical path model corresponding to each preset position array provided by an embodiment of the present application. This method includes:

[0271] Step S61: Adjust the spatial positions of the first reflector to the Nth reflector respectively through the first adjustment component to the Nth adjustment component, so that the spatial positions of the first reflector to the Nth reflector are respectively at the real-space positions corresponding to the 1st preset position to the Nth preset position included in each of the preset position arrays during each of the time periods. Collect the first image, the second image, and the image of the user's eyes corresponding to each of the preset position arrays through the first camera, the second camera, and the camera serving as the fixation target during each of the time periods.

[0272] Among them, the first image, the second image, and the image of the user's eyes corresponding to each of the preset position arrays are used to determine the optical path model corresponding to each of the preset position arrays. If the camera serving as the fixation target is not included, there is no image of the user's eyes, and the optical path model corresponding to each of the preset position arrays is determined only through the first image and the second image corresponding to each of the preset position arrays.

[0273] In the embodiments of the present application, each of the preset position arrays is respectively the 1st preset position array to the L*Mth preset position array, and the 1st preset position array to the L*Mth preset position array respectively correspond to different first images, second images, and images of the user's eyes; each of the time periods is respectively the 1st time period to the L*Mth time period, and the 1st time period to the L*Mth time period respectively correspond to the 1st preset position array to the L*Mth preset position array. Specifically, the spatial positions of the first reflector to the Nth reflector are successively at the real-space positions corresponding to the 1st preset position to the Nth preset position included in the 1st preset position array to the L*Mth preset position array respectively during the 1st time period to the L*Mth time period.

[0274] S62: Determine the optical path model corresponding to each of the preset position arrays according to the first image, the second image, and the image of the user's eyes corresponding to each of the preset position arrays.

[0275] Among them, each of the optical path models at least includes: a visual axis model, a first to an Nth mirror model, a first to an Nth pupil-based first-segment optical path model, a first to an Nth pupil-based last-segment optical path model, a first to an Nth iris-based first-segment optical path model, a first to an Nth iris-based last-segment optical path model, a first to an Nth cornea-based first-segment optical path model, and a first to an Nth cornea-based last-segment optical path model; if the image of the user's eye is not included, the optical path models corresponding to each of the preset position arrays are determined only by the first image and the second image corresponding to each of the preset position arrays. Any one of the preset position arrays corresponds one-to-one to an optical path model. In the embodiments of the present application, the preset position arrays are respectively the first to the L*Mth preset position arrays, and the optical path models corresponding to the preset position arrays are respectively the first to the L*Mth optical path models. The first to the L*Mth preset position arrays correspond to the first to the L*Mth optical path models respectively. The optical path model corresponding to the (i - 1)*M + kth preset position array is referred to as the (i - 1)*M + kth optical path model. There are also a first to an Lth group of optical path models corresponding to the first to the Lth group of preset position arrays respectively. Any one of the groups of optical path models including the first to the Lth group of optical path models contains M optical path models. i takes any positive integer less than or equal to L, and k takes any positive integer less than or equal to M.

[0276] The (i - 1)*M + k optical path model includes the first pupil-based first-segment optical path model to the Nth pupil-based first-segment optical path model, the first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model, the first iris-based first-segment optical path model to the Nth iris-based first-segment optical path model, the first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model, the first cornea-based first-segment optical path model to the Nth cornea-based first-segment optical path model, and the first cornea-based last-segment optical path model to the Nth cornea-based last-segment optical path model, which are used to simulate the shapes of the first pupil-based first-segment optical path to the Nth pupil-based first-segment optical path, the first pupil-based last-segment optical path to the Nth pupil-based last-segment optical path, the first iris-based first-segment optical path to the Nth iris-based first-segment optical path, the first iris-based last-segment optical path to the Nth iris-based last-segment optical path, the first cornea-based first-segment optical path to the Nth cornea-based first-segment optical path, and the first cornea-based last-segment optical path to the Nth cornea-based last-segment optical path during the corresponding time period of the (i - 1)*M + k optical path model; wherein, the time periods corresponding to the first optical path model to the L*M optical path models are the first time period to the L*M time periods respectively; the spatial positions of the first reflector to the Nth reflector are successively at the real spatial positions corresponding to the first preset position to the Nth preset position included in the first preset position array to the L*M preset position arrays during the first time period to the L*M time periods; different spatial positions of the first reflector to the Nth reflector will generate different first pupil-based first-segment optical paths to Nth pupil-based first-segment optical paths, first pupil-based last-segment optical paths to Nth pupil-based last-segment optical paths, first iris-based first-segment optical paths to Nth iris-based first-segment optical paths, first iris-based last-segment optical paths to Nth iris-based last-segment optical paths, first cornea-based first-segment optical paths to Nth cornea-based first-segment optical paths, and first cornea-based last-segment optical paths to Nth cornea-based last-segment optical paths.

[0277] The first camera can collect the light in the first pupil-based last-segment optical path to the light in the Nth pupil-based last-segment optical path during the first time period to the L*M time periods, and can form an image of the pupil on the images of the mirrors of the first reflector to the Nth reflector included in the first image corresponding to the first preset position array to the L*M preset position arrays respectively.

[0278] The first camera can collect the light in the first iris-based final optical path to the light in the Nth iris-based final optical path during the first time period to the L*M time period, and can form the first feature on the image of the iris on the images of the mirrors of the first mirror to the Nth mirror included in the first images corresponding to the first preset position array to the L*M preset position arrays respectively.

[0279] The first camera can collect the light in the first cornea-based final optical path to the light in the Nth cornea-based final optical path during the first time period to the L*M time period, and can form the light spot on the images of the mirrors of the first mirror to the Nth mirror included in the first images corresponding to the first preset position array to the L*M preset position arrays respectively.

[0280] In the method for determining the visual axis equation set, in step S62 above, the method for determining the optical path model corresponding to each preset position array according to the first image, the second image, and the image of the user's eye corresponding to each preset position array includes:

[0281] Step S71: Determine the eyeball-based optical path generation device model corresponding to each preset position array according to the spatial positions of the first mirror to the Nth mirror during each time period. In the embodiments of the present application, each of the preset position arrays is the first preset position array to the L*M preset position arrays, and any one of the first preset position array to the L*M preset position arrays only corresponds to one eyeball-based optical path generation device model.

[0282] Any one of the eye-based optical path generation device models at least includes the first mirror model to the Nth mirror model, the first camera model, and the second camera model. The shapes of the first mirror model to the Nth mirror model, the first camera model, and the second camera model included in any one of the eye-based optical path generation device models are respectively exactly the same as those of the first mirror model to the Nth mirror model, the first camera model, and the second camera model included in the reference eye-based optical path generation device model. The determination method of any one of the eye-based optical path generation device models can refer to the method of determining the reference eye-based optical path generation device model in the reference model in step S41. It should be noted that the respective time periods corresponding to the first preset position array to the L*Mth preset position array are the first time period to the L*Mth time period. In the (i - 1)*M + kth time period, the spatial positions of the first mirror to the Nth mirror are respectively at the real space positions corresponding to the first preset position to the Nth preset position included in the (i - 1)*M + kth preset position array. In the virtual space, the three-dimensional rectangular coordinate system Ⅰ is placed with the first camera model included in the eye-based optical path generation device model corresponding to the (i - 1)*M + kth preset position array as the reference. In reality, the three-dimensional rectangular coordinate system Ⅱ is placed with the first camera as the reference. It is necessary to ensure that the relative position relationship between the bounded surface where the first mirror model to the Nth mirror model included in the eye-based optical path generation device model corresponding to the (i - 1)*M + kth preset position array is located and the three-dimensional rectangular coordinate system Ⅰ is equivalent to: the position relationship between the mirror surfaces of the first mirror to the Nth mirror and the three-dimensional rectangular coordinate system Ⅱ in the (i - 1)*M + kth time period; where i is a positive integer less than L, and k is a positive integer less than or equal to M.

[0283] Step S72: According to the light spots in the first images corresponding to the respective preset position arrays and the light spots in the second images corresponding to the respective preset position arrays, and by means of each of the device models for collecting eye data, determine the corneal models corresponding to the respective preset position arrays.

[0284] In the embodiments of the present application, the respective preset position arrays are the first preset position array to the L*Mth preset position array, and any one of the preset position arrays including the first preset position array to the L*Mth preset position array corresponds to only one corneal model.

[0285] The determination method of any of the corneal models may refer to the method for determining the corneal model as the reference in step S42. The difference lies in that the corneal model corresponding to the (i - 1)*M + k preset position array is determined based on the first image and the second image corresponding to the (i - 1)*M + k preset position array and through the model of the eyeball-based optical path generation device corresponding to the (i - 1)*M + k preset position array, rather than being determined based on the first image and the second image included in the starting data and through the reference model of the eyeball-based optical path generation device. Here, i is a positive integer less than or equal to L, and k is a positive integer less than or equal to M.

[0286] Step S73: Respectively based on the images of the pupils on the mirror surfaces of the reflectors for the reflected fixation targets in the first images corresponding to the respective preset position arrays, and respectively determine the visual axis models corresponding to the respective preset position arrays through the respective device models for collecting eyeball data, or respectively based on the images of the pupils on the images of the user's eyes corresponding to the respective preset position arrays and respectively determine the visual axis models corresponding to the respective preset position arrays through the respective device models for collecting eyeball data; wherein, if there is no image of the user's eyes because there is no camera as the fixation target, then the visual axis model is determined based on the first image, and if there is an image of the user's eyes because there is a camera as the fixation target, then the visual axis model is determined based on the image of the user's eyes.

[0287] In the embodiments of the present application, the respective preset position arrays are respectively the 1st preset position array to the L*Mth preset position array, and any one of the preset position arrays including the 1st preset position array to the L*Mth preset position array corresponds to only one visual axis model.

[0288] If the camera serving as the fixation target is not included, there is no image of the user's eyes. If there is no image of the user's eyes, the method for determining the visual axis model corresponding to the ((i - 1)*M + k)-th preset position array can refer to that described in step S43: the method for determining the reference visual axis model when the eye-based optical path generating device includes the reflector that reflects the fixation target, and use the reference visual axis model obtained by this method as the visual axis model corresponding to the ((i - 1)*M + k)-th preset position array. The difference is that the visual axis model corresponding to the ((i - 1)*M + k)-th preset position array is determined based on the first image corresponding to the ((i - 1)*M + k)-th preset position array and through the eye-based optical path generating device model corresponding to the ((i - 1)*M + k)-th preset position array, rather than being determined based on the first image included in the starting data and through the reference eye-based optical path generating device model.

[0289] If the camera serving as the fixation target is included, there is an image of the user's eyes. The method for determining the visual axis model corresponding to the ((i - 1)*M + k)-th preset position array can refer to that described in step S43: the method for determining the reference visual axis model when the eye-based optical path generating device includes the camera serving as the fixation target, and use the reference visual axis model obtained by this method as the visual axis model corresponding to the ((i - 1)*M + k)-th preset position array. The difference is that the visual axis model corresponding to the ((i - 1)*M + k)-th preset position array is determined based on the image of the user's eyes corresponding to the ((i - 1)*M + k)-th preset position array and through the eye-based optical path generating device model corresponding to the ((i - 1)*M + k)-th preset position array, rather than being determined based on the image of the user's eyes included in the starting data and through the reference eye-based optical path generating device model.

[0290] Step S74: Based on the array composed of the images of the pupils on the first images corresponding to each of the preset position arrays, and through each of the device models for collecting eye data, determine the first to N-th pupil-based first-segment optical path models and the first to N-th pupil-based last-segment optical path models corresponding to each of the preset position arrays; wherein, the array composed of the images of the pupils is the array composed of the images of the pupils respectively included on the images of the mirrors of the first to N-th reflectors in the first image.

[0291] In the embodiments of the present application, each of the device models for collecting eye data is respectively the first device model for collecting eye data to the L*Mth device model for collecting eye data. Any one of the preset position arrays from the first preset position array to the L*Mth preset position array corresponds to N first-segment light path models based on the pupil from the first first-segment light path model based on the pupil to the Nth first-segment light path model based on the pupil, and N last-segment light path models based on the pupil from the first last-segment light path model based on the pupil to the Nth last-segment light path model based on the pupil.

[0292] As Fig. 20 shown, Fig. 20 FIG. is a schematic diagram of a first image corresponding to a 10th preset position array provided by an embodiment of the present application. The first image corresponding to the 10th preset position array shown includes: images of the mirrors of the first mirror 301 to the Nth mirror 310. Any one of the images of the mirrors from the image of the mirror surface of the first mirror 301 to the image of the mirror surface of the Nth mirror 310 contains an image 512T of the user's pupil represented by a circle, an image 513T of the iris represented by a dotted line, and multiple light spots represented by solid dots; any one of the images of the iris contains a marking point 516T, and there is a first light spot marked as 514T among the multiple light spots represented by solid dots on the mirror surface of any one of the mirrors.

[0293] In Fig. 20 the embodiment shown, the array composed of the images of the pupil is the array composed of the images 512T of the user's pupil respectively included on the images of the mirror surface of the first mirror 301 to the image of the mirror surface of the Nth mirror 310.

[0294] As Fig.21 shown, Fig.21 FIG. is a schematic structural diagram of a 10th light path model provided by an embodiment of the present application. The light path model shown includes: a visual axis model 511M10 corresponding to the 10th preset position array, a corneal model 51M10 corresponding to the 10th preset position array, and a device model for generating a light path based on the eye corresponding to the 10th preset position array. The device model for generating a light path based on the eye corresponding to the 10th preset position array includes: a first mirror model 301M10 to a tenth mirror model 310M10, a first camera model 111M10 and a light collection area model A10 of the first camera included therein. In Fig.21 the embodiment of the application shown, the light collection area model A10 of the first camera is represented by a point represented by a "+". Fig.21The first camera model 111M10 in the illustrated application embodiment, the light collection area model A10 of the first camera it contains, and Fig.13 The shapes and functions of the first camera model 111M and the light collection area model A of the first camera in the illustrated application embodiment are exactly the same, and different names are used for mutual distinction. Fig.21 The illustrated 10th optical path model further includes the first pupil-based first-segment optical path model to the Nth pupil-based first-segment optical path model, the first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model, the first iris-based first-segment optical path model to the Nth iris-based first-segment optical path model, the first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model, the first cornea-based first-segment optical path model to the Nth cornea-based first-segment optical path model, and the first cornea-based last-segment optical path model to the Nth cornea-based last-segment optical path model. Specifically, Fig.21 The illustrated 10th optical path model further includes line segments 611M represented by multiple center lines, line segments 612M represented by multiple long dashes, and line segments 613M represented by multiple double-dot dashes (in Fig.21 only one line segment 611M represented by a center line, one line segment 612M represented by a long dash, and one line segment 613M represented by a double-dot dash are marked, and there are also multiple line segments represented by center lines, multiple line segments represented by long dashes, and multiple line segments represented by double-dot dashes that are not marked).

[0295] Among them, among all the center line segments intersecting the plane where the corneal model 51M10 is located, the center line segments intersecting the plane where the first mirror model 301M10 is located to the plane where the Nth mirror model 310M10 is located are respectively used to represent the first pupil-based first-segment light path model to the Nth pupil-based first-segment light path model; among all the center line segments intersecting the point where the light-gathering area model A10 of the first camera is located, the center line segments intersecting the plane where the first mirror model 301M10 is located to the plane where the Nth mirror model 310M10 is located are respectively used to represent the first pupil-based last-segment light path model to the Nth pupil-based last-segment light path model; among all the long dashed line segments intersecting the plane where the corneal model 51M10 is located, the long dashed line segments intersecting the plane where the first mirror model 301M10 is located to the plane where the Nth mirror model 310M10 is located are respectively used to represent the first iris-based first-segment light path model to the Nth iris-based first-segment light path model; among all the long dashed line segments intersecting the point where the light-gathering area model A10 of the first camera is located, the long dashed line segments intersecting the plane where the first mirror model 301M10 is located to the plane where the Nth mirror model 310M10 is located are respectively used to represent the first iris-based last-segment light path model to the Nth iris-based last-segment light path model; among all the double-dashed line segments intersecting the plane where the corneal model 51M10 is located, the double-dashed line segments intersecting the plane where the first mirror model 301M10 is located to the plane where the Nth mirror model 310M10 is located are respectively used to represent the first cornea-based first-segment light path model to the Nth cornea-based first-segment light path model; among all the double-dashed line segments intersecting the point where the light-gathering area model A10 of the first camera is located, the double-dashed line segments intersecting the plane where the first mirror model 301M10 is located to the plane where the Nth mirror model 310M10 is located are respectively used to represent the first cornea-based last-segment light path model to the Nth cornea-based last-segment light path model.

[0296] This application provides a method according to Fig. 20 the array composed of the images of the pupils in the first image corresponding to the 10th preset position array shown in Fig.21 and the light path generation device model based on the eyeball in the embodiment shown in Fig.21Examples of the first pupil-based first-segment optical path model to the Nth pupil-based first-segment optical path model and the first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model in the described embodiments; specifically, the geometric centers of 10 images 512T of the user's pupils included in the array composed of the images of the pupil are respectively set as 10 marked points, and respectively according to Fig. 20 the 10 marked points on the shown image, and according to the first camera model 111M10 through the method of determining the image-based light model mentioned above, determine 10 image-based light models respectively corresponding to the 10 marked points. The straight lines where the 10 image-based light models are located are intercepted into 10 line segments respectively by the planes where the first mirror model 301M10 to the Nth mirror model 310M10 are located and the points where the light collection area model A10 of the first camera is located, and the 10 line segments are respectively called the first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model. The line segments where the first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model are located are respectively used as incident light rays, and the planes included in the first mirror model 301M10 to the Nth mirror model 310M10 are used as reflection surfaces. According to the law of reflection of light, make 10 reflected light rays respectively corresponding to the first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model, and the straight lines where the 10 reflected light rays are located are intercepted into 10 line segments respectively by the planes where the first mirror model 301M10 to the Nth mirror model 310M10 are located and the plane where the cornea model 51M10 is located, and the 10 line segments are successively called the first pupil-based first-segment optical path model to the Nth pupil-based first-segment optical path model.

[0297] It should be noted that the first pupil-based first-segment optical path model to the Nth pupil-based first-segment optical path model and the first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model are all corresponding to the 10th preset position array. There are a total of L*M preset position arrays, and any one of the preset position arrays corresponds to a unique first pupil-based first-segment optical path model to the Nth pupil-based first-segment optical path model and a first pupil-based last-segment optical path model to the Nth pupil-based last-segment optical path model. The jth pupil-based first-segment optical path model and the jth pupil-based last-segment optical path model corresponding to the (i - 1)*M + k preset position array need to be determined according to the first image corresponding to the (i - 1)*M + k preset position array and the optical path generation device model based on the eyeball corresponding to the (i - 1)*M + k preset position array. The detailed determination method is similar and will not be elaborated here.

[0298] Further, in the embodiments of the present application, for any of the preset position arrays, the j-th pupil-based first-segment optical path model and the j-th pupil-based last-segment optical path model each correspond to a line segment and a length parameter measurement value. The length value corresponding to the j-th pupil-based first-segment optical path model of the ((i - 1)*M + k)-th preset position array is the length value of the line segment where the j-th pupil-based first-segment optical path model of the ((i - 1)*M + k)-th preset position array is located; the length value corresponding to the j-th pupil-based last-segment optical path model of the ((i - 1)*M + k)-th preset position array is the length value of the line segment where the j-th pupil-based last-segment optical path model of the ((i - 1)*M + k)-th preset position array is located; j takes any positive integer equal to N.

[0299] Step S75: According to the array composed of the images of the iris on the first image corresponding to each of the preset position arrays, and by using each of the device models for collecting eye data, determine the first iris-based first-segment optical path model to the N-th iris-based first-segment optical path model and the first iris-based last-segment optical path model to the N-th iris-based last-segment optical path model corresponding to each of the preset position arrays; wherein, the array composed of the images of the iris is the array composed of the images of the iris respectively included on the image of the mirror surface of the first reflector to the image of the mirror surface of the N-th reflector in the first image.

[0300] In the embodiments of the present application, each of the device models for collecting eye data is respectively the first device model for collecting eye data to the L*M-th device model for collecting eye data. Any one of the preset position arrays, including the first preset position array to the L*M-th preset position array, corresponds to N iris-based first-segment optical path models, including the first iris-based first-segment optical path model to the N-th iris-based first-segment optical path model, and N iris-based last-segment optical path models, including the first iris-based last-segment optical path model to the N-th iris-based last-segment optical path model.

[0301] In Fig. 20 the shown embodiment, the array composed of the images of the iris is the array composed of the images of the iris 513T respectively included on the image of the mirror surface of the first reflector 301 to the image of the mirror surface of the N-th reflector 310.

[0302] The present application provides a method according to Fig. 20 the array composed of the images of the iris in the first image corresponding to the 10th preset position array shown and Fig.21 the eye-based optical path generation device model corresponding to the 10th preset position array in the shown embodiment to determine Fig.21Embodiments of the first iris-based first-segment optical path model to the Nth iris-based first-segment optical path model and the first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model as shown; specifically, respectively according to 10 marker points 516T on 10 iris images 513T represented by dotted lines included in the array composed of the iris images, and according to the first camera model 111M10 through the method of determining the image-based light model mentioned above, determine 10 image-based light models corresponding to the 10 marker points 516T respectively. The straight lines where the 10 image-based light models are located are intercepted by the planes where the first mirror model 301M10 to the Nth mirror model 310M10 are located and the points where the light collection area model A10 of the first camera is located into 10 line segments, and the 10 line segments are respectively called the first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model. The line segments where the first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model are located are respectively used as incident light rays, and the planes included in the first mirror model 301M10 to the Nth mirror model 310M10 are used as reflection surfaces. According to the law of light reflection, make 10 reflected light rays corresponding to the first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model respectively. The straight lines where the 10 reflected light rays are located are intercepted by the planes where the first mirror model 301M10 to the Nth mirror model 310M10 are located and the plane where the cornea model 51M10 is located into 10 line segments in sequence, and the 10 line segments are called the first iris-based first-segment optical path model to the Nth iris-based first-segment optical path model in sequence.

[0303] It should be noted that the first iris-based first-segment optical path model to the Nth iris-based first-segment optical path model and the first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model are all corresponding to the 10th preset position array. There are a total of L * M preset position arrays, and any one of the preset position arrays corresponds to a unique first iris-based first-segment optical path model to the Nth iris-based first-segment optical path model and a first iris-based last-segment optical path model to the Nth iris-based last-segment optical path model. The jth iris-based first-segment optical path model and the jth iris-based last-segment optical path model corresponding to the (i - 1) * M + k preset position array need to be determined according to the first image corresponding to the (i - 1) * M + k preset position array and the eyeball-based optical path generation device model corresponding to the (i - 1) * M + k preset position array. The detailed determination method is similar and will not be elaborated here.

[0304] Further, in the embodiments of the present application, for any of the preset position arrays, the j-th iris-based first optical path model and the j-th iris-based last optical path model each correspond to a line segment and a length parameter measurement value. The length value corresponding to the j-th iris-based first optical path model of the ((i - 1)*M + k)-th preset position array is the length value of the line segment where the j-th iris-based first optical path model of the ((i - 1)*M + k)-th preset position array is located; the length value corresponding to the j-th iris-based last optical path model of the ((i - 1)*M + k)-th preset position array is the length value of the line segment where the j-th iris-based last optical path model of the ((i - 1)*M + k)-th preset position array is located; j takes any positive integer equal to N.

[0305] Still further, in Fig.21 the embodiments of, the marked points 516T on the image of the iris are marked by identifying the first features on the image on the image 513T of the iris according to the iris image described in step S45.

[0306] Step S76: Respectively according to the arrays composed of the first light spots on the first images corresponding to each of the preset position arrays, and respectively determine the first corneal-based first optical path model to the N-th corneal-based first optical path model and the first corneal-based last optical path model to the N-th corneal-based last optical path model corresponding to each of the preset position arrays through each of the device models for collecting eye data; wherein, the array composed of the first light spots is the array composed of the first light spots respectively included on the images of the mirrors of the first reflector to the N-th reflector in the first image, and the first light spot is the light spot formed in the first image by the light rays in the first detection light rays emitted by the first light-emitting element that are respectively reflected by the first reflector to the N-th reflector along the normal line of the user's eye cornea towards the user's eye cornea and then specularly reflected by the user's eye cornea along the original path and finally enter the lens of the first camera and are received. There is only one first light spot on the image of any one of the reflectors, including the images of the mirrors of the first reflector to the N-th reflector, in each of the first images.

[0307] In the embodiments of the present application, each of the device models for collecting eye data is the first device model for collecting eye data to the L*Mth device model for collecting eye data. Any one of the preset position arrays from the first preset position array to the L*Mth preset position array corresponds to N first-segment light path models based on the cornea from the first first-segment light path model based on the cornea to the Nth first-segment light path model based on the cornea, and N last-segment light path models based on the cornea from the first last-segment light path model based on the cornea to the Nth last-segment light path model based on the cornea.

[0308] In Fig. 20 the embodiment shown, the array composed of the first light spots is the array composed of the first light spots 514T represented by solid dots marked as 514T respectively included in the images of the mirrors of the first mirror 301 to the Nth mirror 310.

[0309] The present application provides a method according to Fig. 20 the array composed of the first light spots in the first image corresponding to the tenth preset position array shown and Fig.21 the device model of the light path generation device based on the eye corresponding to the tenth preset position array in the embodiment shown to determine Fig.21 the embodiments of the first first-segment light path model based on the cornea to the Nth first-segment light path model based on the cornea and the first last-segment light path model based on the cornea to the Nth last-segment light path model based on the cornea in the embodiment shown; specifically, the geometric centers of the 10 first light spots 514T included in the array composed of the first light spots 514T are respectively set as 10 marked points, and respectively according to Fig. 20The 10 marked points on the shown image, and according to the first camera model 111M10, by the method of determining the image-based light ray model mentioned above, determine the 10 image-based light ray models respectively corresponding to the 10 marked points. The planes where the first mirror model 301M10 to the Nth mirror model 310M10 are located and the points where the light ray convergence region model A10 of the first camera is located respectively intercept the straight lines where the 10 image-based light ray models are located into 10 line segments, and respectively call the 10 line segments the first corneal-based final light path model to the Nth corneal-based final light path model. Respectively take the line segments where the first corneal-based final light path model to the Nth corneal-based final light path model are located as incident light rays, take the planes included in the first mirror model 301M10 to the Nth mirror model 310M10 as reflection surfaces, make 10 reflected light rays respectively corresponding to the first corneal-based final light path model to the Nth corneal-based final light path model according to the law of reflection of light, and the planes where the first mirror model 301M10 to the Nth mirror model 310M10 are located and the plane where the corneal model 51M10 is located successively intercept the straight lines where the 10 reflected light rays are located into 10 line segments, and successively call the 10 line segments the first corneal-based initial light path model to the Nth corneal-based initial light path model.

[0310] It should be noted that the above first corneal-based initial light path model to the Nth corneal-based initial light path model, the first corneal-based final light path model to the Nth corneal-based final light path model are all corresponding to the 10th preset position array. There are a total of L*M preset position arrays, and any one of the preset position arrays corresponds to a unique first corneal-based initial light path model to the Nth corneal-based initial light path model, the first corneal-based final light path model to the Nth corneal-based final light path model. The jth corneal-based initial light path model and the jth corneal-based final light path model corresponding to the (i - 1)*M + k preset position array need to be determined according to the first image corresponding to the (i - 1)*M + k preset position array and the eyeball-based light path generation device model corresponding to the (i - 1)*M + k preset position array. The detailed determination method is similar and will not be elaborated here.

[0311] Further, in the embodiments of the present application, for any of the preset position arrays, the j-th corneal-based first-segment optical path model and the j-th corneal-based last-segment optical path model each correspond to a line segment and a length parameter measurement value. The length value corresponding to the j-th corneal-based first-segment optical path model of the ((i - 1)*M + k)-th preset position array is the length value of the line segment where the j-th corneal-based first-segment optical path model of the ((i - 1)*M + k)-th preset position array is located; the length value corresponding to the j-th corneal-based last-segment optical path model of the ((i - 1)*M + k)-th preset position array is the length value of the line segment where the j-th corneal-based last-segment optical path model of the ((i - 1)*M + k)-th preset position array is located; j takes any positive integer equal to N.

[0312] Still further, the present application provides an embodiment of finding the first light spot among multiple high-light spots included in the image of the mirror surface of any of the reflectors from the image of the mirror surface of the first reflector included in the first image to the image of the mirror surface of the N-th reflector. The method includes:

[0313] (1) Regarding the first camera model 111M in the application embodiment shown as Fig.21 the camera model C1 of the first measurement unit in the specific implementation part of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A; and regarding the mirror model array composed of the first mirror model 301M to the N-th mirror model 310M in the embodiments of the present application shown as Figure 5 the mirror model array composed of the first mirror model J1 to the ninth mirror model J9 in the specific implementation part of the patent document with the application number CN202310058420.4. Fig.21 shown as Figure 5 the mirror model array composed of the first mirror model J1 to the ninth mirror model J9 in the specific implementation part of the patent document with the application number CN202310058420.4.

[0314] (2) Regarding the arrays of multiple light spots represented by solid dots on the image of the mirror surface of the first mirror 301 in the application embodiment shown as Fig. 20 to the arrays of multiple light spots represented by solid dots on the image of the mirror surface of the ninth mirror 309 as the first-level array C1Z of reflection point models on the mirror models J1 to J9 in the specific implementation part of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A respectively. Figure 7 shown as

[0315] (3) According to the method in step Q2 of the specific implementation part of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A, the array of multiple light spots represented by solid dots on the image of the mirror surface of the first reflector 301 in the application embodiment shown in Fig. 20 to the array of multiple light spots represented by solid dots on the image of the mirror surface of the ninth reflector 309 is divided into the reflection point model secondary array described in step Q2 of the Chinese patent document with the application number CN202310058420.4.

[0316] As Fig. 22 shown, Fig. 22 This is a schematic diagram of a simplified view of the first image corresponding to the 10th preset position array provided in the embodiment of the present application. The simplified view includes: the image of the mirror surface of the first reflector 301 to the image of the mirror surface of the Nth reflector 310. Fig. 20 On the image of the mirror surface of any one of the reflectors, there are multiple light spots represented by solid dots. The light spots on the image of the mirror surface of any one of the reflectors form a reflection point model primary array. Each reflection point model primary array also includes multiple imaginary light spots represented by hollow circles and the reflection point model secondary array represented by a dotted line frame. Fig. 22 For the simplified view of Fig. 22 shown in Fig. 20 , the image of the mirror surface of the first reflector 301 to the image of the mirror surface of the Nth reflector 310 and the light spots shown in Fig. 22 are all extracted from Fig. 20 . The imaginary light spots shown in Fig. 22 are supplemented according to the light spots shown in Fig. 20 to determine the reflection point model secondary array. In the embodiment shown in Fig. 22 , the reflection point model secondary array includes 100. Each reflection point model secondary array is represented by a dotted line frame. Fig. 22 On the image of the mirror surface of any one of the reflectors in Fig.16 , there are 10 reflection point model secondary arrays, namely the first reflection point model secondary array 301Z, the second reflection point model secondary array 302Z, up to the tenth reflection point model secondary array 310Z, which are arranged in a circular ring. And the arrangement sequence and rule of the shown reflection point model secondary arrays on the mirror surface of each reflector are the same as those of the mirror surface of the first reflector 301 to the mirror surface of the Nth reflector 310 (only the shown reflection point model secondary arrays on the mirror surfaces of the first reflector 301, the second reflector 302, and the tenth reflector 310 are marked in , and the remaining reflection point model secondary arrays are not marked in the figure).

[0317] (4) Refer to the corresponding situation described in step Q3 of the Chinese patent document with the application number CN202310058420.4 and the publication number CN116817784A when the measured surface shape is the pure convex surface. In Fig. 22 each of the shown reflection point model secondary arrays, count in the order from left to right, and establish a one-to-one correspondence between the shown light spot ranked first and Fig.21 the first camera model 111M10 described in the shown application embodiment.

[0318] (5) Use Fig. 22 the light spots corresponding to the first camera model in the first reflection point model secondary array 301Z on the image of the mirror surface of the shown first reflector 301, the light spots corresponding to the first camera model in the second reflection point model secondary array 302Z on the image of the mirror surface of the second reflector 302, and so on until the light spots corresponding to the first camera model in the Nth reflection point model secondary array 310Z on the image of the mirror surface of the Nth reflector 310 as the first light spots respectively.

[0319] Fig. 20 In the shown embodiment, the solid dot marked as 514T is the first light spot 514T.

[0320] Among them, each of the optical path models includes: the optical path generation device model based on the eyeball, the corneal model, the visual axis model, the first to Nth pupil-based first-segment optical path models, the first to Nth pupil-based last-segment optical path models, the first to Nth iris-based first-segment optical path models, the first to Nth iris-based last-segment optical path models, the first to Nth corneal-based first-segment optical path models, and the first to Nth corneal-based last-segment optical path models corresponding to each of the preset position arrays. In the embodiment of the present application, the (i - 1)*M + k optical path model includes the optical path generation device model based on the eyeball, the corneal model, the visual axis model, the first to Nth pupil-based first-segment optical path models, the first to Nth pupil-based last-segment optical path models, the first to Nth iris-based first-segment optical path models, the first to Nth iris-based last-segment optical path models, the first to Nth corneal-based first-segment optical path models, and the first to Nth corneal-based last-segment optical path models corresponding to the (i - 1)*M + k preset position array.

[0321] It should be understood that this article does not limit the implementation sequence of the steps S72 to S76 herein.

[0322] In the method for determining the visual axis equation set, the above step S14 performs data analysis on all the optical path models, and the method for determining the visual axis equation set according to the data analysis results includes:

[0323] Step S81: By performing data analysis on each of the optical path models, determine the first distance parameter measurement value to the Nth distance parameter measurement value, the first angle parameter measurement value to the Nth angle parameter measurement value, and the first rotation angle parameter measurement value to the Nth rotation angle parameter measurement value corresponding to each of the optical path models.

[0324] In the embodiments of the present application, each of the optical path models is respectively the first optical path model to the L*M optical path model. Each of the optical path models can also be denoted as the (i - 1)*M + k optical path model, where i takes any positive integer less than or equal to the total number of groups L of the preset position array, and k takes any positive integer less than or equal to the total number of the preset position arrays M included in each group of the preset position array. Any one of the optical path models including the first optical path model to the L*M optical path model corresponds to the first distance parameter measurement value to the Nth distance parameter measurement value, the first angle parameter measurement value to the Nth angle parameter measurement value, and the first rotation angle parameter measurement value to the Nth rotation angle parameter measurement value; and the first distance parameter measurement value to the Nth distance parameter measurement value corresponding to any one of the optical path models is unique, and the first angle parameter measurement value to the Nth angle parameter measurement value corresponding to any one of the optical path models is unique, and the first rotation angle parameter measurement value to the Nth rotation angle parameter measurement value corresponding to any one of the optical path models is unique. For example, the first distance parameter measurement value corresponding to the first optical path model is not equal to the first distance parameter measurement value corresponding to the second optical path model, nor is it equal to the first distance parameter measurement value corresponding to any other optical path model.

[0325] Step S82: Determine the visual axis equation A according to the first distance parameter measurement value to the Nth distance parameter measurement value, the first angle parameter measurement value to the Nth angle parameter measurement value, the first rotation angle parameter measurement value to the Nth rotation angle parameter measurement value, and the first visual axis parameter A measurement value to the Nth visual axis parameter A measurement value corresponding to each of the optical path models; wherein, the visual axis equation A is used to determine the output value of the visual axis parameter A according to the actual measurement value of the input distance parameter, the actual measurement value of the angle parameter, and the actual measurement value of the rotation angle parameter. The output value of the visual axis parameter A is used to determine the visual axis for eye movement tracking, and the visual axis for eye movement tracking is used to implement the eye movement tracking method.

[0326] This application provides an embodiment for determining the visual axis equation A. The specific method includes:

[0327] (1) Determine the j-th data node AⅠ corresponding to each of the optical path models according to the measured value of the j-th included angle parameter, the measured value of the j-th rotation angle parameter, and the measured value of the j-th visual axis parameter A corresponding to each of the optical path models. Specifically, take the measured value of the j-th included angle parameter corresponding to the ((i - 1)*M + k)-th optical path model as the first independent variable included in the j-th data node AⅠ corresponding to the ((i - 1)*M + k)-th optical path model. In the embodiments of this application, the first independent variable included in the j-th data node AⅠ is written as xj, where j = 1, 2... N; take the measured value of the j-th rotation angle parameter corresponding to the ((i - 1)*M + k)-th optical path model as the second independent variable included in the j-th data node AⅠ corresponding to the ((i - 1)*M + k)-th optical path model. In the embodiments of this application, the second independent variable included in the j-th data node AⅠ is written as yj, where j = 1, 2... N; take the measured value of the j-th visual axis parameter A corresponding to the ((i - 1)*M + k)-th optical path model as the dependent variable included in the j-th data node AⅠ corresponding to the ((i - 1)*M + k)-th optical path model. In the embodiments of this application, the dependent variable included in the j-th data node AⅠ is written as aj, where j = 1, 2... N; j takes any positive integer less than or equal to N. When j takes values from 1 to N respectively, there are the 1st data node AⅠ to the Nth data node AⅠ corresponding to the ((i - 1)*M + k)-th optical path model. In the embodiments of this application, the j-th data node AⅠ corresponding to the ((i - 1)*M + k)-th optical path model is written as (xj, yj, aj), where j = 1, 2... N. In the embodiments of this application, the 1st data node AⅠ to the Nth data node AⅠ corresponding to any of the optical path models are respectively written as (x1, y1, a1) to (xn, yn, an).

[0328] It should be noted that the first data node AⅠ to the Nth data node AⅠ corresponding to any one of the optical path models are different from the first data node AⅠ to the Nth data node AⅠ corresponding to the optical path models other than this optical path model. For example, the first data node AⅠ corresponding to the first optical path model is written as (x1, y1, a1), and the first data node AⅠ corresponding to the second optical path model is also written as (x1, y1, a1). However, the first data node AⅠ corresponding to the first optical path model, that is, (x1, y1, a1), is different from the first data node AⅠ corresponding to the second optical path model, that is, (x1, y1, a1). Because x1, y1, and a1 in the first data node AⅠ corresponding to the first optical path model are not equal to x1, y1, and a1 in the first data node AⅠ corresponding to the second optical path model. The reason is that x1, y1, and a1 in the first data node AⅠ corresponding to the first optical path model are the measurement values of the first included angle parameter, the first rotation angle parameter, and the first visual axis parameter A corresponding to the first optical path model respectively, while x1, y1, and a1 in the first data node AⅠ corresponding to the second optical path model are the measurement values of the first included angle parameter, the first rotation angle parameter, and the first visual axis parameter A corresponding to the second optical path model respectively, and the measurement value of the first included angle parameter corresponding to the first optical path model is not equal to the measurement value of the first included angle parameter corresponding to the second optical path model, and the measurement value of the first rotation angle parameter corresponding to the first optical path model is not equal to the measurement value of the first rotation angle parameter corresponding to the second optical path model, and the measurement value of the first visual axis parameter A corresponding to the first optical path model is not equal to the measurement value of the first visual axis parameter A corresponding to the second optical path model.

[0329] As Fig.24 shown, Fig.24 is a schematic diagram of all the data nodes AⅠ corresponding to all the optical path models in a three-dimensional rectangular coordinate system where the horizontal axis, vertical axis, and height axis are set as the X axis, Y axis, and A axis respectively provided by an embodiment of the present application. It includes: the data nodes AⅠ within the dotted line coil 8001, up to the data nodes AⅠ within the dotted line coil 8015. Any one of the dotted line coils from the dotted line coil 8001 to the dotted line coil 8015 contains 10 points represented by "+" in the coordinate system. Fig.24In any of the embodiments shown, the points in the coordinate system represented by "+" are used to characterize the corresponding points of a data node AⅠ in the coordinate system. The data nodes AⅠ within the dashed-line coil 8001 to the data nodes AⅠ within the coil 8015 are respectively used to characterize the data nodes corresponding to the 1st to 15th optical path models in the embodiments of the present application. For example, the point labeled 800502 in the dashed-line coil 8006 is used to characterize the 2nd data node AⅠ corresponding to the 6th optical path model. This data node AⅠ is written as (x2, y2, a2) in the embodiments of the present application. If the measured value of the 2nd included angle parameter x2 = 0.1, the measured value of the 2nd rotation angle parameter y2 = 36, and the measured value of the 2nd visual axis parameter A a2 = 30 for the 6th optical path model, then this data node AⅠ is specifically (0.1, 36, 30). The distance of the point corresponding to this data node AⅠ from the YA plane axis in this coordinate system is equal to 0.1, the distance from the XA plane is equal to 36, and the distance from the XY plane is equal to 30. In the embodiments of the present application, there are 3 groups of optical path models, namely the 1st group of optical path models to the 3rd group of optical path models. The 1st group of optical path models includes the 1st to 5th optical path models, the 2nd group of optical path models includes the 6th to 10th optical path models, and the 3rd group of optical path models includes the 11th to 15th optical path models.

[0330] (2) Determine the i-th special visual axis equation A corresponding to the i-th group of optical path models through interpolation based on all the data nodes AⅠ corresponding to the i-th group of optical path models. Here, i is any positive integer less than or equal to the total number L of groups of optical path models. Specifically, based on all the data nodes AⅠ corresponding to each of the optical path models included in the i-th group of optical path models, determine the interpolation polynomial corresponding to the i-th group of optical path models through interpolation. The interpolation polynomial corresponding to the i-th group of optical path models passes through all the data nodes AⅠ corresponding to each of the optical path models included in the i-th group of optical path models. The interpolation polynomial corresponding to the i-th group of optical path models is called the i-th special visual axis equation A, where i is any positive integer less than or equal to L. When i takes values 1, 2, up to L respectively, there are the 1st special visual axis equation A, the 2nd special visual axis equation A, up to the L-th special visual axis equation A. In the embodiments of the present application, the i-th special visual axis equation A is written as: ∫Ai(x,y)=ai, i = 1, 2,...L, where i is any positive integer less than or equal to N. When i takes values 1, 2, up to L respectively, there are ∫A1(x,y)=a1, ∫A2(x,y)=a2, up to ∫AL(x,y)=aL; ∫A1, ∫A2, up to ∫AL are different operation rules respectively. The ∫Ai(x,y)=ai can be used to output a determined and unique value of ai after inputting any x value and y value within the input range. The input x value is called the actual measured value of the included angle parameter, and the input y value is called the actual measured value of the rotation angle parameter. The calculated value of ai is used to determine the output value of the visual axis parameter A, and the output value of the visual axis parameter A is used to implement the eye movement tracking scheme. In the embodiments of the present application, L is set to 3, that is to say, the special visual axis equations A included in the embodiments of the present application altogether include 3, namely the 1st special equation A, the 2nd special equation A, and the 3rd special equation A. In Fig.24 the shown embodiments of the application, there are the graphic shape T81 (represented by a grid) corresponding to the 1st special equation A in the coordinate system, the graphic shape T82 (represented by a grid) corresponding to the 2nd special equation A in the coordinate system, and the graphic shape T83 (represented by a grid) corresponding to the 3rd special equation A in the coordinate system. In Fig.24 the shown embodiments of the application, the graphic shape T81 corresponding to the 1st special equation A in the coordinate system passes through all the data nodes AⅠ corresponding to the 1st group of optical path models; the graphic shape T82 corresponding to the 2nd special equation A in the coordinate system passes through all the data nodes AⅠ corresponding to the 2nd group of optical path models; the graphic shape T81 corresponding to the 3rd special equation A in the coordinate system passes through all the data nodes AⅠ corresponding to the 3rd group of optical path models.

[0331] (3)Determine the $i$-th data node $A_{II}$ according to the measured value of the $i$-th distance parameter corresponding to the $i$-th optical path model and the $i$-th special equation $A$. Specifically, take the measured value of the $i$-th distance parameter corresponding to the $i$-th optical path model as the independent variable included in the $i$-th data node $A_{II}$. In the embodiments of the present application, the independent variable included in the $i$-th data node $A_{II}$ is written as $z_i$, where $i = 1, 2, \cdots, L$. Take the $a_i$ value corresponding to the special equation $A$ of the visual axis of the $i$-th optical path model as the dependent variable included in the $i$-th data node $A_{II}$. In the embodiments of the present application, the dependent variable included in the $i$-th data node $A_{II}$ is written as $a_i$, where $i = 1, 2, \cdots, L$. In the embodiments of the present application, the $i$-th data node $A_{II}$ is written as $(z_i, a_i)$, where $i = 1, 2, \cdots, L$. When $i$ takes values from $1$ to $L$ respectively, there are the 1st data node $A_{II}$ to the $L$-th data node $A_{II}$. In the embodiments of the present application, the 1st data node $A_{II}$ to the $L$-th data node $A_{II}$ are written as $(z_1, a_1)$ to $(z_L, a_L)$ respectively.

[0332] (4)Determine the visual axis equation $A$ by interpolation according to the 1st data node $A_{II}$ to the $L$-th data node $A_{II}$. Specifically, according to the 1st data node $A_{II}$ to the $L$-th data node $A_{II}$, that is, $(z_1, a_1)$ to $(z_L, a_L)$, determine the interpolation polynomial by interpolation. This interpolation polynomial passes through the $i$-th data node $A_{II}$, that is, the interpolation polynomial passes through $(z_i, a_i)$, where $i = 1, 2, \cdots, L$. The interpolation polynomial is called the visual axis equation $A$. In this embodiment, the visual axis equation $A$ is written as:

[0333] 。

[0334] The ∫Ai(x, y) = ai included in the visual axis equation A is a value that changes with the changes in the x value and the y value. When the determined x value and y value are substituted, the ai in the i-th data node (zi, ai), where i = 1, 2... L, will be determined. Consequently, the (zi, ai), where i = 1, 2... L, will be determined. Further, the rule ∫A’ in ∫A’(z) = a in the visual axis equation A will be determined. As long as the z values within the range are substituted into ∫A’(z) = a, a definite and unique value of a can be calculated; that is to say, as long as the x value, y value, and z value within the range are substituted into the visual axis equation A, the definite and unique value of a can be determined according to the visual axis equation A; the x value is called the actual measured value of the included angle parameter, the y value is called the actual measured value of the rotation angle parameter, the z value is called the actual measured value of the distance parameter, and the calculated a value is the output value of the visual axis parameter A. For the convenience of subsequent introduction, the visual axis equation A is abbreviated as ∫A(x, y, z) = a. The visual axis equation A is used to determine the output value of the visual axis parameter A, and the output value of the visual axis parameter A is used as the first parameter for determining a straight line in a three-dimensional coordinate system to determine a straight line. The determined straight line is called the visual axis for eye movement tracking, and the visual axis for eye movement tracking can be used to implement the eye movement tracking method. According to the general equation of a straight line Ax + By + Cz + D = 0 in a three-dimensional rectangular coordinate system, it can be known that at least 4 parameters are required to determine a straight line in a three-dimensional coordinate system. The first parameter for determining the straight line is one of these four parameters, and the other three of these four parameters are respectively called the second parameter for determining the straight line, the third parameter for determining the straight line, and the fourth parameter for determining the straight line in the embodiments of the present application.

[0335] Step S83: Determine the visual axis equation B according to the first distance parameter measurement value to the Nth distance parameter measurement value, the first included angle parameter measurement value to the Nth included angle parameter measurement value, the first rotation angle parameter measurement value to the Nth rotation angle parameter measurement value, and the first visual axis parameter B measurement value to the Nth visual axis parameter B measurement value corresponding to each of the optical path models; wherein, the visual axis equation B is used to determine the output value of the visual axis parameter B according to the actual measured value of the input distance parameter, the actual measured value of the included angle parameter, and the actual measured value of the rotation angle parameter. The output value of the visual axis parameter B is used to determine the visual axis for eye movement tracking, and the visual axis for eye movement tracking is used to implement the eye movement tracking method.

[0336] The present application provides an embodiment for determining the visual axis equation B. The specific method includes:

[0337] (1) Determine the j-th data node BⅠ corresponding to each of the optical path models according to the measured value of the j-th included angle parameter, the measured value of the j-th rotation angle parameter, and the measured value of the j-th visual axis parameter B corresponding to each of the optical path models. Specifically, for the detailed method of determining the j-th data node BⅠ corresponding to each of the optical path models, reference can be made to the embodiment of determining the j-th data node AⅠ corresponding to each of the optical path models described in step S82 herein. The detailed method will not be elaborated herein. It should be noted that the 1st data node BⅠ to the Nth data node BⅠ are determined according to the measured values of the 1st visual axis parameter B to the Nth visual axis parameter B respectively, rather than according to the measured values of the 1st visual axis parameter A to the Nth visual axis parameter A. In the embodiment of the present application, the j-th data node BⅠ corresponding to the (i - 1)*M + k optical path model is written as (xj, yj, bj), where j = 1, 2... N.

[0338] (2) Determine the i-th special visual axis equation B corresponding to the i-th group of optical path models by interpolation according to all the data nodes BⅠ corresponding to the i-th group of optical path models, where i takes any positive integer less than or equal to the total number L of groups of optical path models; specifically, for the detailed method of determining the i-th special visual axis equation B corresponding to the i-th group of optical path models, reference can be made to the embodiment of determining the i-th special visual axis equation A corresponding to the i-th group of optical path models described in step S82 herein. The detailed method will not be elaborated herein. It should be noted that the i-th special equation B corresponding to the i-th group of optical path models is determined according to the j-th data node BⅠ respectively, rather than according to the j-th data node AⅠ. In the embodiment of the present application, the i-th special visual axis equation A is written as: ∫Ai(x, y) = ai, where i = 1, 2,... L.

[0339] (3) Determine the i-th data node BⅡ according to the measured value of the i-th distance parameter corresponding to the i-th group of optical path models and the i-th special equation B; specifically, for the method of determining the i-th data node BⅡ, reference can be made to the embodiment of determining the i-th data node AⅡ described in step S82 herein. The detailed method will not be elaborated herein. It should be noted that the i-th data node BⅡ is determined according to the i-th special visual axis equation B, rather than according to the i-th special visual axis equation A. In the embodiment of the present application, the i-th data node BⅡ is written as (zi, ai), where i = 1, 2... L.

[0340] (4) Determine the visual axis equation B by interpolation from the first data node BⅡ to the L-th data node BⅡ; specifically, the method for determining the visual axis equation B can refer to the embodiment of determining the visual axis equation A by interpolation from the first data node AⅡ to the L-th data node AⅡ described in this step S82. It should be noted that the visual axis equation B is determined according to the first data node BⅡ to the L-th data node BⅡ, and no longer determined according to the first data node AⅡ to the L-th data node AⅡ. In the embodiment of the present application, the visual axis equation B is written as ∫B(x, y, z)=b, and ∫B(x, y, z)=b is used to calculate the value of b after substituting the x value, y value, and z value. The substituted x value is called the actual measurement value of the included angle parameter, the substituted y value is called the actual measurement value of the rotation angle parameter, the substituted z value is called the actual measurement value of the distance parameter, and the calculated value of b is called the output value of the visual axis parameter B. The output value of the visual axis parameter B is used as the second parameter of the determined straight line in the three-dimensional coordinate system.

[0341] Step S84: Determine the visual axis equation C according to the first distance parameter measurement value to the N-th distance parameter measurement value, the first included angle parameter measurement value to the N-th included angle parameter measurement value, the first rotation angle parameter measurement value to the N-th rotation angle parameter measurement value, and the first visual axis parameter C measurement value to the N-th visual axis parameter C measurement value corresponding to each optical path model; wherein, the visual axis equation C is used to determine the output value of the visual axis parameter C according to the actual measurement value of the input distance parameter, the actual measurement value of the included angle parameter, and the actual measurement value of the rotation angle parameter. The output value of the visual axis parameter C is used to determine the visual axis for eye movement tracking, and the visual axis for eye movement tracking is used to implement the eye movement tracking function.

[0342] The present application provides an embodiment for determining the visual axis equation C, and the specific method includes:

[0343] (1) Determine the j-th data node CⅠ corresponding to each optical path model according to the j-th included angle parameter measurement value, the j-th rotation angle parameter measurement value, and the j-th visual axis parameter C measurement value corresponding to each optical path model.

[0344] (2) Determine the i-th special visual axis equation C corresponding to the i-th group of optical path models by interpolation from all the data nodes CⅠ corresponding to the i-th group of optical path models, where i takes any positive integer less than or equal to the total number L of the optical path model groups.

[0345] (3) Determine the i-th data node CⅡ according to the i-th distance parameter measurement value corresponding to the i-th group of optical path models and the i-th special equation C.

[0346] (4)Determine the visual axis equation C by interpolation from the first data node CⅡ to the L-th data node CⅡ.

[0347] Specifically, for the detailed method of determining the visual axis equation C, reference can be made to the method of determining the visual axis equation A described in this step S82 or the method of determining the visual axis equation B. The detailed method of determining the visual axis equation C will not be elaborated herein. It should be noted that the visual axis equation C is determined based on the first data node CⅡ to the L-th data node CⅡ. Any one of the data nodes CⅡ, including the first data node CⅡ to the L-th data node CⅡ, is written as the i-th data node CⅡ, where i is a positive integer less than or equal to L. The i-th data node CⅡ is determined according to the i-th special equation C, and the i-th special equation C is determined according to all the data nodes CⅠ corresponding to the i-th optical path model. All the data nodes CⅠ are determined according to the measured values of the j-th included angle parameter, the j-th rotation angle parameter, and the j-th visual axis parameter C corresponding to each optical path model. In the embodiment of the present application, the visual axis equation C is written as ∫C(x, y, z)=c. ∫C(x, y, z)=c is used to calculate the value of c after substituting the x value, y value, and z value. The substituted x value is called the actually measured value of the included angle parameter, the substituted y value is called the actually measured value of the rotation angle parameter, the substituted z value is called the actually measured value of the distance parameter, and the calculated value of c is called the output value of the visual axis parameter C. The output value of the visual axis parameter C is used as the third parameter for determining the straight line in the three-dimensional coordinate system.

[0348] Step S85: Determine the visual axis equation D according to the first distance parameter measurement value to the N-th distance parameter measurement value, the first included angle parameter measurement value to the N-th included angle parameter measurement value, the first rotation angle parameter measurement value to the N-th rotation angle parameter measurement value, and the first visual axis parameter D measurement value to the N-th visual axis parameter D measurement value corresponding to each optical path model; wherein, the visual axis equation D is used to determine the output value of the visual axis parameter D according to the actually measured value of the input distance parameter, the actually measured value of the included angle parameter, and the actually measured value of the rotation angle parameter. The output value of the visual axis parameter D is used to determine the visual axis for eye movement tracking, and the visual axis for eye movement tracking is used to implement the eye movement tracking function.

[0349] The present application provides an embodiment for determining the visual axis equation D. The specific method includes:

[0350] (1)Determine the j-th data node BⅠ corresponding to each optical path model according to the measured values of the j-th included angle parameter, the j-th rotation angle parameter, and the j-th visual axis parameter D corresponding to each optical path model.

[0351] (2) Determine the i-th special visual axis equation D corresponding to the i-th group of optical path models by interpolation method based on all the data nodes D I corresponding to the i-th group of optical path models, where i is any positive integer less than or equal to the total number of optical path model groups L.

[0352] (3) Determine the i-th data node DⅡ according to the i-th distance parameter measurement value corresponding to the i-th group of optical path models and the i-th special equation D.

[0353] (4) Determine the line of sight equation D by interpolation based on the first data node DⅡ to the Lth data node DⅡ.

[0354] Specifically, the detailed method for determining the line of sight equation D may refer to the method for determining the line of sight equation A described in step S82 or the method for determining the line of sight equation B. As for the detailed method for determining the line of sight equation D, it will not be repeated herein. It should be noted that the line of sight equation D is determined based on the first data node DⅡ to the Lth data node DⅡ, and any data node DⅡ including the first data node DⅡ to the Lth data node DⅡ is written as the i-th data node DⅡ, where i is a positive integer less than or equal to L, and the i-th data node DⅡ is determined based on the i-th special equation D, and the i-th special equation D is determined based on all the data nodes DⅠ corresponding to the i-th group of optical path models, and all the data nodes DⅠ are determined based on the j-th angle parameter measurement value, the j-th rotation angle parameter measurement value, and the j-th line of sight parameter D measurement value corresponding to each of the optical path models. In an embodiment of the present application, the visual axis equation D is written as ∫D(x, y, z)=d, and the ∫D(x, y, z)=d is used to substitute the x value, y value and z value to calculate the value of d, the x value introduced is called the actual measured value of the angle parameter, the y value introduced is called the actual measured value of the rotation angle parameter, the z value introduced is called the actual measured value of the distance parameter, and the calculated value of d is called the visual axis parameter D output value, and the visual axis parameter D output value is used as the third parameter of the determined straight line in the three-dimensional coordinate system.

[0355] Step S86: Determine the visual axis equation group according to the visual axis equation A, the visual axis equation B, the visual axis equation C, and the visual axis equation D; wherein the visual axis equation group includes the visual axis equation A, the visual axis equation B, the visual axis equation C, and the visual axis equation D; specifically, in the embodiment of the present application, the visual axis equation A is ∫A(x, y, z)=a, the visual axis equation B is ∫∫B(x, y, z)=b, the visual axis equation C is ∫C(x, y, z)=c, and the visual axis equation D is ∫D(x, y, z)=d, constituting the visual axis equation group. In the embodiment of the present application, the visual axis equation group is written as:

[0356] .

[0357] It should be understood that this document does not limit the implementation order of the steps S82 to S85.

[0358] In the method for determining the visual axis equation group, the method of performing data analysis on each of the optical path models in step S81 to determine the first distance parameter measurement value to the Nth distance parameter measurement value, the first angle parameter measurement value to the Nth angle parameter measurement value, and the first rotation angle parameter measurement value to the Nth rotation angle parameter measurement value corresponding to each of the optical path models includes:

[0359] Step S91: Determine the jth distance parameter measurement value corresponding to each of the optical path models by analyzing the jth first segment optical path model based on the cornea and the jth last segment optical path model based on the cornea contained in each of the optical path models, or determine the jth distance parameter measurement value corresponding to each of the optical path models by analyzing the jth first segment optical path model based on the pupil and the jth last segment optical path model based on the pupil contained in each of the optical path models, or determine the jth distance parameter measurement value corresponding to each of the optical path models by analyzing the jth first segment optical path model based on the iris and the jth last segment optical path model based on the iris contained in each of the optical path models.

[0360] The present application provides three methods for determining the jth distance parameter measurement value corresponding to each optical path model. The method for determining the jth distance parameter measurement value corresponding to each optical path model is referred to as the method for determining the distance parameter measurement value hereinafter.

[0361] The first method for determining the distance parameter measurement value is: determining the jth distance parameter measurement value corresponding to each optical path model by analyzing the jth first segment optical path model based on the cornea and the jth last segment optical path model based on the cornea contained in each optical path model; specifically, adding the length value corresponding to the jth first segment optical path model based on the cornea contained in the (i-1)*M+k optical path model and the length value corresponding to the jth last segment optical path model based on the cornea contained in the (i-1)*M+k optical path model to obtain a value as the jth distance parameter measurement value corresponding to the (i-1)*M+k optical path model; j is any positive integer less than or equal to N.

[0362] The second method for determining the measured value of the distance parameter is as follows: the measured value of the j-th distance parameter corresponding to each of the optical path models is determined by analyzing the j-th pupil-based first-segment optical path model and the j-th pupil-based last-segment optical path model included in each of the optical path models; specifically, the value obtained by adding the length value corresponding to the j-th pupil-based first-segment optical path model included in the ((i - 1)*M + k)-th optical path model and the length value corresponding to the j-th pupil-based last-segment optical path model included in the ((i - 1)*M + k)-th optical path model is used as the measured value of the j-th distance parameter corresponding to the ((i - 1)*M + k)-th optical path model; j takes any positive integer less than or equal to N.

[0363] The third method for determining the measured value of the distance parameter is as follows: the measured value of the j-th distance parameter corresponding to each of the optical path models is determined by analyzing the j-th iris-based first-segment optical path model and the j-th iris-based last-segment optical path model included in each of the optical path models; specifically, the value obtained by adding the length value corresponding to the j-th iris-based first-segment optical path model included in the ((i - 1)*M + k)-th optical path model and the length value corresponding to the j-th iris-based last-segment optical path model included in the ((i - 1)*M + k)-th optical path model is used as the measured value of the j-th distance parameter corresponding to the ((i - 1)*M + k)-th optical path model; j takes any positive integer less than or equal to N.

[0364] It should be noted that in order to obtain accurate measured values of the distance parameter, in specific implementation, all the measured values of the distance parameter, including the measured values of the first distance parameter to the N-th distance parameter corresponding to all the optical path models, can only be determined by the same method for determining the measured value of the distance parameter.

[0365] Step S92: The measured value of the j-th included angle parameter corresponding to each of the optical path models is determined by analyzing the relative positions between the j-th pupil-based first-segment optical path model and the j-th cornea-based first-segment optical path model included in each of the optical path models, or the measured value of the j-th included angle parameter corresponding to each of the optical path models is determined by analyzing the relative positions between the j-th pupil-based last-segment optical path model and the j-th cornea-based last-segment optical path model included in each of the optical path models.

[0366] In the embodiments of the present application, two methods for determining the measured value of the j-th included angle parameter corresponding to each of the optical path models are provided. The method for determining the measured value of the j-th included angle parameter corresponding to each of the optical path models is hereinafter simply referred to as the method for determining the measured value of the included angle parameter.

[0367] The first method for determining the measured value of the included angle parameter is as follows: The measured value of the j-th included angle parameter corresponding to each of the optical path models is determined by analyzing the relative positions between the j-th pupil-based first-segment optical path model and the j-th cornea-based first-segment optical path model included in each of the optical path models; specifically, the angular value of the included angle between the line segment where the j-th pupil-based first-segment optical path model included in the ((i - 1)*M + k)-th optical path model is located and the line segment where the j-th cornea-based first-segment optical path model included in the ((i - 1)*M + k)-th optical path model is located is used as the measured value of the j-th included angle parameter corresponding to the ((i - 1)*M + k)-th optical path model; i takes any positive integer less than or equal to L, k takes any positive integer of M, and j takes any positive integer less than or equal to N.

[0368] The second method for determining the measured value of the included angle parameter is as follows: The measured value of the j-th included angle parameter corresponding to each of the optical path models is determined by analyzing the relative positions between the j-th pupil-based last-segment optical path model and the j-th cornea-based last-segment optical path model included in each of the optical path models; specifically, the angular value of the included angle between the line segment where the j-th pupil-based last-segment optical path model included in the ((i - 1)*M + k)-th optical path model is located and the line segment where the j-th cornea-based last-segment optical path model included in the ((i - 1)*M + k)-th optical path model is located is used as the measured value of the j-th included angle parameter corresponding to the ((i - 1)*M + k)-th optical path model; i takes any positive integer less than or equal to L, k takes any positive integer of M, and j takes any positive integer less than or equal to N.

[0369] Among them, the angular value of the included angle is specifically: Among the 4 included angles between the straight lines where two non-parallel and non-perpendicular line segments are located, it is the angular value of any one of the two equal included angles less than 90°.

[0370] Step S93: The measured value of the j-th rotation angle parameter corresponding to each of the optical path models is determined by analyzing the relative positions between the j-th pupil-based first-segment optical path model, the j-th cornea-based first-segment optical path model, and the j-th iris-based first-segment optical path model included in each of the optical path models.

[0371] The method for determining the measured value of the j-th rotation angle parameter corresponding to each of the optical path models is hereinafter simply referred to as the method for determining the measured value of the rotation angle parameter. The specific method includes:

[0372] (1) The plane passing through the line segment where the j-th pupil-based first-segment optical path model in the (i - 1)*M + k optical path model is located and passing through a point on the line segment where the j-th cornea-based first-segment optical path model in the (i - 1)*M + k optical path model is located is taken as the j-th Plane Ⅰ in the (i - 1)*M + k optical path model. In specific implementation, there are two cases where the line segments where the j-th pupil-based first-segment optical path model and the j-th cornea-based first-segment optical path model respectively in the (i - 1)*M + k optical path model are located are coplanar and approximately coplanar. When they are coplanar, any point on the line segment where the j-th cornea-based first-segment optical path model is located can be taken, and taking any point will not affect the spatial position of the j-th Plane Ⅰ. When they are approximately coplanar, to ensure the accuracy to the maximum extent, the point on the line segment where the j-th cornea-based first-segment optical path model is located is taken as the intersection point of the line segment where the j-th cornea-based first-segment optical path model is located and the curved surface where the cornea model is located. j takes any positive integer less than or equal to N.

[0373] (2) The plane passing through the line segment where the j-th pupil-based first-segment optical path model in the (i - 1)*M + k optical path model is located and passing through a point on the line segment where the j-th iris-based first-segment optical path model in the (i - 1)*M + k optical path model is located is taken as the j-th Plane Ⅱ in the (i - 1)*M + k optical path model. In specific implementation, there are two cases where the line segments where the j-th pupil-based first-segment optical path model and the j-th iris-based first-segment optical path model respectively in the (i - 1)*M + k optical path model are located are coplanar and approximately coplanar. When they are coplanar, any point on the line segment where the j-th iris-based first-segment optical path model is located can be taken, and taking any point will not affect the spatial position of the j-th Plane Ⅱ. When they are approximately coplanar, to ensure the accuracy to the maximum extent, the point on the line segment where the j-th iris-based first-segment optical path model is located is taken as the intersection point of the line segment where the j-th iris-based first-segment optical path model is located and the curved surface where the cornea model is located. j takes any positive integer less than or equal to N.

[0374] (3) The angle value required to rotate the j-th plane Ⅰ included in the (i - 1)*M + k optical path model around the j-th pupil-based first-segment optical path model included in the (i - 1)*M + k optical path model and along the j-th rotation direction indicated by the right-hand rule corresponding to the (i - 1)*M + k optical path model until it first coincides with the j-th plane Ⅱ included in the (i - 1)*M + k optical path model is taken as the measured value of the j-th rotation angle parameter corresponding to the (i - 1)*M + k optical path model. If the j-th plane Ⅰ included in the (i - 1)*M + k optical path model already coincides with the j-th plane Ⅱ included in the (i - 1)*M + k optical path model, then the measured value of the j-th rotation angle parameter corresponding to the (i - 1)*M + k optical path model is equal to zero. Among them, the j-th rotation direction indicated by the right-hand rule corresponding to the (i - 1)*M + k optical path model is specifically that the right hand holds the j-th pupil-based first-segment optical path model included in the (i - 1)*M + k optical path model, the thumb is straightened and points along the direction from the corneal model included in the (i - 1)*M + k optical path model to the j-th reflector model included in the (i - 1)*M + k optical path model, and the direction in which the four fingers are wound is the j-th rotation direction indicated by the right-hand rule corresponding to the (i - 1)*M + k optical path model; j takes any positive integer less than or equal to N.

[0375] In the methods for determining the measured value of the distance parameter, the measured value of the included angle parameter, and the measured value of the rotation angle parameter mentioned above, the (i - 1)*M + k optical path model can represent any one of the optical path models from the 1st optical path model to the L*M optical path models, and the structure of any one of the optical path models can refer to Fig.21 the structure of the 10th optical path model in the application embodiment shown.

[0376] Step S94: Determine the j-th three-dimensional coordinate system included in each optical path model according to the j-th pupil-based first-segment optical path model, the j-th cornea-based first-segment optical path model, and the corneal model included in each optical path model, and determine the measured value of the j-th visual axis parameter A, the measured value of the j-th visual axis parameter B, the measured value of the j-th visual axis parameter C, and the measured value of the j-th visual axis parameter D corresponding to each optical path model by analyzing the relative positions between the j-th three-dimensional coordinate system included in each optical path model and the visual axis model.

[0377] The method for determining the measured value of the j-th visual axis parameter A, the measured value of the j-th visual axis parameter B, the measured value of the j-th visual axis parameter C, and the measured value of the j-th visual axis parameter D corresponding to each optical path model is hereinafter simply referred to as the method for determining the measured value of the visual axis parameter. The method for determining the measured value of the visual axis parameter includes:

[0378] (1) Determine the j-th three-dimensional coordinate system included in each of the optical path models according to the j-th pupil-based first-segment optical path model, the j-th cornea-based first-segment optical path model, and the cornea model included in each of the optical path models.

[0379] The embodiments of the present application provide two methods for determining the j-th three-dimensional coordinate system included in each optical path model.

[0380] The first method for determining the j-th three-dimensional coordinate system included in each optical path model is as follows: Let the j-th three-dimensional coordinate system be a three-dimensional rectangular coordinate system including an X-axis, a Y-axis, and a Z-axis; Let the origin of the j-th three-dimensional coordinate system included in the (i - 1)*M + k optical path model be at the intersection of the line segment where the j-th cornea-based first-segment optical path model included in the (i - 1)*M + k optical path model is located and the plane where the cornea model included in the (i - 1)*M + k optical path model is located; Let the Z-axis of the j-th three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model coincide with the line segment where the j-th cornea-based first-segment optical path model included in the (i - 1)*M + k optical path model is located and the Z-axis of the j-th three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model points to the j-th reflector model included in the (i - 1)*M + k optical path model; Let the line where the Y-axis of the j-th three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model is located intersect the line segment where the j-th pupil-based first-segment optical path model included in the (i - 1)*M + k optical path model is located and the Y-axis of the j-th three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model points to the line segment where the j-th pupil-based first-segment optical path model included in the (i - 1)*M + k optical path model is located; j takes any positive integer less than or equal to N.

[0381] As Fig.23 shown, Fig.23Schematic diagram of the structure of some sub-models included in a 10th optical path model provided by an embodiment of the present application. Some sub-models included in the 10th optical path model shown include: a second mirror model 302M10, a cornea model 51M10, a visual axis model 511M10, a light-gathering area model A10 of a first camera, a 2nd pupil-based first-segment optical path model 611A2M10 (represented by a center line) and a 2nd pupil-based last-segment optical path model (represented by a center line) that intersects it on the plane where the second mirror model 302M10 is located, a 2nd iris-based first-segment optical path model 612A2M10 (represented by a long dash line) and a 2nd iris-based last-segment optical path model (represented by a long dash line) that intersects it on the plane where the second mirror model 302M10 is located, a 2nd cornea-based first-segment optical path model 613A2M10 (represented by a double-dot dash line) and a 2nd cornea-based last-segment optical path model (represented by a double-dot dash line) that intersects it on the plane where the second mirror model 303M10 is located, and a 2nd three-dimensional rectangular coordinate system 32W10. Except for the 2nd three-dimensional rectangular coordinate system 32W10, Fig.23 the above-mentioned models shown are respectively based on Fig.21 the second mirror model 302M10, the cornea model 51M10, the visual axis model 511M10, the light-gathering area model A10 of the first camera, the 2nd pupil-based first-segment optical path model 611A2M10, the 2nd pupil-based last-segment optical path model, the 2nd iris-based first-segment optical path model 612A2M10, the 2nd iris-based last-segment optical path model, the 2nd cornea-based first-segment optical path model 613A2M10, and the 2nd cornea-based last-segment optical path model shown in Fig.23 and Fig.21 due to different viewing angles, so Fig.23 the models shown in

[0382] are different in observation from the models shown in 21. Fig.23In the illustrated embodiment, an embodiment of determining the second three-dimensional rectangular coordinate system 32W10 included in the 10th optical path model is provided. The origin of the second three-dimensional rectangular coordinate system 32W10 included in the 10th optical path model is the intersection point of the line segment where the second iris-based first-segment optical path model 612A2M10' is located and the surface where the corneal model 51M10' is located; the Z-axis of the second three-dimensional rectangular coordinate system 32W10 included in the 10th optical path model coincides with the line segment where the second iris-based first-segment optical path model 612A2M10' is located and points to the second mirror model 32M10'; the straight line where the Y-axis of the second three-dimensional rectangular coordinate system 32W10 included in the 10th optical path model is located intersects the line segment where the second pupil-based first-segment optical path model 611A2M10' is located and points to the line segment where the second pupil-based first-segment optical path model 611A2M10' is located. The determination method of the jth three-dimensional rectangular coordinate system included in the remaining (i - 1)*M + k optical path models is similar and will not be elaborated herein.

[0383] The second method for determining the jth three-dimensional coordinate system included in each optical path model is as follows: Let the jth three-dimensional coordinate system be a three-dimensional rectangular coordinate system including an X-axis, a Y-axis, and a Z-axis; let the origin of the jth three-dimensional coordinate system included in the (i - 1)*M + k optical path model be at the intersection point of the line segment where the jth pupil-based first-segment optical path model included in the (i - 1)*M + k optical path model is located and the plane where the corneal model included in the (i - 1)*M + k optical path model is located; let the Z-axis of the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model coincide with the line segment where the jth pupil-based first-segment optical path model included in the (i - 1)*M + k optical path model is located and the Z-axis of the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model points to the jth mirror model included in the (i - 1)*M + k optical path model; let the straight line where the Y-axis of the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model is located intersect the line segment where the jth cornea-based first-segment optical path model included in the (i - 1)*M + k optical path model is located and the Y-axis of the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model points to the line segment where the jth cornea-based first-segment optical path model included in the (i - 1)*M + k optical path model is located; j takes any positive integer less than or equal to N.

[0384] It should be noted that, in order to obtain the correct measurement value of the visual axis parameter, in the specific implementation, the first to the Nth three-dimensional coordinate systems corresponding to all the optical path models can only be determined by the same method of determining the jth three-dimensional coordinate system included in each optical path model. Only the correct measurement value of the visual axis parameter can determine the correct visual axis equation set, and only the correct visual axis equation set can implement the eye movement tracking method.

[0385] The embodiments of the present application provide two methods for determining the jth three-dimensional coordinate system included in each optical path model. In practical applications, various transformations can be performed according to the methods for determining the jth three-dimensional coordinate system included in each optical path model provided by the embodiments of the present application. For example, equal translation or rotation can be performed on the already determined first to the Nth three-dimensional rectangular coordinate systems included in all the optical path models, or the already determined first to the Nth three-dimensional rectangular coordinate systems included in all the optical path models can be replaced with spherical coordinate systems, cylindrical coordinate systems, etc. The above transformation methods can all achieve the purpose of determining the first to the Nth three-dimensional coordinate systems in each optical path model. It should be understood that the methods for determining the first to the Nth three-dimensional coordinate systems in each optical path model that can achieve the same effect as the methods for determining the jth three-dimensional coordinate system included in each optical path model described in this article are not limited herein.

[0386] (2) Determine the measurement values of the jth visual axis parameter A, the jth visual axis parameter B, the jth visual axis parameter C, and the jth visual axis parameter D corresponding to each optical path model by analyzing the relative positions between the jth three-dimensional coordinate system included in each optical path model and the visual axis model.

[0387] In the embodiments of the present application, the first visual axis parameter measurement value A to the Nth visual axis parameter measurement value A, the first visual axis parameter measurement value B to the Nth visual axis parameter measurement value B, the first visual axis parameter measurement value C to the Nth visual axis parameter measurement value C, and the first visual axis parameter measurement value D to the Nth visual axis parameter measurement value D corresponding to each optical path model are simply referred to as: all visual axis parameter measurement values corresponding to each optical path model. In the embodiments of the present application, two methods for determining all visual axis parameter measurement values corresponding to each optical path model are provided. The method for determining all visual axis parameter measurement values corresponding to each optical path model is simply referred to as the method for determining the visual axis parameter measurement value hereinafter.

[0388] The first method for determining the measured value of the visual axis parameter: Among the first three-dimensional rectangular coordinate system to the Nth three-dimensional rectangular coordinate system and the straight line where the visual axis model is located in any one of the optical path models from the first optical path model to the L*M optical path model, the x coordinate value of the intersection point of the plane corresponding to the equation Z = d1 in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model and the straight line where the visual axis model is located in the (i - 1)*M + k optical path model in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model is taken as the corresponding value of the (i - 1)*M + k optical path model.

[0389] The measured value of the (i - 1)*M*N + (k - 1)*N + j visual axis parameter A; the y coordinate value of the intersection point of the plane corresponding to the equation Z = d1 in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model and the straight line where the visual axis model is located in the (i - 1)*M + k optical path model in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model is taken as the measured value of the jth visual axis parameter B corresponding to the (i - 1)*M + k optical path model; the x coordinate value of the intersection point of the plane corresponding to the equation Z = d2 in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model and the straight line where the visual axis model is located in the (i - 1)*M + k optical path model in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model is taken as the measured value of the jth visual axis parameter C corresponding to the (i - 1)*M + k optical path model; the y coordinate value of the intersection point of the plane corresponding to the equation Z = d2 in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model and the straight line where the visual axis model is located in the (i - 1)*M + k optical path model in the jth three-dimensional rectangular coordinate system included in the (i - 1)*M + k optical path model is taken as the measured value of the jth visual axis parameter D corresponding to the (i - 1)*M + k optical path model; where i takes any positive integer less than or equal to L, k takes any positive integer less than or equal to M, and j takes any positive integer of N. d1 in the equation Z = d1 takes any real number, and d2 in the equation Z = d2 takes any real number not equal to d1. For example, in practical applications, for the convenience of calculation, d1 can be set to 0 and d2 can be set to 100.

[0390] The second method for determining the measured value of the visual axis parameter is as follows: Use the expression of the general equation of the line of ((i - 1)*M*N+(k - 1)*N + j) to describe the line where the visual axis model in the j-th three-dimensional rectangular coordinate system included in the optical path model of (i - 1)*M + k is located, and use the coefficients in the expression of the general equation of the line of ((i - 1)*M*N+(k - 1)*N + j) as the j-th visual axis parameter A, the j-th visual axis parameter B, the j-th visual axis parameter C, and the j-th visual axis parameter D corresponding to the optical path model of (i - 1)*M + k. Specifically, in the embodiments of the present application, the expression of the general equation of the line is written as Ax + By + Cz + D = 0, and the expression of the general equation of the line of ((i - 1)*M*N+(k - 1)*N + j) is written as , i = 1, 2,...L, k = 1, 2,...M, j = 1, 2,...N. The expression of the general equation of the line of ((i - 1)*M*N+(k - 1)*N + j) contains 4 coefficients, and these 4 coefficients are the coefficient , the coefficient , the coefficient , and the coefficient . Use the coefficient , use the coefficient , use the coefficient , and use the coefficient as the measured value of the j-th visual axis parameter A, the measured value of the j-th visual axis parameter B, the measured value of the j-th visual axis parameter C, and the measured value of the j-th visual axis parameter D corresponding to the optical path model of (i - 1)*M + k, respectively. Among them, i takes any positive integer less than or equal to L, k takes any positive integer less than or equal to M, and j takes any positive integer less than or equal to N.

[0391] Among them, j takes any positive integer less than or equal to the total number N of the adjustment components. When j takes 1 to N respectively, there are the measured value of the first distance parameter, the measured value of the first included angle parameter, the measured value of the first rotation angle parameter, the measured value of the first visual axis parameter A, the measured value of the first visual axis parameter B, the measured value of the first visual axis parameter C, and the measured value of the first visual axis parameter D corresponding to each optical path model, up to the measured value of the N-th distance parameter, the measured value of the N-th included angle parameter, the measured value of the N-th rotation angle parameter, the measured value of the N-th visual axis parameter A, the measured value of the N-th visual axis parameter B, the measured value of the N-th visual axis parameter C, and the measured value of the first visual axis parameter D.

[0392] It should be noted that during specific implementation, in order to obtain the correct measurement value of the visual axis parameter, all the measurement values of the visual axis parameters corresponding to each optical path model can only be determined by the same method for determining all the measurement values of the visual axis parameters corresponding to each optical path model. Only the correct measurement value of the visual axis parameter can determine the correct visual axis equation set, and only the correct visual axis equation set can realize the eye movement tracking function.

[0393] The embodiments of the present application provide two methods for determining the measurement value of the visual axis parameter. Essentially, these two methods for determining the measurement value of the visual axis parameter respectively use the position coordinate values of two points in space as the measurement value of the visual axis parameter to describe a straight line in space, and use the coefficients in the general expression of a straight line in a three-dimensional rectangular coordinate system as the measurement value of the visual axis parameter to describe a straight line in space. In practical applications, there are also various methods that can achieve the purpose of describing a straight line in space through the measurement value of the visual axis parameter. For example, using the position coordinate and slope value of a point in space as the measurement value of the visual axis parameter to achieve the purpose of describing a straight line in space. It should be understood that this application does not limit the methods that can achieve the purpose of determining all the measurement values of the visual axis parameters corresponding to each optical path model. It should also be understood that this application does not limit the implementation sequence of steps 91 to 94.

[0394] In practical applications, with the help of a measurement unit and a reflector, and through the visual axis equation set, an eye movement tracking scheme can be realized. In the following text, this eye movement tracking scheme is called Scheme One. The following is an introduction to Scheme One:

[0395] As Fig.25 shown, Fig.25 This is a schematic diagram of a method for using the visual axis equation set provided by the embodiments of the present application. When the user's eyeball 51 gazes at an object 76 in reality, and the third light-emitting element 732 included in the third measurement unit 73 emits third detection light to the outside, and the light diffusely reflected by the user's eyeball 51 can be specularly reflected by the mirror surface 75 and can enter the lens of the third camera 731 included in the third measurement unit 73, an initial optical path '611A based on the pupil (represented by the center line), a final optical path '611B based on the pupil (represented by the center line), an initial optical path '612A based on the iris (represented by the long dash line), a final optical path '612B based on the iris (represented by the long dash line), an initial optical path '613A based on the cornea (represented by the double dash line), and a final optical path '613B based on the cornea (represented by the double dash line) will be generated.

[0396] Among them, the third measurement unit 73 includes the third camera 731 and the third light-emitting element 732. The structure of the third measurement unit 73 is exactly the same as that of the first measurement unit 11. The third camera 731 included in the third measurement unit 73 is equivalent to the first camera 111 included in the first measurement unit 11, and the third light-emitting element 732 included in the third measurement unit 73 is equivalent to the first light-emitting element 112 included in the first measurement unit 11; the mirror surface 75 is the mirror surface of a reflector, and the mirror surface 75 can specularly reflect the incident light; the first-stage light path '611A based on the pupil is the path that the light diffusely reflected from the pupil of the user's eyeball 51 in all directions and can be specularly reflected by the mirror surface 75 and enter the lens of the third camera 731 and be received travels in space from the time it is diffusely reflected from the pupil of the user's eyeball 51 until before it is specularly reflected by the mirror surface 75. This path overlaps a line segment in space; the second-stage light path '611B based on the pupil is the path that the light in the first-stage light path '611A based on the pupil travels in space from the time it is specularly reflected by the mirror surface 75 until it enters the lens of the third camera 731. This path overlaps a line segment in space; the first-stage light path '612A based on the iris is the path that the light diffusely reflected from the first feature on the iris of the user's eyeball 51 in all directions and can be specularly reflected by the mirror surface 75 and enter the lens of the third camera 731 and be received travels in space from the time it is diffusely reflected from the first feature on the iris of the user's eye until before it is specularly reflected by the mirror surface 75. This path overlaps a line segment in space; the second-stage light path '612B based on the iris is the path that the light in the first-stage light path '612A based on the iris travels in space from the time it is specularly reflected by the mirror surface 75 until it enters the lens of the third camera 731. This path overlaps a line segment in space; the second-stage light path '613A based on the cornea is the path that the first detection light rays emitted from the first light-emitting element in all directions and can be specularly reflected by the mirror surface 75 and travel along the normal of the cornea of the user's eye 51D towards the cornea of the user's eye 51 travel in space from the time they are emitted from the first light-emitting element 732 until before they are specularly reflected by the mirror surface 75. This path overlaps a line segment in space; the first-stage light path '613B based on the cornea is the path that the light in the second-stage light path '613A based on the cornea travels in space from the time it is specularly reflected by the mirror surface 75 until it irradiates the cornea of the user's eyeball 51. This path overlaps a line segment in space.

[0397] As long as it is possible to model in the virtual three-dimensional space simulated by a computer the 'first-segment light path' model based on the pupil, the 'last-segment light path' model based on the pupil, the 'first-segment light path' model based on the iris, the 'last-segment light path' model based on the iris, the 'first-segment light path' model based on the cornea, the 'last-segment light path' model based on the cornea, the mirror model, the cornea model of the user's eyeball, and the object model, which are respectively used to restore the real 'first-segment light path' 611A based on the pupil, the 'last-segment light path' 611B based on the pupil, the 'first-segment light path' 612A based on the iris, the 'last-segment light path' 612B based on the iris, the 'first-segment light path' 613A based on the cornea, the 'last-segment light path' 613B based on the cornea, the mirror 75, the cornea of the user's eyeball 51, and the object 76, it is possible to calculate the fixation point of the user's eyeball 51 on the object 76. This fixation point is used as the output result of the first solution mentioned in this application, thereby realizing the eye movement tracking function.

[0398] Specifically, after modeling in the virtual three-dimensional space simulated by a computer the 'first-segment light path' model based on the pupil, the 'last-segment light path' model based on the pupil, the 'first-segment light path' model based on the iris, the 'last-segment light path' model based on the iris, the 'first-segment light path' model based on the cornea, the 'last-segment light path' model based on the cornea, the mirror model, the cornea model of the user's eyeball, and the object model, the method for calculating the fixation point of the user's eyeball 51 on the object 76 includes the following steps:

[0399] (1) In this virtual three-dimensional space simulated by a computer, the sum of the length values of the line segments where the 'first-segment light path' 613A based on the cornea is located and the length values of the line segments where the 'last-segment light path' 613B based on the cornea is located is used as the actual measured value of the distance parameter.

[0400] (2) In this virtual three-dimensional space simulated by a computer, any one of the two angles with an angle value less than 90° among the four angles formed between the straight line where the line segment of the 'first-segment light path' model based on the pupil is located and the straight line where the line segment of the 'first-segment light path' model based on the cornea is located is used as the actual measured value of the angle parameter.

[0401] (3) In the virtual three-dimensional space simulated by the computer, a plane I' is determined according to the pupil-based first-segment light path' model and the cornea-based first-segment light path' model, and a plane II' is determined according to the pupil-based first-segment light path' model and the iris-based first-segment light path' model. Taking the straight line where the pupil-based first-segment light path' model is located as the rotation axis, holding the straight line where the pupil-based first-segment light path' model is located with the right hand, straightening the thumb and pointing it along the direction from the cornea of the user's eyeball 51 to the reflector A, and taking the direction of the four fingers' surrounding as the rotation direction, the rotation angle value required to rotate the plane I' until it first coincides with the plane II' is used as the actual measured value of the rotation angle parameter. If the plane I' and the plane II' already coincide, the actual measured value of the rotation angle parameter is equal to zero. Among them, the plane I' is a plane passing through the straight line where the pupil-based first-segment light path' 611A is located and passing through the intersection point I'. The intersection point I' is the intersection point of the straight line where the cornea-based first-segment light path' 613A is located and the curved surface where the cornea of the user's eye 51 is located. The different plane II' is a plane passing through the straight line where the pupil-based first-segment light path' 611A is located and passing through the intersection point II'. The intersection point II' is the intersection point of the straight line where the iris-based first-segment light path' 612A is located and the curved surface where the cornea of the user's eye 51 is located.

[0402] (4) In the virtual three-dimensional space simulated by the computer, the pupil-based first-segment light path' model and the cornea-based first-segment light path' model are used as references to determine a three-dimensional rectangular coordinate system'. The three-dimensional rectangular coordinate system' is the same as the j-th three-dimensional rectangular coordinate system described in step S94. The pupil-based first-segment light path' model and the cornea-based first-segment light path' model can be regarded as the j-th pupil-based first-segment light path model and the j-th cornea-based first-segment light path model described in step S94 respectively, and the three-dimensional rectangular coordinate system' is determined according to the method of determining the j-th three-dimensional coordinate system included in each light path model described in step S94 based on the pupil-based first-segment light path' model and the cornea-based first-segment light path' model.

[0403] (5) Substitute the measured distance parameter measured value, the included angle parameter measured value, and the rotation angle parameter measured value corresponding to the actual application into the visual axis equations A, B, C, and D respectively, and the output values of the visual axis parameters A, B, C, and D can be calculated respectively. According to the three-dimensional rectangular coordinate system' included in the virtual three-dimensional space simulated by the computer and the calculated output values of the visual axis parameters A, B, C, and D, a straight line is determined, and this straight line is the visual axis model for eye movement tracking.

[0404] Take the intersection point of the straight line where the visual axis line model for eye tracking is located and the object as the calculated fixation point of the user's eyeball 51 on the object 76, and use this fixation point as the output result of the eye tracking method mentioned in this application, thereby realizing the eye tracking function. Currently, the application of the eye tracking function is extensive, such as advertising analysis, psychological research, controlling terminal devices, etc. Taking the control of terminal devices as an example, restoring the computer screen in the vi...

Claims

1. An eyeball-based optical path generating device, characterized in that, The eye-based optical path generating device includes: An image acquisition component; the image acquisition component at least includes a first measurement unit and a second measurement unit; the first measurement unit includes a first camera and a first light-emitting element; the second measurement unit includes a second camera and a second light-emitting element; N adjustment components, which are sequentially the first adjustment component to the Nth adjustment component, and the first reflector to the Nth reflector sequentially installed on the first adjustment component to the Nth adjustment component; N is an integer greater than or equal to 1; A reflector for reflecting the fixation target or a camera serving as the fixation target; the eye-based optical path generating device at least includes any one of the reflector for reflecting the fixation target and the camera serving as the fixation target; the reflector for reflecting the fixation target is used to specularly reflect the light diffusely reflected by the user's eyeball into the lens of the first camera so that the first camera can receive the light diffusely reflected by the user's eyeball and incident into the lens of the first camera, and the reflector for reflecting the fixation target is also used to specularly reflect the light diffusely reflected by the first camera at various angles into the pupil of the user's eyeball, so that the user can see and fixate on the image of the lens of the first camera reflected in the mirror surface of the reflector for reflecting the fixation target; the camera serving as the fixation target is used to acquire an image of the user's eyes when the user's eyeball fixates on the lens of the camera serving as the fixation target, and the image of the user's eyes at least includes an image of the pupil and an image of the iris; A connecting member; the connecting member is used to mount the image acquisition component, the first adjustment component to the Nth adjustment component, the reflector for reflecting the fixation target or the camera serving as the fixation target; Wherein, when the user's eyeball is in a specific spatial region and fixates on the image of the lens of the first camera reflected in the mirror surface of the reflector for reflecting the fixation target or fixates on the lens of the camera serving as the fixation target, the image acquisition component is used to acquire the light reflected on the mirror surfaces of the reflectors including the first reflector to the Nth reflector and the reflector for reflecting the fixation target, thereby forming an image of the user's eyes in the mirror surface; the first adjustment component to the Nth adjustment component are respectively used to adjust the spatial positions of the first reflector to the Nth reflector; the camera serving as the fixation target is used to acquire an image of the user's eyes; the image of the user's eyes in the mirror surface and the image of the user's eyes are used to determine an optical path model; the optical path model is used to determine an optical axis equation set; the optical axis equation set is used to implement a method of eye movement tracking; if the eye-based optical path generating device does not include a camera serving as the fixation target, the optical path model is determined only by the image of the user's eyes in the mirror surface.

2. The eye-based optical path generating device according to claim 1, wherein The first adjustment component to the Nth adjustment component at least respectively include a first adjustment module to an Nth adjustment module, and the first adjustment component to the Nth adjustment component may further include a displacement adjustment member; when the first adjustment component to the Nth adjustment component includes the displacement adjustment member, the first adjustment component to the Nth adjustment component are mounted on the displacement adjustment member, and the displacement adjustment member is mounted on the connection member; when the first adjustment component to the Nth adjustment component does not include the displacement adjustment member, the first adjustment component to the Nth adjustment component are mounted on the connection member; Wherein, the first reflector to the Nth reflector are respectively mounted on the first adjustment module to the Nth adjustment module, and the first adjustment module to the Nth adjustment module are respectively used to adjust the spatial positions of the first reflector to the Nth reflector.

3. The optical path generation device based on the eyeball according to claim 2, characterized in that Any one of the adjustment modules including the first adjustment module to the Nth adjustment module includes: G movement guiding members, and these G movement guiding members are sequentially the first movement guiding member to the Gth movement guiding member, where G is an integer greater than or equal to 1; The first movement guiding member is mounted on the displacement adjustment member or on the connection member, and the (v + 1)th movement guiding member is movably mounted on the vth movement guiding member, where v is a positive integer not greater than G; The first reflector to the Nth reflector are respectively mounted on the Gth movement guiding member included in the first adjustment module to the Gth movement guiding member included in the Nth adjustment module; Wherein, the (v + 1)th movement guiding member can slide or rotate relative to the vth movement guiding member under the action of a driving force.

4. The eyeball-based optical path generating device according to claim 2, wherein The first adjustment component to the Nth adjustment component further include: a position sensor, and the position sensor is used to measure the spatial positions of the first reflector to the Nth reflector to obtain the spatial position information of the first reflector to the Nth reflector; the spatial position information of the first reflector to the Nth reflector is used to determine the optical path model.

5. The optical path generation device based on the eyeball according to claim 1, wherein The image acquisition component includes: A first measurement unit, and the first measurement unit includes: a first camera and a first light-emitting element; the first camera is used to acquire a first image; the first light-emitting element at least includes a first light-emitting portion, and the first light-emitting portion is used to emit first detection light in a scattered manner; A second measurement unit, and the second measurement unit includes: a second camera and a second light-emitting element; the second camera is used to acquire a second image; the second light-emitting element at least includes a second light-emitting portion, and the second light-emitting portion is used to emit second detection light in a scattered manner; Wherein, the user eye image in the mirror surface includes the first image and the second image; at different time points, the image acquisition component will acquire different user eye images in the mirror surface, and different user eye images in the mirror surface include different first images and second images.

6. The first measurement unit and the second measurement unit according to claim 5, characterized in that, Among the first camera and the first light-emitting element included in the first measurement unit, the geometric center of the first light-emitting portion included in the first light-emitting element is located within the light-gathering area of the first camera; among the second camera and the second light-emitting element included in the second measurement unit, the geometric center of the second light-emitting portion included in the second light-emitting element is located within the light-gathering area of the second camera; Wherein, the light-gathering area of any one of the cameras including the first camera and the second camera is a spatial area where the straight lines where all the light rays that can enter the camera lens and can be received by the photosensitive element included in the camera can intersect with each other. The light-gathering area of any one of the cameras is a spatial area inside the camera.

7. The eyeball-based optical path generating device according to claim 5, characterized in that The first light-emitting element is used to emit first detection light rays; the second light-emitting element is used to emit second detection light rays; the first mirror to the Nth mirror and the mirror for reflecting the fixation target are used to specularly reflect the first detection light rays and the second detection light rays to the cornea of the user's eyeball, and specularly reflect the first detection light rays and the second detection light rays specularly reflected by the cornea of the user's eyeball into the lenses of the first camera and the second camera again; the first mirror to the Nth mirror and the mirror for reflecting the fixation target are also used to specularly reflect the light rays diffusely reflected by the pupil and iris of the user's eyeball into the lens of the first camera; the first camera forms a first image based on all the first detection light rays specularly reflected into the lens of the first camera, all the second detection light rays specularly reflected into the lens of the first camera, and the light rays diffusely reflected by the pupil and iris of the user's eyeball specularly reflected into the lens of the first camera; The second camera forms a second image based on all the first detection light rays specularly reflected into the lens of the second camera and all the second detection light rays specularly reflected into the lens of the second camera; Wherein, the first image includes: the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target. Any one of the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target included in the first image includes the image of the pupil, the image of the iris, and a light spot; the second image includes: the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target. All or part of the images of the mirrors of the first mirror to the Nth mirror and the mirror for reflecting the fixation target included in the second image include the image of the pupil, the image of the iris, and a light spot.

8. The eye-based optical path generating device according to any one of claims 1-7, characterized in that, It further includes a computer, which at least includes a controller and an information processor; the controller is used to control the first light-emitting element and the second light-emitting element to emit the first detection light and the second detection light, and control the first camera and the second camera to collect light to obtain the first image and the second image, and the information processor is used to perform data analysis on the first image, the second image, and the spatial position information of the first reflector to the Nth reflector, and determine the optical path model based on the data analysis result; wherein, if the camera as the fixation target is further included, the controller is further used to control the camera as the fixation target to collect light to obtain an image of the user's eyes, and during the process of determining the optical path model, the information processor also needs to perform data analysis on the image of the user's eyes to determine the optical path model.

9. A method for determining an equation set of visual axes, characterized in that, Based on the eyeball-based optical path generation device according to claim 1, the method includes: Presetting L distance parameter values, M included angle parameter values, and N rotation angle parameter values in advance; the L distance parameter values are successively the first distance parameter value to the Lth distance parameter value, L is an integer greater than 1, the M included angle parameter values are successively the first included angle parameter value to the Mth included angle parameter value, M is an integer greater than 1, and the N rotation angle parameter values are successively the first rotation angle parameter value to the Nth rotation angle parameter value, N is an integer greater than or equal to 1; Determine the first group of preset position arrays to the Lth group of preset position arrays according to the first distance parameter value to the Lth distance parameter value, the first included angle parameter value to the Mth included angle parameter value, and the first rotation angle parameter value to the Nth rotation angle parameter value; any group of preset position arrays including the first group of preset position arrays to the Nth group of preset position arrays includes M preset position arrays; Determine the optical path model corresponding to each of the preset position arrays at different time periods; Perform data analysis on all the optical path models, and determine the visual axis equations according to the data analysis result.

10. The method for determining the visual axis equation set according to claim 9, characterized in that, The method for determining the first group of preset position arrays to the Nth group of preset position arrays according to the first distance parameter value to the Lth distance parameter value, the first included angle parameter value to the Mth included angle parameter value, and the first rotation angle parameter value to the Nth rotation angle parameter value includes: Determine the starting data through the eyeball-based optical path generation device; the starting data at least includes the first image and the second image, and if the eyeball-based optical path generation device includes the camera as the fixation target, the starting data further includes the image of the user's eyes; Perform data analysis on the starting data, and determine a reference model based on the data analysis result; the reference model at least includes a reference eyeball-based optical path generation device model, a reference corneal model, a reference visual axis model, a reference pupil center model, and a reference iris feature model; Determine the first group of preset position arrays to the Nth group of preset position arrays according to the reference model.

11. The method for determining the visual axis equation set according to claim 10, characterized in that, The method for determining the starting data by the eye-based optical path generating device includes: Adjust the spatial positions of the first reflector to the Nth reflector from the first adjustment module to the Nth adjustment module to the starting position; when the spatial positions of the first reflector to the Nth reflector are at the starting position, the spots reflected by the user's eye cornea will appear on the images of the mirror surfaces of the reflectors included in the first image and the second image collected by the first camera and the second camera, and when the spatial positions of the first reflector to the Nth reflector are at the starting position, the spots included in the first image and the second image collected by the first camera and the second camera can be used to calculate the shape of the user's eye cornea; When the spatial positions of the first reflector to the Nth reflector are at the starting position, and the user's eyeball is within the specific spatial region and gazes at the image of the lens of the first camera reflected in the mirror surface of the reflector serving as the reflection gaze target or gazes at the camera serving as the gaze target, use the first image, the second image, and the image of the user's eye collected by the first camera, the second camera, and the camera serving as the gaze target respectively as the starting data; Among them, if the eye-based optical path generating device includes the reflector for reflecting the gaze target and does not include the camera serving as the gaze target, the starting data only includes the first image and the second image collected by the second camera; if the eye-based optical path generating device includes the camera serving as the gaze target and does not include the reflector for reflecting the gaze target, the starting data includes the first image, the second image, and the image of the user's eye.

12. The method for determining the visual axis equation set according to claim 10, wherein The method for analyzing the starting data and determining the reference model based on the analysis result of the data includes: Determine the eye-based optical path generating device model as the reference according to the spatial positions of the first reflector to the Nth reflector at the starting position; the eye-based optical path generating device model as the reference at least includes a first camera model, and the first camera model at least includes a light converging region model of the first camera; Determine the corneal model as the reference in the virtual space according to the spots included in the first image and the second image included in the starting data, and the eye-based optical path generating device model as the reference; the corneal model as the reference at least includes a curved surface for restoring the shape of the user's eye cornea; Based on the image of the pupil included in the mirror surface of the reflector that reflects the fixation target in the first image included in the starting data, or based on the image of the pupil included in the image of the user's eye included in the starting data, determine the reference visual axis model in the virtual space through the reference eyeball-based optical path generating device model; if the eyeball-based optical path generating device includes the reflector that reflects the fixation target, the reference visual axis model is determined according to the first image included in the starting data; if the eyeball-based optical path generating device includes the camera as the fixation target, the reference visual axis model is determined according to the image of the user's eye included in the starting data. Determine the reference pupil center model based on the reference visual axis model and the reference corneal model. Based on the iris information included in the mirror surface of the reflector that reflects the fixation target in the first image included in the starting data, or based on the iris information included in the image of the user's eye included in the starting data, determine the reference iris feature model through the reference eyeball-based optical path generating device model; if the eyeball-based optical path generating device includes the reflector that reflects the fixation target, the reference iris feature model is determined according to the first image included in the starting data; if the eyeball-based optical path generating device includes the camera as the fixation target, the reference iris feature model is determined according to the image of the user's eye included in the starting data.

13. The method for determining the visual axis equation set according to claim 10, wherein The method for determining the first group of preset position arrays to the Nth group of preset position arrays based on the reference model includes: Establish a dynamic mirror surface model based on the reference model. Determine all possible combinations formed by taking one distance parameter value, one angle parameter value, and one rotation angle parameter value from L distance parameter values, M angle parameter values, and N rotation angle parameter values respectively; each combination includes one distance parameter value, one angle parameter value, and one rotation angle parameter value. Determine the preset position corresponding to each combination through the dynamic mirror surface model; each preset position corresponds to one distance parameter value, one angle parameter value, and one rotation angle parameter value. Form the preset position arrays by combining the preset positions corresponding to the same distance parameter value and the same angle parameter value, and form the first group of preset position arrays to the Lth group of preset position arrays respectively by combining all the preset position arrays corresponding to the first distance parameter value to all the preset position arrays corresponding to the Lth distance parameter value. Among them, any one of the preset position arrays corresponds to a distance parameter value and an included angle parameter value, and any one of the preset position arrays includes N preset positions, which are successively the 1st preset position to the Nth preset position, and any one of the preset position arrays including the 1st group of preset position arrays to the Lth group of preset position arrays includes M preset position arrays.

14. The method for determining the visual axis equation set according to claim 13, wherein, The method for establishing a dynamic mirror model based on the reference model includes: assuming that the light convergence area model of the first camera included in the reference model is point A, the point where the pupil center model is located is point B, and the point where the iris feature model serving as the reference is located is point C. Make a point O on the curved surface included in the reference cornea model, and draw a normal line of the curved surface where the reference cornea model is located through the point O. Make a point P on the normal line, and draw a plane through the point P. This plane is the dynamic mirror model. Make the plane where the dynamic mirror model is located perpendicular to the angle bisector of angle APO. Make a point Q on the plane where the dynamic mirror model is located, make the angle bisector of angle AQB perpendicular to the plane where the dynamic mirror model is located, and make a point R on the plane where the dynamic mirror model is located, make the angle bisector of angle ARC perpendicular to the plane where the dynamic mirror model is located.

15. The method for determining the visual axis equation set according to claim 9, wherein The method for determining the optical path model corresponding to each preset position array at different time periods includes: Adjust the spatial positions of the first reflector to the Nth reflector respectively through the first adjustment component to the Nth adjustment component, so that the spatial positions of the first reflector to the Nth reflector are respectively at the real space positions corresponding to the 1st preset position to the Nth preset position included in each preset position array during each time period. Collect the first image, the second image, and the image of the user's eyes corresponding to each preset position array through the first camera, the second camera, and the camera serving as the fixation target during each time period; the first image, the second image, and the image of the user's eyes corresponding to each preset position array are used to determine the optical path model corresponding to each preset position array. If the camera serving as the fixation target is not included, there is no image of the user's eyes, and only determine the optical path model corresponding to each preset position array through the first image and the second image corresponding to each preset position array; Determine the optical path model corresponding to each preset position array according to the first image, the second image, and the image of the user's eyes corresponding to each preset position array; Among them, each of the optical path models at least includes: a visual axis model, a first to an Nth mirror model, a first to an Nth pupil-based first-segment optical path model, a first to an Nth pupil-based last-segment optical path model, a first to an Nth iris-based first-segment optical path model, a first to an Nth iris-based last-segment optical path model, a first to an Nth cornea-based first-segment optical path model, and a first to an Nth cornea-based last-segment optical path model; if the image of the user's eye is not included, only the optical path models corresponding to each of the preset position arrays are determined by the first image and the second image corresponding to each of the preset position arrays.

16. The method for determining the visual axis equation set according to claim 15, characterized in that, The method for determining the optical path models corresponding to each of the preset position arrays according to the first image, the second image, and the image of the user's eye corresponding to each of the preset position arrays includes: According to the spatial positions of the first to the Nth mirrors in each of the time periods, respectively determine the eyeball-based optical path generation device models corresponding to each of the preset position arrays; According to the light spots in the first image corresponding to each of the preset position arrays and the light spots in the second image corresponding to each of the preset position arrays, and determine the cornea models corresponding to each of the preset position arrays through each of the device models for collecting eyeball data; Respectively according to the image of the pupil on the mirror surface of the mirror that reflects the fixation target in the first image corresponding to each of the preset position arrays, and respectively determine the visual axis models corresponding to each of the preset position arrays through each of the device models for collecting eyeball data, or respectively according to the image of the pupil on the image of the user's eye corresponding to each of the preset position arrays and respectively determine the visual axis models corresponding to each of the preset position arrays through each of the device models for collecting eyeball data; if there is no image of the user's eye because there is no camera as the fixation target, then determine the visual axis model according to the first image, and if there is an image of the user's eye because there is a camera as the fixation target, then determine the visual axis model according to the image of the user's eye; According to the array composed of the images of the pupils on the first image corresponding to each of the preset position arrays, and determine the first to the Nth pupil-based first-segment optical path models and the first to the Nth pupil-based last-segment optical path models corresponding to each of the preset position arrays through each of the device models for collecting eyeball data; the array composed of the images of the pupils is the array composed of the images of the pupils respectively included on the mirror surface images of the first to the Nth mirrors in the first image. An array composed of the images of the irises on the first image corresponding to each of the preset position arrays, and determining, through each of the device models for collecting eye data, the first to Nth first-segment optical path models based on the iris and the first to Nth last-segment optical path models based on the iris corresponding to each of the preset position arrays; the array composed of the images of the irises is an array composed of the images of the irises respectively included on the images of the mirrors of the first to Nth reflectors in the first image; Respectively, an array composed of the first light spots on the first image corresponding to each of the preset position arrays, and respectively determining, through each of the device models for collecting eye data, the first to Nth first-segment optical path models based on the cornea and the first to Nth last-segment optical path models based on the cornea corresponding to each of the preset position arrays; the array composed of the first light spots is an array composed of the first light spots respectively included on the images of the mirrors of the first to Nth reflectors in the first image, the first light spot is a light spot formed in the first image by the light rays in the first detection light rays emitted by the first light-emitting element that are respectively reflected by the first to Nth reflectors along the normal line of the user's eye cornea towards the user's eye cornea, then reflected by the user's eye cornea along the original path specularly and finally entering the lens of the first camera and being received, and there is only one first light spot on the image of the mirror of any one of the reflectors including the images of the mirrors of the first to Nth reflectors in each of the first images; Wherein, each of the optical path models includes: the optical path generating device model based on the eye, the cornea model, the visual axis model, the first to Nth first-segment optical path models based on the pupil, the first to Nth last-segment optical path models based on the pupil, the first to Nth first-segment optical path models based on the iris, the first to Nth last-segment optical path models based on the iris, the first to Nth first-segment optical path models based on the cornea, and the first to Nth last-segment optical path models based on the cornea corresponding to each of the preset position arrays.

17. The method for determining the visual axis equation set according to claim 9, characterized in that, Performing data analysis on all the optical path models, and the method for determining the visual axis equation set according to the data analysis result includes: By performing data analysis on each of the optical path models, determining the first to Nth distance parameter measurement values, the first to Nth angle parameter measurement values, and the first to Nth rotation angle parameter measurement values corresponding to each of the optical path models; Determine the visual axis equation A based on the first to Nth distance parameter measurement values, the first to Nth angle parameter measurement values, the first to Nth rotation angle parameter measurement values, and the first to Nth visual axis parameter A measurement values corresponding to each of the optical path models; the visual axis equation A is used to determine the output value of the visual axis parameter A according to the actual measurement values of the input distance parameter, the angle parameter, and the rotation angle parameter; Determine the visual axis equation B based on the first to Nth distance parameter measurement values, the first to Nth angle parameter measurement values, the first to Nth rotation angle parameter measurement values, and the first to Nth visual axis parameter B measurement values corresponding to each of the optical path models; the visual axis equation B is used to determine the output value of the visual axis parameter B according to the actual measurement values of the input distance parameter, the angle parameter, and the rotation angle parameter; Determine the visual axis equation C based on the first to Nth distance parameter measurement values, the first to Nth angle parameter measurement values, the first to Nth rotation angle parameter measurement values, and the first to Nth visual axis parameter C measurement values corresponding to each of the optical path models; the visual axis equation C is used to determine the output value of the visual axis parameter C according to the actual measurement values of the input distance parameter, the angle parameter, and the rotation angle parameter; Determine the visual axis equation D based on the first to Nth distance parameter measurement values, the first to Nth angle parameter measurement values, the first to Nth rotation angle parameter measurement values, and the first to Nth visual axis parameter D measurement values corresponding to each of the optical path models; the visual axis equation D is used to determine the output value of the visual axis parameter D according to the actual measurement values of the input distance parameter, the angle parameter, and the rotation angle parameter; Determine the visual axis equation set according to the visual axis equation A, the visual axis equation B, the visual axis equation C, and the visual axis equation D; wherein, the visual axis equation set includes the visual axis equation A, the visual axis equation B, the visual axis equation C, and the visual axis equation D.

18. The method for determining the visual axis equation set according to claim 17, wherein The method for determining the first to Nth distance parameter measurement values, the first to Nth angle parameter measurement values, and the first to Nth rotation angle parameter measurement values corresponding to each of the optical path models by performing data analysis on each of the optical path models includes: Determine the measured value of the j-th distance parameter corresponding to each of the optical path models by analyzing the j-th corneal-based first-segment optical path model and the j-th corneal-based last-segment optical path model included in each of the optical path models, or determine the measured value of the j-th distance parameter corresponding to each of the optical path models by analyzing the j-th pupil-based first-segment optical path model and the j-th pupil-based last-segment optical path model included in each of the optical path models, or determine the measured value of the j-th distance parameter corresponding to each of the optical path models by analyzing the j-th iris-based first-segment optical path model and the j-th iris-based last-segment optical path model included in each of the optical path models; Determine the measured value of the j-th included angle parameter corresponding to each of the optical path models by analyzing the relative positions between the j-th pupil-based first-segment optical path model and the j-th corneal-based first-segment optical path model included in each of the optical path models, or determine the measured value of the j-th included angle parameter corresponding to each of the optical path models by analyzing the relative positions between the j-th pupil-based last-segment optical path model and the j-th corneal-based last-segment optical path model included in each of the optical path models. Determine the measured value of the j-th rotation angle parameter corresponding to each of the optical path models by analyzing the relative positions among the j-th pupil-based first-segment optical path model, the j-th corneal-based first-segment optical path model, and the j-th iris-based first-segment optical path model included in each of the optical path models; Determine the j-th three-dimensional coordinate system included in each of the optical path models according to the j-th pupil-based first-segment optical path model, the j-th corneal-based first-segment optical path model, and the corneal model included in each of the optical path models. Determine the measured value of the j-th visual axis parameter A, the measured value of the j-th visual axis parameter B, the measured value of the j-th visual axis parameter C, and the measured value of the j-th visual axis parameter D corresponding to each of the optical path models by analyzing the relative positions between the j-th three-dimensional coordinate system and the visual axis model included in each of the optical path models; Where j takes any positive integer less than or equal to the total number N of the adjusting components. When j takes 1 up to N respectively, there are the measured value of the first distance parameter, the measured value of the first included angle parameter, the measured value of the first rotation angle parameter, the measured value of the first visual axis parameter A, the measured value of the first visual axis parameter B, the measured value of the first visual axis parameter C, and the measured value of the first visual axis parameter D corresponding to each of the optical path models, up to the measured value of the N-th distance parameter, the measured value of the N-th included angle parameter, the measured value of the N-th rotation angle parameter, the measured value of the N-th visual axis parameter A, the measured value of the N-th visual axis parameter B, the measured value of the N-th visual axis parameter C, and the measured value of the first visual axis parameter D.

19. An eye movement tracking method, characterized in that, Based on the visual axis equation set recited in claim 9, the method includes: Pre-set an eye tracking device, which at least includes a third measurement unit, a computer, and a connecting member; the eye tracking device may further include a fourth measurement unit or a device capable of measuring the corneal spatial position of the user's eyeball, and the eye tracking device can be used to measure the position of the user's eyeball to determine the spatial position information of the user's eyeball; the third measurement unit includes a third camera and a third light-emitting element, and the structure of the third measurement unit is the same as that of the first measurement unit. The third camera included in the third measurement unit is equivalent to the first camera included in the first measurement unit, and the third light-emitting element included in the third measurement unit is equivalent to the first light-emitting element included in the first measurement unit. The third light-emitting element is used to emit a third detection light to the outside; the fourth measurement unit includes a fourth camera and a fourth light-emitting element, and the structure of the fourth measurement unit is the same as that of the first measurement unit. The fourth camera included in the fourth measurement unit is equivalent to the first camera included in the first measurement unit, and the fourth light-emitting element included in the fourth measurement unit is equivalent to the first light-emitting element included in the first measurement unit. The fourth light-emitting element is used to emit a fourth detection light to the outside; When the user's eyeball gazes at an object in reality, the third light-emitting element included in the third measurement unit emits the third detection light to the outside, and the third camera included in the third measurement unit collects the light reflected by the user's eyeball and incident into the lens of the third camera to obtain a third image; Determine a pupil-based light path model, an iris-based light path model, and a cornea-based light path model in a three-dimensional space simulated by a computer according to the third image; In the three-dimensional space simulated by the computer, determine a three-dimensional coordinate system for eye tracking according to the pupil-based light path model, the iris-based light path model, and the cornea-based light path model; Determine the actual measured value of the distance parameter by analyzing the cornea-based light path model, or determine the actual measured value of the distance parameter by analyzing the pupil-based light path model, or determine the actual measured value of the distance parameter by analyzing the cornea-based light path model; determine the actual measured value of the included angle parameter by analyzing the relative positions between the pupil-based light path model and the cornea-based light path model; determine the actual measured value of the rotation angle parameter by analyzing the relative positions among the pupil-based light path model, the cornea-based light path model, and the iris-based light path model; According to the actual measured value of the distance parameter, the actual measured value of the included angle parameter, and the actual measured value of the rotation angle parameter, and according to the visual axis equations, determine the output values of the visual axis parameters A, B, C, and D; Determine the visual axis model for eye tracking based on the output values of the visual axis parameter A, the output value of the visual axis parameter B, the output value of the visual axis parameter C, the output value of the visual axis parameter D, and the three-dimensional coordinate system for eye tracking.

Images