A Dynamic Method for Establishing Iris and Pupil Edge Curve Models Based on an Eye Model

Abstract

This invention discloses a method for dynamically establishing iris and pupil edge curve models based on an eye model, relating to the field of eye-tracking technology. It utilizes existing technologies where multiple cameras in a gaze-direction measurement device simultaneously capture images of the user's eyeballs from different angles. Then, through calculation and analysis, a polynomial function set is established based on the projected iris / pupil edge curves corresponding to the focus models of multiple color cameras at different positions in the gaze-direction model. The first and second position parameters, calculated based on the relative position of the virtual camera's virtual focus and the eye model, are input into the polynomial function set to fit the iris / pupil edge curves. Compared to existing technologies where the shape of the iris / pupil edge curves in the eye model remains unchanged in virtual space, this invention considers real-world refraction, enabling more precise determination of the operator's gaze position on the terminal device screen.

Description

Technical Field

[0001] This invention relates to the field of eye-tracking technology, specifically to a method for dynamically establishing iris and pupil edge curve models based on an eye model. Background Technology

[0002] With the continuous development of science and technology, electronic devices with eye-tracking (also known as gaze tracking) function are widely used in people's daily life and work, bringing great convenience to people's daily life and work, and becoming an indispensable tool for people today.

[0003] Chinese patent application number 2021106099152 provides a gaze-pointing measurement device and a gaze-pointing model establishment method, which can perform high-precision measurement of eye gaze and simultaneously achieve high-precision eye tracking. However, in this patent, the iris and pupil edge curves in the eye model of the operator's eye gaze model are fixed curves, and their shapes do not change due to changes in the relative position of the virtual camera and the eye model. This causes the cone surface constructed in virtual space with the iris or pupil edge curve as the guideline and the virtual focus of the virtual camera as the vertex to not accurately simulate the shape of the cone surface formed by the light reflected from the iris or pupil edge region of the user's eyeball after refraction through the cornea and entering the camera when the camera and the user's eyeball are in the same relative position in the real scene. This leads to errors in gaze tracking, thus affecting accuracy. To this end, we propose a dynamic modeling method for iris and pupil edge curves based on an eye model to accurately simulate the shape of the cone formed by the light reflected from the iris or pupil edge region of the user's eyeball after refraction through the cornea and into the camera, in the relative positions of the user's eye model and the virtual focus in reality. Summary of the Invention

[0004] The purpose of this invention is to provide a method for dynamically establishing iris and pupil edge curve models based on an eye model.

[0005] This invention can be achieved through the following technical solution: a method for dynamically establishing iris and pupil edge curve models based on an eye model, including a gaze pointing measurement device for implementing the method and a gaze pointing model constructed using the gaze pointing measurement device. The dynamic establishment method includes the following steps: Step 1: Based on the position parameters of the focal model and eye coordinate system of the multiple color camera models in each gaze pointing model, and the corresponding projected iris / pupil edge curve shape, obtain interpolation function group I corresponding to each gaze pointing model; the interpolation function group I maps the corresponding iris / pupil edge curve shape and a specific value of the simulated focal third position parameter based on the simulated focal first position parameter and the simulated focal second position parameter. Step 2: By merging the interpolation function group I corresponding to each of the multiple gaze pointing models, an interpolation function group II corresponding to the multiple gaze pointing models is obtained, and then the iris / pupil edge curve corresponding to it is mapped out through the interpolation function group II.

[0006] A further technical improvement of the present invention is that: in step one, firstly, based on the projected iris / pupil edge curve, an arc length ratio-horizontal coordinate mapping function and an arc length ratio-vertical coordinate mapping function corresponding to the color camera focus model are established; Subsequently, based on the three position parameters corresponding to the multiple color camera focus models in each gaze pointing model, as well as the arc length ratio-horizontal coordinate mapping function and the arc length ratio-vertical coordinate mapping function, we obtain the interpolation group containing the horizontal coordinate parameter, the interpolation group containing the vertical coordinate parameter, and the interpolation group containing the third position parameter corresponding to each gaze pointing model. Based on the three interpolation groups, three corresponding interpolation functions are established, and the three interpolation functions together form the interpolation function group I corresponding to each gaze-pointing model.

[0007] A further technical improvement of the present invention is as follows: In step two, based on at least R interpolation function groups corresponding to at least R gaze pointing models in the plurality of gaze pointing models, N' pairs of specific simulated focus first and second position parameters are input to obtain N' groups of mapped iris / pupil edge curves corresponding to each pair of simulated focus first and second position parameters, with each group having at least R iris / pupil edge curves; Fine-tune the iris / pupil edge curves corresponding to the first and second position parameters of each pair of simulated focal points until they overlap, and draw the projected iris / pupil edge curve II based on the set of overlapping iris / pupil edge curves; Based on the N' pairs of specific simulated focal point first and second position parameters corresponding to multiple gaze pointing models, and the corresponding N' projected iris / pupil edge curves II, an interpolation function set II corresponding to the multiple gaze pointing models is obtained. This then maps to the corresponding iris or pupil edge curve II.

[0008] A further technical improvement of the present invention is that: in each group of gaze pointing models, there is at least one gaze pointing model, and the distance between the point where the first camera focus model of each gaze pointing model in a group of gaze pointing models is located and the point where the pupil center point is located is consistent.

[0009] A further technical improvement of the present invention is that the projection iris / pupil edge curve proposed in step one is based on an eye model, which is established by obtaining real user eye data from the photo-taking module in the gaze pointing measurement device.

[0010] A further technical improvement of the present invention is that the shapes of the polynomial function images corresponding to the three interpolation functions are all smooth, and are used to fit the shape of the iris / pupil edge curve in the eye model when the simulated focus of the virtual camera and the eye model are in a relative position within a specific range. This simulates the shape of the cone formed by the light reflected from the iris / pupil edge region of the user's eyeball after refraction through the cornea and entering the camera when the simulated focus of the virtual camera and the eye model are in that relative position, and the real camera and the user's eyeball are in the same relative position.

[0011] A further technical improvement of the present invention is that: the interpolation function group I corresponding to each gaze pointing model in the same group of gaze pointing models theoretically overlaps with each other in the same eye coordinate system. However, local intersections occur during the actual mapping process. By fine-tuning, the shapes of the group of iris edge curves are mutually symmetrical, and a new spline is used to outline the group of fine-tuned iris edge curves to obtain the projected iris edge curve II corresponding to the group of iris edge curves.

[0012] Compared with the prior art, the present invention has the following beneficial effects: The method provided by this invention is based on images obtained by multiple cameras simultaneously capturing images of the user's eyeball from different angles in an existing gaze pointing measurement device. Then, through calculation and analysis, a polynomial function set is established based on the projected iris / pupil edge curves corresponding to the focus models of multiple color cameras at different positions in the gaze pointing model. The first and second position parameters, calculated based on the relative position of the virtual focus of the virtual camera and the eye model, are input into the polynomial function set to fit the iris / pupil edge curve. Therefore, in virtual space, the cone-shaped surface formed by connecting the fitted iris / pupil edge curve and the virtual focus can accurately simulate the shape of the cone formed by the light reflected from the iris / pupil edge region of the user's eyeball at the corresponding relative position of the user's eye model and the virtual focus, after refraction through the cornea and entering the camera. This method simulates and estimates the shape of the iris / pupil edge region in the image obtained by the user's cornea refracting light reflected from the iris / pupil and then entering the camera lens under specific relative positions in reality. Compared with the existing technology, where the shape of the iris / pupil edge curve remains unchanged in the eye model in virtual space, this invention takes into account the refraction in reality, which makes it more accurate to determine the position of the operator's gaze on the terminal device screen. Attached Figure Description

[0013] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0014] Figure 1 This is a flowchart illustrating the implementation process of the method of the present invention; Figure 2 This is a schematic diagram of the target pointing model of the present invention; Figure 3 For the present invention Figure 1 Enlarged view of details in area A in the middle; Figure 4 For the present invention Figure 1 Enlarged view of details in area B; Figure 5 This is a schematic diagram of a camera module structure according to the present invention; Figure 6 For the present invention Figure 4 A magnified view of a section at point C; Figure 7 A flowchart illustrating the method for establishing iris marking lines according to the present invention; Figure 8 This is a schematic diagram illustrating the principle of marking iris texture in this invention; Figure 9 This is a schematic diagram of the iris pattern marking line model structure constructed according to the present invention; Figure 10 This is a schematic diagram illustrating the principle of placing the eye coordinate system in the eye model according to the present invention; Figure 11 This is a schematic diagram of the method for determining the projected iris / pupil edge curve group model according to the present invention; Figure 12 This is a schematic diagram of an image captured by the color camera of the present invention; Figure 13 This is a schematic diagram showing the relative relationship between the eye coordinate system model of the present invention and the reflective surfaces of each iris edge; Figure 14 This is a schematic diagram showing the relative positional relationship between the eye coordinate system and the projected iris edge curve of the present invention; Figure 15 This is a schematic diagram of a complete gaze-direction model of the present invention; Figure 16 This is a schematic diagram showing the overlap of facial coordinate systems in two gaze-pointing models within a set according to the present invention. Figure 17 This is a diagram showing the relative positions of multiple elements in the gaze-pointing model of the present invention; Figure 18 This is a schematic diagram showing the starting point marking of the projected iris edge curve in this invention; Figure 19This is a schematic diagram of the trajectory state for determining the arc length ratio parameter according to the present invention; Figure 20 This is a schematic diagram of the image shape of the arc length ratio-horizontal and vertical coordinate mapping function of the projected iris edge curve in this invention. Figure 21 This is a schematic diagram of the function image shape of the corresponding iris curve under the arc length ratio parameter setting of the present invention; Figure 22 This is a schematic diagram of the function image shape of the arc length ratio-x-axis mapping function corresponding to a set of gaze pointing models in this invention; Figure 23 This is a schematic diagram of the function image shape of the x-coordinate difference function corresponding to a gaze pointing model in this invention when different arc length ratio parameters are set; Figure 24 This is a schematic diagram of the image shape of the horizontal coordinate difference function under the corresponding position parameters and arc length ratio parameters of the present invention; Figure 25 This is a schematic diagram of the image shape of the ordinate difference function under the corresponding position parameters and arc length ratio parameters of the present invention. Figure 26 This is a schematic diagram of the function image state of the interpolation function group I corresponding to the gaze pointing model in the present invention in the eye coordinate system; Figure 27 This is a schematic diagram showing the coordinate points corresponding to the first and second position parameters of the sixteen pairs of simulated focal points in this invention; Figure 28 This is a schematic diagram showing the relative positions of the sixteen sets of iris edge curves in this invention; Figure 29 This is a schematic diagram showing the relative position state after the relative position of the iris edge curve is finely adjusted according to the present invention. Detailed Implementation

[0015] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0016] Please see Figures 2-4 As shown, this embodiment provides a gaze-pointing model for technical preparation for subsequent solution descriptions. The gaze-pointing model includes a camera module model T11 and an eye model T12. Among them, the camera module model T11 includes the infrared camera component model T1120 and the color camera array model: The infrared camera component model T1120 includes at least a first camera model T1121, and the first camera model T1121 includes at least a first camera focus model T1121' and a first camera optical axis model T1121''. The color camera array model includes N color camera models located around the infrared camera component T1120, namely the first color camera model T1101, the second color camera model T1102, and so on up to the Nth color camera model T1116. Each color camera model contains a color camera focus model, labeled as the first color camera focus model T1101', the second color camera focus model T1102', and so on up to the Nth color camera focus model T1116'; where N is a positive integer greater than 1, and in this example, the total number of color camera models N is 16. The "+" in the first camera model T1121 indicates the first camera focus model T1121'; The optical axis model T1121'' of the first camera is a ray that originates from the focal point model T1121' of the first camera and is perpendicular to the first camera model; The "+" signs in the first color camera model T1101 and the sixteenth color camera model T1116 represent the first color camera focus model T1101' and the sixteenth color camera focus model T1116', respectively.

[0017] The eye model T12 includes at least: a pupil center point model T1251, an eye coordinate system model T1252 located based on the pupil center point model T1251 and the first camera focus model T1121', a visual axis model TT1253 located based on the eye coordinate system model T1252, and 16 corresponding projected iris / pupil edge curves obtained by projecting the aforementioned 16 color camera focus models onto the eye coordinate system model T1252, respectively labeled as the first projected iris / pupil edge curve T1401, the second projected iris / pupil edge curve T1402, up to the Nth projected iris / pupil edge curve T1416; It should be noted that the projected iris / pupil edge curve includes the projected iris edge curve and the projected pupil edge curve. In the method mentioned in this article, the data processing methods for the projected iris edge curve and the projected pupil edge curve are similar, and the properties and functions of the interpolation function group obtained after processing are also similar. Therefore, the projected iris edge curve and the projected pupil edge curve are collectively referred to as the projected iris / pupil edge curve. The aforementioned camera module model is a 3D model built based on a camera module within a gaze-oriented model, wherein... Figures 5-6The patent document with application number 2021106099152 describes a physical photographing module T21 and an infrared camera component T2120 installed on the photographing module T21. The infrared camera assembly T2120 includes at least a first camera T2121, which has a first camera focus T2121' and a first camera optical axis T2121'', as well as a focus T2101' of the first color camera, a focus T2102' of the second color camera, and a focus T2116' of the sixteenth color camera. The camera module model T11 is a 3D model of the camera module T21; The relative positional relationships between the first camera focus model T1121', the first camera optical axis model T1121'', the first color camera focus model T1101', the second color camera focus model T1102', and up to the sixteenth color camera focus model T1116' in the camera module model T11 are the same as the relative positional relationships between the first camera focus T2121', the first camera optical axis T2121'', and the focus T2101' of the first color camera, the second color camera focus T2102', and up to the sixteenth color camera focus T2116' in the camera module T21.

[0018] The camera module model T11 is built on the gaze pointing model. Through the eye model T12 with a specific positional relationship, it simulates the relative positional relationship between the camera module T21 and the user's eye T22 when the real camera module T21 measures the user's eye T22.

[0019] The eye model T12 was built based on data obtained from measurements of the real user's eyeballs T22 taken by the camera module T21. Figure 5 and Figure 6 The diagram shows the user's eyeball T22, the center point of the pupil T2251 on the user's eyeball, and the visual axis T2253 of the user's eyeball. The two circular dashed lines on the user's eyeball T22 represent the junction area between the iris and sclera on the eyeball. The junction area between the pupil and the iris is represented by a dashed line because it is on the back of the eyeball.

[0020] After the visual axis T2253 of the user's eyeball coincides with the optical axis T2121'' of the first camera in the camera module T21, the infrared measurement component T2120 in the camera module T21 measures the distance from the center point T2251 of the pupil of the user's eyeball to the infrared measurement component T2120 and marks the distance value as S. Here, the center point T2251 of the pupil on the user's eyeball is the intersection of the visual axis T2253 of the user's eyeball and the outermost layer of the cornea on the eye. The method described above, which ensures that the optical axis T2121'' of the first camera in the camera module T21 is collinear with the visual axis T2253 of the user's eyeball, can be found in the specification of the patent document with application number 2021106099152. Figure 14 , Figure 15 and Figure 16 Corresponding explanation; The method for measuring the distance between the center point T2251 of the user's pupil and the focal point T1121' of the first camera in the infrared measurement component T2120 of the aforementioned camera module T21 can be found in the specification of the patent document with application number 2021106099152. Figure 13 , Figure 17 as well as Figures 27-3 The explanation corresponding to 3.

[0021] Based on the distance value S obtained from the above measurements, the pupil center point model T1251 is determined in the gaze pointing model T11. In the real scene, since the pupil center point T2251 is the intersection of the visual axis T2253 of the user's eyeball and the outermost layer of the cornea, the pupil center point T2251 must be on the visual axis T2253. Since the visual axis T2253 is collinear with the optical axis T2121'' of the first camera when the camera module measures the user's eyeball, the pupil center point T2251 is on the optical axis T2121'' of the first camera. Moreover, the distance value S from the pupil center point T2251 to the infrared camera component T2120 is known through measurement. Therefore, the relative positional relationship between the pupil center point T2251 and the camera module T21 can be determined. Reference Figure 2 and Figure 4 In the gaze-pointing model, a pupil center point T1251 is drawn on the optical axis model T1121'' of the first camera of the photo-taking module model T11. The distance from the pupil center point T1251 to the focal point model T1121' of the first camera is equal to the distance value S measured in reality. The relative position of the pupil center point of the user's eyeball and the photo-taking module can be simulated by the relative position of the photo-taking module model T11 and the pupil center point T1251.

[0022] like Figure 7 As shown, the method for establishing iris marking lines based on data obtained from measuring the real user's eyeball using the first camera in the image-taking module specifically includes: Step S21: Identify the iris pattern in the image based on the brightness and color of the image obtained by the first camera in the camera module capturing the user's eyes.

[0023] Step S22: Mark the feature information in the iris pattern using marker point T3001, such as... Figure 8 As shown, Figure 8This is a schematic diagram illustrating the principle of marking iris textures according to an embodiment of this application. Each time the first camera captures an image of the same user's same eye, it identifies the same feature in the iris texture of the user's eye in similar images and uses it as a marker point T3001.

[0024] Step S23: Based on the position of the marker point, determine the relative positional relationship between the ray and the coordinate system of the first camera when the ray incident on the focal model T1121' of the first camera at marker point T3001. Construct the iris pattern marking line model T3002 based on the ray, as follows: Figure 9 As shown; For specific details regarding steps S21-S23 above, please refer to steps S71-S73 in the specification section of the patent document for application number 2021106099152.

[0025] like Figure 10 As shown, the eye coordinate system is determined based on the pupil center point and the iris marking line: In the gaze-pointing model, the eye coordinate system T1252 is used to determine the relative positional relationship between the eye model and the camera module model T11; Figure 10 The diagram shows the pupil center point T1251, the origin T1254 of the eye coordinate system T1252, the AZ axis, the AX axis, and the AY axis. It also shows the optical axis model of the first camera T1121'' and the iris pattern marker line T3002.

[0026] Specifically, the direction from the pupil center point T1251 to the focal point T1121' of the first camera model is the AZ axis direction of the eye coordinate system. The origin T1254 of the eye coordinate system lies on the optical axis T1121'' of the first camera. Taking the AZ axis direction as the forward direction, the origin T1254 of the eye coordinate system is behind the pupil center point T1251. The distance between the origin T1254 of the eye coordinate system and the pupil center point T1251 is a constant value, approximately equal to the distance from the intersection of the visual axis and the cornea along the visual axis direction in the human eyeball to the center of the iris. The AX axis of the eye coordinate system intersects the iris ridge marker line T3002.

[0027] Based on the focus and eye coordinate system model of each color camera, a corresponding number of projected iris / pupil edge curves are determined to obtain a set model of projected iris / pupil edge curves. In this embodiment, the number is set to 16. like Figure 11 As shown, the method for determining the projected iris / pupil edge curve group model specifically includes: Step S31: Obtain multiple iris / pupil edge reflection surfaces through multiple color cameras in the color camera array; like Figure 12The diagram shown illustrates the images captured by the color camera. In this example, the color camera array of the camera module contains 16 color camera models, namely the first color camera model T1101 to the sixteenth color camera model T1116. These 16 color camera models correspond to 16 real color cameras that simultaneously capture images of the user's eyes, resulting in 16 eye images. The diagram also generates iris / pupil edge reflection surfaces that correspond one-to-one with the first color camera model T1101 to the sixteenth color camera model T1116, namely the first iris / pupil edge reflection surface T1301 to the sixteenth iris / pupil edge reflection surface T1316.

[0028] The iris / pupil edge reflection surface is the shape of the surface formed by light rays reflected from the characteristic area of ​​the iris-sclera junction or the characteristic area of ​​the pupil-iris junction when the color camera in the color camera array of the simulated camera module takes an image of the user's eye.

[0029] Step S32: Intersect multiple iris edge reflective surfaces with the set plane to obtain multiple one-to-one corresponding iris edge curves.

[0030] like Figure 13 The diagram showing the relative relationship between the eye coordinate system model T1252 and each iris edge reflection surface illustrates that the reflection surfaces from the first iris edge to the sixteenth iris edge intersect with the AX AY plane 541 of the eye coordinate system, resulting in 16 corresponding intersection lines, which are the first iris edge curve and the sixteenth iris edge curve.

[0031] like Figure 14 The diagram illustrates the relative positional relationship between the eye coordinate system and the projected iris edge curves. On the AX AY plane 541 of the eye coordinate system, there are the first iris edge curve T1401, the second iris edge curve 1402, and so on up to the sixteenth iris edge curve 1416. These 16 iris edge curves represent the light rays emitted from the junction of the iris and sclera in the user's eyeball, refracted by the transparent cornea surrounding the junction, and then incident on 16 color cameras. In the gaze-direction model, the shapes of the light rays incident on the 16 color cameras are represented by 16 curved surfaces, such as... Figure 15 As shown, because the light rays entering the 16 color cameras are refracted by the cornea, they are not overlapping and overlap each other. If the light rays were not refracted, they would not overlap or would overlap together.

[0032] The shape, morphology and... are not shown in this article. Figure 14The projected iris edge curves are similar in shape, but differ in that the projected pupil edge curve is a closed curve resembling a circle. Furthermore, the sixteen projected pupil edge curves corresponding to the first color camera model to the sixteenth color camera model do not overlap but intersect with each other. The method for converting an eye image captured by any color camera into an iris edge reflection surface is similar to the method for obtaining iris marking lines, both utilizing the method described in the patent document of application number 2021106099152. Figures 27 to 3 The explanation corresponding to 3 describes a method for calculating the angle of light based on the position of a point.

[0033] like Figure 15 The diagram shows a complete gaze pointing model, which includes an eye coordinate system model T1252, a camera module model T11, a user face model 568, and a face coordinate system model 563 located based on the face model.

[0034] The relative positions of each gaze towards the coordinate system model 563 and the photography module model T11 within the model are different.

[0035] In this embodiment, a set of gaze pointing models is set up, which contains R gaze pointing models. Any one of the gaze pointing models is marked as gaze pointing model r, and 1≤r≤R.

[0036] In this example, a set of gaze pointing models is set up to contain two gaze pointing models, that is, R is set to 2, such as Figure 16 The diagram shown illustrates the alignment of facial coordinate system 563 in two gaze-pointing models within a set. This set of gaze-pointing models includes gaze-pointing model 1 and gaze-pointing model 2. The distance S from the first camera focus model in each gaze-pointing model to the corresponding pupil center point T1251 is approximately 300mm. The reason for this approximation is that each gaze-pointing model is obtained by measuring the user's eyes based on the position of the camera module according to pre-set rules. Therefore, the actual measured data will have a slight deviation from the expected data.

[0037] As for how to obtain the model for each gaze in a set, please refer to steps S41 to S42, S51 to S52 and S61 to S62 mentioned in the specification of the patent document of application number 2021106099152.

[0038] In the process of constructing the iris / pupil edge curves mentioned above, it is necessary to accurately simulate the constantly changing relative positions of the camera and the user's eyeballs in the real scene, so as to construct a cone-shaped surface formed by connecting the iris / pupil edge curves and the virtual focus in a way that closely resembles reality. In this embodiment, such as Figure 1 As shown, this invention provides a method for dynamically establishing iris and pupil edge curve models based on an eye model. This method is based on images obtained by multiple cameras in a real-world gaze pointing measurement device simultaneously capturing images of the user's eyeball from different angles. By calculating and analyzing the projected iris / pupil edge curves corresponding to the focus models of multiple color cameras at different positions in the gaze pointing model, a set of polynomial functions is established. The first / second position parameters calculated based on the relative position of the virtual focus of the virtual camera and the eye model are then input into the set of polynomial functions to fit the iris / pupil edge curves. The method for dynamically establishing iris and pupil edge curve models based on an eye model specifically includes the following steps: Step S41: Based on the position parameters of the focal model and the shape of the projected iris / pupil edge curve of the multiple color camera models in each gaze pointing model, fit an interpolation function set corresponding to each gaze pointing model. This interpolation function set can map the corresponding iris / pupil edge curve shape and the simulated focal third position parameter of a specific value to the simulated focal first position parameter and the simulated focal second position parameter within the range. This step can be broken down into the following three aspects: S411: Based on the projected iris / pupil edge curves corresponding to the multiple color camera focus models in each gaze-pointing model, establish the arc length ratio-horizontal coordinate mapping function and the arc length ratio-vertical coordinate mapping function corresponding to the color camera focus model; Reference Figure 17 It shows the first color camera focus model T1101' in the first gaze pointing model in the first group of gaze pointing models, the corresponding projected iris edge curve T1401 and projected pupil edge curve T1501 on the XY plane in the eye coordinate system T1252, and any point p in the point set composed of all points on the projected iris edge curve T1401 and any point p' in the point set composed of all points on the projected pupil edge curve T1501.

[0039] In this paper, we define a set of gaze pointing models as containing a total of R gaze pointing models, where R is a positive integer greater than or equal to 1, and any gaze pointing model is numbered as gaze pointing model r, where 1 ≤ r ≤ R.

[0040] For example, if a set of eye-pointing models contains a total of 3 eye-pointing models, namely the first eye-pointing model in the set, up to the third eye-pointing model, then the total number of eye-pointing models in this set, R, is equal to 3, including eye-pointing model 1 through eye-pointing model 3. Any one of these eye-pointing models is denoted as eye-pointing model r, and r takes any positive integer greater than or equal to 1 and less than or equal to 3.

[0041] On the projection iris / pupil edge curve corresponding to any color camera focus model in the gaze-directed model r: Based on the mapping relationship between the arc length ratio parameter 'a' of any point P on the projected iris / pupil edge curve and the x-coordinate parameter 'x' of that point in the eye coordinate system, establish an arc length ratio-x-coordinate mapping function. ; Based on the mapping relationship between the arc length ratio parameter 'a' of any point P on the projected iris / pupil edge curve and the ordinate parameter 'y' of that point in the eye coordinate system, establish an arc length ratio-ordinate mapping function. ; Among them, or In the code, the X and Y characters in Xn and Yn are used to distinguish between the arc length ratio-x-axis mapping function and the arc length ratio-y-axis mapping function; r represents the mapping function that corresponds to the gaze-directing model r in this group. For example, when r is 1, the gaze-directing model 1 only corresponds to... or Correspondingly.

[0042] n represents the nth gaze-directing model in this group, that is, the gaze-directing model n corresponds to. For example, when r is 1 and n is 2, it means that in gaze-directing model 1, from the first color camera focus model to the second color camera focus model in the Nth color camera focus model, only the second color camera focus model corresponds to the first color camera focus model. or Correspondingly, the number n of the color camera focus model corresponds only to the arc length ratio-horizontal coordinate mapping function and the arc length ratio-vertical coordinate mapping function with the same number n, where 1≤n≤N and n is an integer.

[0043] The arc length ratio parameter 'a' is the ratio of the arc length traversed from the starting point A to any point on the projected iris / pupil edge curve to the total arc length of the projected iris / pupil edge curve, given a pre-determined set of starting points on the curve. Therefore, the arc length ratio parameter 0 ≤ a ≤ 1.

[0044] When determining the starting point A, there are two cases: determining the starting point A for the edge curve of the projected iris that is not closed, and determining the starting point A for the edge curve of the projected pupil that is closed. For the first scenario, refer to Figure 18 When determining the starting point A of the projection iris edge curve T14N that does not close the shape corresponding to a gaze pointing to the model, one set of endpoints from the two sets of endpoints corresponding to the beginning and end of a set of projection iris edge curves within the model can be determined as the starting point A.

[0045] Reference Figure 19 When determining the starting point A of the projection pupil edge curve T15N that is closed in shape corresponding to a gaze pointing to a model, any point on one of the projection pupil edge curves T15N can be taken as the starting point A. Using the starting point A as the tangent, the tangent point that is closer to the starting point A among the two tangent points on the other projection pupil edge curves that are parallel to the starting point A can be taken as the starting point A.

[0046] When determining the arc length ratio parameter a of the projected pupil edge curve T15N of a set of closed curves, the arc length ratio parameter a can be obtained by looking in the opposite direction of the Z-axis arrow of the eye coordinate system, starting from the starting point A, and taking the arc length traversed by any point P in the clockwise direction as the ratio of the arc length of the projected pupil edge curve where any point P is located.

[0047] Obviously, the above method for determining the arc length ratio parameter a is just an example. Its purpose is only to convert the position of any point on the curve into a numerical value. Other methods for converting the position of any point on the curve into a numerical value include the curvature ratio of the point on the curve, the arc length value traversed by the point on the curve, etc., all of which should be regarded as equivalent transformations of the arc length ratio parameter a.

[0048] The x-coordinate parameter X is the x-coordinate of any point P on the projected iris / pupil edge curve in the XY plane of the eye coordinate system.

[0049] The ordinate parameter Y is the ordinate value of any point P on the projected iris / pupil edge curve in the XY plane of the eye coordinate system.

[0050] It can be viewed as a mapping relationship between the arc length ratio parameter a corresponding to any point P on the edge curve of the projected iris / pupil corresponding to the focal point model of the nth color camera in model r, and the horizontal coordinate parameter X corresponding to that point.

[0051] This can be viewed as a mapping relationship between the arc length ratio parameter a corresponding to any point P on the projection iris / pupil edge curve of the nth color camera focus model in model r and the corresponding ordinate parameter Y.

[0052] Reference Figure 20 A set of eyes pointing to the corresponding model Figure 17 The first gaze in the model points to the first color camera focus model T1101' in the model, and the corresponding arc length ratio-horizontal coordinate mapping function and arc length ratio-vertical coordinate mapping function of the projected iris edge curve T1401 on the XY plane of the eye coordinate system are shown. It should be noted that the corresponding arc length ratio-horizontal coordinate or vertical coordinate mapping function of the projected pupil edge curve is similar.

[0053] and This can be viewed as expressing the shape of the projected iris / pupil edge curve corresponding to the nth color camera focus model in the model r using two functions. This is to facilitate the calculation and fitting in subsequent steps.

[0054] S412: Based on the first position parameters, second position parameters, third position parameters, arc length ratio-horizontal coordinate mapping function, and arc length ratio-vertical coordinate mapping function corresponding to the multiple color camera focus models in each gaze pointing model, we obtain the interpolation group containing the horizontal coordinate parameter, the interpolation group containing the vertical coordinate parameter, and the interpolation group containing the third position parameter corresponding to the gaze pointing model.

[0055] The first, second, and third position parameters are represented by the symbols ε, ζ, and L, respectively, and are used to determine the relative positional relationship between the color camera focus model and the eye coordinate system in the gaze pointing model. Alternatively, in practical applications, the relative positional relationship between the simulated focus and the eye coordinate system can be determined, and the iris / pupil edge curve can be generated based on the obtained relative positional relationship using the methods mentioned above. In this example, the first position parameter ε and the second position parameter ζ represent the angles between the orthographic projections of the line connecting the color camera focus model or simulated focus and the origin of the eye coordinate system onto the YZ and XZ planes of the eye coordinate system and the connecting line, respectively. It should be noted that when the x-value of the point coordinate of the color camera focus model or simulated focus in the eye coordinate system is positive, the first position parameter ε is positive, and vice versa; when the y-value of the point coordinate is positive, the second position parameter ζ is positive, and vice versa; the third position parameter L represents the distance between the color camera focus model or simulated focus and the origin of the eye coordinate system. Obviously, there are multiple methods to determine the relative position of the color camera focus model or simulated focus with respect to the eye coordinate system using three parameters. For example, when the eye coordinate system is a three-dimensional Cartesian coordinate system, the X, Y, and Z coordinate values ​​of the color camera focus model or simulated focus in the eye coordinate system should all be regarded as equivalent transformations of this application when used to achieve the purpose of this paper.

[0056] In this application, in any one of the gaze-pointing models r in a set of gaze-pointing models, from the first color camera focus model to the Nth color camera focus model, the corresponding parameters are set as follows: From the first color camera focus model, the first position parameter Up to the first position parameter of the Nth color camera focus model ; From the second position parameter of the focus model of the first color camera Up to the second position parameter of the Nth color camera focus model ; From the third position parameter of the first color camera focus model Up to the third position parameter of the Nth color camera focus model .

[0057] From the arc length ratio-horizontal coordinate mapping function corresponding to the focus model of the first color camera function value Up to the arc length ratio-horizontal coordinate mapping function corresponding to the N-color camera focus model. function value ; From the arc length ratio-ordinate mapping function corresponding to the focus model of the first color camera function value Up to the arc length ratio-ordinate mapping function corresponding to the N-color camera focus model. function value Furthermore, the meaning of 'r' in each positional parameter is the same as that mentioned above.

[0058] Only the first positional parameter Second positional parameter Third positional parameter The five parameters, namely the arc length ratio-horizontal coordinate mapping function and the arc length ratio-vertical coordinate mapping function, are derived from this color camera focus model and correspond to each other only when all five parameters have the same value as r in the nth color camera focus model in the gaze pointing model r of this set of gaze pointing models, and when all five parameters have the same value as r.

[0059] For example, when r and n are 1 and 2 respectively, it represents the first gaze pointing model in this set of gaze pointing models, that is, the second color camera focus model in gaze pointing model 1 only corresponds to: the first position parameter of the second color camera focus model, the second position parameter of the second color camera focus model, the third position parameter of the second color camera focus model, the arc length ratio-horizontal coordinate mapping function, and the arc length ratio-vertical coordinate mapping function.

[0060] In any one of the gaze-pointing models r in a set of gaze-pointing models: Based on the n color camera focus models respectively , and These are respectively formed into a set of numbers, that is, interpolations containing the horizontal coordinate, namely: , ... The interpolation values ​​containing the horizontal coordinate parameter for each of the n color camera focus models within this gaze-direction model form the interpolation set containing the horizontal coordinate parameter for that gaze-direction model, denoted as: ; Based on the n color camera focus models respectively , and These are respectively formed into a set of numbers, that is, interpolations containing the ordinate, as follows: , ...... In a total of n gaze-pointing models, the interpolations corresponding to the n color camera focus models, each containing a ordinate parameter, form the interpolation set corresponding to the gaze-pointing model, denoted as: ; Based on the n color camera focus models respectively , , These are respectively formed into a set of numbers, that is, interpolations containing a third positional parameter, namely: , ...... In a total of n gaze-pointing models, the interpolations corresponding to the n color camera focus models, each containing the third position parameter, form the interpolation group corresponding to the gaze-pointing model, denoted as: ; Among them, when , and The r in the model can only correspond to each other when the r in the gaze-pointing model takes the same number; The interpolation sets containing the horizontal coordinate parameter, the interpolation sets containing the vertical coordinate parameter, and the interpolation sets containing the third position parameter are used to establish corresponding polynomial functions, respectively denoted as: horizontal coordinate interpolation function, vertical coordinate interpolation function, and third position parameter interpolation function; The polynomial functions corresponding to the horizontal axis interpolation function, the vertical axis interpolation function, and the third position parameter interpolation function have smooth shapes and are used to fit the shape of the iris / pupil edge curve in the eye model when the simulated focus of the virtual camera and the eye model are at a certain relative position within a specific range. Using this iris / pupil edge curve as the guideline and the simulated focus as the vertex, a conical surface is constructed. This conical surface is used to simulate the shape of the light rays reflected from the iris / pupil edge region of the user's eyeball after refraction through the cornea and entering the camera when the simulated focus of the virtual camera and the eye model are at that relative position, corresponding to the real camera and the user's eyeball being at the same relative position.

[0061] S413: Based on the interpolation sets of the horizontal coordinate, vertical coordinate, and third position parameter corresponding to each gaze-pointing model, the following functions are obtained: horizontal coordinate interpolation function, vertical coordinate interpolation function, and third position parameter interpolation function. These three are used to form interpolation function group I corresponding to each gaze-pointing model. Interpolation function group I can map the shape of the projected iris / pupil edge curve and a specific value of the simulated focus third position parameter based on the simulated focus first and second position parameters within a range. Specifically, based on the interpolation groups containing the horizontal coordinate parameters corresponding to the focus models of the n color cameras... Interpolation with x-axis in , ...... Establish the x-coordinate interpolation function corresponding to the gaze-directing model r. ; Based on the interpolation groups corresponding to the focus models of n color cameras, including the ordinate parameter. Interpolation with ordinate in , ...... Establish the ordinate interpolation function corresponding to the gaze-directing model r. ; Based on the interpolation groups corresponding to the focus models of n color cameras, each containing a third position parameter. Interpolation with third position parameter , ...... Establish an interpolation function containing a third position parameter corresponding to the gaze-directing model r. ; It should be noted that the mathematical methods for establishing interpolation functions based on interpolation mentioned above include, but are not limited to, Newton's interpolation method and Taylor's interpolation method. Unlike the spline curve interpolation function established based on point coordinates, the abscissa interpolation function, ordinate interpolation function, and interpolation function containing a third position parameter mentioned in this article are all spline surface interpolation functions established based on point coordinates.

[0062] Regarding the x-axis interpolation function or ordinate interpolation function Or third position parameter interpolation function The geometric description of the shape is as follows: interpolation function with x-axis For example: Step S4131: In any one of the gaze pointing models r in a set of gaze pointing models, the corresponding interpolation group containing the horizontal coordinate parameter In the middle, the interpolation of the x-coordinates corresponding to the focus models of n color cameras is performed. , ...... Let these points be the coordinates of points 1, 2, ..., n respectively. In the Cartesian coordinate system, plot the z-axis coordinates of points 1, 2, ..., n respectively. The z-axis coordinates of points 1, 2, ..., n must all have the same value to ensure that the function is meaningful, and 0 ≤ a ≤ 1. like Figure 21 As shown, based on the first gaze in the first group pointing to model 1 The interpolation group corresponding to 1, including the x-axis parameter. When the value of the arc length ratio parameter a is set to 0.5, points 1, 2, ..., 16 are plotted in the Cartesian coordinate system. The shape of the function graph in the figure corresponds to the iris edge curve.

[0063] Step S4132: Refer to Figure 21 In the Cartesian scale, the shape of the smooth surface X43 connecting points 1, 2, ..., n is the interpolation function. The shape of the function graph in Cartesian coordinates when a is set to 0.5.

[0064] Figure 22 Here is a graph of the function mapping function of arc length ratio - x-axis corresponding to a set of gaze directions on the model: / ...... The corresponding function graph contains 16 curves, labeled as curve 21, curve 22, and so on, up to curve 36. When the value of 'a' takes different values ​​from 0 to 1, substituting these values ​​into the z-coordinates of points 1, 2, ..., n will result in different changes in the z-coordinates. Therefore, when the value of 'a' takes different values ​​from 0 to 1, the interpolation function... The shape of a figure will change differently in the Cartesian coordinate system, as shown in the reference. Figure 23 The first gaze in the first group points to model 1. The interpolation function corresponding to 1 with a coordinate. The shapes of the function graphs corresponding to the arc length ratio parameter a taking the values ​​of 0, 0.25, 0.5, 0.75 and 1 are represented by surfaces 41, 42, 43, 44 and 45, respectively.

[0065] y-coordinate interpolation function Geometric methods for describing the shape of a graph and interpolation functions for the x-coordinate. The geometric methods for describing the shape of the graphic are similar and will not be described in detail here; Third position parameter interpolation function Geometric methods for describing the shape of a graph and interpolation functions for the x-coordinate. The geometric methods for describing the shape of the figure are similar, the difference lies in the third position parameter interpolation function. The function has only two variables, ε and ζ, which correspond exactly to the X and Y axes in a Cartesian coordinate system. The third position parameter interpolation function... Unlike interpolation functions that include horizontal and vertical coordinates, the shape of the third position parameter interpolation function in the Cartesian coordinate system remains unchanged.

[0066] The following is a detailed description of the interpolation function set containing the position parameters of the color camera and the corresponding mapped iris edge curve shape parameters: For each gaze-pointing model r in a set of gaze-pointing models, the interpolation function set I is determined by the x-axis interpolation function. y-axis interpolation function And the third position parameter interpolation function Together they form; Therefore, the interpolation function set I corresponding to each gaze pointing model r in a set of gaze pointing models can be written as:

[0067] The interpolation function group can only correspond to each other if the value of r in the interpolation function group is consistent with the value of r in the model to which the gaze is directed. The domain of the interpolation function set I is the set of all points in the common orthographic projection region of the graphic shape in the eye coordinate system corresponding to the interpolation function set I of the eye-pointing model r on the XY screen of the eye coordinate system, for each pair of ε and ζ values, and 0≤a≤1.

[0068] To facilitate understanding, specific parameter settings are used to describe how the shape of the iris / pupil edge curve and the connection surface of the simulated focal point, under the conditions of the relative positional relationship between the camera and the user's eye under the first, second, and third positional parameters, and the reflection of light from the iris and sclera junction area in the user's eye and refraction by the transparent cornea, can simulate the shape of the conical surface formed by light entering the camera lens in reality. Special cases: Taking the gaze direction model 1 when r is 1 as an example, refer to Figure 17 The diagram shows the iris edge curve T1401 obtained based on the focus model T1101' of the first color camera in model 1, which is based on the gaze direction.

[0069] The first color camera focus model and the first position parameter corresponding to the eye coordinate system. Second position parameter Substitute the x-coordinate interpolation function corresponding to this color camera focus model Then, the mapping relationship between a and x in this x-coordinate interpolation function is equivalent to the arc length ratio - x-coordinate mapping function. The mapping relationship between 'a' and 'x' in the figure is referred to... Figure 20 The image at the top is The graph of the function is such that when the arc length ratio parameter 'a' takes five uniform values ​​from 0 to 1 (i.e., 0, 0.25, 0.5, 0.75, 1), these five values ​​of the arc length ratio parameter 'a' are substituted into the x-axis interpolation function. This yields 5 sets of numbers, namely , , , , .

[0070] Reference Figure 24 Using these 5 sets of numbers as the coordinates of points in the Cartesian coordinate system (where the first position parameter ε, the second position parameter ζ, and the x-coordinate value of the point in the eye coordinate system are used as the x, y, and z coordinates of the point respectively, and the value of the arc length ratio a is not used as the coordinate value of the point), we can get points 5101, 5201, 5301, 5401, and 5501 in the Cartesian coordinate system. Figure 24 The diagram also shows the first and second position parameters taken from the values ​​of [missing information]. and Until the end of the process and At that time, and substitute them into the equations respectively. Afterwards, the mapping relationship between a and x is equivalent to... Until Then, based on the arc length ratio a taking the same values ​​of 0, 0.25, 0.5, 0.75, and 1, the Cartesian coordinates reflect the following: from points 5102, 5202, 5302, 5402, and 5502, all the way to points 5116, 5216, 5316, 5416, and 5516. Figure 21 The document also presents the x-coordinate interpolation functions for arc length ratios 'a' of 0, 0.25, 0.5, 0.75, and 1. The shapes of the figures in the Cartesian coordinate system are, namely, surface X41, surface X42, surface X43, surface X44, and surface X45.

[0071] Following a similar method, direct the same gaze towards the model's first position parameter. ( =-5.049), second position parameter ( Substituting (=49.699) into the interpolation function corresponding to the ordinate of this color camera focus model... Furthermore, when the arc length ratio parameter 'a' takes the same five values ​​(0, 0.25, 0.5, 0.75, and 1), and these five values ​​of 'a' are substituted into the ordinate interpolation function... This yields 5 sets of numbers, namely: , , , , .

[0072] Reference Figure 25 Using these 5 sets of numbers as the coordinates of points in the Cartesian coordinate system (where the first position parameter ε, the second position parameter ζ, and the y-coordinate value of the point in the eye coordinate system are used as the x, y, and z coordinates of the point respectively, and the value of the arc length ratio a is not used as the coordinate value of the point), we can get points 6101, 6201, 6301, 6401, and 6501 in the Cartesian coordinate system. Figure 25 The diagram also shows the first and second position parameters taken from the values ​​of [missing information]. and Until the end of the process and At that time, and substitute them into the equations respectively. Afterwards, the mapping relationship between a and x is equivalent to... Until Then, based on the arc length ratio a taking the same values ​​of 0, 0.25, 0.5, 0.75, and 1, the Cartesian coordinates reflect the following: from points 6102, 6202, 6302, 6402, and 6502, all the way to points 6116, 6216, 6316, 6416, and 6516. Figure 25 The paper also presents the ordinate interpolation functions for arc length ratios 'a' of 0, 0.25, 0.5, 0.75, and 1. The shapes of the figures in the Cartesian coordinate system are, namely, surface Y41, surface XY2, surface Y43, surface Y44, and surface Y45.

[0073] Import the eye model and the simulated focus model into the simulated space, and direct the gaze towards the first and second position parameters corresponding to model 1. and Substitute the third position parameter interpolation function into the corresponding interpolation function group I. This yields the distance between the hypothetical simulated focal point and the center point of the pupil in the eye model, i.e., the third position parameter. =290.631

[0074] In the simulated space, the parameters of the first, second, and third position parameters, i.e. , , The relative positions of the simulated focal point and the eye coordinate system in the eye model are converted into the following: the angles between the line connecting the simulated focal point and the origin of the eye coordinate system and the orthographic projection of the line on the YZ and XZ planes of the eye coordinate system are 5.049 and 49.669, respectively. The x-value of the simulated focal point on the eye coordinate system is negative, and the y-value is positive. The distance from the simulated focal point to the center of the pupil of the eye model is 290.631 mm.

[0075] Focusing on the first position parameter corresponding to Model 1 Second position parameter After inputting the interpolation functions for the x-coordinate and y-coordinate of the model corresponding to the gaze direction, according to... Figure 24 and Figure 25The z-coordinates of the five pairs of points shown in the figure (5101 and 6101, 5201 and 6201, 5301 and 6301, 5401 and 6401, 5501 and 6501) in the Cartesian coordinate system are respectively used as the x and y coordinates of point I, point II, point III, point IV, and point V in the XY plane of the eye coordinate system. That is, the coordinates of point I, point II, point III, point IV, and point V in the eye coordinate system are (2.005, -5.668), (-0.726, -6.072), (-3.261, -4.971), (-4.862, -2.716), and (-5.135, 0.039). Connecting points I to IV using spline 81 yields an iris edge curve, the shape of which approximates... Figure 17 The projected iris edge curve T1401. Therefore: the conical shape constructed using this iris edge curve as the guideline and the simulated focal point as the vertex can be approximated as: the gaze pointing towards model 1. The shape of the iris edge reflection surface corresponding to the focal point model of the first color camera in model 1 is approximately the same as the shape of the gaze pointing model 1. When the iris edge reflects the curved surface in 1, the light reflected from the junction of the iris and sclera in the user's eye and refracted by the transparent cornea forms a cone shape that enters the lens of the first camera.

[0076] Similarly, an eye model and a simulated focus model are imported into the simulated space, and the gaze is directed to the first and second position parameters corresponding to model 1. and , and Until and Substituting each value into the interpolation function group I corresponding to this gaze-pointing model yields: the exact relative position between the simulated focal point and the eye coordinate system, the iris edge curve at the relative position, and the cone shape constructed with the iris edge curve as the directrix and the simulated focal point as the vertex. These values ​​can also simulate the cone shape formed by light reflected from the iris and sclera at the same relative position between the camera focal point and the user's eye in reality, refracted by the transparent cornea, and then incident on the camera lens.

[0077] General situation: Then, input any ε and ζ values ​​within the domain of the interpolation function group I corresponding to model 1; for example, take a random input with ε=5 and ζ=-29.

[0078] Substituting ε=5 and ζ=-29 into the third position parameter interpolation function The value of the specific third position parameter L is obtained, namely L=288.33; When a takes the values ​​0, 0.25, 0.5, 0.75, and 1 respectively, substitute ε=5 and ζ=-29 into the x-coordinate interpolation function. We obtained five corresponding values: 1.823, -0.852, -3.355, -4.966, and -5.266.

[0079] When a takes the values ​​0, 0.25, 0.5, 0.75, and 1 respectively, substituting ε=5 and ζ=-29 into the ordinate interpolation function... We obtained five corresponding values: 0.902, -1.786, -3.957, -4.975, and -4.581.

[0080] The final coordinates of the five points on the eye coordinate system, namely point I', point II', point III', point IV', and point V', are (1.823, 0.902), (-0.852, -1.786), (-3.355, -3.957), (-4.966, -4.975), and (-5.266, -4.581), respectively.

[0081] Because the x-coordinate value of point I' (1.823, 0.902) is 1.823, it can be seen as: setting the arc length ratio parameter a to 0 and substituting it into the x-coordinate interpolation function. To obtain the graphical shape corresponding to the function, refer to... Figure 24 Find the surface X41 in the curve, and find the height coordinate value corresponding to the point ε=5, ζ=-29 on the surface X41, which is 1.823.

[0082] The x-coordinate, y-coordinate, and height coordinates of points 5401, 5402, ..., 5416 that determine the shape of surface X41 in the Cartesian coordinate system are respectively: the values ​​of the first positional relationship between the gaze-directing model 1 and the eye coordinate system from the first color camera focus model to the sixteenth color camera focus model, and the values ​​of the second positional relationship between these 16 color camera focus models and the eye coordinate system, as well as the corresponding iris / pupil edge curves, as shown in the reference. Figure 15 The x-coordinate of point I' in the eye coordinate system is calculated by fitting the x-coordinate of all points with an arc length ratio of 0. In other words, the x-coordinate of point I' in the eye coordinate system is fitted based on the existing known conditions.

[0083] Similarly, the y-coordinate value of point I' (1.823, 0.902) is 0.902, which can be seen as: setting the arc length ratio parameter a to 0 and substituting it into the y-coordinate interpolation function. To obtain the graphical shape corresponding to the function, refer to... Figure 25Find the surface Y41 in the curve, and find the high coordinate value corresponding to the point ε=5, ζ=-29 on the surface Y41, which is 0.902.

[0084] Decide Figure 25 The x-coordinate, y-coordinate, and height coordinates of points 6401, 6402, ..., 6416 in the Cartesian coordinate system of the Y41 mid-curve are respectively: the values ​​of the first positional relationship between the gaze-directing model 1 and the eye coordinate system from the first color camera focus model to the sixteenth color camera focus model; the values ​​of the second positional relationship between these 16 color camera focus models and the eye coordinate system; and the corresponding iris / pupil edge curves, as shown in the reference. Figure 18 The y-coordinate of point A in the eye coordinate system is calculated by fitting the values ​​of all points with an arc length ratio of 0 to the y-coordinate of the point A. In other words, the y-coordinate of point A in the eye coordinate system is fitted based on existing known conditions.

[0085] Similarly, the coordinate values ​​of points II', III', IV', and V' are also determined by setting the arc length ratio parameter a to 0.25, 0.5, 0.75, and 1 respectively, and by using the aforementioned... and the aforementioned The coordinates of points B, C, D, and E in the eye coordinate system are obtained through mapping. That is, the coordinates of points B, C, D, and E in the eye coordinate system are fitted based on existing known conditions.

[0086] Therefore, the iris edge curve obtained by connecting points I', II', III', IV', and V' with splines is obtained by: the relative positions of the gaze pointing to the focal point model of the first color camera in model 1, all the way to the focal point model of the sixteenth color camera, and the eye coordinate system, as well as the corresponding projected iris edge curve shapes, and fitting them according to the relative positions of the virtual simulated focal point and the eye coordinate system (ε=5, ζ=-29). (The distance L between the simulated focal point and the pupil center point in the eye coordinate system is also fitted according to the existing known conditions.)

[0087] Therefore, by importing an eye model and a simulated focus model into the simulated space, and setting the relative positions of the virtual focus and the eye coordinate system to ε=5 and ζ=-29, the iris edge curve obtained by connecting points I', II', III', IV', and V' is used as the guideline. The cone shape made with the simulated focus whose relative position with the eye coordinate system satisfies ε=5, ζ=-29, and L=288.33 as the vertex can simulate the cone shape formed by light reflected from the iris and sclera in the user's eye and refracted by the transparent cornea when the camera focus and the user's eye are in the same relative position.

[0088] Similarly, when the first and second position parameters of the simulated focal point and eye coordinate system, namely ε and ζ, are not equal to 5 and -29, but are within the domain of the interpolation function group I corresponding to the gaze pointing model, the values ​​of the corresponding iris edge curve and the third position parameter can be fitted.

[0089] Step S42: By merging the interpolation function group I corresponding to each gaze pointing model in a set of gaze pointing models, we obtain the interpolation function group II corresponding to each set of gaze pointing models. The iris / pupil edge curve mapped by the interpolation function group II is more accurate and can describe more information about the iris edge than the iris / pupil edge curve mapped by the interpolation function group I corresponding to each gaze pointing model.

[0090] The following details step S42 using the processing of the iris edge curve as an example, while the method for processing the pupil edge curve is similar. Reference Figure 26 By substituting an arbitrary pair of first and second position parameters, i.e., arbitrary values ​​of ε and ζ, into the interpolation function group I corresponding to each gaze pointing model in the same set of gaze pointing models, the shapes of the corresponding iris edge curves obtained in the same eye coordinate system can overlap. However, the resulting iris edge curves have both intersecting and non-intersecting parts.

[0091] The overlap is possible because substituting a pair of ε and ζ values ​​into the interpolation function for the third position parameter corresponding to each gaze-pointing model in the same set yields approximately equal values ​​for the third position parameter L. Substituting the pair of ε and ζ values ​​into the interpolation functions for the horizontal and vertical coordinates corresponding to each gaze-pointing model in the same set yields a set of iris edge curves in an eye coordinate system. Under the relative position of the eye coordinate system and the simulated focal point, satisfying this pair of ε, ζ values ​​and approximately equal L values, the shapes of the conical surfaces constructed with the simulated focal point as the vertex and these iris edge curves as the directrix correspond to those in reality. When the relative position of the user's eyeball and the camera satisfies this pair of ε and ζ values ​​and approximately equal L values, the shapes formed by the light rays reflected from the junction of the iris and sclera in the user's eyeball and refracted by the transparent cornea, then entering the focal point of the camera lens, overlap. Therefore, the conical surfaces corresponding to these sets of iris edge curves can approximately overlap.

[0092] The explanation for the overlapping and non-overlapping parts: The iris edge curve obtained by inputting any ε and ζ values ​​into the interpolation function set of a gaze-pointing model can be seen as fitting the boundary region between the iris and sclera, partially obscured by the eyelids, captured simultaneously by color cameras at different positions. Therefore, it can only simulate the boundary region of the iris and sclera that can be captured. The interpolation function sets corresponding to different gaze-pointing models within the same set can be seen as obtaining curves obtained under different positional relationships between the eyeball and face when the user's eyeball and the camera module are in approximately the same position. Therefore, the boundary region between the iris and sclera, partially obscured by the eyelids, captured in the images will have different parts. Thus, the iris edge curves obtained by substituting any ε and ζ values ​​into the interpolation function set corresponding to the same gaze-pointing model will have both overlapping and non-overlapping parts.

[0093] The specific process of merging interpolation function group I in step S42 to obtain interpolation function group II corresponding to each gaze-direction model includes the following steps: S421: Based on at least R interpolation function sets corresponding to at least R gaze pointing models in a set of gaze pointing models, input N' pairs of specific simulated focus first and second position parameters to obtain N' sets of mapped iris / pupil edge curves corresponding to each pair of simulated focus first and second position parameters, with each set containing at least R iris / pupil edge curves: S4211: Based on the maximum R gaze pointing models contained in each gaze pointing model group, determine the first and second position parameters of the simulated focus for N' pairs of specific values ​​within the common intersection range of the domains of the corresponding interpolation function groups.

[0094] Set N'=N, that is, set N pairs of specific simulated focus first and second position parameters, where N is equal to the number of color cameras in the model that the gaze is pointing to, and name them respectively as the first pair of simulated focus first and second position parameters, the second pair of simulated focus first and second position parameters, ... the Nth pair of simulated focus first and second position parameters.

[0095] Taking the first pair of simulated focus and first and second position parameters as an example: let the first position parameter of the first pair of simulated focus be the average value of the first position parameters of the simulated focus of the first color camera of all gaze-directed models in that group, that is... , ...... The average value, i.e. Similarly, let the second position parameters of the first pair of simulated focal points be... , ...... The average value, i.e. ; That is, a set of gazes pointing to the model corresponds to: the first pair of simulated focal points and the first and second position parameters are... and Similarly, the first and second position parameters of the second pair of simulated focal points are: and Until the Nth pair of simulated focal points, the first and second position parameters are... and ; In this example, the first and second position parameters of the first pair of simulated focal points are respectively... and Up to the sixteenth pair of simulated focal points and first and second position parameters and The values ​​are set as follows: (-5.049, 45.792), (-20.692, 29.611), (-34.397, 12.904), (-45.792, -5.049), (12.904, 34.397), (-2.029, 16.759), (-16.759, -2.029), (-29.963, -20.692), (29.611, 20.692), (16.759, 2.029), (2.029, -16.759), (-12.904, -34.397), (45.792, 7.610), (34.397, -12.904), (20.692, -29.611), (5.049, -45.792). like Figure 27 As shown, the first and second position parameters of these 16 pairs of simulated focal points are respectively represented as x and y coordinate values ​​on the coordinate system as points 701, 702, and up to point 716.

[0096] S4212: Substitute the first and second position parameters of the simulated focal point for specific values ​​of N' into each gaze pointing model in this group to obtain N' groups, each with at least R iris edge curves: Each and , and ...... and Substituting each gaze pointing model in this group of gaze pointing models into the corresponding interpolation function group, we get the horizontal and vertical interpolation functions, respectively: , ...... This yields a total of r groups, with each group containing n pairs of arc length ratios, representing the mapping functions for the horizontal and vertical axes.

[0097] Following the method provided in step S413 above, n sets of corresponding iris edge curves are obtained based on each pair of arc length ratio-horizontal coordinate mapping functions and arc length ratio-vertical coordinate mapping functions, with R curves in each set.

[0098] Among them, the n sets of iris edge curves are: and , and ...... and Based on a set of interpolation functions mapped from a set of gaze-directing models, and according to the above argument, a set of iris edge curves can overlap.

[0099] Reference Figure 28 This example shows 16 sets of iris edge curves, which are: the first and second position parameters of the simulated focal point. and Up to the sixteenth pair of simulated focal points and first and second position parameters and The interpolation function groups for the gaze-pointing model were respectively substituted into the first group of iris edge curves 801, the second group of iris edge curves 802, and so on up to the sixteenth group of iris edge curves 816, and the iris edge curves in each group are approximately overlapping.

[0100] S422: Fine-tune a set of mapped iris / pupil edge curves corresponding to the first and second position parameters of each pair of simulated focal points until they overlap, and generate the projected iris / pupil edge curve II based on the overlapping set of iris / pupil edge curves. The shape of each spline corresponding to the set of iris edge curves is fine-tuned to overlap as much as possible. In this paper, the goal of fine-tuning is to compromise the shapes of the set of iris edge curves.

[0101] After fine-tuning the splines corresponding to the iris edge curves in the same group of N' iris edge curves until they are nearly overlapping, a new spline is used to outline the fine-tuned group of iris edge curves, resulting in the projected iris edge curve II corresponding to each group of iris edge curves. Figure 29 It is based on Figure 28 The first set of iris edge curves 801, the second set of iris edge curves 802, and so on up to the sixteenth set of iris edge curves 816, are used to obtain the projected iris edge curves II 901, II 902, and II 916. The projected iris edge curves II are more accurate because they are based on the corresponding set of iris edge curves.

[0102] S423: Based on the N' pairs of specific simulated focal point first and second position parameters corresponding to a set of gaze pointing models and the corresponding N' projected iris / pupil edge curves II, obtain the interpolation function group II corresponding to the set of gaze pointing models; the interpolation function group II includes the horizontal coordinate interpolation function II, the vertical coordinate interpolation function II and the third position parameter interpolation function II corresponding to the set of gaze pointing models.

[0103] Focusing a group of eyes on the first pair of simulated focal points and the first and second position parameters corresponding to the model. and The first and second position parameters of the second pair of simulated focal points and ...the first and second position parameters of the Nth pair of simulated focal points and N pairs of specific simulated focus and first and second position parameters are considered as: the first and second position parameters of the focus model of the first color camera corresponding to a single gaze pointing to the model, i.e. and The first and second position parameters of the focus model of the second color camera, i.e. and ...the first and second position parameters of the Nth color camera focus model, i.e. and .

[0104] The N pairs of specific simulated focal points corresponding to the gaze-pointing model and the N projected iris / pupil edge curves II, which correspond one-to-one with the first and second position parameters of the simulated focal points, are regarded as: the N pairs of color camera focal point models corresponding to the gaze-pointing model and the N projected iris / pupil edge curves, which correspond one-to-one with the first and second position parameters of the simulated focal points, respectively.

[0105] Following the method mentioned in step S41 of establishing the horizontal and vertical coordinate interpolation functions in the interpolation function group based on the N pairs of color camera focus models and their corresponding first and second position parameters, the horizontal coordinate interpolation function II is established based on the N pairs of specific simulated focus first and second position parameters and their corresponding N pairs of projected iris / pupil edge curves. The ordinate is connected to the interpolation function II. .

[0106] The implementation of the third position parameter interpolation function II: The first group of eyes was directed at the model corresponding to... With the Substituting the third position parameter interpolation function into the interpolation function group corresponding to each gaze pointing model in this set of gaze pointing models, that is: , ...... Obtain the values ​​of the corresponding r third positional parameters, i.e. , ...... The average value of the third position parameter is obtained by taking the average value. .

[0107] Using the same method, and , and ...... and Substituting these values ​​into the interpolation functions for the third position parameters of the respective gaze-directing models, we obtain the average values ​​of the corresponding N-1 third position parameters, which are respectively , ...... .

[0108] Based on the first and second position parameters of the first pair of simulated focal points corresponding to the model of this group of gazes. and The first and second position parameters of the second pair of simulated focal points and ...the first and second position parameters of the Nth pair of simulated focal points and and their respective corresponding , ...... Following the method mentioned in step S41 for establishing the third position parameter interpolation function, the third position parameter interpolation function II is established. Specifically, an interpolation group containing the third position parameter corresponding to the gaze-pointing model is first established, that is, Then, based on the interpolation group containing the third position parameter, establish the third position parameter interpolation function II corresponding to the gaze pointing model of that group, that is... .

[0109] Then, the interpolation function group II corresponding to each group of gazes pointing to the model can be written as:

[0110] Here, the domain of ε and ζ values ​​is the intersection of the domains of ε and ζ values ​​for the interpolation function set corresponding to each gaze-directing model. .

[0111] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for dynamically establishing iris and pupil edge curve models based on an eye model, comprising a gaze pointing measurement device for implementing the method and a gaze pointing model constructed using the gaze pointing measurement device, characterized in that, The dynamic creation method includes the following steps: Step 1: Based on the position parameters of the focal model and eye coordinate system of the multiple color camera models in each gaze pointing model, and the corresponding projection iris or pupil edge curve shape, obtain interpolation function group I corresponding to each gaze pointing model; the interpolation function group I maps the corresponding iris or pupil edge curve shape and a specific value of the simulation focal third position parameter according to the first and second position parameters of the simulated focal point. In step one, firstly, based on the projected iris or pupil edge curve, an arc length ratio-horizontal coordinate mapping function and an arc length ratio-vertical coordinate mapping function are established for the color camera focus model. Then, based on the three position parameters corresponding to the multiple color camera focus models in each gaze pointing model, as well as the arc length ratio-horizontal coordinate mapping function and the arc length ratio-vertical coordinate mapping function, interpolation sets containing horizontal coordinate parameters, interpolation sets containing vertical coordinate parameters, and interpolation sets containing the third position parameter are obtained for each gaze pointing model. Based on these three interpolation sets, three corresponding interpolation functions are established, and these three interpolation functions together constitute each gaze pointing model. The model corresponds to interpolation function group I; wherein, interpolation function group I includes a third position parameter interpolation function, which is used to determine the simulated focus third position parameter of the specific value based on the first position parameter and the second position parameter; wherein, the first position parameter, the second position parameter, and the third position parameter are used to determine the relative positional relationship between the simulated focus and the eye coordinate system; wherein, the three interpolation functions are polynomial functions; wherein, the projected iris or pupil edge curve includes: projected iris edge curve and projected pupil edge curve; wherein, the gaze pointing model includes a first camera focus model, the first camera focus is used to determine the position of the pupil center point; Step 2: By merging the interpolation function group I corresponding to each gaze pointing model in the multiple gaze pointing models, an interpolation function group II corresponding to the multiple gaze pointing models is obtained. Then, the corresponding iris or pupil edge curve II is mapped out through the interpolation function group II. In Step 2, based on at least R interpolation function groups corresponding to at least R gaze pointing models in the multiple gaze pointing models, N' pairs of specific simulated focus first and second position parameters are input to obtain N' groups of iris or pupil edge curves corresponding to each pair of simulated focus first and second position parameters, with each group containing at least R iris or pupil edge curves. The iris or pupil edge curves corresponding to each pair of simulated focus first and second position parameters are fine-tuned until they overlap, and the projected iris or pupil edge curve II is drawn based on the overlapping iris or pupil edge curves. Based on the N' pairs of specific simulated focus first and second position parameters corresponding to the multiple gaze pointing models, and the corresponding N' projected iris or pupil edge curves II, the interpolation function group II corresponding to the multiple gaze pointing models is obtained, and the corresponding iris or pupil edge curve II is mapped out. Here, R is a positive integer greater than or equal to 1.

2. The method for dynamically establishing iris and pupil edge curve models based on an eye model according to claim 1, characterized in that, In each of the multiple gaze pointing models, there is at least one gaze pointing model, and the distance between the point where the first camera focus model of each gaze pointing model is located and the point where the pupil center point is located is consistent.

3. The method for dynamically establishing iris and pupil edge curve models based on an eye model according to claim 1, characterized in that, The projected iris or pupil edge curve proposed in step one is based on an eye model, which is established by obtaining real user eye data from the camera module in the gaze pointing measurement device.

4. The method for dynamically establishing iris and pupil edge curve models based on an eye model according to claim 1, characterized in that, The polynomial function images corresponding to the three interpolation functions are all smooth in shape and are used to fit the shape of the iris or pupil edge curve in the eye model when the simulated focus of the virtual camera and the eye model are in a relative position within a specific range. This simulates the shape of the cone formed by the light reflected from the iris or pupil edge region of the user's eyeball after refraction through the cornea and entering the camera when the simulated focus of the virtual camera and the eye model are in that relative position, and the real camera and the user's eyeball are in the same relative position.

5. The method for dynamically establishing iris and pupil edge curve models based on an eye model according to claim 1, characterized in that, In the multiple gaze-pointing models, the interpolation function group I corresponding to each gaze-pointing model theoretically overlaps the shape of each iris edge curve obtained in the same eye coordinate system. However, in the actual mapping process, local intersections occur. The iris or pupil edge curves mapped to each pair of specific color camera position parameters are finely adjusted until they overlap. Based on the overlapping iris or pupil edge curves, the projected iris / pupil edge curve II is determined, thus obtaining the projected iris edge curve II corresponding to this set of iris edge curves.

Images