An eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction

Abstract

The application provides an eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction, and relates to the technical field of visual line tracking. The method comprises the following steps: based on the characteristics of the iris on the camera imaging, a space iris circle is reconstructed, the iris imaging elliptical cone is standardized through coordinate system transformation, and the iris imaging elliptical cone in the new coordinate system is obtained; based on the iris imaging elliptical cone, two circular sections are determined; based on the normal vectors of the two circular sections, the normal vector of the OCI plane is determined; based on the normal vector of the OCI plane, the vector of the three-dimensional cornea center and the camera optical center is determined, and the positions of the three-dimensional iris center, the three-dimensional cornea center and the cornea reflection point are further determined; taking the eyeball structure and the imaging relationship as constraints, based on the iris radius and the distance from the three-dimensional cornea center to the three-dimensional iris center, the three-dimensional cornea center and the three-dimensional iris center are obtained, and the optical axis of the eyeball is estimated through the three-dimensional cornea center and the three-dimensional iris center. The application can reduce the complexity of the eyeball parameter calculation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of gaze tracking, and particularly to an eyeball parameter positioning and eyeball optical axis estimation method and device based on iris reconstruction. BACKGROUND

[0002] Gaze tracking is a technology for analyzing the gaze direction or gaze point of a user from the facial or eye features of the user, and can realize automatic interaction and attention analysis based on vision. At present, gaze tracking has been applied in the fields of human-computer interaction, medical diagnosis, virtual reality and intelligent transportation. Gaze tracking methods are generally divided into appearance-based methods and model-based methods. The appearance-based method generally trains a mapping model between the appearance of a face or an eye and a gaze, to predict the gaze information corresponding to a new appearance. This method has lower requirements for system configuration and is more robust, but is affected by individual differences, head posture, gaze range and environmental differences, and the gaze accuracy is usually more than 3°. Realizing truly unconstrained gaze estimation still faces great challenges. At present, most gaze tracking devices use model-based tracking technology, which has wide application prospects.

[0003] The model-based method is generally divided into a two-dimensional mapping-based method and a three-dimensional model-based method. The two-dimensional mapping-based method is to establish a two-dimensional mapping function between eye movement characteristics and gaze landing points for gaze estimation. The three-dimensional model-based method is to solve three-dimensional eyeball parameters such as corneal center and eyeball center according to the eyeball structure and geometric imaging model, to estimate the eyeball optical axis, and then estimate the three-dimensional gaze according to the kappa angle. The three-dimensional model-based method is a method based on corneal reflection and pupil refraction, which reconstructs the eyeball optical axis by estimating the three-dimensional corneal center and three-dimensional pupil center, can track the head posture under the configuration of multiple cameras and multiple light sources, and simplifies the user calibration process.

[0004] In recent years, three-dimensional gaze estimation methods based on depth sensors have developed rapidly because three-dimensional gaze estimation models can be simplified with depth information, but in order to improve tracking accuracy, user calibration needs to calibrate the eyeball radius and the eyeball center and inner corner point vector. This method requires many parameters to be calibrated, and it is difficult to accurately detect facial features and estimate head posture in a simple system. Three-dimensional gaze estimation methods include methods based on corneal reflection and pupil refraction, methods based on depth sensors, and methods based on facial features. The method based on corneal reflection and pupil refraction has certain requirements for system configuration, and simplifying user calibration and improving gaze estimation performance usually comes at the expense of system complexity. Although the method based on depth sensors and the method based on facial features can estimate three-dimensional gaze using a single camera, in fact, the user calibration is relatively complex, and it is difficult to accurately estimate the three-dimensional eyeball center and head posture from a two-dimensional face image, especially the method based on facial features, which also needs to preset some eyeball invariant parameters with fixed values. These make the gaze estimation performance of the two methods not as good as the method based on corneal reflection and pupil refraction. It is challenging to achieve accurate three-dimensional gaze estimation based on simple user calibration in a simple system configuration. SUMMARY

[0005] In order to solve the technical problems of the three-dimensional gaze estimation method in the prior art, such as the requirement for system configuration, complex user calibration, and complex eyeball parameter calculation, embodiments of the present application provide an eyeball parameter positioning and eyeball optical axis estimation method and device based on iris reconstruction. The technical solution is as follows:

[0006] On the one hand, an eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction is provided, which is realized by an eyeball parameter positioning and eyeball optical axis estimation device based on iris reconstruction. The method comprises:

[0007] S1, based on the characteristics of the iris on the camera imaging, reconstruct the spatial iris circle, standardize the iris imaging elliptical cone through coordinate system transformation, and obtain the iris imaging elliptical cone in the new coordinate system;

[0008] S2, based on the iris imaging elliptical cone in the new coordinate system, determine two circular sections;

[0009] S3, calculate the intersection of the iris imaging elliptical cone and the two circular sections to determine the three-dimensional iris center;

[0010] S4, define the OCI plane of the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center; determine the normal vector of the OCI plane based on the normal vectors of the two circular sections; determine the vector of the three-dimensional corneal center and the camera optical center based on the normal vector of the OCI plane;

[0011] S5, determining the positions of the three-dimensional iris center, the three-dimensional cornea center and the corneal reflection point based on the vector of the three-dimensional cornea center and the camera optical center and the reflection relationship of the single light source;

[0012] S6, estimating the iris radius with individual differences and the distance from the three-dimensional cornea center to the three-dimensional iris center according to the positions of the three-dimensional iris center, the three-dimensional cornea center and the corneal reflection point, taking the eyeball structure and the imaging relationship as constraints; obtaining the three-dimensional cornea center and the three-dimensional iris center based on the iris radius and the distance from the three-dimensional cornea center to the three-dimensional iris center, and estimating the optical axis of the eyeball through the three-dimensional cornea center and the three-dimensional iris center.

[0013] In another aspect, an eyeball parameter positioning and eyeball optical axis estimation device based on iris reconstruction is provided, which is applied to an eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction, and the device comprises:

[0014] An acquisition unit is configured to reconstruct a spatial iris circle based on the characteristics of the iris on the camera imaging, standardize the iris imaging elliptical cone through coordinate system transformation, and obtain the iris imaging elliptical cone in a new coordinate system;

[0015] A first determination unit is configured to determine two circular sections based on the iris imaging elliptical cone in the new coordinate system;

[0016] A second determination unit is configured to calculate the intersection of the iris imaging elliptical cone and the two circular sections, and determine the three-dimensional iris center;

[0017] A third determination unit is configured to define an OCI plane of the three-dimensional cornea center, the three-dimensional iris center and the camera optical center, determine the normal vector of the OCI plane based on the normal vectors of the two circular sections, and determine the vector of the three-dimensional cornea center and the camera optical center based on the normal vector of the OCI plane;

[0018] A fourth determination unit is configured to determine the positions of the three-dimensional iris center, the three-dimensional cornea center and the corneal reflection point based on the vector of the three-dimensional cornea center and the camera optical center and the reflection relationship of the single light source;

[0019] An estimation unit is configured to estimate the iris radius with individual differences and the distance from the three-dimensional cornea center to the three-dimensional iris center according to the positions of the three-dimensional iris center, the three-dimensional cornea center and the corneal reflection point, taking the eyeball structure and the imaging relationship as constraints; obtain the three-dimensional cornea center and the three-dimensional iris center based on the iris radius and the distance from the three-dimensional cornea center to the three-dimensional iris center, and estimate the optical axis of the eyeball through the three-dimensional cornea center and the three-dimensional iris center.

[0020] In another aspect, an iris reconstruction based eye parameter positioning and eye optical axis estimation device is provided, comprising: a processor; a memory having computer readable instructions stored thereon, the computer readable instructions, when executed by the processor, implement any one of the above iris reconstruction based eye parameter positioning and eye optical axis estimation methods.

[0021] In another aspect, a computer readable storage medium is provided, the storage medium having at least one instruction stored therein, the at least one instruction being loaded and executed by a processor to implement any one of the above iris reconstruction based eye parameter positioning and eye optical axis estimation methods.

[0022] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0023] The embodiments of the present application first reconstruct a spatial iris circle based on the characteristics of the iris on the camera imaging, standardize the iris imaging elliptical cone through coordinate system transformation to obtain the iris imaging elliptical cone in the new coordinate system; determine two circular sections based on the iris imaging elliptical cone in the new coordinate system; secondly, calculate the intersection of the iris imaging elliptical cone and the two circular sections to determine the three-dimensional iris center; define an OCI plane of the three-dimensional corneal center, the three-dimensional iris center and the camera optical center; determine the normal vector of the OCI plane based on the normal vectors of the two circular sections; determine the vector of the three-dimensional corneal center and the camera optical center based on the normal vector of the OCI plane; determine the positions of the three-dimensional iris center, the three-dimensional corneal center and the corneal reflection point based on the vector of the three-dimensional corneal center and the camera optical center and the reflection relationship of a single light source; finally, estimate the iris radius with individual differences and the distance from the three-dimensional corneal center to the three-dimensional iris center according to the positions of the three-dimensional iris center, the three-dimensional corneal center and the corneal reflection point, with the eye structure and the imaging relationship as constraints; obtain the three-dimensional corneal center and the three-dimensional iris center based on the iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center, and estimate the optical axis of the eye through the three-dimensional corneal center and the three-dimensional iris center.

[0024] The present application reconstructs the spatial iris circle by using the characteristics of the iris on the imaging surface of the camera, normalizes the iris imaging elliptical cone to a coordinate system with the direction of the axis of the iris imaging elliptical cone as the Z' axis, the direction of the long axis of the ellipse as the Y' axis and the direction of the short axis of the ellipse as the X' axis through coordinate system transformation, projects the circular section taken on the iris imaging elliptical cone to a straight line on the X'OZ' plane in the coordinate system, and the two cases of the circular section are symmetrical about the axis of the iris imaging elliptical cone, which is convenient for solving the normal vector and the center of the circular section. The normal vector of the circular section is determined by rotating the ellipse on the X'OY' plane to obtain the straight line equation of the circular section on the X'OZ' plane, and then the direction vector perpendicular to the straight line is taken as the normal vector, which can determine two solutions according to the imaging characteristics of the iris. The center of the circular section is solved by defining a circular section passing through the point of X'=0 and Z'=1 to determine the center and radius of the circle, and then representing the three-dimensional iris center coordinates in the camera coordinate system according to the similarity principle, which are related to the spatial iris radius.

[0025] ​According to the basic structure of the eyeball, the line connecting the three-dimensional corneal center and the three-dimensional iris center, i.e. the optical axis direction of the eyeball, is perpendicular to the spatial iris plane, and the spatial iris edge point should be coplanar with the three-dimensional iris center, thereby establishing 2n-2 equations. According to the reflection relationship of a light source on the outer surface of the cornea in the system, one equation is established, and a total of 2n-1 equations are established. In order to accurately locate the eyeball parameters, the parameters in the above equations are appropriately transformed; the three-dimensional iris center in the equation is related to the spatial iris radius and can be represented by the spatial iris radius. The embodiment of the application innovatively defines an OCI plane containing the three-dimensional corneal center, the three-dimensional iris center and the camera optical center, and determines the normal vector of the OCI plane according to two solutions of the spatial iris normal vector, thereby intersecting with the corneal reflection plane of a light source to determine the vector OC of the three-dimensional corneal center and the camera optical center. The estimation of the three-dimensional corneal center is simplified to solving a scale factor, which is represented by the three-dimensional iris center and the distance from the three-dimensional corneal center to the three-dimensional iris center according to the geometric imaging relationship, thereby reducing the demand for system configuration. The spatial iris edge points are imaged in the camera, so each spatial iris edge point can be represented by an imaging point multiplied by a scale factor, and the spatial iris is approximately circular, so according to the distance from the spatial iris edge point to the three-dimensional iris center being equal to the spatial iris radius, the scale factor can be represented by the three-dimensional iris center and the spatial iris radius. The corneal reflection point is imaged in the camera, so the reflection point of the system light source on the cornea can be represented by a bright spot coordinate multiplied by a scale factor, and the corneal reflection point is on the outer surface of the cornea, according to the structure of the eyeball, under the assumption of the spatial iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center, the corneal radius can be represented, and according to the distance from the corneal reflection point to the three-dimensional corneal center being equal to the corneal radius, the scale factor can be represented by the three-dimensional corneal center, the spatial iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center.

[0026] Based on the above parameter transformation, the parameters in the equation are converted to two unknowns, i.e. the spatial iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center, through the known distance constraint and the visual parameter, and both of the two parameters are eyeball invariant parameters, which have a theoretical range, so in the parameter optimization, on the one hand, the unknown parameters are easy to set the initial value; on the other hand, compared with optimizing the above multiple scale factors, optimizing the two eyeball invariant parameters reduces the calculation complexity, improves the stability and convergence. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

[0028] Figure 1 is a flow chart of an eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction provided by the embodiments of the present application.

[0029] Figure 2 is a schematic diagram of an eyeball basic structure provided by the embodiments of the present application.

[0030] Figure 3 is a schematic diagram of iris geometric imaging relationship provided by the embodiments of the present application.

[0031] Figure 4 is a schematic diagram of a circle section cut on an elliptical cone provided by the embodiments of the present application.

[0032] Figure 5 is a schematic diagram of OCI plane construction provided by the embodiments of the present application.

[0033] Figure 6 is a schematic diagram of imaging relationship of iris edge points provided by the embodiments of the present application.

[0034] Figure 7 is a schematic diagram of corneal reflection relationship of a light source provided by the embodiments of the present application.

[0035] Figure 8 is a block diagram of an eyeball parameter positioning and eyeball optical axis estimation device based on iris reconstruction provided by the embodiments of the present application.

[0036] Figure 9 is a schematic diagram of a structure of an eyeball parameter positioning and eyeball optical axis estimation device based on iris reconstruction provided by the embodiments of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the present application will be described below with reference to the drawings.

[0038] In the embodiments of the present application, the words such as "example", "for example" are used to represent as an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two options.

[0039] In the embodiments of the present application, "image" and "picture" can be used interchangeably, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized.

[0040] In the embodiments of the present application, sometimes the subscript such as W1 can be written in the form of non-subscript such as W1, and the meanings expressed are consistent when the distinction is not emphasized.

[0041] In order to make the technical problems, technical solutions and advantages to be solved by the present application more clear, the following will be described in detail in conjunction with the drawings and specific embodiments.

[0042] The embodiment of the present application provides an eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction, which can be realized by an eyeball parameter positioning and eyeball optical axis estimation device based on iris reconstruction. The eyeball parameter positioning and eyeball optical axis estimation device based on iris reconstruction can be a terminal or a server. As shown in the flow chart of the eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction, the processing flow of the method can include the following steps: Figure 1

[0043] S1, based on the characteristics of the iris on the camera imaging, reconstruct the spatial iris circle, standardize the iris imaging elliptical cone through coordinate system transformation, and obtain the iris imaging elliptical cone in the new coordinate system.

[0044] Among them, the iris is a flat circular thin film in the middle layer of the eyeball wall, which can be regarded as a spatial circular target, and the imaging in the camera is approximately elliptical. As shown in the flow chart of the eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction, the processing flow of the method can include the following steps: Figure 2 As shown in the flow chart of the eyeball parameter positioning and eyeball optical axis estimation method based on iris reconstruction, the processing flow of the method can include the following steps:

[0045] Optionally, the specific implementation process of S1 includes S11-S13:

[0046] S11, based on the geometric imaging relationship of the iris in the camera, the iris imaging elliptical cone is constructed with the camera optical center as the vertex;

[0047] Among them, the elliptical feature of the iris on the image plane is assumed to be an ellipse center , the length of the long semi-axis of the ellipse , the length of the short semi-axis of the ellipse ​and the angle between the major axis of the ellipse and the horizontal direction To represent the geometric imaging relationship of the iris in a camera, it can be depicted as an elliptical cone with the camera's optical center O as its vertex and the iris imaging ellipse as its base, as shown in the attached figure. Figure 3 As shown, the equation of the elliptic cone with the iris imaging ellipse as its base is expressed by the following formula (1):

[0048] (1)

[0049] in, Let A be a point on the elliptic cone, and let A represent the first coefficient of the equation, B the second coefficient, C the third coefficient, D the fourth coefficient, E the fifth coefficient, and F the sixth coefficient. The coefficients of these equations can be obtained using the parameters of the iris imaging ellipse. and camera focal length Calculated.

[0050] The first coefficient A of the equation is represented by the following formula (2):

[0051] (2)

[0052] The second coefficient B of the equation is expressed by the following formula (3):

[0053] (3)

[0054] The third coefficient C of the equation is expressed by the following formula (4):

[0055] (4)

[0056] The fourth coefficient D of the equation is expressed by the following formula (5):

[0057] (5)

[0058] The fifth coefficient E of the equation is expressed by the following formula (6):

[0059] (6)

[0060] The sixth coefficient F of the equation is expressed by the following formula (7):

[0061] (7)

[0062] S12, in the original camera coordinate system O-XYZ, the iris imaging elliptical cone is represented by a real symmetric matrix; a diagonalization matrix of the real symmetric matrix is calculated; eigenvalues of the diagonalization matrix are obtained according to the diagonalization matrix; a new coordinate system O-X'Y'Z' is constructed based on the original camera coordinate system O-XYZ;

[0063] wherein, in the original camera coordinate system O-XYZ, the iris imaging elliptical cone can be represented by a real symmetric matrix:

[0064] wherein, ;

[0065] the diagonalization matrix of the real symmetric matrix satisfies ; ;

[0066] wherein, the diagonalization matrix , is the first eigenvalue of the real symmetric matrix; is the second eigenvalue of the real symmetric matrix; is the third eigenvalue of the real symmetric matrix; wherein the normalized eigenvector corresponding to the first eigenvalue can be represented as ; the normalized eigenvector corresponding to the second eigenvalue can be represented as ; and the normalized eigenvector corresponding to the third eigenvalue can be represented as .

[0067] S13, in the new coordinate system O-X'Y'Z', a standardized iris imaging elliptical cone is constructed based on the eigenvalues of the diagonalization matrix.

[0068] wherein, the Z' axis is consistent with the direction of the axis of the iris imaging elliptical cone, the Y' axis is consistent with the direction of the major axis of the ellipse, and the X' axis is consistent with the direction of the minor axis; as shown in the accompanying drawings, the O-XYZ' coordinate system and the O-X'Y'Z' coordinate system satisfy: Figure 3 , that is, the iris imaging elliptical cone can be represented as:

[0069] wherein, the eigenvalues of the real symmetric matrix satisfy the following ordering rules: , , and .

[0070] S2, based on the iris imaging elliptical cone in the new coordinate system, two circular sections are determined.

[0071] ​In one feasible implementation, in the coordinate system O-X'Y'Z', a plane parallel to the X'OY' plane will intersect an ellipse on the iris elliptical cone. There are two cases where a circular cross-section is intersected on the iris elliptical cone. Based on the above standardized operation, the circular cross-section projected onto the X'OZ' plane is a straight line, as shown in the attached figure. Figure 4 As shown. The equations of the projection lines of the two circular sections onto the X'OZ' plane are expressed by the following formula (8):

[0072] (8)

[0073] in, This is a constant, determined by the radius of the circular section being cut. The normal vector corresponding to the circular section can be expressed as: The normal vectors of the two circular sections are obtained by normalizing the normal vectors of the circular sections: The normalized normal vectors of the two circular cross-sections are transformed back to the original coordinate system O-XYZ to obtain the normal vectors of the circular cross-sections in the camera coordinate system O-XYZ: .

[0074] Among them, such as Figure 4 As shown, the projection of the iris elliptic cone onto the X'OZ' plane results in two edges. According to the equation of the iris elliptic cone, the equations of the two edges are expressed by the following formula (9):

[0075] (9)

[0076] S3. Calculate the intersection of the iris imaging elliptical cone with the two circular cross sections to determine the three-dimensional iris center.

[0077] In one feasible implementation, the equations of the projection lines of the two circular cross sections onto the X'OZ' plane and the equations of the two edges are established simultaneously, and then... This allows us to obtain the intersection of the iris imaging elliptical cone with two circular cross sections.

[0078] Optionally, the calculation process of the intersection point of the iris imaging elliptical cone and the two circular sections is expressed by the following formulas (10)-(13):

[0079] (10)

[0080] (11)

[0081] (12)

[0082] (13)

[0083] in, Denotes the first eigenvalue of a real symmetric matrix; Denotes the second eigenvalue of a real symmetric matrix; The third eigenvalue of a real symmetric matrix is ​​represented. The coordinates of the intersection point of the elliptical cone edge 1 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the first circular section with intercept 1; The coordinates of the intersection point of the elliptical cone edge 2 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the first circular section with intercept 1; The coordinates of the intersection point of the elliptical cone edge 1 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the second circular section with intercept 1; The coordinates represent the intersection of the elliptical cone edge 2 projected onto the X'OZ' plane by the iris elliptical cone and the straight line projected onto the X'OZ' plane by the second circular section corresponding to intercept 1.

[0084] Among them, the intersection of the iris imaging elliptical cone and the two circular cross sections. to The radius corresponding to the circular cross section can be calculated using the following formula (14):

[0085] (14)

[0086] in, is the first eigenvalue of a real symmetric matrix; This is the second eigenvalue of a real symmetric matrix; is the third eigenvalue of a real symmetric matrix;

[0087] Assume the radius of the spatial iris circle is According to the principle of similar triangles, we know that: ;but ;in, This indicates the radius corresponding to the circular cross-section.

[0088] From equation (8), it can be seen that the equation for the spatial iris circle cross-section can be expressed as the spatial iris circle radius is... It can be represented as follows: Intersecting with the edge of the iris elliptic cone, we get: , , , The three-dimensional iris center is obtained by observing the intersection of the edges of the iris elliptical cone.

[0089] Optionally, the three-dimensional iris center is represented by the following formulas (15)-(16):

[0090] (15)

[0091] (16)

[0092] in, This represents the center of the first three-dimensional iris in the coordinate system O-X'Y'Z'; This represents the center of the second three-dimensional iris in the coordinate system O-X'Y'Z'; Denotes the first eigenvalue of a real symmetric matrix; Denotes the second eigenvalue of a real symmetric matrix; The third eigenvalue of a real symmetric matrix is ​​represented. The elliptic cone edge 1, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane from the first circular section; The elliptic cone edge 2, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane from the first circular section; The elliptic cone edge 1, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane by the second circular section; The elliptic cone edge 2, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane by the second circular section; r represents the radius of the spatial iris circle.

[0093] Since there are two cases for cutting a circular cross-section on the iris elliptic cone, there are also two solutions for the three-dimensional iris center. and Transforming them to the original camera coordinate system O-XYZ, we obtain the three-dimensional iris center in the camera coordinate system O-XYZ as follows: , According to the spatial geometric imaging relationship, the unique correct solution should satisfy that the y-component and z-component of the normal vector of the circular section have opposite signs. Therefore, it can be obtained from... and Select a unique solution from the options, and use It can be seen from equations (15) and (16) that the three-dimensional iris center Available spatial iris radius Show it.

[0094] S4. Define the OCI plane with the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center; determine the normal vector of the OCI plane based on the normal vectors of the two circular sections; determine the vectors of the three-dimensional corneal center and the camera optical center based on the normal vector of the OCI plane.

[0095] Optionally, the specific implementation process of S4 includes S41-S44:

[0096] S41. Based on the structure of the eyeball, construct an OCI plane that includes the camera optical center, the three-dimensional corneal center, and the three-dimensional iris center, based on the camera optical center, the iris imaging ellipse center, and the iris normal vector.

[0097] Based on the structure of the eyeball, the optical axis of the eyeball passes through the three-dimensional corneal center and the three-dimensional iris center, and the normal vector of the spatial iris is in the same direction as the optical axis of the eyeball.

[0098] S42. Based on the normal vectors of the two circular cross sections, construct the normal vector of the OCI plane through the symmetric relationship with the straight line connecting the optical center of the camera and the center of the iris imaging ellipse;

[0099] Among them, such as Figure 5 The diagram shows the construction of the OCI plane; there are two circular sections cut from the iris elliptical cone, assuming the normal vectors of the two circular sections are respectively... and This indicates that, due to their connection to the camera's optical center... and the center of the iris imaging ellipse The line is symmetric; the normal vector of the OCI plane is expressed by the following formula (17):

[0100] (17)

[0101] S43. Based on the light source reflection relationship and the collinear relationship between the reflection point, the camera optical center, and the Pulchin spot, determine the normal vector of the light source reflection plane;

[0102] Among them, as attached Figure 6 The image shows the imaging relationship of iris edge points, spatial iris edge points. Through the camera's optical center The image is captured by the camera as Because the optical axis of the eyeball and the normal vector of the iris in space are in the same direction, the three-dimensional corneal center... With the 3D Iris Center The line connecting the two is perpendicular to the iris plane, and their relationship can be expressed by the following formula (18):

[0103] (18)

[0104] Among them, each iris edge point All are related to the 3D Iris Center Within the same iris plane, the conditions are expressed by the following formula (19):

[0105] (19)

[0106] in, The spatial iris edge point is represented as , This is the proportionality coefficient. For imaging points.

[0107] In this model, the iris is approximately circular, and the distance from each edge point of the spatial iris to the center of the three-dimensional iris is equal to the radius of the iris, which can be expressed as: Convert the iris radius expression into a proportional figure. The quadratic equation in one variable can be expressed by the following formula (20):

[0108] (20)

[0109] Among them, the solution of the above proportional coefficient can be determined by using formula (18) as a constraint condition, then the spatial iris edge point Three-dimensional iris center can be used and iris radius express.

[0110] S44. Determine the vectors of the three-dimensional corneal center and the camera optical center based on the normal vector of the light source reflection plane and the normal vector of the OCI plane.

[0111] One feasible implementation method is, for example Figure 7 The diagram shows the corneal reflection relationship of a light source; when using a single light source, the light source... Reflection occurs on the outer surface of the cornea, and the reflection point is... Through the camera's optical center The image is captured by the camera as According to the reflection relationship of the light source, the incident ray is the light source. With reflection point The line connecting the points, with the normal being the center of the three-dimensional cornea. With reflection point The line connecting the two points, with the reflected ray being the reflection point. With camera optical center The lines connecting the points. According to the law of reflection, there exists a light source. Reflection point Three-dimensional corneal center and camera optical center The four points are collinear. Due to the reflection point... Camera optical center With Pulchin If the three points are collinear, then the normal vector of the light source's reflecting plane is expressed by the following formula (21):

[0112] (twenty one)

[0113] S5. Based on the vector relationship between the three-dimensional corneal center and the camera optical center and the reflection relationship of a single light source, determine the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point.

[0114] Optionally, the specific implementation process of S5 includes S51-S55:

[0115] S51. Based on the vectors of the three-dimensional corneal center and the camera optical center, construct the relationship between the vectors of the three-dimensional corneal center and the camera optical center.

[0116] Since the angle of incidence equals the angle of reflection, the reflection point is... ,light source Camera optical center and three-dimensional corneal center The following relationship (22) must be satisfied:

[0117] (twenty two)

[0118] Among them, due to the three-dimensional corneal center and camera optical center Not only in the OCI plane, but also in the light source reflection plane, according to formulas (17) and (20), The unit direction vector is: .

[0119] S52. Based on the vector relationship between the three-dimensional corneal center and the camera optical center, the scaling factor of the three-dimensional corneal center is solved by setting the distance conditions satisfied by the three-dimensional corneal center and the three-dimensional iris center; the position of the three-dimensional corneal center is obtained based on the scaling factor.

[0120] Among them, the three-dimensional corneal center It can be expressed by the following formula (23):

[0121] (twenty three)

[0122] in, This is the proportionality coefficient.

[0123] Among them, assuming a three-dimensional corneal center To the 3D Iris Center The distance is ,satisfy: ; convert it to an unknown as The quadratic equation of .

[0124] Optionally, the process of solving for the proportional coefficient of the three-dimensional corneal center is expressed by the following formula (24):

[0125] (twenty four)

[0126] in, The scaling factor representing the three-dimensional corneal center; The unit vector representing the three-dimensional corneal center and the camera optical center; Represents the coordinates of the three-dimensional iris center; Represents the three-dimensional corneal center To the 3D Iris Center The distance.

[0127] Since the center of the three-dimensional cornea should be behind the center of the three-dimensional iris, therefore Taking the largest solution, the three-dimensional corneal center Three-dimensional iris center can be used and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris express.

[0128] S53. Based on the collinear relationship between the corneal reflection point, the camera optical center, and the bright spot, construct the relationship between the corneal reflection point and the bright spot;

[0129] Among them, due to the reflection point Camera optical center With bright spots Three points collinear, corneal reflex point It can be expressed by the following formula (25):

[0130] (25)

[0131] in, This is the proportionality coefficient.

[0132] S54. Based on the relationship between the corneal reflection point and the bright spot, by setting the distance condition between the corneal reflection point and the three-dimensional corneal center, solve for the scaling factor of the corneal reflection point; based on the scaling factor, obtain the position of the corneal reflection point.

[0133] In one feasible implementation, due to corneal reflex points With the three-dimensional corneal center The distance condition is expressed by the following formula (26):

[0134] (26)

[0135] corneal reflex point With the three-dimensional corneal center The distance condition formula is converted into coefficients. The quadratic equation of .

[0136] The process of solving the proportional coefficient of the corneal reflective point is shown in the following formula (27):

[0137] (27)

[0138] In this case, the light source is reflected at the very front of the cornea, therefore Taking the minimum solution, the corneal reflex point is... Three-dimensional corneal center can be used iris radius and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris express.

[0139] S55. Based on the relationship that the incident angle equals the reflection angle, determine that the three-dimensional corneal center, the camera optical center, and the corneal reflection point meet the conditions. By meeting the conditions, obtain the position of the three-dimensional iris center.

[0140] Among them, a system of equations is established based on equations (18)-(19) and (22), and the number of equations is . Through transformation, the system of equations contains only two unknowns: iris radius. and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris Using the least squares method, the iris radius... and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris The following conditions must be met:

[0141]

[0142] in, , , ;in, This indicates that the three-dimensional corneal center is satisfied. With the 3D Iris Center The line connecting the iris plane is perpendicular to the optimization objective function constrained by the iris plane. This indicates that each iris edge point is satisfied. All are related to the 3D Iris Center The objective function constrained within the same iris plane; This represents the objective function that satisfies the constraint that the incident angle equals the reflection angle during the reflection of the light source on the outer surface of the cornea.

[0143] S6. Based on the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point, and constrained by the eye structure and imaging relationship, estimate the iris radius, which varies with individuals, and the distance from the three-dimensional corneal center to the three-dimensional iris center; based on the iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center, obtain the three-dimensional corneal center and the three-dimensional iris center, and estimate the optical axis of the eyeball through the three-dimensional corneal center and the three-dimensional iris center.

[0144] Among them, the iris radius is solved. and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris Subsequently, the ocular parameters referred to in this invention are: three-dimensional corneal center and 3D Iris Center Based on the structure of the eyeball, the three-dimensional corneal center is used. and 3D Iris Center The optical axis of the eyeball can be estimated.

[0145] This invention first reconstructs the spatial iris circle based on the characteristics of the iris in camera imaging. Then, it standardizes the iris imaging elliptical cone through coordinate system transformation to obtain the iris imaging elliptical cone in the new coordinate system. Based on the iris imaging elliptical cone in the new coordinate system, two circular cross-sections are determined. Next, the intersection points of the iris imaging elliptical cone and the two circular cross-sections are calculated to determine the three-dimensional iris center. An OCI plane is defined with respect to the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center. Based on the normal vectors of the two circular cross-sections, the normal vector of the OCI plane is determined. Based on the normal vector of the OCI plane, the three-dimensional corneal center and the phase... The vector of the camera's optical center is determined; based on the vectors of the 3D corneal center and the camera's optical center and the reflection relationship of a single light source, the positions of the 3D iris center, the 3D corneal center, and the corneal reflection point are determined; finally, based on the positions of the 3D iris center, the 3D corneal center, and the corneal reflection point, and constrained by the eye structure and imaging relationship, the individual-variable iris radius and the distance from the 3D corneal center to the 3D iris center are estimated; based on the iris radius and the distance from the 3D corneal center to the 3D iris center, the 3D corneal center and the 3D iris center are obtained, and the optical axis of the eyeball is estimated through the 3D corneal center and the 3D iris center.

[0146] This invention reconstructs the spatial iris circle using the characteristics of the iris on the camera's imaging plane. Through coordinate system transformation, the iris imaging elliptical cone is standardized to a coordinate system with the iris imaging elliptical cone's axis as the Z' axis, the major axis as the Y' axis, and the minor axis as the X' axis. In this coordinate system, the circular cross-section intercepted on the iris imaging elliptical cone projects onto the X'OZ' plane as a straight line. Furthermore, the two cases of intercepting the circular cross-section are symmetrical about the iris imaging elliptical cone's axis, facilitating the calculation of the normal vector and center of the circular cross-section. The normal vector of the circular cross-section is determined by rotating the ellipse in the X'OY' plane to determine the equation of the straight line on the X'OZ' plane. The direction vector perpendicular to this equation is then used as the normal vector, which can determine two solutions based on the iris imaging characteristics. The solution for the center of the circular cross section is to first define a circular cross section passing through the point X'=0, Z'=1 to determine the center and radius of the circle. Then, based on the principle of similarity, the coordinates of the three-dimensional iris center with a spatial iris radius of r are characterized. Finally, the coordinates of the three-dimensional iris center in the camera coordinate system are obtained through coordinate system transformation, which is related to the spatial iris radius.

[0147] In this embodiment of the invention, based on the basic structure of the eyeball, the line connecting the three-dimensional corneal center and the three-dimensional iris center is the optical axis of the eyeball, which is perpendicular to the spatial iris plane, and the edge points of the spatial iris should be coplanar with the three-dimensional iris center. Thus, 2n-2 equations are established. Based on the reflection relationship of a light source on the outer surface of the cornea in the system, one equation is established, for a total of 2n-1 equations. To accurately locate the eyeball parameters, this invention appropriately transforms the parameters in the above equations; the three-dimensional iris center in the equations is related to the spatial iris radius and can be represented by the spatial iris radius. This embodiment of the invention innovatively defines an OCI plane containing the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center. The normal vector of the OCI plane is determined based on two solutions of the spatial iris normal vector, and thus intersects with the corneal reflection plane of a light source, determining the vector OC between the three-dimensional corneal center and the camera optical center. The estimation of the three-dimensional corneal center is simplified to solving for a scaling factor, which is represented by the three-dimensional iris center and the distance from the three-dimensional corneal center to the three-dimensional iris center based on the geometric imaging relationship, reducing the requirements for system configuration. Since the spatial iris edge points are imaged in the camera, each spatial iris edge point can be represented by multiplying the imaged point by a scaling factor. The spatial iris is approximately circular, and since the distance from the spatial iris edge point to the center of the three-dimensional iris is equal to the spatial iris radius, the scaling factor can be represented by the center of the three-dimensional iris and the spatial iris radius. Similarly, the corneal reflection points are imaged in the camera, so the reflection points of the system light source on the cornea can be represented by multiplying the bright spot coordinates by a scaling factor. The corneal reflection points are located on the outer surface of the cornea. Based on the eye structure, and assuming the spatial iris radius and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris, the corneal radius can be expressed. Therefore, since the distance from the corneal reflection point to the center of the three-dimensional cornea is equal to the corneal radius, the scaling factor can be represented by the center of the three-dimensional cornea, the spatial iris radius, and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris.

[0148] Based on the above parameter transformation, the parameters in the equations are transformed into two unknowns through known distance constraints and visual parameters: the spatial iris radius and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris. Both of these parameters are eye-invariant parameters with their theoretical range. Therefore, in parameter optimization, on the one hand, the unknown parameters are easy to set initial values; on the other hand, compared with optimizing the above multiple proportional coefficients, optimizing these two eye-invariant parameters reduces computational complexity and improves stability and convergence.

[0149] Figure 8 This is a block diagram illustrating an eye parameter localization and optical axis estimation device based on iris reconstruction, according to an exemplary embodiment. The device is used in an eye parameter localization and optical axis estimation method based on iris reconstruction. (Refer to...) Figure 8The device includes an acquisition unit 810, a first determination unit 820, a second determination unit 830, a third determination unit 840, a fourth determination unit 850, and an estimation unit 860. Wherein:

[0150] The acquisition unit 810 is used to reconstruct the spatial iris circle based on the features of the iris in camera imaging, and to standardize the iris imaging elliptical cone through coordinate system transformation to obtain the iris imaging elliptical cone in the new coordinate system.

[0151] The first determining unit 820 is used to determine two circular cross sections based on the iris imaging elliptical cone in the new coordinate system;

[0152] The second determining unit 830 is used to calculate the intersection of the iris imaging elliptical cone and the two circular cross sections to determine the three-dimensional iris center.

[0153] The third determining unit 840 is used to define the OCI plane with the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center; determine the normal vector of the OCI plane based on the normal vectors of the two circular cross sections; and determine the vectors of the three-dimensional corneal center and the camera optical center based on the normal vector of the OCI plane.

[0154] The fourth determining unit 850 is used to determine the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point based on the vector relationship between the three-dimensional corneal center and the camera optical center and the reflection relationship of a single light source;

[0155] The estimation unit 860 is used to estimate the iris radius, which varies with individuals, and the distance from the three-dimensional corneal center to the three-dimensional iris center, based on the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point, constrained by the eye structure and imaging relationship; based on the iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center, the three-dimensional corneal center and the three-dimensional iris center are obtained, and the optical axis of the eyeball is estimated through the three-dimensional corneal center and the three-dimensional iris center.

[0156] Optionally, the acquisition unit 810 is configured to:

[0157] Using the camera's optical center as the vertex, an iris imaging elliptical cone is constructed based on the geometric imaging relationship of the iris within the camera.

[0158] In the original camera coordinate system O-XYZ, the iris imaging elliptical cone is represented by a real symmetric matrix; the diagonalization matrix of the real symmetric matrix is ​​calculated; the eigenvalues ​​of the diagonalization matrix are obtained based on the diagonalization matrix; a new coordinate system O-X'Y'Z' is constructed based on the original camera coordinate system O-XYZ.

[0159] In the new coordinate system O-X'Y'Z', a standardized iris imaging elliptic cone is constructed based on the eigenvalues ​​of the diagonalized matrix.

[0160] Optionally, the calculation process of the intersection point of the iris imaging elliptical cone and the two circular cross sections is expressed by the following formulas (1)-(4):

[0161] (1)

[0162] (2)

[0163] (3)

[0164] (4)

[0165] in, Denotes the first eigenvalue of a real symmetric matrix; Denotes the second eigenvalue of a real symmetric matrix; The third eigenvalue of a real symmetric matrix is ​​represented. The coordinates of the intersection point of the elliptical cone edge 1 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the first circular section with intercept 1; The coordinates of the intersection point of the elliptical cone edge 2 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the first circular section with intercept 1; The coordinates of the intersection point of the elliptical cone edge 1 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the second circular section with intercept 1; The coordinates represent the intersection of the elliptical cone edge 2 projected onto the X'OZ' plane by the iris elliptical cone and the straight line projected onto the X'OZ' plane by the second circular section corresponding to intercept 1.

[0166] Optionally, the three-dimensional iris center is represented by the following formulas (5)-(6):

[0167] (5)

[0168] (6)

[0169] in, This represents the center of the first three-dimensional iris in the coordinate system O-X'Y'Z'; This represents the center of the second three-dimensional iris in the coordinate system O-X'Y'Z'; Denotes the first eigenvalue of a real symmetric matrix; Denotes the second eigenvalue of a real symmetric matrix; The third eigenvalue of a real symmetric matrix is ​​represented. The elliptic cone edge 1, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane from the first circular section; The elliptic cone edge 2, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane from the first circular section; The elliptic cone edge 1, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane by the second circular section; The elliptic cone edge 2, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane by the second circular section; r represents the radius of the spatial iris circle.

[0170] Optionally, the third determining unit 840 is configured to:

[0171] Based on the structure of the eyeball, an OCI plane containing the camera optical center, the center of the iris imaging ellipse, and the iris normal vector is constructed.

[0172] Based on the normal vectors of the two circular cross sections, the normal vector of the OCI plane is constructed through the symmetric relationship with the straight line connecting the optical center of the camera and the center of the iris imaging ellipse;

[0173] Based on the light source reflection relationship and the collinear relationship between the reflection point, the camera optical center, and the Pulchin spot, determine the normal vector of the light source reflection plane;

[0174] Based on the normal vector of the light source reflection plane and the normal vector of the OCI plane, determine the vectors of the three-dimensional corneal center and the camera optical center.

[0175] Optionally, the fourth determining unit 850 includes:

[0176] Based on the vectors of the 3D corneal center and the camera optical center, the relationship between the vectors of the 3D corneal center and the camera optical center is constructed.

[0177] Based on the vector relationship between the 3D corneal center and the camera optical center, the scaling factor of the 3D corneal center is solved by setting the distance conditions satisfied by the 3D corneal center and the 3D iris center; based on the scaling factor, the position of the 3D corneal center is obtained.

[0178] Based on the collinear relationship between the corneal reflection point, the camera optical center, and the bright spot, the relationship between the corneal reflection point and the bright spot is constructed.

[0179] Based on the relationship between the corneal reflection point and the bright spot, the scaling factor of the corneal reflection point is solved by setting the distance condition between the corneal reflection point and the three-dimensional corneal center; based on the scaling factor, the position of the corneal reflection point is obtained.

[0180] Based on the relationship that the incident angle equals the reflection angle, the conditions for the three-dimensional corneal center, the camera optical center, and the corneal reflection point to be satisfied are determined. By satisfying these conditions, the position of the three-dimensional iris center is obtained.

[0181] Optionally, the process of solving the proportional coefficient of the three-dimensional corneal center is represented by the following formula (7):

[0182] (7)

[0183] in, The scaling factor representing the three-dimensional corneal center; The unit vector representing the three-dimensional corneal center and the camera optical center; Represents the coordinates of the three-dimensional iris center; Represents the three-dimensional corneal center To the 3D Iris Center The distance;

[0184] The process of solving the proportional coefficient of the corneal reflective point is shown by the following formula (8):

[0185] (8)

[0186] in, The proportionality coefficient representing the corneal reflectance point; Indicates the coordinates of the bright spot; Indicates the radius of the iris.

[0187] This invention first reconstructs the spatial iris circle based on the characteristics of the iris in camera imaging. Then, it standardizes the iris imaging elliptical cone through coordinate system transformation to obtain the iris imaging elliptical cone in the new coordinate system. Based on the iris imaging elliptical cone in the new coordinate system, two circular cross-sections are determined. Next, the intersection points of the iris imaging elliptical cone and the two circular cross-sections are calculated to determine the three-dimensional iris center. An OCI plane is defined with respect to the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center. Based on the normal vectors of the two circular cross-sections, the normal vector of the OCI plane is determined. Based on the normal vector of the OCI plane, the three-dimensional corneal center and the phase... The vector of the camera's optical center is determined; based on the vectors of the 3D corneal center and the camera's optical center and the reflection relationship of a single light source, the positions of the 3D iris center, the 3D corneal center, and the corneal reflection point are determined; finally, based on the positions of the 3D iris center, the 3D corneal center, and the corneal reflection point, and constrained by the eye structure and imaging relationship, the individual-variable iris radius and the distance from the 3D corneal center to the 3D iris center are estimated; based on the iris radius and the distance from the 3D corneal center to the 3D iris center, the 3D corneal center and the 3D iris center are obtained, and the optical axis of the eyeball is estimated through the 3D corneal center and the 3D iris center.

[0188] This invention reconstructs the spatial iris circle using the characteristics of the iris on the camera's imaging plane. Through coordinate system transformation, the iris imaging elliptical cone is standardized to a coordinate system with the iris imaging elliptical cone axis as the Z' axis, the major axis as the Y' axis, and the minor axis as the X' axis. In this coordinate system, the circular cross-section intercepted on the iris imaging elliptical cone projects onto the X'OZ' plane as a straight line. Furthermore, the two cases of intercepting the circular cross-section are symmetrical about the iris imaging elliptical cone axis, facilitating the solution of the normal vector and center of the circular cross-section. The solution for the normal vector of the circular cross-section involves rotating the ellipse in the X'OY' plane to determine the equation of the straight line on the X'OZ' plane, and then using the direction vector perpendicular to it as the normal vector. This allows for the determination of two solutions based on the iris imaging characteristics. The solution for the center of the circular cross-section involves first defining a circular cross-section passing through a point X'=0, Z'=1 to determine the center and radius of the circle. Then, the radius of the spatial iris is characterized using the principle of similarity. The three-dimensional coordinates of the iris center are obtained, and then the three-dimensional coordinates of the iris center in the camera coordinate system are obtained through coordinate system transformation. It is related to the spatial iris radius.

[0189] In this embodiment of the invention, based on the basic structure of the eyeball, the line connecting the three-dimensional corneal center and the three-dimensional iris center is the optical axis of the eyeball, which is perpendicular to the spatial iris plane, and the edge points of the spatial iris should be coplanar with the three-dimensional iris center. Thus, 2n-2 equations are established. Based on the reflection relationship of a light source on the outer surface of the cornea in the system, one equation is established, for a total of 2n-1 equations. To accurately locate the eyeball parameters, this invention appropriately transforms the parameters in the above equations; the three-dimensional iris center in the equations is related to the spatial iris radius and can be represented by the spatial iris radius. This embodiment of the invention innovatively defines an OCI plane containing the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center. The normal vector of the OCI plane is determined based on two solutions of the spatial iris normal vector, and thus intersects with the corneal reflection plane of a light source, determining the vector OC between the three-dimensional corneal center and the camera optical center. The estimation of the three-dimensional corneal center is simplified to solving for a scaling factor, which is represented by the three-dimensional iris center and the distance from the three-dimensional corneal center to the three-dimensional iris center based on the geometric imaging relationship, reducing the requirements for system configuration. Since the spatial iris edge points are imaged in the camera, each spatial iris edge point can be represented by multiplying the imaged point by a scaling factor. The spatial iris is approximately circular, and since the distance from the spatial iris edge point to the center of the three-dimensional iris is equal to the spatial iris radius, the scaling factor can be represented by the center of the three-dimensional iris and the spatial iris radius. Similarly, the corneal reflection points are imaged in the camera, so the reflection points of the system light source on the cornea can be represented by multiplying the bright spot coordinates by a scaling factor. The corneal reflection points are located on the outer surface of the cornea. Based on the eye structure, and assuming the spatial iris radius and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris, the corneal radius can be expressed. Therefore, since the distance from the corneal reflection point to the center of the three-dimensional cornea is equal to the corneal radius, the scaling factor can be represented by the center of the three-dimensional cornea, the spatial iris radius, and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris.

[0190] Based on the above parameter transformation, the parameters in the equations are transformed into two unknowns through known distance constraints and visual parameters: the spatial iris radius and the distance from the center of the three-dimensional cornea to the center of the three-dimensional iris. Both of these parameters are eye-invariant parameters with their theoretical range. Therefore, in parameter optimization, on the one hand, the unknown parameters are easy to set initial values; on the other hand, compared with optimizing the above multiple proportional coefficients, optimizing these two eye-invariant parameters reduces computational complexity and improves stability and convergence.

[0191] Figure 9 This is a schematic diagram of the structure of an eyeball parameter localization and eyeball optical axis estimation device based on iris reconstruction provided in an embodiment of the present invention, as shown below. Figure 9 As shown, the eye parameter localization and eye optical axis estimation device based on iris reconstruction may include the above-mentioned... Figure 8The illustrated device is an eye parameter localization and optical axis estimation device based on iris reconstruction. Optionally, the eye parameter localization and optical axis estimation device 910 based on iris reconstruction may include a first processor 2001.

[0192] Optionally, the eye parameter localization and eye optical axis estimation device 910 based on iris reconstruction may also include a memory 2002 and a transceiver 2003.

[0193] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0194] The following is combined Figure 9 A detailed introduction to each component of the 910 eye parameter localization and optical axis estimation device based on iris reconstruction is provided below:

[0195] The first processor 2001 is the control center of the iris reconstruction-based eye parameter localization and eye optical axis estimation device 910. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0196] Optionally, the first processor 2001 can perform various functions of the iris reconstruction-based eye parameter localization and eye optical axis estimation device 910 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0197] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 9 CPU0 and CPU1 are shown in the diagram.

[0198] In a specific implementation, as one example, the eye parameter localization and eye optical axis estimation device 910 based on iris reconstruction may also include multiple processors, for example... Figure 9The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0199] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0200] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently, and may be connected to the eye axis estimation device 910 via an interface circuit based on iris reconstruction eye parameter localization. Figure 9 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0201] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0202] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 9 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0203] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the eye axis estimation device 910 via an interface circuit based on iris reconstruction-based eye parameter localization. Figure 9(Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0204] It should be noted that, Figure 9 The structure of the eye parameter localization and eye optical axis estimation device 910 based on iris reconstruction shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0205] Furthermore, the technical effect of the eye parameter localization and eye optical axis estimation device 910 based on iris reconstruction can be referred to the technical effect of the eye parameter localization and eye optical axis estimation method based on iris reconstruction described in the above method embodiments, and will not be repeated here.

[0206] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0207] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0208] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0209] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0210] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0211] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0212] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0213] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0214] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0215] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0216] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0217] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0218] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for locating eyeball parameters and estimating the optical axis of the eyeball based on iris reconstruction, characterized in that, The method includes: S1. Based on the characteristics of the iris in camera imaging, reconstruct the spatial iris circle, and standardize the iris imaging ellipse cone through coordinate system transformation to obtain the iris imaging ellipse cone in the new coordinate system. S2. Based on the iris imaging elliptical cone in the new coordinate system, determine two circular cross sections; S3. Calculate the intersection of the iris imaging elliptical cone with the two circular sections to determine the three-dimensional iris center; S4. Define the OCI plane with the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center; determine the normal vector of the OCI plane based on the normal vectors of the two circular cross sections; determine the vectors of the three-dimensional corneal center and the camera optical center based on the normal vector of the OCI plane. S5. Based on the vector relationship between the three-dimensional corneal center and the camera optical center and the reflection relationship of a single light source, determine the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point; Specifically, S5, based on the vector relationship between the three-dimensional corneal center and the camera optical center, and the reflection relationship of a single light source, determines the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point, including: S51. Based on the vectors of the three-dimensional corneal center and the camera optical center, construct the relationship between the vectors of the three-dimensional corneal center and the camera optical center. S52. Based on the vector relationship between the three-dimensional corneal center and the camera optical center, the scaling factor of the three-dimensional corneal center is solved by setting the distance conditions satisfied by the three-dimensional corneal center and the three-dimensional iris center; the position of the three-dimensional corneal center is obtained based on the scaling factor. The process of solving the proportional coefficient of the three-dimensional corneal center is represented by the following formula (1): （1） in, The scaling factor representing the three-dimensional corneal center; The unit vector representing the three-dimensional corneal center and the camera optical center; Represents the coordinates of the three-dimensional iris center; Represents the three-dimensional corneal center To the 3D Iris Center The distance; S53. Based on the collinear relationship between the corneal reflection point, the camera optical center, and the bright spot, construct the relationship between the corneal reflection point and the bright spot; S54. Based on the relationship between the corneal reflection point and the bright spot, by setting the distance condition between the corneal reflection point and the three-dimensional corneal center, solve for the scaling factor of the corneal reflection point; based on the scaling factor, obtain the position of the corneal reflection point. The process of solving the proportional coefficient of the corneal reflective point is shown by the following formula (2): （2） in, The proportionality coefficient representing the corneal reflectance point; Indicates the coordinates of the bright spot; Indicates the iris radius; S55. Based on the relationship that the incident angle equals the reflection angle, determine that the three-dimensional corneal center, the camera optical center, and the corneal reflection point meet the conditions. By meeting the conditions, obtain the position of the three-dimensional iris center. S6. Based on the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point, and constrained by the eye structure and imaging relationship, estimate the iris radius, which varies with individuals, and the distance from the three-dimensional corneal center to the three-dimensional iris center; based on the iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center, obtain the three-dimensional corneal center and the three-dimensional iris center, and estimate the optical axis of the eyeball through the three-dimensional corneal center and the three-dimensional iris center.

2. The method for iris reconstruction-based localization of eyeball parameters and estimation of eyeball optical axis according to claim 1, characterized in that, S1 reconstructs the spatial iris circle based on the features of the iris in camera imaging, and standardizes the iris imaging ellipse cone through coordinate system transformation to obtain the iris imaging ellipse cone in the new coordinate system, including: S11. Using the camera's optical center as the vertex, construct an iris imaging elliptical cone based on the geometric imaging relationship of the iris in the camera; S12. Under the original camera coordinate system O-XYZ, the iris imaging elliptical cone is represented by a real symmetric matrix; the diagonalization matrix of the real symmetric matrix is ​​calculated; the eigenvalues ​​of the diagonalization matrix are obtained based on the diagonalization matrix; a new coordinate system O-X'Y'Z' is constructed based on the original camera coordinate system O-XYZ. S13. In the new coordinate system O-X'Y'Z', construct the standardized iris imaging elliptic cone based on the eigenvalues ​​of the diagonalized matrix.

3. The method for iris reconstruction-based localization of eyeball parameters and estimation of eyeball optical axis according to claim 1, characterized in that, The calculation process of the intersection point of the iris imaging elliptical cone and the two circular cross sections is expressed by the following formulas (3)-(6): （3） （4） （5） （6） in, Denotes the first eigenvalue of a real symmetric matrix; Denotes the second eigenvalue of a real symmetric matrix; The third eigenvalue of a real symmetric matrix is ​​represented. The coordinates of the intersection point of the elliptical cone edge 1 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the first circular section with intercept 1; The coordinates of the intersection point of the elliptical cone edge 2 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the first circular section with intercept 1; The coordinates of the intersection point of the elliptical cone edge 1 projected onto the X'OZ' plane and the line projected onto the X'OZ' plane corresponding to the second circular section with intercept 1; The coordinates represent the intersection of the elliptical cone edge 2 projected onto the X'OZ' plane by the iris elliptical cone and the straight line projected onto the X'OZ' plane by the second circular section corresponding to intercept 1.

4. The method for iris reconstruction-based localization of eyeball parameters and estimation of eyeball optical axis according to claim 1, characterized in that, The three-dimensional iris center is represented by the following formulas (7)-(8): （7） （8） in, This represents the center of the first three-dimensional iris in the coordinate system O-X'Y'Z'; This represents the center of the second three-dimensional iris in the coordinate system O-X'Y'Z'; Denotes the first eigenvalue of a real symmetric matrix; Denotes the second eigenvalue of a real symmetric matrix; The third eigenvalue of a real symmetric matrix is ​​represented. The elliptic cone edge 1, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane from the first circular section; The elliptic cone edge 2, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane from the first circular section; The elliptic cone edge 1, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane by the second circular section; The elliptic cone edge 2, representing the projection of the iris elliptic cone onto the X'OZ' plane, and its radius are... The coordinates of the intersection point of the lines projected onto the X'OZ' plane by the second circular section; r represents the radius of the spatial iris circle.

5. The method for iris reconstruction-based localization of eyeball parameters and estimation of eyeball optical axis according to claim 1, characterized in that, The definition of S4 is based on the OCI plane formed by the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center; the normal vector of the OCI plane is determined based on the normal vectors of the two circular cross sections. Based on the normal vector of the OCI plane, the vectors for determining the 3D corneal center and camera optical center are included: S41. Based on the structure of the eyeball, construct an OCI plane that includes the camera optical center, the three-dimensional corneal center, and the three-dimensional iris center, based on the camera optical center, the center of the iris imaging ellipse, and the iris normal vector. S42. Based on the normal vectors of the two circular cross sections, construct the normal vector of the OCI plane through the symmetric relationship with the straight line connecting the optical center of the camera and the center of the iris imaging ellipse; S43. Based on the light source reflection relationship and the collinear relationship between the reflection point, the camera optical center, and the Pulchin spot, determine the normal vector of the light source reflection plane; S44. Determine the vectors of the three-dimensional corneal center and the camera optical center based on the normal vector of the light source reflection plane and the normal vector of the OCI plane.

6. A device for iris reconstruction-based eye parameter localization and optical axis estimation, wherein the device is used to implement the iris reconstruction-based eye parameter localization and optical axis estimation method as described in any one of claims 1-5, characterized in that, The device includes: The acquisition unit is used to reconstruct the spatial iris circle based on the features of the iris in the camera image, and to standardize the iris imaging ellipse cone through coordinate system transformation to obtain the iris imaging ellipse cone in the new coordinate system. The first determining unit is used to determine two circular cross sections based on the iris imaging elliptical cone in the new coordinate system; The second determining unit is used to calculate the intersection of the iris imaging elliptical cone and the two circular sections to determine the three-dimensional iris center; The third determining unit is used to define the OCI plane with the three-dimensional corneal center, the three-dimensional iris center, and the camera optical center; to determine the normal vector of the OCI plane based on the normal vectors of the two circular cross sections; and to determine the vectors of the three-dimensional corneal center and the camera optical center based on the normal vector of the OCI plane. The fourth determining unit is used to determine the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point based on the vector relationship between the three-dimensional corneal center and the camera optical center and the reflection relationship of a single light source; The estimation unit is used to estimate the iris radius, which varies from person to person, and the distance from the three-dimensional corneal center to the three-dimensional iris center, based on the positions of the three-dimensional iris center, the three-dimensional corneal center, and the corneal reflection point, constrained by the eye structure and imaging relationship; based on the iris radius and the distance from the three-dimensional corneal center to the three-dimensional iris center, the three-dimensional corneal center and the three-dimensional iris center are obtained, and the optical axis of the eyeball is estimated through the three-dimensional corneal center and the three-dimensional iris center.

7. A device for iris reconstruction-based eyeball parameter localization and optical axis estimation, characterized in that, The eye parameter localization and eye optical axis estimation device based on iris reconstruction includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 5.

Images