Gaze direction measurement device and gaze direction model establishment method

Abstract

The technical scheme of the application discloses a gaze direction measuring device and a gaze direction model establishing method. The gaze direction measuring device comprises a geometric positioning cap, an adjusting assembly and an image acquisition assembly mounted on the adjusting assembly. The geometric positioning cap is used for being worn on the head of a user. The image acquisition assembly comprises at least a photographing module. When the user wears the geometric positioning cap, the image acquisition assembly is used for acquiring a geometric positioning cap image, a face image of the user and an eye image of the user. The adjusting assembly is used for adjusting the spatial position of the photographing module. The geometric positioning cap image, the face image and the eye image are used for determining a gaze direction model. The gaze direction measuring device has high measuring accuracy, and when the gaze direction model is established, high-precision eye movement tracking can be realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of eye movement tracking, and more particularly, to a gaze direction measurement device and a gaze direction model establishing method. BACKGROUND

[0002] With the continuous development of science and technology, electronic devices with eye movement tracking (also known as gaze tracking) function are widely used in people's daily life and work, bringing great convenience to people's daily life and work, and becoming an indispensable important tool for people today.

[0003] How to design a high-precision gaze direction measurement device for establishing a gaze direction model to facilitate electronic devices to realize high-precision eye movement tracking is a problem to be solved in the technical field of eye movement tracking. SUMMARY

[0004] Therefore, the present application provides a gaze direction measurement device and a gaze direction model establishing method, and the scheme is as follows:

[0005] A gaze direction measurement device, comprising:

[0006] a geometric positioning cap for wearing on the head of a user;

[0007] an adjusting assembly and an image acquisition assembly mounted on the adjusting assembly; the image acquisition assembly at least comprises a photographing module;

[0008] When the user wears the geometric positioning cap, the image acquisition assembly is used to acquire a geometric positioning cap image, a face image of the user and an eye image of the user; the adjusting assembly is used to adjust the spatial position of the photographing module; and the geometric positioning cap image, the face image and the eye image are used to determine a gaze direction model.

[0009] Preferably, in the above-mentioned gaze direction measurement device, the adjusting assembly comprises a skeleton and an adjusting module arranged on the skeleton.

[0010] The photographing module is mounted on the adjusting module, and the adjusting module is used to adjust the spatial position of the photographing module.

[0011] Preferably, in the above-mentioned gaze direction measurement device, the adjusting module comprises N motion guide components, the N motion guide components are sequentially a first motion guide component to an Nth motion guide component, and N is a positive integer greater than 2.

[0012] The first motion guide component is fixed on the skeleton, and the i+1 motion guide component is movably mounted on the ith motion guide component, i being a positive integer not greater than N-1;

[0013] The photographing module is mounted on the Nth motion guide component.

[0014] Preferably, in the above eye gaze direction measurement device, the i+1 motion guide component is capable of translational motion or rotation relative to the ith motion guide component.

[0015] When the i+1 motion guide component is capable of translational motion relative to the ith motion guide component, the two motion guide components are controlled to move translationally by a first electromagnetic drive device, one of the two motion guide components being fixed with a magnet and the other being fixed with a coil; the first electromagnetic drive device comprises the magnet and the coil, and is used to cause the two motion guide components to move relative to each other based on the electromagnetic force between the magnet and the coil when the coil is charged.

[0016] When the i+1 motion guide component is capable of rotation relative to the ith motion guide component, the two motion guide components are controlled to rotate by a second electromagnetic drive device, the second electromagnetic drive device comprising a motor.

[0017] Preferably, in the above eye gaze direction measurement device, the adjustment module further comprises a position sensor for detecting the relative position between the motion guide components.

[0018] Preferably, in the above eye gaze direction measurement device, the image acquisition assembly further comprises at least one three-dimensional scanner; the three-dimensional scanner is used to scan the three-dimensional shape of the face of the user by infrared laser.

[0019] The three-dimensional scanner is arranged on the skeleton or on the photographing module.

[0020] Preferably, in the above eye gaze direction measurement device, the photographing module comprises:

[0021] a camera support;

[0022] an infrared camera assembly mounted on the camera support, the infrared camera assembly comprising a first camera, a second camera and an infrared point light source; the first camera and the second camera are used to acquire an infrared light image of the eye image.

[0023] a color camera array mounted on the camera holder, the color camera array comprising a plurality of color cameras arranged around the infrared camera assembly, the color cameras configured to capture visible light images of the geometric positioning cap, the face image, and the eye image;

[0024] at least one infrared illumination source mounted on the camera holder or the infrared camera assembly.

[0025] Preferably, in the above eye gaze direction measurement device, the infrared illumination source is configured to emit first infrared detection light, the first camera is configured to form a first image based on the first infrared detection light reflected by the eyeball of the user, the first image comprising infrared light iris information and infrared light pupil information of the eyeball of the user, and the optical axis of the first camera is adjusted to intersect at the pupil center point of the eye based on the infrared light pupil information in the first image; the infrared point light source is configured to emit second infrared detection light, and the second camera is configured to form a second image based on the second infrared detection light reflected by the cornea of the user, the second image comprising a highlight point of the second infrared detection light reflected by the cornea of the user.

[0026] Preferably, in the above eye gaze direction measurement device, the infrared light image of the eye image comprises the first image and the second image.

[0027] Preferably, in the above eye gaze direction measurement device, the second camera and the infrared point light source are arranged on both sides of the optical axis of the first camera, the line connecting the lens focal point of the second camera and the infrared point light source intersects the optical axis of the first camera, the line is perpendicular to the visual axis of the user, and the distance between the infrared point light source and the optical axis of the first camera is equal to the distance between the lens focal point of the second camera and the optical axis of the first camera.

[0028] Preferably, in the above eye gaze direction measurement device, the color camera is capable of receiving visible light band light for capturing a third image, the third image comprising the junction line features of the iris and the sclera of the user.

[0029] Preferably, in the above eye gaze direction measurement device, the visible light image of the eye image comprises the third image.

[0030] Preferably, in the above eye gaze direction measurement device, the geometric positioning cap comprises:

[0031] a wearing component configured to be worn on the head of the user;

[0032] a geometric shape identification component arranged outside the wearing component;

[0033] The relative position between the face of the user and the photographing module is determined based on the images of the geometric shape identification components in the geometric positioning cap images.

[0034] Preferably, in the above eye gaze direction measurement device, the geometric shape identification components include a plurality of visible point light sources with different colors, which are arranged around the outside of the wearing component.

[0035] The relative position between the face of the user and the photographing module is determined based on at least two geometric positioning cap images collected by the photographing module, and each geometric positioning cap image includes a plurality of images of the same geometric shape identification components.

[0036] Preferably, in the above eye gaze direction measurement device, a computer is further included, which has an information processor for image data analysis on the geometric positioning cap images, the face images and the eye images, and based on the image data analysis result, the relative position between the face of the user and the photographing module is determined, the spatial position of the photographing module is adjusted by the adjusting assembly to adjust the relative position, the corresponding geometric positioning cap images, face images and eye images under different relative positions are obtained to determine the eye gaze direction model under different relative positions.

[0037] The application further provides an eye gaze direction model establishment method based on an eye gaze direction measurement device, which includes the following steps:

[0038] When the user wears the geometric positioning cap, the geometric positioning cap images, the face images of the user and the eye images of the user are collected by the image collection assembly;

[0039] The geometric positioning cap images, the face images and the eye images are subjected to image data analysis, and based on the image data analysis result, the relative position between the face of the user and the photographing module is determined.

[0040] The spatial position of the photographing module is adjusted by the adjusting assembly to adjust the relative position, the corresponding geometric positioning cap images, face images and eye images under different relative positions are obtained to determine the eye gaze direction model under different relative positions; wherein the eye gaze direction model includes the visual axis model, the pupil center point model, the iris texture marking line model, the eye coordinate system and the projection iris edge curve group model corresponding to the relative position.

[0041] Preferably, in the above gaze direction model establishing method, the photographing module comprises: a camera support; an infrared camera assembly mounted on the camera support, the infrared camera assembly comprising: a first camera, a second camera and an infrared point light source; a color camera array mounted on the camera support, the color camera array comprising a plurality of color cameras located around the infrared camera assembly; and at least one infrared illumination light source.

[0042] The method for collecting the geometric positioning cap image, the face image of the user and the eye image of the user comprises:

[0043] The spatial position of the photographing module is adjusted by the adjusting assembly so that the visual axis of the user and the optical axis of the first camera satisfy the coincidence condition.

[0044] After the coincidence condition is satisfied, the relative position of the photographing module and the face of the user is changed, and first measurement data under different relative positions is obtained, the first measurement data comprising a face image and an eye image corresponding to the relative position; when the images under different relative positions are collected, the optical axis of the first camera intersects the pupil center point of the user.

[0045] Preferably, in the above gaze direction model establishing method, the method for making the visual axis of the user and the optical axis of the first camera satisfy the coincidence condition comprises:

[0046] The first image of the eye of the user is collected by the first camera, the first image comprising infrared pupil information;

[0047] Based on the first image, the spatial position of the photographing module is adjusted by the adjusting assembly to drive the first camera to move so that the optical axis of the first camera coincides with the visual axis of the user.

[0048] Preferably, in the above gaze direction model establishing method, the method for determining the relative position of the face of the user and the photographing module comprises:

[0049] Based on the first measurement data, the motion law of the eye of the user relative to the face reference is determined, the face reference comprising the geometric positioning cap position and the face three-dimensional model;

[0050] According to the motion law and the face reference, a standard model is constructed, the standard model comprising a plurality of preset virtual positions of the photographing module, a face coordinate system of the user and an eye center coordinate system;

[0051] The three mutually perpendicular coordinate axes in the face coordinate system are FX axis, FY axis and FZ axis respectively; the FZ axis faces the photographing module, and the FX FY plane is arranged opposite to the photographing module.

[0052] Preferably, in the gaze direction model establishing method, the method for determining the movement rule of the user's eye relative to the face reference comprises:

[0053] Based on a plurality of sets of the first measurement data, a plurality of direction models corresponding one-to-one are determined; the direction model comprises an eye axis model, a face coordinate system, and a relative relationship between the two;

[0054] All the direction models are superimposed based on the face coordinate system to obtain an eye axis model of the eye looking in different directions with the user's face as the reference, and the intersection of all the eye axis models is a center model of the user's eye; wherein the center model of the user's eye can determine the eye center coordinate system.

[0055] Preferably, in the gaze direction model establishing method, the first measurement data comprises the face image and the corresponding geometric positioning cap image photographed by the color camera array;

[0056] The method for determining the direction model comprises:

[0057] Based on the corresponding geometric positioning cap image of the face image, the relative position relationship between the user's face and the photographing module is determined;

[0058] Based on the relative position relationship between the user's face and the photographing module, an eye axis model and a face coordinate system are constructed, and the eye axis model, the face coordinate system, and the relative relationship between the two constitute the direction model.

[0059] Preferably, in the gaze direction model establishing method, a left eye center coordinate system is established, and three mutually perpendicular coordinate axes in the left eye center coordinate system are LX axis, LY axis, and LZ axis; the center of the user's left eye is the origin of the left eye center coordinate system, the LZ axis is parallel to the FZ axis, and the LY axis is parallel to the FY axis;

[0060] The method for constructing the standard model comprises:

[0061] 35 array-arranged virtual positions are set; each virtual position corresponds to a three-dimensional model of the photographing module; in the three-dimensional model of the photographing module corresponding to each virtual position, the optical axis of the first camera intersects at the origin of the left eye center coordinate system;

[0062] An angle between a projection of a line connecting any point in the left eye center coordinate system and the origin on the LY LZ plane and the line is γ, and an angle between a projection of the line on the LX LZ plane and the line is δ, then angle coordinates (γ, δ) of the optical axis of the first camera in the left eye center coordinate system in the 35 virtual positions are (45°, 30°), (30°, 30°), (15°, 30°), (0°, 30°), (-15°, 30°), (-30°, 30°), (-45°, 30°), (45°, 15°), (30°, 15°), (15°, 15°), (0°, 15°), (-15°, 15°), (-30°, 15°), (-45°, 15°), (45°, 0°), (30°, 0°), (15°, 0°), (0°, 0°), (-15°, 0°), (-30°, 0°), (-45°, 0°), (45°, -15°), (30°, -15°), (15°, -15°), (0°, -15°), (-15°, -15°), (-30°, -15°), (-45°, -15°), (45°, -30°), (30°, -30°), (15°, -30°), (0°, -30°), (-15°, -30°), (-30°, -30°), (-45°, -30°).

[0063] Preferably, in the above gaze direction model establishing method, the method for determining the gaze direction model in different relative positions comprises:

[0064] Based on the standard model, the spatial position of the photographing module is adjusted by the adjusting assembly, so that it is at a real space position corresponding to each virtual position, and the face image and the eye image corresponding to each real space position are collected by the photographing module;

[0065] Based on the face image and the eye image, the gaze direction model corresponding to different real space positions is determined.

[0066] Preferably, in the above gaze direction model establishing method, the eye image comprises a visible light image collected by the color camera array and an infrared light image collected by the infrared camera assembly;

[0067] The projection iris edge curve group model is determined based on the visible light image;

[0068] The iris texture mark line model is determined based on the infrared light image.

[0069] Preferably, in the above eye-gaze direction model establishing method, the model of the set of projected iris edge curves comprises a model of a plurality of projected iris edge curves, the projected iris edge curve being an intersection line of an iris edge reflection surface and a set plane in an eye coordinate system, the set plane being a plane in which an AX axis and an AY axis of the eye coordinate system lie, and the plane is opposite to the photographing module;

[0070] The method for determining the model of the set of projected iris edge curves comprises:

[0071] A plurality of iris edge reflection surfaces are obtained by the plurality of color cameras in the color camera array.

[0072] A plurality of one-to-one corresponding projected iris edge curves are obtained by intersecting the plurality of iris edge reflection surfaces with the set plane.

[0073] Preferably, in the above eye-gaze direction model establishing method, a first image of the user's eye is captured by the first camera.

[0074] The method for determining the model of the iris texture marking line comprises:

[0075] The iris texture in the first image is identified according to image brightness and color in the first image.

[0076] The feature information in the iris texture is marked by marking points.

[0077] The model of the iris texture marking line is determined according to the positions of the marking points, the relative positional relationship between the light and the first camera coordinate system when the light corresponding to the marking points is incident on the first camera, and the model of the iris texture marking line is constructed based on the light.

[0078] As can be seen from the above description, in the technical scheme of the present application, the eye-gaze direction measuring device comprises: a geometric positioning cap, which is used to be worn on the head of a user; an adjusting assembly and an image acquisition assembly installed on the adjusting assembly; the image acquisition assembly at least comprises a photographing module; wherein, when the user wears the geometric positioning cap, the image acquisition assembly is used to acquire a geometric positioning cap image, a face image of the user and an eye image of the user; the adjusting assembly is used to adjust the spatial position of the photographing module; the geometric positioning cap image, the face image and the eye image are used to determine an eye-gaze direction model. The eye-gaze direction measuring device has high measurement accuracy, and when the eye-gaze direction model is established, high-precision eye movement tracking can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on the provided drawings.

[0080] The structures, proportions, sizes, etc. shown in the drawings of the present specification are only used to cooperate with the content disclosed in the present specification, to be understood and read by those skilled in the art, and are not used to limit the conditions that can be implemented by the present application, so they do not have technical significance. Any modification of structure, change of proportion relationship or adjustment of size, without affecting the effects and purposes that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application.

[0081] Figure 1 A structural schematic diagram of a gaze direction measuring device provided by an embodiment of the present application is provided.

[0082] Figure 2 A structural schematic diagram of an adjusting assembly in a gaze direction measuring device provided by an embodiment of the present application is provided.

[0083] Figure 3 A partial enlarged view of a first movement guide component in the gaze direction measuring device is provided. Figure 2

[0084] A partial enlarged view of a second movement guide component in the gaze direction measuring device is provided. Figure 4 Figure 2 A partial enlarged view of a third movement guide component in the gaze direction measuring device is provided.

[0085] Figure 5 Figure 2 A partial enlarged view of a fourth movement guide component in the gaze direction measuring device is provided.

[0086] Figure 6 A partial enlarged view of a fifth movement guide component in the gaze direction measuring device is provided. Figure 2

[0087] A partial enlarged view of a sixth movement guide component in the gaze direction measuring device is provided. Figure 7 Figure 2 A schematic diagram of the driving principle of the translational motion between two movement guide components provided by an embodiment of the present application is provided.

[0088] Figure 8 Figure 2 A schematic diagram of the driving principle of the translational motion between two movement guide components provided by an embodiment of the present application is provided.

[0089] Figure 9 A schematic diagram of the driving principle of the translational motion between two movement guide components provided by an embodiment of the present application is provided. Figure 10

[0090] Figure 11 ​​​​​A driving principle diagram of rotation between two motion guide components is provided for an embodiment of the present application.

[0091] Figure 12 A structure diagram of a three-dimensional scanner is provided for an embodiment of the present application.

[0092] Figure 13 A structure diagram of a photographing module of a gaze direction measuring device is provided for an embodiment of the present application.

[0093] Figure 14 A diagram of a first image is provided for an embodiment of the present application.

[0094] Figure 15 A structure diagram of a first camera is provided for an embodiment of the present application.

[0095] Figure 16 A principle diagram of alignment of a user pupil center point and a lens center of the first camera is provided for an embodiment of the present application.

[0096] Figure 17 A structure diagram of an infrared camera assembly is provided for an embodiment of the present application.

[0097] Figure 18 A scene diagram of a square pattern photographed by a camera is provided for an embodiment of the present application.

[0098] Figure 19 A diagram of an image photographed by a camera is provided for an embodiment of the present application.

[0099] Figure 20 A structure diagram of a geometric positioning cap is provided for an embodiment of the present application.

[0100] Figure 21 A geometric positioning cap image photographed by a first color camera;

[0101] Figure 22 A geometric positioning cap image photographed by a 16th color camera;

[0102] Figure 23 A diagram of a camera optical axis is provided for an embodiment of the present application.

[0103] Figure 24 A three-dimensional model diagram of a photographing module and a geometric positioning cap is provided for an embodiment of the present application.

[0104] Figure 25 A three-dimensional model of a photographing module and a first geometric positioning cap image relative position diagram is provided for an embodiment of the present application.

[0105] Figure 26 A module coordinate system diagram is provided for an embodiment of the present application.

[0106] Figure 27-33 A schematic diagram of a principle of forming a light ray angle for a point position calculation provided by an embodiment of the present application;

[0107] Figure 34 A schematic diagram of a method for establishing a gaze direction model provided by an embodiment of the present application;

[0108] Figure 35 A flowchart of a method for image acquisition provided by an embodiment of the present application;

[0109] Figure 36 A schematic diagram of a method for making a visual axis of a user coincide with an optical axis of the first camera provided by an embodiment of the present application;

[0110] Figure 37 A schematic diagram of a principle of a method for establishing a facial coordinate system provided by an embodiment of the present application;

[0111] Figure 38 A schematic diagram of a method for determining a motion rule of an eye of the user relative to a facial reference provided by an embodiment of the present application;

[0112] Figure 39 A schematic diagram of a method for determining the gaze direction model provided by an embodiment of the present application;

[0113] Figure 40 A schematic diagram of a gaze direction model provided by an embodiment of the present application;

[0114] Figure 41 A schematic diagram of another gaze direction model provided by an embodiment of the present application;

[0115] Figure 42 A schematic diagram of a principle of coincidence of a plurality of gaze direction models relative to a facial coordinate system provided by an embodiment of the present application;

[0116] Figure 43 A schematic diagram of a principle of establishing a left eye center coordinate system provided by an embodiment of the present application;

[0117] Figure 44 A schematic diagram of a standard model provided by an embodiment of the present application;

[0118] Figure 45 A schematic diagram of a method for determining a gaze direction model under different relative positions provided by an embodiment of the present application;

[0119] Figure 46 A schematic diagram of a method for determining an iris texture mark line model provided by an embodiment of the present application;

[0120] Figure 47A schematic diagram of a principle of marking iris texture provided by an embodiment of the present application;

[0121] Figure 48 A schematic diagram of a principle of determining the relative position relationship between the iris texture marking line and the photographing module provided by an embodiment of the present application;

[0122] Figure 49 A schematic diagram of the relative position between the eye visual axis and the pupil center point provided by an embodiment of the present application;

[0123] Figure 50 A schematic diagram of a principle of determining the relative position relationship between the pupil center point and the photographing module provided by an embodiment of the present application;

[0124] Figure 51 A schematic diagram of a principle of placing the eye coordinate system in the eye model provided by an embodiment of the present application;

[0125] Figure 52 A schematic diagram of a method of determining the projected iris edge curve group model provided by an embodiment of the present application;

[0126] Figure 53 A schematic diagram of the image captured by the color camera in an embodiment of the present application;

[0127] Figure 54 A schematic diagram of the relative relationship between the eye coordinate system of the left eye and each iris edge reflection surface in an embodiment of the present application;

[0128] Figure 55 A schematic diagram of the relative relationship between the eye coordinate system of the left eye and the projected iris edge curve in an embodiment of the present application;

[0129] Figure 56 A schematic diagram of a gaze direction model provided by an embodiment of the present application. DETAILED DESCRIPTION

[0130] The embodiments of the present application will be described in detail below with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0131] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with the accompanying drawings and specific embodiments.

[0132] As shown in Figure 1 Figure 1 ​A structure schematic diagram of a gaze direction measuring device provided in an embodiment of the present application, the gaze direction measuring device comprises:

[0133] A geometric positioning cap 11 for being worn on a head of a user; the geometric positioning cap 11 is a separate component for marking a position of the head of the user;

[0134] An adjusting assembly 12 and an image acquisition assembly 13 mounted on the adjusting assembly 12; the image acquisition assembly 13 at least comprises a photographing module 131;

[0135] Wherein, when the user wears the geometric positioning cap 11, the image acquisition assembly 13 is used for acquiring a geometric positioning cap image, a face image of the user and an eye image of the user; the adjusting assembly 12 is used for adjusting a spatial position of the photographing module 131; the geometric positioning cap image, the face image and the eye image are used for determining a gaze direction model.

[0136] The gaze direction measuring device provided in the technical scheme of the present application, after the user faces the photographing module 131 on the adjusting assembly 12 and wears the geometric positioning cap 11, the position of the head and the geometric positioning cap 11 is relatively stable, and in the case of slight shaking of the head, the geometric positioning cap 11 does not move relative to the head, and thus the geometric positioning cap image, the face image and the eye image can be acquired by the image acquisition assembly 13.

[0137] As shown in Figure 2 , Figure 2 A structure schematic diagram of an adjusting assembly in a gaze direction measuring device provided in an embodiment of the present application, the adjusting assembly 12 comprises: a framework 121 and an adjusting module 122 arranged on the framework 121; wherein the photographing module 131 is mounted on the adjusting module 122, and the adjusting module 122 is used for adjusting a spatial position of the photographing module 131. By the adjusting module 122, the relative position of the geometric positioning cap 11 worn on the head of the user and the photographing module 131 can be adjusted, so as to acquire the geometric positioning cap image, the face image and the eye image corresponding to different relative positions.

[0138] In the embodiment of the present application, the adjusting module 122 comprises: N motion guide components, the N motion guide components are sequentially a first motion guide component to an Nth motion guide component, N is a positive integer greater than 2; the first motion guide component is fixed on the framework 121, and an i+1th motion guide component is movably mounted on an ith motion guide component, i is a positive integer not greater than N-1; wherein the photographing module 131 is mounted on the Nth motion guide component. Wherein, the relative movement between two motion guide components is translation or rotation.

[0139] By the N-dimension motion guiding components, the translational and rotational movement of the photographing module 131 in different dimensions can be realized, the adjustment of the spatial position of the photographing module 131 can be realized, and then the geometric positioning cap image, the face image and the eye image under different relative positions between the photographing module 131 and the face of the user can be collected.

[0140] In Figure 2 In the embodiment shown, the adjustment module 122 includes six motion guiding components, which are sequentially a first motion guiding component 1221, a second motion guiding component 1222, a third motion guiding component 1223, a fourth motion guiding component 1224, a fifth motion guiding component 1225 and a sixth motion guiding component 1226.

[0141] As Figure 3 shown, Figure 3 To Figure 2 be a partial enlarged view of the first motion guiding component in Figure 2 and Figure 3 shown, the adjustment module 122 includes two first motion guiding components 1221 fixedly installed on the framework 121, and the two first motion guiding components 1221 are arranged in parallel. The first motion guiding component 1221 has a first linear guide rail 1221a. Specifically, the framework 121 is a cubic frame, and the two first motion guiding components 1221 are respectively installed on two edge frames of the cubic frame which are coplanar and parallel. Among them, the cubic frame can be a metal frame or an alloy frame, etc.

[0142] As Figure 4 shown, Figure 4 To Figure 2 be a partial enlarged view of the second motion guiding component in Figure 2-4 , the second motion guiding component 1222 is movably installed on the first motion guiding component 1221. Specifically, the two ends of the second motion guiding component 1222 are movably installed on the two first motion guiding components 1221 through the first sliding blocks 1222b respectively. In Figure 4 the manner shown, the left end of the second motion guiding component 1222 is fixed with a first sliding block 1222b, and the first sliding block 1222b at the left end is movably installed on the two first linear guide rails 1221a corresponding to the left end. The right end of the second motion guiding component 1222 is fixed with another first sliding block 1222b, and the first sliding block 1222b at the right end is movably installed on the two first linear guide rails 1221a corresponding to the right end. The second motion guiding component 1222 has a second linear guide rail 1222a.

[0143] As Figure 5 shown, Figure 5 To Figure 2A partial enlarged view of the third motion guide component, combined with... Figure 2 , Figure 4 and Figure 5 As shown, the third motion guide component 1223 is movably mounted on the second motion guide component 1222. Specifically, the third motion guide component includes a relatively fixed second slider 1223a and a third slider 1223b. The second slider 1223a is movably mounted on the second linear guide rail 1222a, and the third slider 1223b is movably mounted on the fourth motion guide component 1224.

[0144] like Figure 6 As shown, Figure 6 for Figure 2 A partial enlarged view of the fourth motion guide component, combined with... Figure 2 , Figure 5 and Figure 6 As shown, the fourth motion guide component 1224 is movably mounted on the third motion guide component 1223. Specifically, the fourth motion guide component 1224 includes a third linear guide rail 1224a and a first cylindrical bushing 1224b fixed to one end of the third linear guide rail 1224a. The third linear guide rail 1224a is movably mounted on the third motion guide component 1223 via a third slider 1223b.

[0145] like Figure 7 As shown, Figure 7 for Figure 2 A partial enlarged view of the fifth motion guide component, combined with... Figure 2 , Figure 6 and Figure 7 As shown, the fifth motion guide component 1225 is movably mounted on the fourth motion guide component 1224. Specifically, the fifth motion guide component 1225 includes a first cylindrical shaft 1225a and a second cylindrical bushing 1225b that are relatively fixed. The first cylindrical shaft 1225a is rotatably mounted in the first cylindrical bushing 1224b.

[0146] like Figure 8 As shown, Figure 8 for Figure 2 A partial enlarged view of the 6th motion guide component, combined with... Figure 2 , Figure 7 and Figure 8 As shown, the sixth motion guide component 1226 is movably mounted on the fifth motion guide component 1225. Specifically, the sixth motion guide component 1226 includes a second cylindrical shaft 1226a, which is rotatably mounted in a second cylindrical bushing 1225b. The second cylindrical shaft 1226a is fixed relative to the imaging module 131.

[0147] It should be noted that the number and mechanical structure of the motion guide components in the adjustment module 122 can be set based on requirements to achieve adjustment of the spatial position of the photographing module 131, and are not limited to Figure 2-8 In the embodiment of the present application, the i+1th motion guide component can be translated or rotated relative to the ith motion guide device. When the two can be relatively translated, the driving principle is as shown in Figure 9 and Figure 10 When the two can be relatively rotated, the driving principle is as shown in Figure 11 .

[0148] As shown in Figure 9 and Figure 10 As shown in Figure 9 and Figure 10 is a schematic diagram of the driving principle of translation between two motion guide components provided in the embodiment of the present application, wherein Figure 9 is a structural schematic diagram when the i+1th motion guide component 22 and the ith motion guide component 21 are separated, Figure 10 is a structural schematic diagram after the i+1th motion guide component 22 and the ith motion guide component 21 are installed and combined. When the i+1th motion guide component 22 can be translated relative to the ith motion guide component 21, the translation between the two motion guide components is controlled by a first electromagnetic driving device. Of the two motion guide components, one is fixed with a magnet 01, and the other is fixed with a coil 02. The first electromagnetic driving device includes the magnet 01 and the coil 02, and is used to make the two motion guide components relatively move based on the electromagnetic force between the magnet 01 and the coil 02 when the coil 02 is charged.

[0149] In the manner shown in Figure 9 and Figure 10 For example, the i+1th motion guide component 22 is fixed with a coil 02, and the ith motion guide component 21 is fixed with a magnet 01. The ith motion guide component 21 has a linear guide rail, and a plurality of magnets 01 are fixedly arranged in the middle of the linear guide rail. The i+1th motion guide component 22 has a slider which is movably installed on the linear guide rail. The slider has a coil 02 in the middle. Obviously, in other manners, the magnet 01 can be fixed on the i+1th motion guide component 22, and the coil 02 can be fixed on the ith motion guide component 21.

[0150] As shown in Figure 11 Figure 11 ​A driving principle diagram of rotation between two motion guide components is provided for the embodiment of the present application. When the i+1th motion guide component 22 can rotate relative to the ith motion guide component 21, rotation control between the two motion guide components is performed through the second electromagnetic driving device 23. The second electromagnetic driving device 23 includes a motor.

[0151] Among the first motion guide component 1221 to the sixth motion guide component 1226, a total of five groups of electromagnetic driving devices are needed, which are the first group of electromagnetic driving devices to the fifth group of electromagnetic driving devices.

[0152] The first group of electromagnetic driving devices acts between the first motion guide component 1221 and the second motion guide component 1222, and drives the relative motion of the first motion guide component 1221 and the second motion guide component 1222 through electromagnetic force.

[0153] The second group of electromagnetic driving devices acts between the second motion guide component 1222 and the third motion guide component 1223, and drives the relative motion of the second motion guide component 1222 and the third motion guide component 1223 through electromagnetic force.

[0154] The third group of electromagnetic driving devices acts between the third motion guide component 1223 and the fourth motion guide component 1224, and drives the relative motion of the third motion guide component 1223 and the fourth motion guide component 1224 through electromagnetic force.

[0155] The fourth group of electromagnetic driving devices acts between the fourth motion guide component 1224 and the fifth motion guide component 1225, and drives the relative motion of the fourth motion guide component 1224 and the fifth motion guide component 1225 through electromagnetic force.

[0156] The fifth group of electromagnetic driving devices acts between the fifth motion guide component 1225 and the sixth motion guide component 1226, and drives the relative motion of the fifth motion guide component 1225 and the sixth motion guide component 1226 through electromagnetic force.

[0157] Because the sixth motion guide component 1226 and the photographing module 131 are relatively fixedly installed, the photographing module 131 can change the relative position with the skeleton 121, change the spatial position, and further change the relative position with the geometric positioning cap 11 worn through the common action of the multiple groups of electromagnetic driving devices and the multiple motion guide components.

[0158] In order to be able to detect the relative position parameter of the i+1th motion guide component 22 relative to the ith motion guide component 21, and accurately detect the relative position between the motion guide components, as shown in FIG. 8, the relative position parameter of the i+1th motion guide component 22 relative to the ith motion guide component 21 is detected through the first sensor 24 and the second sensor 25. Figure 9-11As shown, the adjusting module 122 further comprises a position sensor 30 for detecting the relative position between the motion guide components. The position sensor 30 comprises a resistance wire 31 and a brush 32. When the brush 32 contacts the resistance wire 31 at different positions, the resistance of the measuring circuit is different, and the relative position parameters of the two motion guide components can be determined by the resistance value, thereby facilitating accurate movement parameter control.

[0159] The resistance wire 31 and the brush 32 are respectively fixed on the i+1th motion guide component 22 and the ith motion guide component 21. When the relative position of the i+1th motion guide component 22 relative to the ith motion guide component 21 changes, the relative position of the resistance wire 31 and the brush 32 also changes, and the resistance value between the brush 32 and the resistance wire 31 also changes. The relative position of the i+1th motion guide component 22 relative to the ith motion guide component 21 can be determined by the change of the resistance value.

[0160] In the embodiment, there are five groups of position sensors 30, which are the first group of position sensors, the second group of position sensors, the third group of position sensors, the fourth group of position sensors, and the fifth group of position sensors. The first group of position sensors are used to determine the relative position between the first motion guide component 1221 and the second motion guide component 1222. The second group of position sensors are used to determine the relative position between the second motion guide component 1222 and the third motion guide component 1223. The third group of position sensors are used to determine the relative position between the third motion guide component 1223 and the fourth motion guide component 1224. The fourth group of position sensors are used to determine the relative position between the fourth motion guide component 1224 and the fifth motion guide component 1225. The fifth group of position sensors are used to determine the relative position between the fifth motion guide component 1225 and the sixth motion guide component 1226.

[0161] The motion direction is determined by the motion guide components, the driving force is provided by the electromagnetic driving device, and the motion state is detected by the position sensor, so that the photographing module 131 and the skeleton 121 can be adjusted to any spatial position within a set range, thereby adjusting the relative position of the photographing module 131 and the geometric positioning cap 11 based on the demand.

[0162] In Figure 9 and Figure 10In the illustrated configuration, the (i+1)th motion guide component 22 is capable of translational movement relative to the (i)th motion guide component 21. Specifically, the (i+1)th motion guide component 22 can move along the linear guide rail in the (i)th motion guide component 21. In this configuration, the resistance wire 31 is arranged along the linear guide rail in the (i)th motion guide component 21, and the brush is fixed on the (i+1)th motion guide component 22, moving as the (i+1)th motion guide component 22 moves. Figure 11 In the illustrated configuration, the (i+1)th motion guide component 22 is rotatable relative to the (i)th motion guide component 21. The resistance wire loop 31 is fixed on the cylindrical bushing of the (i)th motion guide component 21 and surrounds the cylindrical axis of the (i+1)th motion guide component 22. The brush 32 is fixed on the cylindrical axis surrounding the (i+1)th motion guide component 22 and can rotate with the rotation of the cylindrical axis of the (i+1)th motion guide component 22.

[0163] In the embodiments of this application, such as Figure 1 and Figure 2 As shown, the image acquisition component 13 further includes at least one 3D scanner 132; the 3D scanner 132 is used to scan the three-dimensional shape of the user's face using infrared laser; the facial image also includes the three-dimensional shape of the face. The 3D scanner 132 is disposed on the skeleton 121 or on the camera module 131. Figure 1 and Figure 2 In the illustrated configuration, six of the aforementioned 3D scanners 132 are provided. Figure 2 Taking the cube frame as an example, with the skeleton 121 as an example, all 3D scanners 132 are fixed on the cube frame. In the rectangular frame on one side of the cube frame used to install the adjustment module 122, during image acquisition, a 3D scanner 132 is fixed at the top and bottom ends of the inner side of the vertically placed left frame, a 3D scanner 132 is fixed at the top and bottom ends of the inner side of the vertically placed right frame, and a 3D scanner 132 is fixed in the middle of the inner side of each of the two horizontally placed frames.

[0164] When multiple 3D scanners 132 are present, it is not limited to fixing all 3D scanners 132 to the frame 121. Some 3D scanners 132 can be mounted and fixed to the frame 121, some can be mounted and fixed to the imaging module 131, or all 3D scanners 132 can be mounted and fixed to the imaging module 131. The 3D scanners 132 can be set on the frame 121 or on the imaging module 131 as needed, and this application embodiment does not specifically limit this.

[0165] like Figure 12 As shown, Figure 12 This is a schematic diagram of the structure of a 3D scanner provided in an embodiment of this application.Figure 12 The 3D scanner 132 shown includes a laser emitter 1321 and a laser receiver 1322. The 3D scanner emits laser light through the laser emitter 1321 and receives the reflected laser light after it has been blocked by an object through the laser receiver 1322. Since the angle parameters of the laser emitted by the laser emitter 1321 and the angle parameters of the laser received by the laser receiver 1322 are known, and the relative position parameters of the laser emitter 1321 and the laser receiver 1322 are also known, the relative position parameters of the object blocking the laser light relative to the 3D scanner 132 can be determined based on these known angle and relative position parameters, thus achieving the purpose of scanning the shape of the object. Therefore, the 3D scanner can scan the 3D shape of a user's face, including facial image information such as eyelids, nose, and ears.

[0166] In the 3D scanner 132, the laser emitter 1321 emits infrared light to avoid the laser affecting the user's vision when scanning the user's facial image. Not limited to the laser receiver 1322 receiving lasers emitted by the laser emitter 1321 in the same 3D scanner, in multiple 3D scanners 132, the laser receiver 1322 of one 3D scanner 132 can receive lasers emitted by the laser emitters 1321 of other 3D scanners 132 to form a 3D point cloud, facilitating accurate and rapid acquisition of 3D images of the user's face.

[0167] In this embodiment of the application, the user's eye image includes an infrared image and a visible light image of the eye.

[0168] like Figure 13 As shown, Figure 13 This is a schematic diagram of the structure of a gaze pointing measurement device imaging module provided in an embodiment of this application. The imaging module 131 shown includes: a camera bracket 131a; an infrared camera assembly 131b mounted on the camera bracket 131a, the infrared camera assembly 131b including: a first camera 131b1, a second camera 131b2, and an infrared point light source 131b3; the first camera 131b1 and the second camera 131b2 are used to acquire infrared light images of the eye image; a color camera array mounted on the camera bracket 131a, the color camera array including a plurality of color cameras 131c located around the infrared camera assembly 131b; the color cameras 131c are used to acquire visible light images of the geometric positioning cap image, the facial image, and the eye image; and at least one infrared illumination source 131d.

[0169] The infrared illumination source 131d is mounted on the camera bracket 131a and / or the infrared camera assembly 131b.

[0170] The components in the photographing module 131 are fixedly arranged as a whole, and the relative positions of the internal components are unchanged. The infrared point light source 131b3 emits infrared light of a preset wavelength. The light emitting hole of the infrared point light source 131b3 can be circular with a diameter of 0.5 mm. Apparently, the shape and size of the light emitting hole of the infrared point light source 131b3 can be set based on requirements, and are not limited to the description in the embodiments of the present application.

[0171] In the embodiments of the present application, the color camera array is provided with 16 color cameras 131c arranged in a 4x4 array around the infrared camera assembly 131b. Apparently, the number and arrangement of the color cameras 131c can be set based on requirements, and are not limited to the arrangement shown in the figure. Figure 13

[0172] In the embodiments of the present application, five infrared illuminating light sources 131d can be provided, four of which are fixedly arranged on the camera bracket 131a around the infrared camera assembly 131b, and the other one is fixedly arranged on the infrared camera assembly 131b. Apparently, the number and arrangement of the infrared illuminating light sources 131d can be set based on requirements, and are not limited to the arrangement shown in the figure. Figure 13

[0173] The first camera 131b1 and the second camera 131b2 are both infrared cameras, which can only receive infrared light for infrared imaging. The infrared illuminating light source 131d is used to emit first infrared detection light. After the first infrared detection light irradiates the iris and pupil of the user's eye, the reflected light can be received by the first camera 131b1 to form a first image, and the first image can distinguish the iris texture and pupil of the eye.

[0174] The iris texture of Asians is brown, which can receive most visible light, but it is difficult to reflect visible light. Therefore, a common visible light camera cannot receive the reflection of the iris of Asians, and cannot capture a clear iris texture. In the embodiments of the present application, the infrared illuminating light source 131d emits first infrared detection light, which is not easily absorbed by the iris of Asians and reflects light. The reflected light is received by the first camera 131b1, so that an image that can clearly distinguish the iris texture and pupil of the eye is captured. Based on the first image, the relative position of the user's eye and the first camera 131b1 can be determined.

[0175] The first camera 131b1 forms a first image based on the first infrared detection light reflected by the eyeball of the user. The first image is as shown in the figure. Figure 14 Figure 14 ​​​A schematic diagram of a first image is provided for the embodiments of the present application, the first image comprising infrared light iris information and infrared light pupil information of the eyeball of the user. Based on the infrared light pupil information in the first image, the optical axis of the first camera 131b1 is adjusted to intersect at the pupil center point of the eye.

[0176] As shown in Figure 15 , Figure 15 A structural schematic diagram of a first camera is provided for the embodiments of the present application. A visible light point light source T151 is installed in the first camera 131b1 to indicate the position of the first camera 131b1, so as to facilitate the user to fix the first camera 131b1 with the gaze. The visible light point light source T151 can move, and when it moves to the optical axis (indicated by the dashed line in Figure 15 ) of the first camera 131b1 and emits visible light, the visible light can be irradiated to the user's eye through the lens center of the first camera 131b1. After the visible light point light source leaves the optical axis, it does not block the light from entering the lens T152, so as to facilitate the first camera 131b1 to image.

[0177] The barrel side wall of the first camera 131b1 has a through hole, and the visible light point light source T151 is located in the lens barrel. The visible light point light source T151 is fixed on a connecting rod T153. The connecting rod T153 extends to the outside of the lens barrel through the through hole and is fixed on an electromagnetic driving device. The visible light point light source T151 is moved by the electromagnetic driving device. Other ways of indicating the position of the first camera 131b1 without installing the visible light point light source on the first camera 131b1 are also possible. The embodiments of the present application are not limited to the way of installing the visible light point light source on the first camera 131b1.

[0178] As shown in Figure 16 , Figure 16 A principle schematic diagram of the alignment of the pupil center point of the user and the lens center of the first camera is provided for the embodiments of the present application. According to the infrared light pupil information in the first image, the spatial position of the photographing module 131 is adjusted, and the first camera 131b1 is rotated, so that the optical axis of the first camera 131b1 intersects at the pupil center point of the user.

[0179] Taking the left eye of the user as an example, first, as shown in the upper left eye of the user as an example, first, as shown in the upper Figure 16 , according to the infrared light pupil information of the left eye in the first image, the position of the left eye pupil center point in the first image is determined, and then, as shown in the lower Figure 16As shown in the middle and lower images, the first camera 131b1 is rotated to take a first image again, and the pupil center point of the left eye is in the center of the first image. At this time, the optical axis of the first camera 131b1 intersects the pupil center point of the left eye of the user, and the visual axis of the user intersects the lens center of the first camera 131b1. Based on this, the relative position of the left eye of the user and the first camera 131b1 can be determined.

[0180] In the first image, the pupil center point is in the center of the first image, and the optical axis of the first camera is the visual axis when the user's eye directly looks at the lens center of the first camera. Taking the left eye as an example, according to the pupil feature of the left eye in the first image taken by the first camera, the position of the pupil center point in the first image is determined. The first camera is rotated around the lens focal point of the first camera, so that the optical axis of the first camera is collinear with the visual axis of the left eye. At this time, the first camera takes the left eye again, and the pupil center point of the left eye is in the center of the first image, so that the optical axis of the first camera is collinear with the visual axis of the left eye. Thus, the relative position of the visual axis and the photographing module can be determined. Correspondingly, in the virtual space, the optical axis model of the first camera and the visual axis model coincide in the three-dimensional model of the photographing module.

[0181] The infrared point light source 131b3 is used to emit second infrared detection light. In the embodiment of the application, the second camera 131b2 only receives the second infrared detection light of the corresponding wave band of the infrared point light source 131b3. The second camera 131b2 is used to form a second image based on the second infrared detection light reflected by the cornea of the user's eye, and the second image includes a highlight point of the second infrared detection light reflected by the cornea of the user's eye. The infrared light image of the eye image includes the first image and the second image. The position of the highlight point in the second image can be used to determine the distance from the user's eye to the second camera 131b2, so as to determine the relative position of the user's eye and the photographing module 131.

[0182] As shown in Figure 17 , the second camera 131b2 is used to form a second image based on the second infrared detection light reflected by the cornea of the user's eye, and the second image includes a highlight point of the second infrared detection light reflected by the cornea of the user's eye. The infrared light image of the eye image includes the first image and the second image. The position of the highlight point in the second image can be used to determine the distance from the user's eye to the second camera 131b2, so as to determine the relative position of the user's eye and the photographing module 131. Figure 17A structural schematic diagram of an infrared camera assembly provided in an embodiment of the present application is shown in the above. The infrared camera assembly 131b includes a first camera 131b1, a second camera 131b2, and an infrared point light source 131b3. In order to accurately calculate the distance from the user's eyes to the second camera 131b2, the second camera 131b2 and the infrared point light source 131b3 are arranged on both sides of the optical axis T171 of the first camera 131b1; the line T172 connecting the lens focal point of the second camera 131b2 and the infrared point light source 131b3 intersects the optical axis T171 of the first camera 131b1, and the line T172 is perpendicular to the optical axis T171 of the first camera 131b1; the distance between the infrared point light source 131b3 and the optical axis T172 of the first camera 131b1 is equal to the distance between the lens focal point of the second camera 131b2 and the optical axis of the first camera 131b1.

[0183] In an embodiment of the present application, the images captured by the first camera 131b1, the second camera 131b2, and the color camera 131c are all distortion-free images, or are converted into distortion-free images through an algorithm.

[0184] As shown in Figure 18 and Figure 19 , the color camera 131c is arranged on the optical axis T171 of the first camera 131b1, and the color camera 131c is arranged on the optical axis T172 of the second camera 131b2. Figure 18 A scene schematic diagram in which a camera captures a square pattern is provided in an embodiment of the present application. Figure 19 A schematic diagram of an image captured by a camera is provided in an embodiment of the present application. In the present example, any camera-captured or converted distortion-free image needs to satisfy: as shown in Figure 18 a square pattern plane T182 composed of uniform and equal-size squares is captured by a camera T181, and the optical axis T183 of the camera T181 is perpendicular to the plane T182 during the capturing; each square in the captured or processed image is still a square, and the sizes of all the squares are the same. The squares in the image need to satisfy that the four sides are equal in length and perpendicular to adjacent sides. If the captured image is a distorted image as shown in Figure 19 , it needs to be converted into a distortion-free image through an algorithm.

[0185] In an embodiment of the present application, the color camera 131c can receive visible light band light rays, and is used to capture a third image including the junction line features of the user's iris and sclera; wherein the visible light image of the eye image includes the third image.

[0186] As shown in Figure 20 , the color camera 131c is arranged on the optical axis T171 of the first camera 131b1, and the color camera 131c is arranged on the optical axis T172 of the second camera 131b2. Figure 20This is a schematic diagram of a geometric positioning cap provided in an embodiment of this application. The geometric positioning cap 11 includes: a wearable component T201 for wearing on a user's head; and a geometric shape marking component T202 disposed on the outer side of the wearable component T201. The relative position of the user's face and the photographing module 131 is determined based on the image of the geometric shape marking component T202 in the geometric positioning cap image. The geometric positioning cap 11 is wearable on the user's head, lightweight, and fits snugly against the user's skull. It maintains a relatively fixed position with the head even when the user slightly shakes their head, without changing their relative position.

[0187] The geometric shape identification component T202 can be configured to include multiple visible point light sources with different emitting colors, and the visible point light sources are arranged around the outer perimeter of the wearable component T201; based on at least two images of the geometric positioning cap captured by the camera module, the relative position of the user's face and the camera module 131 is determined; each image of the geometric positioning cap includes multiple images of the same geometric shape identification component T202.

[0188] In this embodiment, the geometric positioning cap 11 has seven visible point light sources with different emitting colors, serving as seven geometric shape marking components T202. These seven visible point light sources are designated as the first visible point light source D1 to the seventh visible point light source D7. These seven visible point light sources are installed and fixed at different positions on the wearable component T201. The number of visible point light sources can be set according to requirements and is not limited to seven.

[0189] The camera module 131 is configured to have 16 color cameras 131c, which are designated as color cameras numbered 1 through 16. The number of color cameras 131c can be set according to requirements and is not limited to 16.

[0190] like Figure 21 and Figure 22 As shown, Figure 21 Image of the geometric positioning cap taken by the first color camera. Figure 22 The geometric positioning cap image is captured by the 16th color camera. This is achieved by simultaneously capturing one geometric positioning cap image each from the 1st and 16th color cameras. The geometric positioning cap image captured by the 1st color camera is designated as the first geometric positioning cap image, and the image captured by the 16th color camera is designated as the second geometric positioning cap image. Any two color cameras 131c of the same model but located at different positions can be used to capture the first and second geometric positioning cap images; it is not limited to the 1st and 16th color cameras.

[0191] There are images of at least three identical visible point light sources on the geometric positioning cap 11 in the first geometric positioning cap image and the second geometric positioning cap image, for example, images of the 1st visible point light source to the 3rd visible point light source are common in the first geometric positioning cap image and the second geometric positioning cap image.

[0192] A three-dimensional rectangular coordinate system is placed in the first color camera three-dimensional model, referred to as the first color camera three-dimensional rectangular coordinate system, and the three coordinate axes of the first color camera three-dimensional rectangular coordinate system are respectively referred to as the SX axis, the SY axis, and the SZ axis. The planes between any two of the three coordinate axes are respectively the SX SZ plane, the SY SZ plane, and the SX SY plane. The SZ axis is parallel to and coincides with the optical axis of the first color camera, the SX axis is parallel to the longer side of the rectangular photosensitive element of the first color camera, and the origin of the first color camera three-dimensional rectangular coordinate system is at the focal point of the lens of the first color camera.

[0193] As shown in Figure 23 , Figure 23 is a schematic diagram of the optical axis of the camera in this application. The dashed line in the Figure 23 is the light ray that forms the most central pixel point in the image formed by the camera receiving light rays to shoot images at various angles after passing through the camera lens.

[0194] The angles of the visible light rays emitted by the 1st visible point light source, the 2nd visible point light source, and the 3rd visible point light source and entering the lens of the first color camera are determined from the positions of the center points of the characteristic information of the 1st visible point light source, the 2nd visible point light source, and the 3rd visible point light source in the first geometric positioning cap image. Through the same method, the angles of the visible light rays emitted by the 1st visible point light source, the 2nd visible point light source, and the 3rd visible point light source and entering the lens of the second color camera are determined.

[0195] In an embodiment of the present application, the angle between the projection of the light emitted by the first visible light point source and entering the lens of the first color camera on the SX SZ plane of the first color camera three-dimensional rectangular coordinate system and the light is 11°, the angle between the projection of the light on the SY SZ plane of the first color camera three-dimensional rectangular coordinate system and the light is 15°, which represents the angle between the light and the first color camera three-dimensional rectangular coordinate system is (11, 15); the angle between the projection of the light emitted by the second visible light point source and entering the lens of the first color camera on the SX SZ plane of the first color camera three-dimensional rectangular coordinate system and the light is 12°, the angle between the projection of the light on the SY SZ plane of the first color camera three-dimensional rectangular coordinate system and the light is 17°, which represents the angle between the light and the first color camera three-dimensional rectangular coordinate system is (12, 17); the angle between the projection of the light emitted by the third visible light point source and entering the lens of the first color camera on the SX SZ plane of the first color camera three-dimensional rectangular coordinate system and the light is 15°, the angle between the projection of the light on the SY SZ plane of the first color camera three-dimensional rectangular coordinate system and the light is 10°, which represents the angle between the light and the first color camera three-dimensional rectangular coordinate system is (15, 10).

[0196] As shown in Figure 24 and Figure 25 , Figure 24 is a three-dimensional model schematic view of a photographing module and a geometric positioning cap provided in an embodiment of the present application, Figure 25 is a three-dimensional model of a photographing module and a relative position schematic view of a first geometric positioning cap image T251 provided in an embodiment of the present application. A computer is used to establish a virtual three-dimensional space, and a light pointing measurement device with the same shape as in reality is established. In the three-dimensional model T241 of the photographing module of the light pointing measurement device, the lens focal point of the first color camera three-dimensional model T244 is taken as an endpoint, and three straight lines are drawn according to the angles (11, 15), (12, 17) and (15, 10) of the first color camera three-dimensional rectangular coordinate system respectively, which are defined as the first straight line L1, the second straight line L2 and the third straight line L3 respectively, and are used to simulate the light emitted by the first visible light point source, the second visible light point source and the third visible light point source and entering the lens of the first color camera.

[0197] In the same way, the light emitted by the first visible light point source, the second visible light point source and the third visible light point source and entering the lens of the 16th color camera is simulated respectively, and the three corresponding light entering the lens of the 16th color camera are the fourth straight line L4, the fifth straight line L5 and the sixth straight line L6. Among them, T242 represents the three-dimensional model of the geometric positioning cap, and T243 represents the three-dimensional model of the 16th color camera.

[0198] Because the light emitted by the first visible point light source can be regarded as two light rays respectively entering the lens center of the first color camera and the sixteenth color camera in the light emitted by a point in space, a first straight line L1 simulating the light emitted by the first visible point light source and entering the first color camera intersects a fourth straight line simulating the light emitted by the first visible point light source and entering the sixteenth color camera, and the intersection point simulates the position of the real first visible point light source relative to the positions of the first color camera and the sixteenth color camera. Similarly, the positions of the second visible point light source and the third visible point light source can also be simulated by the first color camera three-dimensional model and the sixteenth color camera three-dimensional model, so as to determine the relative position of the face of the user wearing the geometric positioning cap relative to the first color camera and the sixteenth color camera, that is, the relative position of the face of the user to the photographing module.

[0199] As Figure 26 shown, Figure 26 a schematic diagram of a module coordinate system provided by an embodiment of the present application, the color camera array on the photographing module 131 has a plurality of color cameras 131c. The simultaneous photographing of the eyes of the user by the color cameras 131c at different positions can obtain the images of the face and eyes of the user under different angles. A virtual three-dimensional rectangular coordinate system is placed in the photographing module 131, which is referred to as a module coordinate system. To avoid repetition, the X axis, Y axis and Z axis of the three-dimensional rectangular coordinate system are referred to as WX axis, WY axis and WZ axis respectively, and the planes formed by any two of the three coordinate axes are referred to as WX WZ plane, WY WZ plane and WX WY plane respectively.

[0200] The origin WO of the module coordinate system is on the optical axis of the first camera, and the WZ axis of the module coordinate system is parallel to the optical axis of the first camera. Moreover, it is satisfied that when looking from the origin of the module coordinate system along the direction of the optical axis of the first camera 131b1 to the first camera 131b1, the lens of the first camera 131b1 can be seen, that is, the front of the first camera 131b1 can be seen. The distance from the origin WO of the module coordinate system to the focal point of the lens of the first camera 131b1 is set to 30 cm. Moreover, the WX WY plane is parallel to the line connecting the focal point of the lens of the second camera 131b2 and the infrared point light source 131b3. It should be noted that all the coordinate systems in the present application are three-dimensional rectangular coordinate systems.

[0201] The plurality of color cameras 131c in the photographing module 131 are arranged in a regular array. For example, if there are 16 color cameras 131c, the 16 color cameras 131c are arranged in a 4x4 array along the optical axis of the first camera 131b1, and are sequentially numbered as the first color camera to the sixteenth color camera. The optical axis of each color camera 131c intersects at the origin WO. Each color camera 131c is fixedly installed on the photographing module 131 and has a fixed relative position.

[0202] The projection of a line connecting any point in the module coordinate system to the origin WO on the WY-WZ plane has an angle α with the line. If the coordinate value of the any point on the WX axis in the module coordinate system is positive, the angle α is positive. If the coordinate value of the any point on the WX axis is negative, the angle α is negative. The projection of the line on the WX-WZ plane has an angle β with the line. If the coordinate value of the any point on the WY axis is positive, the angle β is positive. If the coordinate value of the any point on the WY axis is negative, the angle β is negative.

[0203] In the color camera array, the angles of the optical axes of the first color camera to the sixteenth color camera with the projections on the WY-WZ plane and the WX-WZ plane are expressed in the format (α, β) and are respectively (45°, 45°), (15°, 45°), (-15°, 45°), (-45°, 45°), (45°, 15°), (15°, 15°), (-15°, 15°), (-45°, 15°), (45°, -15°), (15°, -15°), (-15°, -15°), (-45°, -15°), (45°, -45°), (15°, -45°), (-15°, -45°), and (-45°, -45°).

[0204] The eye direction measurement device also includes a computer having an information processor configured to analyze image data of the geometric positioning cap image, the face image, and the eye image, determine the relative position of the face of the user and the photographing module based on the image data analysis result, adjust the spatial position of the photographing module through the adjusting assembly to adjust the relative position, obtain the corresponding geometric positioning cap image, face image, and eye image under different relative positions, and determine the eye direction model under different relative positions.

[0205] The information processor can receive signals measured by the position sensor and can also control the electromagnetic drive device, the imaging module, and the 3D scanner. The computer also has an information storage medium for storing images acquired by the imaging module and the 3D scanner. The eye-pointing model establishment method described in the following embodiments is implemented through the information processor.

[0206] The information processor is used to determine the angle of visible light emitted from the first, second, and third visible light source points and incident on the lens of the first color camera, based on the positions of the center points of the feature information of the first, second, and third visible light source points in the first geometric positioning cap image. The information processor also determines the angle of visible light emitted from the first, second, and third visible light source points and incident on the lens of the 16th color camera using the same method.

[0207] Taking the center point of the first visible point light source in the first geometric positioning cap image as an example:

[0208] First, using an image recognition algorithm, the feature information of the first visible light source in the first geometric positioning cap image is determined according to the brightness and color of the image. Then, the center point of the first visible light source in the first geometric positioning cap image is determined. Since the image corresponding to the first visible light source in the first geometric positioning cap is either circular or elliptical, determining the center of the circle or ellipse determines the exact position of the center point of the first visible light source in the first geometric positioning cap image. Finally, using the method of calculating the light angle by corresponding image point positions, the angle between the light from the pixel corresponding to the center point in the first geometric positioning cap image and the camera is determined.

[0209] like Figure 27-33 As shown, Figure 27-33 This application provides a schematic diagram illustrating the principle of calculating the angle of a ray by calculating the position of a point, and the implementation method is as follows:

[0210] First, such as Figure 27 As shown, a piece of paper with a Cartesian coordinate system is printed with high precision. This Cartesian coordinate system includes an origin, a horizontal axis, and a vertical axis, both with scales. On the horizontal axis, scales to the right of the origin are positive, and scales to the left are negative, with values ​​farther from the origin having larger absolute values. On the vertical axis, scales above the origin are positive, and scales below the origin are negative, with values ​​farther from the origin having larger absolute values. It is necessary to ensure that the length of each smallest scale unit is equivalent to 1 cm, 1 mm, 0.1 mm, 0.01 mm, or 1 μm.

[0211] Then, as shown in Figure 28 , the paper is photographed by the camera without distortion. Before photographing, the camera is adjusted to be completely in line with the table on the paper, so that the optical axis of the camera intersects the origin of the coordinate system on the paper, and the optical axis of the camera is perpendicular to the plane on which the paper is located, and the horizontal pixels of the image photographed by the camera are parallel to the horizontal axis of the coordinate system on the paper, and the distance between the focal point of the lens of the camera and the plane on which the paper is located is adjusted to a known value.

[0212] Based on Figure 28 the settings, the image of the paper is photographed by the camera, and a rectangular coordinate system image as shown in Figure 29 is obtained, and the shape of the rectangular coordinate system image is rectangular.

[0213] As shown in Figure 30 , the scene of the paper photographed by the camera is restored in the virtual space built by the computer, and the size of the paper model T301 and the rectangular coordinate system on the paper model T301 are the same as the real paper, and the relative position relationship between the focal point of the lens of the camera model T302 and the optical axis and the paper model T301 is the same as the relative position relationship between the real camera and the paper. In the rectangular coordinate system image obtained by the camera photographing the paper, the four edges of the rectangular coordinate system image are found, and the horizontal scale value and the vertical scale value of the point intersected by each edge on the horizontal axis or the vertical axis of the rectangular coordinate system are read.

[0214] As shown in Figure 31 , in the virtual space built by the computer, a rectangle T303 is made with the four points on the horizontal axis and the vertical axis of the rectangular coordinate system in the paper model T301, and the four edges of the rectangle T303 intersect the four points.

[0215] The rectangular area is the area on the picture that can be photographed within the field of view of the camera, and the boundary of the rectangular area is the boundary of the field of view of the camera. The rectangular area has four end points, which represent the upper left corner, the upper right corner, the lower left corner, and the lower right corner of the rectangular image obtained by photographing the paper. Four straight lines are drawn with the four points of the rectangular area as end points and the focal point of the lens of the camera as another end point. The four straight lines respectively represent the light rays reflected and incident into the lens of the camera from the corresponding points on the picture in the photograph when the photograph is taken. The four straight lines enclose a four-pyramid three-dimensional region, which is the field of view of the camera.

[0216] The rectangular coordinate system image is imported into the virtual space built by the computer, and the rectangular coordinate system image is scaled proportionally so that the four edges of the rectangular coordinate system image coincide with the four edges of the rectangle, and the direction of the rectangular coordinate system image is consistent with that of the paper model, that is, the horizontal axis of the coordinate system in the rectangular coordinate system image is parallel to the horizontal axis of the coordinate system in the paper model, and the arrow directions of the coordinate axes are the same, and the vertical axis of the coordinate system in the rectangular coordinate system image is parallel to the vertical axis of the coordinate system in the paper model, and the arrow directions of the coordinate axes are the same.

[0217] Since the image captured by the distorted camera is distortion-free, the rectangular coordinate system in the rectangular coordinate system image coincides with the coordinate system in the paper model. In the virtual space built by the computer, a straight line is connected between an arbitrary point in the rectangular coordinate system image and the focal point of the camera, and the straight line is referred to as straight line one. In the paper model, a point identical to the arbitrary point is found, and a straight line is connected between the point and the focal point of the lens of the camera three-dimensional model, and the straight line is referred to as straight line two. Straight line one coincides with straight line two, that is, the angle between straight line one and the camera three-dimensional model is the same as the angle between straight line two and the camera three-dimensional model. Straight line two simulates the light ray reflected on the paper in the real space and entering the real camera, and the angle between straight line two and the camera three-dimensional model is the angle between the light ray reflected on the paper in the real space and entering the real camera and the real camera.

[0218] Therefore, as shown in Figure 32 in the virtual space, the rectangle is established in the camera three-dimensional model, the image captured by the camera is imported into the virtual space, and after being scaled proportionally and fitted in the rectangle, as shown in Figure 33 the property of the line connecting an arbitrary point on the image and the focal point of the camera is equivalent to that of straight line one, which is used to determine the angle relationship between the light ray forming the point in the image and the camera when the light ray enters the camera lens.

[0219] The gaze direction measurement device can be used to establish a gaze direction model, and the gaze direction model can be used to realize automatic control of an electronic device by tracking the rotation of the user's eyes, and realize an automatic control scheme of the electronic device by the eyes. For example, a plurality of different standard gaze direction models can be stored in the electronic device, different standard gaze direction models correspond to different control instructions, and different control instructions can control the electronic device to perform different functions. When the gaze direction model corresponding to the user's eye image collected in real time by the electronic device matches a standard gaze direction model, the electronic device can be controlled to perform the corresponding function.

[0220] Based on the above-mentioned gaze direction measurement device, another embodiment of the present application further provides a gaze direction model establishment method, as shown in Figure 34 the gaze direction model establishment methodFigure 34 A schematic diagram of a gaze direction model establishment method provided by an embodiment of the present application, the method comprising:

[0221] Step S11: When the user wears the geometric positioning cap, the geometric positioning cap image, the face image of the user and the eye image of the user are collected by the image collection component.

[0222] After the user wears the geometric positioning cap, the user faces the photographing module of the gaze direction measurement device, and when the head is slightly shaken, the geometric positioning cap and the head remain relatively stable.

[0223] Step S12: Image data analysis is performed on the geometric positioning cap image, the face image and the eye image, and based on the image data analysis result, the relative position of the face of the user and the photographing module is determined.

[0224] Step S13: The spatial position of the photographing module is adjusted by the adjusting component to adjust the relative position, and the corresponding geometric positioning cap image, face image and eye image under different relative positions are obtained to determine the gaze direction model under different relative positions.

[0225] The gaze direction model comprises an optical axis model corresponding to the relative position, a pupil center point model, an iris texture marking line model, an eye coordinate system and a projection iris edge curve group model.

[0226] The photographing module comprises a camera support, an infrared camera assembly mounted on the camera support, the infrared camera assembly comprising a first camera, a second camera and an infrared point light source, a color camera array mounted on the camera support, the color camera array comprising a plurality of color cameras located around the infrared camera assembly, and at least one infrared illumination light source. The specific implementation of the photographing module can refer to the description of the above embodiment, and the method embodiment will not be repeated.

[0227] In the gaze direction model establishment method, the geometric positioning cap image, the face image of the user and the eye image of the user are collected as shown in Figure 35 Figure 35 A flowchart of an image collection method provided by an embodiment of the present application, the method comprising:

[0228] Step S21: The spatial position of the photographing module is adjusted by the adjusting component to make the visual axis of the user and the optical axis of the first camera satisfy the coincidence condition.

[0229] ​Step S22: After the overlap condition is met, change the relative position of the camera module and the user's face to obtain first measurement data under different relative positions.

[0230] The first measurement data includes facial and eye images at corresponding relative positions. In other words, when images are acquired at different relative positions, the optical axis of the first camera intersects at the center point of the user's pupil.

[0231] In step S22 above, the method of changing the relative position of the camera module and the user's face includes: a first method, keeping the focus of the first camera of the camera module fixed while the user's head rotates; a second method, keeping the user's head fixed while controlling the movement of the camera module, such as the user controlling the movement of the camera module by driving the adjustment component through a remote control.

[0232] like Figure 36 As shown, Figure 36 This application provides a schematic diagram of a method for ensuring that the user's line of sight coincides with the optical axis of the first camera, the method comprising:

[0233] Step S31: Acquire a first image of the user's eye using a first camera, the first image including infrared pupil information.

[0234] When the user looks directly at the first camera, the information processor sends a shooting command to the camera module to control the first camera to capture an image of the eye and obtain the first image. Taking the left eye image as an example, the first image includes information about the left pupil. Without adjustment, the center point of the left pupil is generally not in the exact center of the first image.

[0235] Step S32: Based on the first image, adjust the spatial position of the camera module through the adjustment component to move the first camera so that the optical axis of the first camera coincides with the user's line of sight.

[0236] The information processor identifies the features of the left eye pupil in the first image. The first camera is rotated around its focal point. After the optical axis of the first camera intersects the center point of the left eye pupil using an adjustment module, the information processor sends another image capture command to the first camera. The first camera then captures another image of the left eye. In this second image, the center point of the left eye pupil is located in the exact center of the first image. At this point, the optical axis of the first camera intersects the center point of the left eye pupil, and the user's visual axis intersects the center of the first camera lens. Therefore, the visual axis coincides with the optical axis of the first camera; in this case, the visual axis is the optical axis of the first camera.

[0237] After aligning the optical axis and visual axis of the first camera, the relative position of the imaging module and the user's face is changed. At each relative position, the imaging module captures an image of the geometric positioning cap and an image of the face. As described in the above embodiment, two color cameras can simultaneously capture images of the first and second geometric positioning caps to determine the relative positional relationship between the three-dimensional model of the geometric positioning cap and the imaging module, thereby determining the relative positional relationship between the user's facial coordinate system and the imaging module.

[0238] The method for determining the relative position of the user's face and the camera module includes:

[0239] First: Based on the first measurement data, determine the movement pattern of the user's eyes relative to a facial reference.

[0240] The facial reference includes the position of the geometric positioning cap and the three-dimensional facial model.

[0241] Then, based on the motion patterns and the aforementioned facial references, a standard model is constructed.

[0242] The standard model includes multiple preset virtual positions of the camera module, the user's facial coordinate system, and the eye center coordinate system. The facial coordinate system has three mutually perpendicular axes: the FX axis, the FY axis, and the FZ axis; the FZ axis points towards the virtual position, and the FX and FY planes are set relative to the virtual position.

[0243] like Figure 37 As shown, Figure 37 This is a schematic diagram illustrating the principle of a method for establishing a facial coordinate system provided in an embodiment of this application. A three-dimensional model of the geometric positioning cap can be established based on a captured image of the cap. Figure 37 The "+" sign indicates a 3D model of the geometric shape identifier component, representing the 3D model of the geometric positioning cap. A 3D Cartesian coordinate system is placed within the 3D model of the geometric positioning cap as a facial coordinate system. The origin of the facial coordinate system is located at the geometric center of the 3D model of the geometric positioning cap. The FZ axis is parallel to the front of the geometric positioning cap, i.e., the front of the user's face when wearing it. When the user wears the geometric positioning cap, the FX and FY planes are positioned opposite the camera module, with the FY axis pointing upwards.

[0244] like Figure 38 As shown, Figure 38 This application provides a schematic diagram of a method for determining the motion pattern of a user's eyes relative to a facial reference, the method comprising:

[0245] Step S41: Based on multiple sets of the first measurement data, determine multiple pointing models that correspond one-to-one.

[0246] Specifically, the camera module collects a set of first measurement data at different relative positions to the user's face, thereby obtaining multiple sets of first measurement data. The pointing model includes a visual axis model, a facial coordinate system, and the relative relationship between the two.

[0247] Step S42: Overlap all the pointing models with the facial coordinate system as the reference to obtain visual axis models with the eyes looking in different directions based on the user's face. The intersection of all visual axis models is the center model of the user's eyes.

[0248] The central model of the user's eye determines the eye's central coordinate system. The intersection of all visual axis models is the rotation point of the eyeball, and the largest angle with the facial coordinate system is the maximum range of motion of the user's eye.

[0249] In this embodiment of the application, the first measurement data includes the facial image captured by the color camera array and the geometric positioning cap image corresponding to the facial image.

[0250] like Figure 39 As shown, Figure 39 This application provides a schematic diagram of a method for determining the pointing model, the method comprising:

[0251] Step S51: Based on the geometric positioning cap image corresponding to the facial image, determine the relative positional relationship between the user's face and the camera module.

[0252] Step S52: Based on the relative positional relationship between the user's face and the camera module, construct a visual axis model and a facial coordinate system, and combine the visual axis model, the facial coordinate system, and their relative relationship to form the pointing model.

[0253] like Figure 40 As shown, Figure 40 This is a schematic diagram of a pointing model provided in an embodiment of this application. Figure 40 The dashed line represents the visual axis. In the virtual space constructed by the computer, each set of first measurement data can correspond to a visual axis model and a facial coordinate system. Therefore, each set of first measurement data can construct a corresponding pointing model based on the corresponding visual axis model, facial coordinate system, and their relative relationship.

[0254] For example, in the first way, the user rotates the head, and in the rotation of the head, the eyes rotate relative to the face, so that the eyes always look straight at the lens center of the first camera. As described in the above embodiment, the user can keep the relative position of the visual axis and the optical axis of the first camera by looking straight at the visible point light source integrated in the first camera. In the process of head rotation, the first camera continuously captures the images of the user's eyes in different relative positions (including the above-mentioned first image). According to the first image, the first camera is continuously rotated with the lens focal point as the rotation point, so as to always ensure that the optical axis of the first camera intersects the pupil center point of the user's eyes.

[0255] With the head rotating in one direction, before the eyes can no longer continue to rotate relative to the head and can no longer look straight at the lens center of the first camera, the user can control the first color camera and the sixteenth color camera in the photographing module to capture the geometric positioning cap to obtain the first geometric positioning cap image and the second geometric positioning cap image, so as to calculate the relative position relationship between the face and the photographing module, and finally obtain the visual axis of the left eye and the face coordinate system, thereby determining the pointing model.

[0256] As shown in Figure 41 , Figure 41 Another schematic diagram of a pointing model provided by the embodiment of the present application is shown in Figure 41 The dashed line represents the visual axis, and the user changes the head rotation direction to determine another pointing model as shown in Figure 41 The user rotates the head in different directions to obtain a plurality of different pointing models, such as rotating in N different directions, and N pointing models can be obtained based on the above-mentioned manner.

[0257] As shown in Figure 42 , Figure 42 A principle schematic diagram in which a plurality of pointing models provided by the embodiment of the present application coincide with the face coordinate system is shown in Figure 42 The dashed line represents the visual axis model of the eyes looking in different directions, and the intersection of all dashed lines represents the eye center model.

[0258] As shown in Figure 43 , Figure 43 A principle schematic diagram for establishing the left eye center coordinate system provided by the embodiment of the present application is shown in

[0259] In the embodiment of the present application, the left eye center coordinate system is established, and the three mutually perpendicular coordinate axes in the left eye center coordinate system are LX axis, LY axis and LZ axis; the center of the user's left eye (the center of the left eye) is the origin of the left eye center coordinate system, and the LZ axis is parallel to the FZ axis, and the LY axis is parallel to the FY axis.

[0260] The method for constructing the standard model comprises:

[0261] As Figure 44 shown, Figure 44 A standard model schematic diagram provided by the embodiment of the application can set a plurality of virtual positions to form a virtual position array of the photographing module. Each virtual position T443 corresponds to a three-dimensional model of the photographing module. For example, 35 array-arranged virtual positions T443 are set. In the three-dimensional model of the photographing module corresponding to each virtual position T443, the optical axis of the first camera intersects the origin of the left-eye center coordinate system T442. As can be set, the 35 virtual positions of the photographing module are sequentially the first photographing module virtual position to the 35th photographing module virtual position. The virtual position T443 is set based on the left-eye center coordinate system T442.

[0262] When looking at the photographing module along the LZ axis in the left-eye center coordinate system, the virtual position array is arranged in 5 rows and 7 columns. As can be set, the first (left end of the first row) of the first row is the first photographing module virtual position, and the last (right end of the first row) is the 7th photographing module virtual position. The last (right end of the last row) of the last row is set as the 35th photographing module virtual position. Obviously, the number and arrangement of the virtual positions T443 can be set based on requirements, and are not limited to the manner described in the embodiment of the application.

[0263] The projection of the line connecting any point in the left-eye center coordinate system T442 and the origin on the LY LZ plane and the line are set as an angle γ. If the coordinate value of the any point on the LX axis is positive, the angle γ is positive. If the coordinate value of the any point on the LX axis is negative, the angle γ is negative. The projection of the line on the LX LZ plane and the line are set as an angle δ. If the coordinate value of the any point on the LY axis is positive, the angle δ is positive. If the coordinate value of the any point on the LY axis is negative, the angle δ is negative.

[0264] The angle coordinates (γ, δ) of the optical axis of the first camera in the left eye center coordinate system in the 35 virtual positions are (45°, 30°), (30°, 30°), (15°, 30°), (0°, 30°), (-15°, 30°), (-30°, 30°), (-45°, 30°), (45°, 15°), (30°, 15°), (15°, 15°), (0°, 15°), (-15°, 15°), (-30°, 15°), (-45°, 15°), (45°, 0°), (30°, 0°), (15°, 0°), (0°, 0°), (-15°, 0°), (-30°, 0°), (-45°, 0°), (45°, -15°), (30°, -15°), (15°, -15°), (0°, -15°), (-15°, -15°), (-30°, -15°), (-45°, -15°), (45°, -30°), (30°, -30°), (15°, -30°), (0°, -30°), (-15°, -30°), (-30°, -30°), and (-45°, -30°), respectively, which correspond to the first to the 35th photographing modules, respectively. The angle coordinates facilitate data calculation and improve model accuracy.

[0265] As shown in Figure 45 Figure 45 A method for determining gaze direction models in different relative positions provided by the embodiments of the present application is shown in the schematic diagram, which comprises the following steps:

[0266] Step S61: Based on the standard model, the spatial position of the photographing module is adjusted by the adjusting assembly so that it is in the real space position corresponding to each virtual position. The face image and eye image corresponding to each real space position are collected by the photographing module.

[0267] Step S62: Based on the face image and eye image, the gaze direction model corresponding to different real space positions is determined.

[0268] ​In real space, keeping the user's head still, the spatial position of the camera module is adjusted so that its real-world position corresponds one-to-one with its virtual position. This can be achieved through the combined action of an information processor, electromagnetic drive device, and motion sensor, adjusting the camera module to change its relative position to the geometric positioning cap worn on the user's head. This ensures that the relative position of the geometric positioning cap and the camera module in real space matches the virtual position of the camera module's 3D model, which is based on the 3D model of the geometric positioning cap. Then, after capturing the first image of the left eye through the first camera, the first camera is rotated using its intersection point as the rotation point, so that its optical axis intersects at the center point of the left pupil.

[0269] In this embodiment, the eye image includes a visible light image captured by the color camera array and an infrared light image captured by the infrared camera component. The projected iris edge curve model is determined based on the visible light image. The iris ridge marker line model is determined based on the infrared light image.

[0270] In this embodiment, after adjusting the optical axis of the first camera to intersect with the user's pupil, the first camera is controlled to re-capture an eye image, obtaining a new first image. The second camera captures the infrared light reflected from the corneal region of the left eye by an infrared point light source, thus obtaining a second image.

[0271] Color cameras 1 through 16 simultaneously capture images of the eye area, obtaining 16 visible light images of the eye, denoted as the 1st to 16th visible light images. Based on these 16 visible light images, the first image, and the second image, a model of the visual axis, iris morphology marker lines, pupil center point, eye coordinate system, and projected iris edge curve group can be obtained in the corresponding gaze pointing model. The gaze pointing model also includes the 3D shape of the user's face acquired by a 3D scanner at the corresponding real-world spatial location.

[0272] In the real space, while keeping the user's head position still, the spatial position of the camera module is adjusted to correspond one-to-one with the virtual position in the standard model, thereby obtaining multiple gaze pointing models. In this way, 35 gaze pointing models are obtained corresponding to the actual spatial positions of 35 virtual positions.

[0273] As described above, a first image of the user's eye is captured using the first camera. At this time, the method for determining the iris pattern marker line model is as follows: Figure 46 As shown.

[0274] like Figure 46 As shown, Figure 46 A schematic diagram of a method for determining the iris pattern marker line model provided in this application embodiment, the method comprising:

[0275] Step S71: Identify the iris pattern in the first image based on the image brightness and color in the first image.

[0276] The information processor uses a preset image recognition algorithm to identify the iris pattern in the first image based on the image brightness and color.

[0277] Step S72: Mark the feature information in the iris pattern using marker points, such as... Figure 47 As shown, Figure 47 This is a schematic diagram illustrating the principle of marking iris textures, provided in an embodiment of this application.

[0278] Step S73: Based on the position of the marker point, determine the formation of the first image. The marker point corresponds to the relative positional relationship between the light ray and the first camera coordinate system when the light ray is incident on the first camera. Construct the iris pattern marker line model based on the light ray. The first camera coordinate system can be determined based on the module coordinate system.

[0279] Based on the method of calculating the angle of light rays from the point positions, the relative positional relationship between the light rays corresponding to the marked points in the first image and the coordinate system of the first camera is calculated. These light rays are the iris pattern marker lines. This allows the determination of the relative positional relationship between the iris pattern marker lines and the imaging module, such as... Figure 48 As shown.

[0280] like Figure 48 As shown, Figure 48 This is a schematic diagram illustrating the principle of determining the relative positional relationship between the iris pattern marker line and the camera module, provided in an embodiment of this application. In virtual space, the relative position of the iris pattern marker line and the camera module can be determined by the iris pattern marker line model T483 determined by the first camera model T481 and the first image T482 in the three-dimensional model of the camera module.

[0281] In this embodiment, when the user's eyes are directly looking at the center of the first camera lens, and the optical axis of the first camera intersects at the center point of the pupil, the second camera forms a second image based on the reflection of infrared light emitted by the infrared point light source in the pupil. The relative positional relationship between the pupil center point and the imaging module can be determined based on the second image. The pupil center point is the intersection of the user's visual axis and the outermost layer of the cornea.

[0282] like Figure 49 As shown, Figure 49A schematic diagram of the relative position of the eye visual axis and the pupil center point provided by the embodiments of the present application, the real visual axis T491 of the user's eye T493 is perpendicular or approximately perpendicular to the section T492 of the pupil center point on the eyeball. The user's eyes straight look at the first camera lens center, and the first camera optical axis intersects the pupil center point of the eye, so that the real visual axis coincides with the first camera optical axis.

[0283] The infrared light emitted by the infrared point light source and irradiated to the pupil center point of the eye is reflected by the pupil center point of the eye, and the perpendicular line intersecting the section of the pupil center point is the normal line, that is, the real visual axis and the optical axis of the first camera are the normal lines. Based on the reflection law: 1, the reflected light, the incident light and the normal line are in the same plane; 2, the reflected light and the incident light are on both sides of the normal line; 3, the reflection angle is equal to the incident angle. Therefore, the lens center of the second camera, the lens center of the first camera, the optical axis of the first camera and the infrared point light source are in the same plane; the lens center of the second camera and the infrared point light source are respectively on both sides of the optical axis of the first camera, that is, on both sides of the normal line; the line connecting the lens center of the second camera and the infrared point light source is perpendicular to the optical axis of the first camera, and the distance from the lens center of the second camera to the optical axis of the first camera is equal to the distance from the infrared point light source to the optical axis of the first camera. Therefore, the infrared light emitted by the infrared point light source and irradiated to the pupil center point of the eye intersects the lens center of the second camera according to the reflection law of light.

[0284] As shown in Figure 50 , Figure 50 A principle diagram for determining the relative position relationship between the pupil center point and the photographing module provided by the embodiments of the present application, according to the reflection light 504 of the second camera shooting the pupil center point of the eye 505 reflecting the infrared point light source 131b3, the second image 501 is obtained, then the angle relationship between the reflection light 504 of the infrared point light source reflected by the pupil center point and the second camera coordinate system is calculated according to the point position calculation forming light angle method, according to the angle relationship, the included angle A (that is, the incident angle) between the light line irradiated to the pupil center point (that is, the incident light 506) and the visual axis 503 (that is, the normal line) is calculated, and the distance S from the lens focal point of the first camera model to the pupil center point is calculated according to the incident angle A and the distance L from the infrared point light source 131b3 to the second camera 131b2. Figure 50 In the formula, 502 is the optical axis of the first camera 131b1. The distance S is calculated according to the following formula:

[0285]

[0286] Since the center point of the pupil is the intersection of the visual axis of the eye and the outermost layer of the cornea, the center point of the pupil must be on the visual axis. Because the visual axis is collinear with the optical axis of the first camera model, the center point of the pupil is on the optical axis of the first camera model. Furthermore, the distance S from the center point of the pupil to the focal point of the first camera is known. Therefore, the relative positional relationship between the center point of the pupil and the camera module can be determined.

[0287] An eye coordinate system is established within the eye model based on the visual axis, pupil center point, and iris texture reflection surface. Taking the left eye as an example, the three mutually perpendicular coordinate axes in the left eye's coordinate system are the AX axis, AY axis, and AZ axis. After determining the relative positions of the visual axis, iris texture marking lines, pupil center point, and the photographic module model in virtual space, the left eye's coordinate system is then established.

[0288] The direction along the visual axis from the center of the pupil to the focal point of the first camera model is the AZ axis direction of the left eye's coordinate system. The origin of the left eye's coordinate system is on the visual axis, and the distance from the center of the pupil is a predetermined distance. Taking the AZ axis direction as the forward direction, the origin of the left eye's coordinate system is behind the center of the pupil. The distance between the origin of the left eye's coordinate system and the center of the pupil is a fixed value, approximately equal to the distance from the intersection of the visual axis and the cornea along the visual axis direction in the human eyeball to the center of the iris. The AX axis of the left eye's coordinate system intersects the iris pattern marking line.

[0289] like Figure 51 As shown, Figure 51 This is a schematic diagram illustrating the principle of placing an eye coordinate system in an eye model according to an embodiment of this application. The eye coordinate system 515 of the left eye is used to determine the relative positional relationship between the eyeball model and the three-dimensional model 514 of the photography module in the model. Figure 51 The diagram shows the origin 511, AZ axis, AX axis, and AY axis of the eye coordinate system 515 for the left eye, as well as the visual axis 512 and the iris pattern marking line 513.

[0290] In this embodiment, the projected iris edge curve group model includes models of multiple projected iris edge curves; that is, the projected iris edge curve group is composed of multiple projected iris edge curves. Each projected iris edge curve is the intersection line between the iris edge reflection surface and a set plane in the eye coordinate system. The set plane is the plane containing the AX and AY axes in the eye coordinate system, and this plane is opposite to the imaging module.

[0291] like Figure 52 As shown, Figure 52 A schematic diagram of a method for determining the projected iris edge curve group model provided in this application embodiment, the method comprising:

[0292] Step S81: obtaining a plurality of iris edge reflection curves by a plurality of color cameras in the color camera array.

[0293] Step S82: intersecting the plurality of iris edge reflection curves with the set plane to obtain a plurality of one-to-one corresponding projection iris edge curves.

[0294] The iris edge reflection curve is a shape image of a curve formed by the feature area of the intersection line between the iris and the sclera reflecting light into the color camera when the color camera in the photographing module array captures the image of the user's eye.

[0295] As shown in Figure 53 , Figure 53 is a schematic diagram of an image captured by a color camera in an embodiment of the present application. In this example, the color camera array of the photographing module has a total of 16 color cameras, which are the 1st color camera to the 16th color camera. The user's eye is captured simultaneously to obtain 16 eye images, and one-to-one corresponding iris edge reflection curves are generated, which are the 1st iris edge reflection curve to the 16th iris edge reflection curve.

[0296] As shown in Figure 54 and Figure 55 , Figure 54 is a schematic diagram of the relative relationship between the eye coordinate system of the left eye and each iris edge reflection curve in an embodiment of the present application, Figure 55 is a schematic diagram of the relative relationship between the eye coordinate system of the left eye and the projection iris edge curve 551 in an embodiment of the present application. The 16 iris edge reflection curves 542 are intersected with the AxA Y plane 541 of the eye coordinate system of the left eye, thereby obtaining the corresponding 16 projection iris edge curves.

[0297] In an embodiment of the present application, the method of converting an eye picture captured by any color camera into an iris edge reflection curve and the method of obtaining an iris texture marker line are similar, both of which use the point position calculation method to form the light angle.

[0298] As shown in Figure 56 , Figure 56 is a schematic diagram of a gaze direction model provided by an embodiment of the present application. The visual axis 566, the iris texture marker line 564, the pupil center point 562, the eye coordinate system 561, the projection iris edge curve group 567, the face coordinate system 563, and the three-dimensional shape 568 of the face scanned by the three-dimensional scanner together form a gaze direction model. Figure 56 A three-dimensional model of the photographing module 565 is also shown.

[0299] It should be noted that, in an embodiment of the present application, only the left eye photographing is taken as an example for description, and the right eye gaze direction model principle is the same, which will not be described herein again.

[0300] The various embodiments described in this specification are presented by way of example, or used in conjunction with examples. Each embodiment can be implemented, or used in conjunction with, other embodiments, and each embodiment can be implemented independently of any other embodiment.

[0301] It should be noted that, in the description of the present application, it needs to be understood that the terms "upper", "lower", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. When a component is considered to be "connected" to another component, it can be directly connected to the other component or there can be a component disposed therebetween.

[0302] It should also be noted that, in this document, relational terms such as first and second, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by an indefinite article "a" or "an" does not exclude the existence of additional identical elements in the process, method, article, or apparatus.

[0303] The above description of disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A gaze pointing measuring device, characterized in that, The gaze pointing measurement device includes: A geometric positioning cap, which is worn on a user's head; An adjustment component and an image acquisition component mounted on the adjustment component; the image acquisition component includes at least a camera module. When a user wears the geometric positioning cap, the image acquisition component is used to acquire images of the geometric positioning cap, the user's face, and the user's eyes; the adjustment component is used to adjust the spatial position of the camera module; the geometric positioning cap image, the face image, and the eye image are used to determine the gaze pointing model; The camera module includes: a camera bracket; an infrared camera assembly mounted on the camera bracket, the infrared camera assembly including: a first camera, a second camera, and an infrared point light source; the first camera and the second camera are used to capture infrared light images of the eye image; a color camera array mounted on the camera bracket, the color camera array including multiple color cameras located around the infrared camera assembly; the color cameras are used to capture visible light images of the geometric positioning cap image, the face image, and the eye image; and at least one infrared illumination source, the infrared illumination source being mounted on the camera bracket and / or the infrared camera assembly. The computer has an information processor for performing image data analysis on the geometric positioning cap image, the facial image, and the eye image. Based on the image data analysis results, it determines the relative position of the user's face and the camera module. The computer then adjusts the spatial position of the camera module using an adjustment component to adjust the relative position, acquiring the geometric positioning cap image, the facial image, and the eye image corresponding to different relative positions to determine the gaze pointing model for each relative position. The gaze pointing model includes a visual axis model, a pupil center point model, an iris pattern marker line model, an eye coordinate system, and a projected iris edge curve group model corresponding to the relative position. The method for determining the iris pattern marking line model by acquiring a first image of the user's eye through the first camera includes: identifying iris patterns in the first image based on the image brightness and color; marking feature information in the iris patterns with marking points; determining the formation of the first image based on the position of the marking points; calculating the relative positional relationship between the light rays corresponding to the marking points and the coordinate system of the first camera when they are incident on the first camera; and constructing the iris pattern marking line model based on the light rays. The projected iris edge curve group model is a model including multiple projected iris edge curves. The projected iris edge curve is the intersection line between the iris edge reflection surface and a set plane in the eye coordinate system. The set plane is a preset plane in the eye coordinate system, which is opposite to the camera module. The method for determining the projected iris edge curve group model includes: acquiring multiple iris edge reflection surfaces through multiple color cameras in the color camera array; intersecting the multiple iris edge reflection surfaces with the set plane to obtain multiple one-to-one corresponding projected iris edge curves.

2. The gaze pointing measuring device according to claim 1, characterized in that, The adjustment component includes: a frame and an adjustment module disposed on the frame; The camera module is mounted on the adjustment module, which is used to adjust the spatial position of the camera module.

3. The gaze pointing measuring device according to claim 2, characterized in that, The adjustment module includes N motion guide components, which are sequentially named from the 1st motion guide component to the Nth motion guide component, where N is a positive integer greater than 2. The first motion guide component is fixed on the frame, and the (i+1)th motion guide component is movably mounted on the ith motion guide component, where i is a positive integer not greater than N-1. The camera module is mounted on the Nth motion guide component.

4. The gaze pointing measuring device according to claim 3, characterized in that, The (i+1)th motion guide component can translate or rotate relative to the i-th motion guide device; When the (i+1)th motion guide component can translate relative to the ith motion guide component, the two motion guide components are controlled to translate through a first electromagnetic drive device. One of the two motion guide components is fixed with a magnet, and the other motion guide component is fixed with a coil. The first electromagnetic drive device includes the magnet and the coil. When the coil is charged, the first electromagnetic drive device is used to make the two motion guide components move relative to each other based on the electromagnetic force between the magnet and the coil. When the (i+1)th motion guide component is able to rotate relative to the ith motion guide component, the rotation between the two motion guide components is controlled by a second electromagnetic drive device, which includes a motor.

5. The gaze pointing measuring device according to claim 3, characterized in that, The adjustment module further includes a position sensor, which is used to detect the relative position between motion guide components.

6. The gaze pointing measuring device according to claim 2, characterized in that, The image acquisition component also includes at least one 3D scanner; the 3D scanner is used to scan the 3D shape of the user's face using infrared laser. The 3D scanner is mounted on the skeleton or on the camera module.

7. The gaze pointing measuring device according to claim 1, characterized in that, The infrared illumination source is used to emit a first infrared detection light; the first camera forms a first image based on the first infrared detection light reflected from the user's eyeball, the first image including infrared iris information and infrared pupil information of the user's eyeball; based on the infrared pupil information in the first image, the optical axis of the first camera is adjusted to intersect with the center point of the pupil of the eye; The infrared point light source is used to emit a second infrared detection beam; The second camera is used to form a second image based on the second infrared detection light reflected by the user's cornea, the second image including the highlight points of the second infrared detection light reflected by the user's cornea; The infrared image of the eye image includes the first image and the second image.

8. The gaze pointing measuring device according to claim 1, characterized in that, The second camera and the infrared point light source are positioned on opposite sides of the optical axis of the first camera; the line connecting the focal point of the second camera lens and the infrared point light source intersects the optical axis of the first camera, and the line is perpendicular to the optical axis of the first camera. The distance between the infrared point light source and the optical axis of the first camera is equal to the distance between the focal point of the second camera and the optical axis of the first camera.

9. The gaze pointing measuring device according to claim 1, characterized in that, The color camera is capable of receiving visible light to capture a third image, which includes the boundary features of the user's iris and sclera. The visible light image of the eye image includes the third image.

10. The gaze pointing measuring device according to claim 1, characterized in that, The geometric positioning cap includes: Wearable components for wearing on a user's head; Geometric shape marking component disposed on the outer side of the wearable component; Specifically, the relative position of the user's face and the camera module is determined based on the image of the geometric shape identifier component in the geometric positioning cap image.

11. The gaze pointing measuring device according to claim 10, characterized in that, The geometric shape marking component includes multiple visible point light sources with different emitting colors, and the visible point light sources are arranged around the outer perimeter of the wearable component; Based on at least two geometric positioning cap images captured by the camera module, the relative position of the user's face and the camera module is determined; each geometric positioning cap image includes multiple images of the same geometric shape identifier component.

12. A method for establishing a gaze-direction model, characterized in that, Based on the gaze pointing measurement device of claim 1, the method includes: When the user wears the geometric positioning cap, the image acquisition component acquires images of the geometric positioning cap, the user's face, and the user's eyes. The method for acquiring the geometric positioning cap image, the user's facial image, and the user's eye image includes: adjusting the spatial position of the camera module using the adjustment component so that the user's visual axis coincides with the optical axis of the first camera; after satisfying the coincidence condition, changing the relative position of the camera module and the user's face to acquire first measurement data under different relative positions, the first measurement data including the facial image and eye image corresponding to the relative position; when acquiring images under different relative positions, the optical axis of the first camera intersects at the center point of the user's pupil; Image data analysis is performed on the geometric positioning cap image, the facial image, and the eye image. Based on the image data analysis results, the relative position of the user's face and the camera module is determined. The spatial position of the camera module is adjusted by the adjustment component to adjust the relative position, and the geometric positioning cap image, the facial image and the eye image corresponding to different relative positions are obtained to determine the gaze pointing model under different relative positions.

13. The method for establishing a gaze-direction model according to claim 12, characterized in that, Methods for ensuring that the user's line of sight coincides with the optical axis of the first camera include: A first image of the user's eye is captured by a first camera, and the first image includes infrared pupil information. Based on the first image, the spatial position of the camera module is adjusted by the adjustment component to move the first camera so that the optical axis of the first camera coincides with the user's line of sight.

14. The method for establishing a gaze-direction model according to claim 12, characterized in that, The method for determining the relative position of the user's face and the camera module includes: Based on the first measurement data, the motion pattern of the user's eyes relative to a facial reference is determined, the facial reference including the position of the geometric positioning cap and a three-dimensional facial model; Based on the motion patterns and the facial reference, a standard model is constructed. The standard model includes multiple preset virtual positions of the camera module, the user's facial coordinate system and eye center coordinate system, and each virtual position corresponds to a three-dimensional model of the camera module. In the facial coordinate system, the three mutually perpendicular coordinate axes are the FX axis, FY axis, and FZ axis; the FZ axis faces the virtual position, and the FX and FY planes are set relative to the virtual position.

15. The method for establishing a gaze-direction model according to claim 14, characterized in that, The method for determining the motion pattern of the user's eyes relative to a facial reference includes: Based on multiple sets of the first measurement data, a number of pointing models are determined in a one-to-one correspondence; the pointing model includes the visual axis model, the facial coordinate system, and the relative relationship between the two. All the pointing models are overlapped with the facial coordinate system as a reference to obtain visual axis models of the eyes looking in different directions with the user's face as a reference. The intersection of all visual axis models is the center model of the user's eyes; wherein, the center model of the user's eyes can determine the eye center coordinate system.

16. The method for establishing a gaze-direction model according to claim 15, characterized in that, The first measurement data includes the facial image captured by the color camera array and the geometric positioning cap image corresponding to the facial image; The method for determining the pointing model includes: Based on the geometric positioning cap image corresponding to the facial image, the relative positional relationship between the user's face and the camera module is determined; Based on the relative positional relationship between the user's face and the camera module, a visual axis model and a facial coordinate system are constructed, and the visual axis model, the facial coordinate system, and their relative relationship are combined to form the pointing model.

17. The method for establishing a gaze-direction model according to claim 14, characterized in that, Establish a left-eye center coordinate system with three mutually perpendicular coordinate axes: LX axis, LY axis, and LZ axis. The center of the user's left eye is the origin of the left-eye center coordinate system. The LZ axis is parallel to the FZ axis, and the LY axis is parallel to the FY axis. Methods for constructing standard models include: The virtual positions are set up in an array of 35 arrays; in the three-dimensional model of the camera module corresponding to each virtual position, the optical axis of the first camera intersects at the origin of the left eye center coordinate system; Let γ be the angle between the projection of the line connecting any point in the left eye center coordinate system to the origin on the LY-LZ plane and the line connecting the origin, and let δ be the angle between the projection of the line connecting the origin on the LX-LZ plane and the line connecting the origin. Then, the angular coordinates (γ, δ) of the optical axis of the first camera in the left eye center coordinate system in the 35 virtual positions are (45°, 30°), (30°, 30°), (15°, 30°), (0°, 30°), (-15°, 30°), (-30°, 30°), (-45°, 30°), (45°, 15°), (30°, 15°), (15°, 15°), (0°, 15°), (-15°, 15°), (-30°, 15°), (-45°, 15°), (45°, 0°). (30°, 0°), (15°, 0°), (0°, 0°), (-15°, 0°), (-30°, 0°), (-45°, 0°), (45°, -15°), (30°, -15°), (15°, -15°), (0°, -15°), (-15°, -15°), (-30°, -15°), (-45°, -15°), (45°, -30°), (30°, -30°), (15°, -30°), (0°, -30°), (-15°, -30°), (-30°, -30°), (-45°, -30°).

18. The method for establishing a gaze-direction model according to claim 14, characterized in that, Methods for determining the gaze pointing model under different relative positions include: Based on the standard model, the spatial position of the camera module is adjusted by the adjustment component so that it is in the real space position corresponding to each of the virtual positions. The camera module then captures facial and eye images corresponding to each of the real space positions. Based on the facial image and the eye image, a gaze pointing model corresponding to different real-world spatial locations is determined.

19. The method for establishing a gaze-direction model according to claim 18, characterized in that, The eye image includes a visible light image captured by the color camera array and an infrared light image captured by the infrared camera component; The model of the projected iris edge curve group is determined based on the visible light image; The iris pattern marking line model is determined based on the infrared image.

20. The method for establishing a gaze-direction model according to claim 19, characterized in that, The defined plane also includes the plane containing the AX and AY axes in the eye coordinate system.

Images