An eye tracking system

Abstract

The application discloses an eye tracking system, which comprises a first light source, a reflection module and an angle adjusting module. The reflection module is located on the propagation path of the first light beam and is used for reflecting the first light beam emitted by the first light source. The angle adjusting module comprises at least one adjusting unit, a carrier body comprises at least one contact unit, the adjusting unit is connected with the contact unit one by one, and the contact unit is driven to move so as to adjust the angle of the reflection surface, so that the reflection surface reflects the first light beam to a target illumination area. The eye tracking system provided by the embodiment of the application adjusts the angle of the reflection surface on the carrier body through the angle adjusting module, reflects the first light beam to the target illumination area, realizes tracking of the first light beam on the position of the eyes or the face of a user, and only needs to cover the position area of the eyes or the face of the user in the target illumination area, without the need of illuminating and covering the whole head box range, so that the utilization rate of the light source energy is improved.

Description

Technical Field

[0001] This application relates to the field of eye-tracking technology, and more particularly to an eye-tracking system. Background Technology

[0002] Current telemetry eye-tracking technology requires using infrared light-emitting diodes (LEDs) to illuminate the user's face, then using a camera to capture the image, and finally using algorithms to analyze the light spots reflected from the eyes in the image.

[0003] For telemetry eye-tracking technology, a large head movement space is required, i.e., a large "headbox" range. The headbox range can be understood as the area within which the telemetry eye-tracking device can capture eye movement and perform eye-tracking calculations. To illuminate a sufficiently large "headbox" range while ensuring high-brightness illumination throughout, related technologies typically use a single high-power LED as the light source, or a combination of multiple lower-power LEDs.

[0004] Specifically, to ensure that the light beam can evenly illuminate the head-mounted display area, a light mask is usually used to adjust the light path, thereby ensuring the uniformity and brightness of light illumination within the head-mounted display area. However, since the position of the light mask is usually fixed, the illumination range of the light beam is also fixed and cannot be flexibly adjusted according to the user's eye or facial position. Therefore, to prevent areas within the head-mounted display area from being uncovered by light, a single high-power LED or a combination of multiple lower-power LEDs is usually used to illuminate simultaneously to provide sufficient beam coverage.

[0005] However, since the area occupied by the user's eyeballs and face is relatively small compared to the entire head box area, most of the light beam incident on the head box area fails to effectively illuminate the user's eyeballs and face area, thus increasing the waste of light source energy, resulting in low light energy utilization and high power consumption of the light source. Summary of the Invention

[0006] This application provides an eye-tracking system to improve the utilization rate of light energy and reduce light energy loss.

[0007] This application provides an eye-tracking system, including a first light source, a reflection module, an angle adjustment module, and at least one processor;

[0008] The first light source is used to emit a first beam of light;

[0009] The reflection module is located on the propagation path of the first beam. The reflection module includes a carrier and a reflecting surface located on one side surface of the carrier. The reflecting surface is used to reflect the first beam.

[0010] The angle adjustment module includes at least one adjustment unit;

[0011] The carrier includes at least one contact unit located on the side surface of the carrier opposite to the reflective surface;

[0012] The adjustment unit is connected to the contact unit in a one-to-one correspondence. The adjustment unit is used to drive the contact unit to move in order to adjust the angle of the reflective surface so that the reflective surface reflects the first light beam to the target illumination area.

[0013] The at least one processor is configured to acquire facial and / or eye data, wherein the facial and / or eye data is used to determine adjustment information; and send the adjustment information to the angle adjustment module to control the movement of the adjustment unit.

[0014] Optionally, at least one of the adjustment units includes a first adjustment unit and a second adjustment unit;

[0015] At least one of the contact units includes a first contact unit and a second contact unit;

[0016] The first adjustment unit is connected to the first contact unit, and the second adjustment unit is connected to the second contact unit;

[0017] The distance between the vertical projection of the first contact unit on the reflective surface and the center of the reflective surface is greater than 0;

[0018] The distance between the vertical projection of the second contact unit on the reflective surface and the center of the reflective surface is greater than 0.

[0019] Optionally, the first adjustment unit is used to drive the first contact unit to move along a first direction;

[0020] The second adjustment unit is used to drive the second contact unit to move along the second direction;

[0021] The first direction and the second direction are parallel or intersecting.

[0022] Optionally, at least one of the adjustment units further includes a third adjustment unit, and at least one of the contact units further includes a third contact unit, wherein the third adjustment unit is connected to the third contact unit;

[0023] The lines connecting the first contact unit, the second contact unit, and the third contact unit form a triangle.

[0024] Optionally, the center of the reflective surface is located within the region bounded by the vertical projection of the triangle onto the reflective surface.

[0025] Optionally, the first adjustment unit is used to drive the first contact unit to move along a first direction;

[0026] The second adjustment unit is used to drive the second contact unit to move along the second direction;

[0027] The third adjustment unit is used to drive the third contact unit to move along a third direction;

[0028] The first direction, the second direction, and the third direction are parallel to each other.

[0029] Optionally, the first direction, the second direction, and the third direction are all parallel to the direction of the carrier pointing to the reflective surface.

[0030] Optionally, the first adjustment unit is used to drive the first contact unit to move along a first direction;

[0031] The second adjustment unit is used to drive the second contact unit to move along the second direction;

[0032] The third adjustment unit is used to drive the third contact unit to move along a third direction;

[0033] The first direction, the second direction, and the third direction are all different from each other.

[0034] Optionally, the angle between the first direction and the direction in which the carrier points to the reflective surface is the first angle;

[0035] The angle between the second direction and the direction in which the carrier points to the reflective surface is the second included angle;

[0036] The angle between the third direction and the direction in which the carrier points to the reflective surface is the third angle;

[0037] The first included angle, the second included angle, and the third included angle are equal.

[0038] Optionally, the eye-tracking system further includes a light source adjustment module;

[0039] The light source adjustment module is connected to the first light source and is used to drive the first light source to move along the propagation direction of the first beam.

[0040] Optionally, the eye-tracking system further includes a second light source;

[0041] The second light source is used to emit a second beam of light, and the area of ​​the illumination region of the second beam of light is larger than the area of ​​the target illumination region.

[0042] Optionally, the reflective surface may include a curved surface.

[0043] Optionally, the adjustment unit includes a drive structure and a universal ball joint;

[0044] The drive structure is movably connected to the contact unit via the universal ball joint, and is used to drive the contact unit to move.

[0045] Optionally, the drive structure includes a motor and a lead screw assembly;

[0046] The motor is connected to the universal ball joint via the lead screw assembly.

[0047] Optionally, the angle adjustment module includes at least one support unit;

[0048] The support unit is connected to the contact unit, and the support unit is used to support the contact unit.

[0049] The eye-tracking system provided in this application adjusts the angle of the reflective surface on the carrier through an angle adjustment module to modulate the propagation direction of the first light beam reflected by the reflective surface. This first light beam is then reflected to the target illumination area, enabling the first light beam to track the user's eye or face position and adapt to changes in the user's position. The target illumination area only needs to cover the user's eye or face area to meet the illumination requirements of eye tracking. This allows the first light beam to be concentrated on the area of ​​the user's eye or face that needs to be tracked, eliminating the need to illuminate the entire head-mounted device. This avoids ineffective illumination of other areas within the head-mounted device, reduces the illumination range of the light source, and ensures that the light source illuminates the user's eyes, thereby reducing light energy loss that does not contribute to eye tracking and improving the utilization rate of the light source energy.

[0050] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of the structure of an eye-tracking system provided in an embodiment of this application;

[0053] Figure 2 This is a schematic diagram of the structure of a first light source provided in an embodiment of this application;

[0054] Figure 3 This is a schematic diagram of another first light source provided in an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of the structure of a reflective surface provided in an embodiment of this application;

[0056] Figure 5 This is a schematic diagram of another eye-tracking system provided in an embodiment of this application;

[0057] Figure 6 This is a schematic diagram of the structure of a reflection module provided in an embodiment of this application;

[0058] Figure 7 This is a schematic diagram of another reflection module provided in an embodiment of this application;

[0059] Figure 8 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0060] Figure 9 This is a schematic diagram of another reflection module provided in an embodiment of this application;

[0061] Figure 10 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0062] Figure 11 This is a schematic diagram of the structure of another reflection module provided in an embodiment of this application;

[0063] Figure 12 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0064] Figure 13 This is a schematic diagram of the structure of another reflection module provided in an embodiment of this application;

[0065] Figure 14 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0066] Figure 15 This is a schematic diagram of the structure of another reflection module provided in an embodiment of this application;

[0067] Figure 16 This is a schematic diagram of the structure of another reflection module provided in an embodiment of this application;

[0068] Figure 17 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0069] Figure 18A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0070] Figure 19 This is a schematic diagram of the structure of an adjustment unit provided in an embodiment of this application;

[0071] Figure 20 This is a schematic diagram of another adjustment unit provided in an embodiment of this application;

[0072] Figure 21 This is a schematic diagram of the structure of another adjustment unit provided in an embodiment of this application;

[0073] Figure 22 This is a schematic diagram of another adjustment unit provided in an embodiment of this application;

[0074] Figure 23 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0075] Figure 24 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0076] Figure 25 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0077] Figure 26 A schematic diagram of the structure of another eye-tracking system provided in the embodiments of this application;

[0078] Figure 27 This is a flowchart illustrating a control method for an eye-tracking system provided in an embodiment of this application. Detailed Implementation

[0079] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0080] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0081] Figure 1 This is a schematic diagram of the structure of an eye-tracking system provided in an embodiment of this application, as shown below. Figure 1 As shown, the eye-tracking system provided in this embodiment includes a first light source 10, a reflection module 30, and an angle adjustment module 40. The first light source 10 emits a first light beam 01. The reflection module 30 is located in the propagation path of the first light beam 01. The reflection module 30 includes a carrier 31 and a reflective surface 32 located on one side surface of the carrier 31. The reflective surface 32 reflects the first light beam 01. The angle adjustment module 40 includes at least one adjustment unit 41. The carrier 31 includes at least one contact unit 310 located on the side surface of the carrier 31 facing away from the reflective surface 32. The adjustment unit 41 is connected to the contact unit 310 in a one-to-one correspondence. The adjustment unit 41 drives the contact unit 310 to move, thereby adjusting the angle of the reflective surface 32 so that the reflective surface 32 reflects the first light beam 01 to the target illumination area P0.

[0082] Specifically, such as Figure 1 As shown, the first light source 10 is used to emit a first beam 01 to the reflection module 30.

[0083] The first light source 10 may include a light-emitting body, which may include infrared light excitation devices such as infrared light-emitting diodes (LEDs). The wavelength of the light beam emitted by the infrared light-emitting diode is usually between 700nm and 1mm, which is suitable for non-contact detection. It also has the characteristics of small size, low power consumption and long life, and is suitable for equipment that works for a long time.

[0084] In other embodiments, within human safety limits, the light emitter can also use devices such as infrared lasers, which typically have higher light intensity and narrower beam width, making them suitable for applications requiring higher precision in illuminating the area.

[0085] Furthermore, in addition to the light-emitting body, the first light source 10 may also include an optical adjustment system. The function of the optical adjustment system is to use optical elements such as lenses, mirrors, prisms, and gratings to adjust the shape, energy distribution, wavefront, or phase of the light beam emitted by the light-emitting body through optical phenomena such as refraction, reflection, interference, and diffraction.

[0086] In an optional embodiment, Figure 2 This is a schematic diagram of the structure of a first light source provided in an embodiment of this application, as shown below. Figure 2 As shown, the first light source 10 includes a light emitter 11 and a beam expander 12. The beam expander 12 serves as an optical adjustment system to diffuse the light beam emitted from the light emitter 11, thereby expanding the beam. This allows the first light beam 01 emitted from the first light source 10 to cover a larger area, which is beneficial for achieving uniform illumination within the entire target illumination area P0.

[0087] In another alternative embodiment, Figure 3 A schematic diagram of another first light source provided in the embodiments of this application is shown below. Figure 3 As shown, the first light source 10 includes a light emitter 11, a beam expander 12, a first collimating lens 13, and a second collimating lens 14. The beam expander 12, the first collimating lens 13, and the second collimating lens 14 form an optical adjustment system. The beam expander 12 is used to diffuse the light beam emitted from the light emitter 11 to achieve beam expansion. The first collimating lens 13 and the second collimating lens 14 work together to collimate the expanded light beam, ensuring that the first light beam 01 has good collimation before reaching the reflection module 30. This avoids the problem of local over-brightness or under-brightness, improves the image quality captured by the image sensor, and at the same time, through the diffusion effect of the beam expander 12 and the collimation processing of the collimating lens, the waste of the light beam can be reduced, the utilization rate of the light source can be improved, and the overall power consumption can be reduced.

[0088] It should be noted that, Figure 3 This description uses only an example of an optical adjustment system including a lens, but it is not limited to this. In other embodiments, the optical adjustment system may also include optical elements such as mirrors, prisms, and gratings. For example, the optical adjustment system may also include a mirror, which can flexibly change the propagation direction of the light beam emitted by the light source to ensure that the light beam can accurately illuminate the reflection module 30. The flexible adjustment of the mirror helps to optimize the layout of the optical elements inside the first light source 10, thereby reducing the space occupied by the first light source 10. This application does not specifically limit this aspect.

[0089] It should be noted that in related technologies, the light source directly illuminates the user's eyes, and an image sensor captures images of the illuminated eye and surrounding area for eye-tracking algorithm analysis. However, direct illumination can easily lead to uneven brightness distribution within the illuminated area, resulting in some areas being too bright and others too dark. This makes it difficult to obtain clear images of the user's eyes, thus affecting the accuracy and stability of eye tracking. Furthermore, direct illumination makes it difficult to precisely control the distribution and shape of the light beam, failing to meet specific lighting requirements.

[0090] Based on the above-mentioned technical problems, in this embodiment, as follows: Figure 1 As shown, the carrier 31 of the reflection module 30 supports and carries the reflective surface 32, which is used to reflect the first beam 01 emitted by the first light source 10 to change the propagation direction of the first beam 01.

[0091] By designing the shape of the reflective surface 32, the light beam reflected by the reflective surface 32 can be shaped so that the light beam reflected by the reflective surface 32 can meet preset requirements. These preset requirements may include a suitable illumination range of the reflected light beam pointing towards the user's eyes, uniform brightness distribution within the illumination range, appropriate brightness level within the illumination range (e.g., achieving high brightness), or other lighting requirements, so as to obtain a clear image of the user's eyes for eye tracking calculation, thereby improving the accuracy and stability of eye tracking.

[0092] In some embodiments, the reflective surface 32 can be planar, which simplifies the manufacturing process and helps reduce costs.

[0093] In other embodiments, the reflective surface 32 may also be curved to provide more design freedom and greater optimization capabilities.

[0094] The surface can be concave or convex, and its shape can be designed as a quadratic surface, spherical surface, aspherical surface, freeform surface, or free-form surface, etc., to achieve precise control of the reflected beam.

[0095] Figure 4 This is a schematic diagram of the structure of a reflective surface provided in an embodiment of this application, such as... Figure 4 As shown, optionally, the reflecting surface 32 is a freeform surface. Compared with a freely constructed surface, the variables of a freeform surface follow a certain expression, which makes it easier for design software or machining software to generate, render and optimize the freeform surface. At the same time, compared with the expressions of quadratic surfaces, spheres and aspherical surfaces, the expression of a freeform surface contains more sub-terms and coefficients. These sub-terms and coefficients can be flexibly adjusted to provide more variables and higher design freedom, thereby enabling better setting of the shape of the freeform surface.

[0096] Furthermore, freeform surfaces can be of various types, such as extended polynomials, Zernike polynomials, or Q-type freeform surfaces.

[0097] like Figure 4 As shown, taking the representation of a freeform surface using an extended polynomial as an example, the expression of the extended polynomial of a freeform surface can include:

[0098]

[0099] Where N is the total number of polynomial coefficients in the series, and A i Let be the coefficients of the i-th extended polynomial, which is a power series in the x and y directions, wherein there are two 1-degree terms, including x and y; and three 2-degree terms, including x. 2 xy and y 2 The cubic term has four terms, including x. 3 x 2 y, xy 2 and y 3 And so on, with the highest degree being 20, the maximum total number of polynomial aspherical coefficients is 230.

[0100] Furthermore, in the extended polynomial above, r is the radial distance from the aspherical axis, and z is the sag, which can be understood as the height of a point on the surface along the normal direction. In the field of optics, the aspherical sag can be represented by Z(x, y), where (x, y) are the coordinates of a point on the surface, c = 1 / R is the vertex curvature, R is the vertex radius of curvature, and k represents the conic coefficient.

[0101] Furthermore, to ensure the dimensionless nature of the polynomial coefficients, x and y can be normalized by dividing the data values ​​at positions such as x and y by the normalization radius to obtain dimensionless polynomial coefficients.

[0102] In this embodiment, the surface shape and parameters of the reflective surface 32 can be designed according to the preset illumination range, illumination brightness, and the position where the beam needs to be projected with high quality. Ultimately, the first beam 01 emitted by the first light source 10 can accurately illuminate the preset illumination range after being reflected by the reflective surface 32, and maintain better brightness, illuminance distribution, size and shape within the illumination range.

[0103] It should be noted that the carrier 31 can be integrally formed with the reflective surface 32 to reduce the number of parts, simplify the assembly process, make the structure of the reflective module 30 more compact, and improve the stability and reliability of the reflective module 30.

[0104] In some embodiments, the carrier 31 can be formed by injection molding, and then on one side surface of the carrier 31 (e.g. Figure 1The reflective surface 32 is located on the right side surface of the carrier 31. The reflective film is attached by processes such as physical vapor deposition or chemical vapor deposition, or the reflective film is directly attached to one side surface of the carrier 31 to form the reflective surface 32.

[0105] In other embodiments, a carrier 31 can be formed by processing a metal material (such as aluminum, magnesium and alloy materials) using a computer numerical control (CNC) machine tool, and then a bright reflective surface 32 can be formed by processing one side of the carrier 31 through processes such as polishing.

[0106] In addition, the carrier 31 can also be designed separately from the reflective surface 32, which makes it convenient for the carrier 31 and the reflective surface 32 to be replaced and repaired independently, and different shapes of reflective surfaces 32 can be selected according to actual needs, so as to adapt to different application requirements.

[0107] The reflective surface 32 can be fixed to the carrier 31 with glue or screws. This application embodiment does not specifically limit this.

[0108] In addition, the reflective surface 32 can be set as a fully reflective surface to reflect nearly 100% of infrared light, thereby reducing energy loss of the beam during the reflection process and improving light utilization.

[0109] Continue to refer to Figure 1 A contact unit 310 is provided on the side surface of the carrier 31 that is away from the reflective surface 32. The contact unit 310 refers to the contact part located on the carrier 31.

[0110] The angle adjustment module 40 includes at least one adjustment unit 41, which is connected to the contact unit 310 in a one-to-one correspondence. This allows each adjustment unit 41 to independently drive the contact unit 310 connected to it to move, thereby adjusting the position of each contact unit 310 on the carrier 31 through the adjustment unit 41. This precisely adjusts the tilt angle of the reflective surface 32 on the carrier 31, thereby modulating the propagation direction of the reflected light beam by the reflective surface 32, so that the reflective surface 32 can reflect the first light beam O1 to the target illumination area P0.

[0111] The target illumination area P0 is a preset illumination range, which is a specific area that needs to be illuminated. This area can be the area where the user's eyes or face are located, to ensure that the user's eyes can be fully and evenly illuminated, so that subsequent image acquisition and processing can proceed smoothly.

[0112] Based on the above technical solution, in this embodiment, the angle of the reflective surface 32 on the carrier 31 is adjusted by the angle adjustment module 40 to modulate the propagation direction of the first beam 01 reflected by the reflective surface 32, and the first beam 01 is reflected to the target illumination area P0, thereby enabling the first beam 01 to track the user's eye or face position to adapt to changes in the user's position. At this time, the target illumination area P0 only needs to cover the user's eye or face position area to meet the illumination requirements of eye tracking. Therefore, the first beam 01 can be concentrated on the user's eye or face position area that needs to be tracked, without needing to cover the entire head box area with light, thereby avoiding ineffective illumination of other areas within the head box area, reducing light energy loss that does not contribute to eye tracking, and thus improving the utilization rate of light source energy.

[0113] In this way, due to the improved utilization rate of the light source, the first light source 10 can use a low-power light emitter. For example, a single low-power LED can meet the lighting requirements of eye tracking, thereby reducing the power consumption of the eye tracking system.

[0114] In this embodiment, the target lighting area P0 includes a circular area with a diameter of 15cm or a rectangular area of ​​15cm*7cm, but is not limited to this. In other embodiments, the size and shape of the target lighting area P0 can be set according to actual needs, and this application embodiment does not specifically limit this.

[0115] In addition, the contact unit 310 may include, but is not limited to, contact points, contact shafts, and contact surfaces.

[0116] Furthermore, the contact point refers to a point-like contact area. At this time, a point contact can be formed between the adjustment unit 41 and the carrier 31. By changing the position of the contact point through the angle adjustment module 40, it is helpful to fine-tune the angle of the reflective surface 32 and ensure the accurate reflection of the first beam 01.

[0117] The contact shaft refers to the shaft-shaped contact part. At this time, the adjustment unit 41 and the carrier 31 can be in line contact with each other in a tangential manner. By changing the position of the contact shaft through the angle adjustment module 40, it helps to provide multi-dimensional adjustment capabilities and can be applied to occasions that require complex adjustments.

[0118] The contact surface refers to the surface-shaped contact area. At this time, the adjustment unit 41 and the carrier 31 can have a curved tangent surface contact. The contact surface can achieve a large area of ​​uniform contact with the adjustment unit 41, which helps to provide uniform support force and ensure the stability of the reflective surface 32.

[0119] Different contact units 310 may adopt different contact types. For example, multiple contact units 310 may include at least two of the following: contact points, contact shafts, and contact surfaces. This application embodiment does not specifically limit this.

[0120] It should be noted that the adjustment unit 41 can drive the contact unit 310 to move linearly. In some embodiments, the adjustment unit 41 can also drive the contact unit 310 to move in a curve, as long as the reflection angle of the reflective surface 32 to the first beam 01 can be changed. This application does not specifically limit this.

[0121] The eye-tracking system provided in this application adjusts the angle of the reflective surface on the carrier through an angle adjustment module to modulate the propagation direction of the first light beam reflected by the reflective surface. This first light beam is then reflected to the target illumination area, enabling the first light beam to track the user's eye or face position and adapt to changes in the user's position. The target illumination area only needs to cover the user's eye or face area to meet the illumination requirements of eye tracking. This allows the first light beam to be concentrated on the area of ​​the user's eye or face that needs to be tracked, eliminating the need to illuminate the entire head-mounted device. This avoids ineffective illumination of other areas within the head-mounted device, reduces light energy loss that does not contribute to eye tracking, and thus improves the utilization rate of the light source energy.

[0122] Figure 5 This is a schematic diagram of another eye-tracking system provided in an embodiment of this application. Figure 6 This is a schematic diagram of the structure of a reflection module provided in an embodiment of this application, with reference to... Figure 5 and Figure 6 At least one adjustment unit 41 includes a first adjustment unit 411 and a second adjustment unit 412, and at least one contact unit 310 includes a first contact unit 311 and a second contact unit 312. The first adjustment unit 411 is connected to the first contact unit 311, and the second adjustment unit 412 is connected to the second contact unit 312. The distance d1 between the vertical projection of the first contact unit 311 on the reflective surface 32 and the center O of the reflective surface 32 is greater than 0, and the distance d2 between the vertical projection of the second contact unit 312 on the reflective surface 32 and the center of the reflective surface 32 is greater than 0.

[0123] Specifically, such as Figure 5 and Figure 6 As shown, two contact units 310 can be provided on the carrier 31, namely the first contact unit 311 and the second contact unit 312.

[0124] Correspondingly, the angle adjustment module 40 consists of two adjustment units 41, which are a first adjustment unit 411 connected to the first contact unit 311 and a second adjustment unit 412 connected to the second contact unit 312.

[0125] Wherein, the distance d1 between the vertical projection of the first contact unit 311 on the reflective surface 32 and the center O of the reflective surface 32 is greater than 0, and the distance d2 between the vertical projection of the second contact unit 312 on the reflective surface 32 and the center of the reflective surface 32 is greater than 0, so that the first adjustment unit 411 and the second adjustment unit 412 apply driving force to the carrier 31 at different positions away from the center O of the reflective surface 32 (i.e., the positions where the first contact unit 311 and the second contact unit 312 are located), thereby enabling more flexible and precise control of the tilt angle of the reflective surface 32, ensuring that the first beam 01 reflected by the reflective surface 32 can accurately illuminate the target illumination area P0.

[0126] It should be noted that the center O of the reflecting surface 32 is the point of application of the resultant gravitational force of the Earth on every tiny part of the reflecting surface 32. When the shape of the reflecting surface 32 is a symmetrical regular shape, the center O of the reflecting surface 32 can be the geometric center of the reflecting surface 32. For example, when the shape of the reflecting surface 32 is circular, the center of the reflecting surface 32 is the center O.

[0127] Figure 7 This is a schematic diagram of another reflection module provided in an embodiment of this application. Figure 8 This is a schematic diagram of the structure of another eye-tracking system provided in an embodiment of this application. Figure 9 This is a schematic diagram of another reflection module provided in an embodiment of this application, as shown below. Figures 7-9 As shown, optionally, the first adjustment unit 411 is used to drive the first contact unit 311 to move along the first direction X; the second adjustment unit 412 is used to drive the second contact unit 312 to move along the second direction Y, wherein the first direction X and the second direction Y are parallel or intersecting.

[0128] Specifically, such as Figures 7-9 As shown, the first adjustment unit 411 is used to drive the first contact unit 311 to move linearly along the first direction X, and the second adjustment unit 412 is used to drive the second contact unit 312 to move linearly along the second direction Y. The linear movement can ensure that the contact unit 310 moves accurately in the predetermined direction and has good repeatability, which is beneficial to improving the position control accuracy of the contact unit 310. At the same time, the linear movement can also reduce the driving structure of the first adjustment unit 411 and the second adjustment unit 412, which is beneficial to reducing the complexity and cost of the system.

[0129] Furthermore, such as Figure 5 and Figure 7As shown, the first direction X and the second direction Y can intersect, that is, the first direction X and the second direction Y are not parallel, but form a certain angle. At this time, the first contact unit 311 and the second contact unit 312 of the carrier 31 move in different directions. By combining the moving distance of the first contact unit 311 in the first direction X and the moving distance of the second contact unit 312 in the second direction Y, the angle adjustment of the reflective surface 32 in more dimensions can be realized, which is beneficial to adapting to various position changes of the user.

[0130] It should be noted that this embodiment achieves the multi-dimensional angle adjustment of the reflective surface 32 using only two contact units 310. The number of contact units 310 is small, which also helps to reduce the complexity and cost of the system.

[0131] Furthermore, the specific directions of the first direction X and the second direction Y, as well as the included angle between them, can be set according to actual needs, and this application embodiment does not impose specific limitations on this.

[0132] Continue to refer to Figure 8 and Figure 9 The first direction X and the second direction Y can also be set in parallel, which can avoid complex linkage effects caused by the difference between the first direction X and the second direction Y, and help simplify the control logic.

[0133] Figure 10 This is a schematic diagram of the structure of another eye-tracking system provided in an embodiment of this application. Figure 11 This is a schematic diagram of another reflection module provided in an embodiment of this application, with reference to... Figure 10 and Figure 11 At least one adjustment unit 41 further includes a third adjustment unit 413, and at least one contact unit 310 further includes a third contact unit 313. The third adjustment unit 413 is connected to the third contact unit 313. The lines connecting the first contact unit 311, the second contact unit 312 and the third contact unit 313 form a triangle J.

[0134] Specifically, such as Figure 10 and Figure 11 As shown, three contact units 310 can be provided on the carrier 31, namely the first contact unit 311, the second contact unit 312 and the third contact unit 313.

[0135] Correspondingly, the angle adjustment module 40 consists of three adjustment units 41, namely, the first adjustment unit 411 connected to the first contact unit 311, the second adjustment unit 412 connected to the second contact unit 312, and the third adjustment unit 413 connected to the third contact unit 313.

[0136] The lines connecting the first contact unit 311, the second contact unit 312, and the third contact unit 313 form a triangle J. At this time, the first contact unit 311, the second contact unit 312, and the third contact unit 313 are not on the same straight line, which can provide more degrees of freedom for the angle adjustment of the reflective surface 32, thereby achieving more precise angle adjustment in three-dimensional space, which is beneficial to adapting to various positional changes of the user's face or eyes.

[0137] Continue to refer to Figure 11 Optionally, the center O of the reflecting surface 32 is located within the region bounded by the vertical projection of triangle J onto the reflecting surface.

[0138] Specifically, such as Figure 11 As shown, the first contact unit 311, the second contact unit 312, and the third contact unit 313 are arranged around the center O of the reflective surface 32. At this time, the first contact unit 311, the second contact unit 312, and the third contact unit 313 are located in different directional areas of the center O. This helps to ensure that the reflective surface 32 is subjected to uniform force during the angle adjustment process, avoids local stress concentration that could cause deformation or damage to the reflective surface 32, and makes the angle adjustment of the reflective surface 32 more balanced and stable.

[0139] Figure 12 This is a schematic diagram of the structure of another eye-tracking system provided in an embodiment of this application. Figure 13 This is a schematic diagram of another reflection module provided in an embodiment of this application, with reference to... Figure 12 and Figure 13 Optionally, the first adjustment unit 411 is used to drive the first contact unit 311 to move along the first direction X; the second adjustment unit 412 is used to drive the second contact unit 312 to move along the second direction Y; and the third adjustment unit 413 is used to drive the third contact unit 313 to move along the third direction Z. The first direction X, the second direction Y, and the third direction Z are parallel to each other. Specifically, as shown... Figure 12 and Figure 13 As shown, the first adjustment unit 411 is used to drive the first contact unit 311 to move linearly along the first direction X, the second adjustment unit 412 is used to drive the second contact unit 312 to move linearly along the second direction Y, and the third adjustment unit 413 is used to drive the third contact unit 313 to move linearly along the third direction Z. The linear movement can ensure that the contact unit 310 moves accurately in the predetermined direction and has good repeatability, which is beneficial to improving the position control accuracy of the contact unit 310. At the same time, the linear movement can also reduce the drive structure of the adjustment unit 41, which is beneficial to reducing the complexity and cost of the system.

[0140] Furthermore, such as Figure 12 and Figure 13As shown, the first direction X, the second direction Y, and the third direction Z are parallel to each other. This avoids the complex linkage effect caused by the differences between the first direction X, the second direction Y, and the third direction Z, and helps to simplify the control logic.

[0141] Continue to refer to Figure 12 Optionally, the first direction X, the second direction Y, and the third direction Z can all be parallel to the direction from the carrier 31 to the reflective surface 32.

[0142] Among them, such as Figure 12 As shown, the moving directions of the first contact unit 311, the second contact unit 312 and the third contact unit 313 are all parallel to the direction from the carrier 31 to the reflective surface 32. The control algorithm for adjusting the angle of the reflective surface 32 can be simpler, which is beneficial to improving the system response speed.

[0143] Figure 14 This is a schematic diagram of the structure of another eye-tracking system provided in an embodiment of this application. Figure 15 This is a schematic diagram of another reflection module provided in an embodiment of this application, as shown below. Figure 14 and Figure 15 As shown, optionally, the first direction X, the second direction Y, and the third direction Z can also intersect with the direction of the carrier 31 pointing to the reflective surface 32 to provide a higher degree of adjustment freedom. This application embodiment does not specifically limit this.

[0144] Figure 16 This is a schematic diagram of another reflection module provided in an embodiment of this application, as shown below. Figure 16 As shown, the first direction X, the second direction Y, and the third direction Z are all different.

[0145] Specifically, such as Figure 16 As shown, by setting the first direction X, the second direction Y, and the third direction Z to be different, it is beneficial to improve the flexibility of adjusting the 32-degree angle of the reflective surface, meet different usage needs, and adapt to a wider range of usage scenarios.

[0146] Meanwhile, if one of the adjustment units 41 fails, the other two adjustment units 41 can still maintain a certain degree of three-dimensional angle control, keep the eye-tracking system running, thereby enhancing the system's robustness and improving its operational stability.

[0147] The specific directions of the first direction X, the second direction Y, and the third direction Z can be set according to actual needs. For example, such as... Figure 16 As shown, the first direction X, the second direction Y, and the third direction Z all have a certain angle with the direction X0 of the carrier 31 pointing to the reflective surface 32. The angles can be the same or different, and this application embodiment does not specifically limit this.

[0148] Continue to refer to Figure 16 Optionally, the angle between the first direction X and the direction X0 from the carrier 31 to the reflective surface 32 is the first included angle α; the angle between the second direction Y and the direction X0 from the carrier 31 to the reflective surface 32 is the second included angle β; and the angle between the third direction Z and the direction X0 from the carrier 31 to the reflective surface 32 is the third included angle γ; the first included angle α, the second included angle β, and the third included angle γ are equal. Wherein, as... Figure 16 As shown, by setting the first included angle α, the second included angle β, and the third included angle γ to be equal, the moving directions of the first contact unit 311, the second contact unit 312, and the third contact unit 313 can have a certain symmetry, which is beneficial to simplify the control algorithm, reduce the computational burden, and improve the system response speed.

[0149] At the same time, when the adjustment unit 41 adjusts the angle of the reflective surface 32, it also helps to ensure that the reflective surface 32 is subjected to balanced force during the adjustment process, thereby avoiding local stress concentration that could lead to deformation or damage to the reflective surface 32.

[0150] Figure 17 A schematic diagram of another eye-tracking system provided in this application embodiment is shown below. Figure 17 As shown, optionally, the eye-tracking system also includes a light source adjustment module 50, which is connected to the first light source 10 and is used to drive the first light source 10 to move along the propagation direction of the first beam 01.

[0151] Where the position of the target illumination area P0 remains unchanged, for the first light source 10 with divergent characteristics, the first light beam 01 emitted by the first light source 10 will gradually diverge during propagation. As the propagation distance increases, the size of the light spot formed in the user's eye image after the first light beam 01 illuminates the target illumination area P0 will gradually increase.

[0152] Therefore, when the distance between the user's eyes or face and the reflective surface 32 changes, the size of the light spot formed in the user's eye or face image after the first beam 01 is reflected by the reflective surface 32 will also change, thus affecting the imaging quality of the image sensor and consequently the accuracy of eye tracking. Based on the actual position of the target illumination area P0, the light source adjustment module 50 can adjust the illumination position of the first light source 10 to form a suitable light spot size in the user's eye or face image, facilitating eye tracking calculations.

[0153] For example, when the distance between the user's eyes or face and the reflective surface 32 is close, the light spot formed by the first beam 01 in the image of the user's eyes or face after being reflected by the reflective surface 32 will be small, or the light spot of the first beam 01 will not be able to cover the target illumination area P0, which may result in insufficient illumination in some areas of the target illumination area P0, so that the actual illumination range cannot cover the entire eye area required for eye tracking, resulting in the loss of important features in the image.

[0154] When the distance between the user's eyes or face and the reflective surface 32 is far, the light spot formed in the image of the user's eyes or face after the first beam 01 is reflected by the reflective surface 32 will be large, or the actual illumination range of the first beam 01 will be larger than the size of the target illumination area P0, which is not conducive to improving light utilization.

[0155] Based on the aforementioned technical problems, in this embodiment, a light source adjustment module 50 is provided to drive the first light source 10 to move along the propagation direction of the first beam 01, adjust the distance between the first light source 10 and the reflecting surface 32, and thereby change the optical path of the first beam 01 between the first light source 10 and the reflecting surface 32. This allows the size of the light spot of the first beam 01 in the image of the user's eyes or face to be adjusted appropriately according to the distance between the user's eyes or face and the reflecting surface 32, and ensures that the actual illumination range formed by the first beam 01 in the user's eyes or face at different distances is consistent with the range of the required target illumination area P0.

[0156] When the distance between the user's eyes or face and the reflective surface 32 is relatively close, the light source adjustment module 50 can drive the first light source 10 to move along the propagation direction of the first beam 01, thereby increasing the distance between the first light source 10 and the reflective surface 32, so that the light spot formed at the distance of the user's eyes or face after the first beam 01 is reflected by the reflective surface 32 becomes larger.

[0157] When the distance between the user's eyes or face and the reflective surface 32 is far, the light source adjustment module 50 can drive the first light source 10 to move along the propagation direction of the first beam 01 to reduce the distance between the first light source 10 and the reflective surface 32, so that the light spot formed at the distance of the user's eyes or face after the first beam 01 is reflected by the reflective surface 32 becomes smaller.

[0158] In this way, the size of the light spot in the image of the user's eyes or face can be adjusted according to the distance between the user's eyes or face and the reflective surface 32. It can also ensure that the actual illumination range formed by the first beam 01 on the user's eyes or face at different distances can be consistent with the range of the required target illumination area P0, so as to adapt to various distance changes between the user's face and the eye tracking device.

[0159] Furthermore, the first beam 01 emitted by the first light source 10 gradually diverges during propagation, and as the propagation distance increases, the brightness of the first beam 01 in the target illumination area P0 will also decrease.

[0160] Therefore, the first light source 10 has a better illumination range. In this embodiment, the first light source 10 is driven to move along the propagation direction of the first beam 01 by the light source adjustment module 50, and the distance between the first light source 10 and the reflective surface 32 is adjusted, thereby changing the optical path of the first beam 01 between the first light source 10 and the reflective surface 32. This also keeps the better illumination range of the first light source 10 at the distance position where the user's eyes or face are located, ensuring that the first light source 10 can provide better illumination for the user's eyes or face at different distances.

[0161] Optionally, the light source adjustment module 50 may include a driving device such as a linear motor, but is not limited to this.

[0162] Furthermore, the connection between the light source adjustment module 50 and the first light source 10 can be a mechanical connection, so that the first light source 10 can move under the driving force provided by the light source adjustment module 50. This application embodiment does not specifically limit this.

[0163] Figure 18 A schematic diagram of another eye-tracking system provided in this application embodiment is shown below. Figure 18 As shown, optionally, the eye-tracking system also includes a second light source 20, which emits a second beam 02, the area of ​​which is illuminated by the second beam 02 being larger than the area of ​​the target illumination area P0.

[0164] Specifically, such as Figure 18 As shown, a second light source 20 is provided, and the area of ​​the illumination region of the second beam 02 emitted by the second light source 20 is larger than the area of ​​the target illumination region P0, so that the second light source 20 can provide a larger illumination range.

[0165] Thus, when the first beam 01 provided by the first light source 10 cannot meet the lighting requirements, causing the system to be unable to recognize the user's face or eye position, the second light source 20 is turned on to expand the lighting range, thereby providing additional lighting and achieving a wider lighting range. This helps the system recognize the user's face or eye position and enhances the working stability and robustness of the eye-tracking system.

[0166] Among them, the second light source 20 can directly emit the second beam 02 towards the head box range (i.e., the user's face or eyes), which helps to simplify the system structure.

[0167] In addition, the second light source 20 may include devices such as infrared light-emitting diodes (LEDs) or infrared lasers, without specific limitations here.

[0168] Figure 19 This is a schematic diagram of the structure of an adjustment unit provided in an embodiment of this application, with reference to... Figure 1 and Figure 19 Optionally, the adjustment unit 41 includes a drive structure 42 and a universal ball shaft 43. The drive structure 42 is movably connected to the contact unit 310 through the universal ball shaft 43 and is used to drive the contact unit 310 to move.

[0169] Specifically, such as Figure 19 As shown, the drive structure 42 is used to provide driving force. The universal ball shaft 43 is connected to the drive structure 42, so that the universal ball shaft 43 can move in a certain direction under the action of the driving force. At the same time, the universal ball shaft 43 is connected to the contact unit 310, so that the contact unit 310 can be moved by the universal ball shaft 43, thereby adjusting the angle of the reflective surface 32.

[0170] The universal ball joint 43 allows the contact unit 310 to have a certain degree of rotational freedom, so that when the adjustment unit 41 drives the corresponding contact unit 310 to move, it can avoid excessive local stress that could cause deformation of the reflection module 30, ensure that the contact unit 310 can move smoothly, and improve the stability and reliability of the reflection module 30 when adjusting the angle.

[0171] Figure 20 A schematic diagram of another adjustment unit provided in an embodiment of this application is shown below. Figure 20 As shown, optionally, the drive structure 42 includes a motor 421 and a lead screw assembly 422, with the motor 421 connected to the universal ball shaft 43 via the lead screw assembly 422.

[0172] Specifically, such as Figure 20 As shown, the motor 421 can drive the lead screw assembly 422 by rotating the output shaft. The lead screw assembly 422 converts the rotational motion of the motor 421 into linear motion. The shaft cage of the universal ball shaft 43 can be set on the carrier 31, and the ball head of the universal ball shaft 43 can be set at the front end of the lead screw assembly 422 and installed in the ball cage. This enables the lead screw assembly 422 to drive the universal ball shaft 43 to move in a certain direction, thereby driving the contact unit 310 to move in the direction of movement of the universal ball shaft 43.

[0173] The motor 421 may include a stepper motor or a servo motor, etc., and this application embodiment does not specifically limit it.

[0174] In addition, the lead screw assembly 422 may include two parts: a screw and a nut, and its specific structure can be set according to actual needs.

[0175] In an optional embodiment, Figure 21 This is a schematic diagram of another adjustment unit provided in an embodiment of this application, as shown below. Figure 21 As shown, the lead screw assembly 422 includes a first nut 4221 and a first screw 4222. The first screw 4222 is connected to a universal ball shaft 43. The first nut 4221 and the first screw 4222 form a threaded connection. The motor 421 is connected to the first nut 4221 and is used to drive the first nut 4221 to rotate. The first nut 4221 applies a radial torque to the first screw 4222 so that the first screw 4222 moves in a straight line under the action of the torque. The first screw 4222 drives the universal ball shaft 43 to move, thereby adjusting the tilt angle of the reflective surface.

[0176] Optional, see reference Figure 21 The lead screw assembly 422 also includes a first limiting member 4225, which is used to limit the maximum movement position of the first screw 4222 to prevent the first screw 4222 from exceeding the predetermined movement range and avoid damage.

[0177] The first limiting member 4225 can be a linear guide rail, which limits the maximum movement position of the first screw 4222 by a slide on the linear guide rail, but is not limited to this.

[0178] In another alternative embodiment, Figure 22 This is a schematic diagram of another adjustment unit provided in an embodiment of this application, as shown below. Figure 22 As shown, the lead screw assembly 422 includes a second nut 4224 and a second screw 4223. The second nut 4224 is connected to a universal ball shaft 43, and the second nut 4224 and the second screw 4223 form a threaded connection. A motor 421 is connected to the second screw 4223. The motor 421 is used to drive the second screw 4223 to rotate, so that the second nut 4224 moves in a straight line. The second nut 4224 drives the universal ball shaft 43 to move, so as to adjust the tilt angle of the reflective surface.

[0179] Optional, see reference Figure 22 The lead screw assembly 422 also includes a second limiting member 4226, which is used to limit the maximum movement position of the second nut 4224 to prevent the second nut 4224 from exceeding the predetermined movement range and avoid damage.

[0180] The second limiting member 4226 can be a limiting groove, the inside of which is slidably connected to the second nut 4224 to limit the position of the second nut 4224, but is not limited to this.

[0181] In the above embodiments, by setting the lead screw assembly to include a nut and a screw, the rotational motion of the motor is converted into linear motion, thereby improving the movement accuracy of the contact unit in the linear direction and thus improving the adjustment accuracy of the tilt angle of the reflector.

[0182] The above examples only illustrate the specific structures of two lead screw assemblies. Assuming that the motor provides driving force to the lead screw assembly and the lead screw assembly can drive the universal ball shaft to move, the embodiments of this application do not specifically limit the structure of the lead screw assembly.

[0183] Optionally, a displacement sensor may also be provided in the lead screw assembly 422 to acquire or record the distance moved by the first screw 4222 or the second nut 4224, so as to determine whether the reflective surface has been adjusted to a suitable tilt angle based on the current distance moved, which helps to improve the adjustment accuracy of the reflective surface angle.

[0184] The displacement sensor may include a Hall sensor to enable non-contact measurement, which helps extend the system's lifespan.

[0185] In other embodiments, the displacement sensor may also be other devices that can provide feedback on displacement information, feedback on position information, or perform position detection. This application does not specifically limit the specific devices used in this embodiment.

[0186] In another optional embodiment, the drive structure 42 may further include a linear motor. In this case, the linear motor can be directly connected to the universal ball shaft 43 to directly drive the universal ball shaft 43 to move linearly in a predetermined direction. In this case, there is no need to set up an intermediate transmission mechanism, which simplifies the system structure.

[0187] It should be noted that the specific structure of the driving structure 42 is not limited to the above embodiments, and the embodiments of this application do not specifically limit it.

[0188] Figure 23 A schematic diagram of another eye-tracking system provided in this application embodiment is shown below. Figure 23 As shown, optionally, the carrier 31 includes at least one contact unit 310, which is located on the side surface of the carrier 31 facing away from the reflective surface 32. The angle adjustment module 40 includes at least one support unit 44, which is correspondingly connected to the contact unit 310 and is used to support the contact unit 310.

[0189] Specifically, such as Figure 23 As shown, a contact unit 310 is provided on the side surface of the carrier 31 facing away from the reflective surface 32. The contact unit 310 refers to the contact part located on the carrier 31.

[0190] The angle adjustment module 40 also includes at least one support unit 44, which is connected to the contact unit 310. The support unit 44 is used to provide support force to the contact unit 310 so that the position of the contact unit 310 remains unchanged during the adjustment of the position of the adjustment unit 41, or in other words, it does not make large linear or curved movements, which helps to improve the reliability of the reflection 32 when adjusting the angle.

[0191] The contact unit 310 may include structures such as contact points, contact shafts, and contact surfaces. The specific arrangements of the contact points, contact shafts, and contact surfaces can be referred to the above embodiments, and will not be repeated here.

[0192] Furthermore, the specific structure of the support unit 44 can also be set according to actual needs. The support unit 44 may include support components such as support shafts, but this application embodiment does not specifically limit this.

[0193] It should be noted that, Figure 23 The example shown illustrates a structure in which the carrier 31 includes a contact unit 310. In other embodiments, the contact unit 310 may be of other numbers, which are not specifically limited here.

[0194] In addition, the number of contact units 310 can also be set according to actual needs. For example, the carrier 31 includes m contact units 310 connected to the support unit 44 and n contact units 310 connected to the adjustment unit 41, where m≥0 and n≥1. This application embodiment does not make specific limitations on this.

[0195] In other alternative embodiments, the eye-tracking system further includes at least one processor (not shown) configured to acquire facial and / or eye data, wherein the facial and / or eye data is used to determine adjustment information; and send adjustment information to the angle adjustment module to control the movement of the adjustment unit.

[0196] In this application embodiment, facial and / or eye data can be understood as the user's facial image or eye image or signal. Generally, the user's facial and / or eye data can be collected by devices such as infrared cameras, DVS cameras, 2D cameras or 3D sensors.

[0197] Facial and / or eye data can be used to determine regulatory information in the following ways:

[0198] S1: Determine eye position based on facial and / or eye data;

[0199] Specifically, at least one processor determines the user's eye position based on the user's facial and / or eye data. The eye position can be understood as the location of the user's eyes in space, which can be a coordinate point or a spatial region that can be expressed by coordinates.

[0200] S1: Determine the eye position based on facial and / or eye data, which may include: S11: Determine eye features based on facial and / or eye data, S12: Determine the eye position based on eye features.

[0201] Specifically, eye features can include the user's pupil position, pupil size, eyelid position, and spot location. Analyzing eye data involves at least one processor using algorithms and learning models to process and determine these features. For example, based on an acquired user eye image, an image analysis model can be used to determine the pupil position and size. The user's eye position can be further obtained through algorithmic and learning model analysis of the eye features.

[0202] S2: Determine the target illumination area based on the eye position;

[0203] Specifically, at least one processor can determine the target lighting area that needs to be illuminated by the light source based on the user's eye position. The user's eye position should be within the target lighting area, and when the target lighting area is illuminated by the light source, the user's eye position will inevitably be illuminated.

[0204] S3: Determine adjustment information based on the target lighting area;

[0205] Specifically, at least one processor can determine the position of the target illumination area in space based on the position of the user's face or eyes in space, and determine adjustment information based on the position of the target illumination area in space. The processor controls the adjustment unit to move through the adjustment information, so that the angle of the reflective surface is adjusted. After adjustment, the tilt angle of the reflective surface of the reflective module can be satisfied to reflect the first beam to the target illumination area.

[0206] In this embodiment of the application, S3: Determining adjustment information based on the target lighting area includes:

[0207] S311: Determine the target position of the adjustment unit based on the target lighting area;

[0208] Specifically, the target illumination area is the region within the head-mounted sensor that is illuminated by light emitted from the light source. The head-mounted sensor range refers to the spatial area within which the user's head can move automatically during eye-tracking calculations by the telemetry eye tracker. The target illumination area can be represented by the position information of the midpoint of the illuminated area in space, or by the position information of a corner point of the illuminated area, etc. The target position of the adjustment unit is the position information of the adjustment unit when the light from the light source illuminates the target illumination area.

[0209] The methods for determining the target position of the adjustment unit based on the target illumination area include, but are not limited to, the following: One approach is to match the target position of the adjustment unit with the target illumination area in a preset mapping information database. This database stores the correlation between the target illumination area and the target position of the adjustment unit. The preset mapping information database can be constructed during the calibration phase of the light source adjustment system. The construction process of the preset mapping information database can be as follows: The position information of the adjustment unit is continuously adjusted through program instructions, causing light to be reflected to different illumination areas P1, P2, P3, P4, ..., Pn, etc., to construct the correlation between the target positions of the adjustment units corresponding to each target illumination area, and these correlations are stored in the preset mapping information database. The larger the value of n, the more data is generated regarding the correlation between the illumination area position and the adjustment unit position information, which is more conducive to matching the target position corresponding to the user's facial position.

[0210] Another approach is to determine the target position of the adjustment unit through an algorithmic model or a deep learning model. In other words, relevant information about the target lighting area is input into the algorithmic model and / or deep learning model to determine the target position to which the adjustment unit should move.

[0211] Of course, in other cases, the target position of the adjustment unit can also be a combination of the two situations mentioned above. For example, first match the target position of the adjustment unit according to the preset mapping information library. If a suitable target position cannot be matched, then determine the target position of the adjustment unit according to the algorithm model or deep learning model.

[0212] S32: Determine adjustment information based on the target position of the adjustment unit;

[0213] Specifically, after at least one processor determines the target position where the adjustment unit should be, it generates adjustment information based on the target position. The processor controls the adjustment unit to move to the target position by executing the adjustment information. During the movement of the adjustment unit, the pose of the reflective surface will change, thereby adjusting the emission direction of the reflected light to adjust the light illumination area so that the reflected light illuminates the target illumination area.

[0214] The process of controlling the movement of the adjustment unit based on adjustment information can be as follows: the motor in the adjustment unit is started based on adjustment information, and the motor drives the transmission device, so that the lead screw in the transmission device pushes the ball head of the ball shaft to move, so as to achieve the purpose of adjusting the pose of the reflector.

[0215] For example, taking the example of three contact units on the carrier, with each contact unit corresponding to one of the three adjustment units, at least one processor of the eye-tracking system can be configured to perform the following steps, including but not limited to:

[0216] The three control units are named S1, S2, and S3, respectively. At a certain time t0, the spatial coordinates of a specific point on each of the three control units can be expressed by the following formula:

[0217] p1(S1) = (a1x, a1y, a1z);

[0218] p1(S2) = (a2x, a2y, a2z);

[0219] p1(S3) = (a3x, a3y, a3z).

[0220] The specific point on the adjustment unit can be the center point of the ball head in the adjustment unit, but it is not limited to this.

[0221] In other embodiments, a point on the adjustment unit that can reflect the change in the position of the contact unit can be selected as a specific point on the adjustment unit, and this application embodiment does not specifically limit this.

[0222] Furthermore, when each regulating unit performs a unit displacement, the spatial vector of the displacement of a specific point on the regulating unit can be expressed as:

[0223]

[0224] When the three regulating units perform displacements of distances k1, k2, and k3 respectively, the spatial vector of the displacement of a specific point on the regulating unit can be expressed as:

[0225]

[0226] At this point, the spatial coordinates of a specific point on the adjustment unit after displacement can be expressed as:

[0227]

[0228] Furthermore, the position and angle information of the reflecting surface can be determined by the aforementioned spatial coordinates.

[0229] In an exemplary embodiment, when all three adjustment units move along the same coordinate axis, for example, when all three adjustment units move along the direction from the support body to the reflective surface, the spatial vector of the displacement of a specific point on the adjustment unit when each adjustment unit performs a unit displacement can be expressed as:

[0230]

[0231] When the three regulating units perform displacements of distances k1, k2, and k3 respectively, the spatial vector of the displacement of a specific point on the regulating unit can be expressed as:

[0232]

[0233] At this point, the spatial coordinates of a specific point on the adjustment unit after displacement can be expressed as:

[0234] p2(S1)=(a1x, a1y, a1z+k1);

[0235] p2(S2)=(a2x, a2y, a2z+k2);

[0236] p2(S3)=(a3x, a3y, a3z+k3).

[0237] Furthermore, the position and angle information of the reflecting surface can be determined by the aforementioned spatial coordinates.

[0238] Figure 24 This is a schematic diagram of the structure of another eye-tracking system provided in an embodiment of this application. Figure 25 This is a schematic diagram of the structure of another eye-tracking system provided in an embodiment of this application, with reference to... Figure 24 and Figure 25 When the target illumination area P0 is located in area P1, the spatial coordinates of a specific point on the three adjustment units can be represented by the following formula:

[0239] p1(S1) = (a1x, a1y, a1z);

[0240] p1(S2) = (a2x, a2y, a2z);

[0241] p1(S3) = (a3x, a3y, a3z).

[0242] Furthermore, when the target illumination area P0 needs to be moved to area P2, the spatial coordinates of specific points on the three adjustment units after displacement can be expressed as:

[0243] p2(S1)=(a1x, a1y, a1z+k1);

[0244] p2(S2)=(a2x, a2y, a2z+k2);

[0245] p2(S3)=(a3x, a3y, a3z+k3).

[0246] Through proper design, within a reasonable target illumination area P0 within the head-mount box, one or more of the following mapping relationships exist or approximately exist:

[0247]

[0248] ...

[0249]

[0250] or,

[0251]

[0252] ...

[0253]

[0254] The above mapping relationship can be a mapping between some continuous functions or a mapping between discrete data in two databases. This application does not specifically limit this.

[0255] In an optional embodiment, the example is a continuous function mapping, with reference to [reference needed]. Figure 24 and Figure 25 Three illumination points are extracted in the target illumination area P0. The three illumination points can be located at the center of the target illumination area P0 and on the two opposite edges of the target illumination area P0, but are not limited to this.

[0256] There are three characteristic rays that illuminate the three illumination points respectively. Correspondingly, there are also three matching characteristic rays in the first beam emitted from the first light source. Therefore, the eye-tracking system can include:

[0257] Assuming in the initial state, for example, as Figure 24 As shown, when the target illumination area P0 is located in area P1, the surface shape equation of the reflecting surface is f. z1 (x,y,z).

[0258] Then passed as well as After the vector transformation, that is, when the three adjustment units perform displacements of distances k1, k2, and k3 respectively, the reflecting surface satisfies the new surface shape equation f. z2 (x,y,z).

[0259] Right now,

[0260] The three characteristic rays l1, l2, and l3 emitted from the first light source reach f. z2 Then, the reflection will follow the law of reflection on its surface, that is, the reflected ray, the incident ray and the normal are in the same plane, the reflected ray and the incident ray are on opposite sides of the normal, and the angles with the interface normal are equal, that is, the angle of incidence and the angle of reflection are equal.

[0261] The three reflected rays l′1, l′2, and l′3 coincide with or nearly coincide with the three characteristic rays of the target illumination area P0, for example, the average angular deviation between the reflected rays and the characteristic rays is within 0.1°.

[0262] Thus, for a changing target illumination area P0, the mapping relationship between P0 and k1, k2 and k3 can be determined by matching its characteristic rays with the characteristic rays emitted from the first light source.

[0263] In another optional embodiment, taking the mapping relationship as a mapping of discrete data in two databases as an example, during the initial calibration or use of the eye-tracking system, for different target illumination areas P0, there are specific values ​​k1, k2 and k3 corresponding to a specific target illumination area P0. These specific values ​​can be preset, generated when calibrating the eye-tracking system, or collected and stored in real time during use, or generated using certain algorithms.

[0264] For example, Figure 26 This is a schematic diagram of the structure of another eye-tracking system provided in an embodiment of this application, with reference to... Figure 26 When a specific value is generated during the calibration of the eye-tracking system, information can be sampled according to program instructions. When the user's eyes or face are in positions P1, P2, P3, P4, ..., Pn, the values ​​of k1, k2, and k3 can be adjusted so that the beam reflected by the reflective surface can illuminate the corresponding position. At this time, the program records the position information of the illuminated area and the corresponding values ​​of k1, k2, and k3.

[0265] Understandably, n refers to the preset number of regions. The larger the value of n, the more preset regions there are, and the more data needs to be sampled. In this case, the determined data for k1, k2, and k3 are more accurate. Figure 26 The example only uses four preset areas, but it is not limited to this. In other embodiments, the number of preset areas can be set according to actual needs. This application does not specifically limit this.

[0266] Furthermore, for other unpreset and uncollected area location information, during the use of the eye-tracking system, the values ​​of k1, k2, and k3 can be estimated through a preset algorithm model and compared with the actual results. If the actual area illuminated by the beam reflected by the reflective surface is inaccurate, the accurate values ​​of k1, k2, and k3 are obtained and recorded, and the algorithm model is corrected in reverse. The methods for correcting the algorithm model in reverse include, but are not limited to, deep learning, etc. This application embodiment does not specifically limit this.

[0267] Figure 27 A flowchart illustrating a control method for an eye-tracking system provided in this application embodiment is shown below. Figure 27 As shown, an exemplary control method for an eye-tracking system may include:

[0268] When the user's eyes or face move to a new area, an image sensor is used to capture an image and determine whether the location of the user's eyes or face can be determined from the image.

[0269] If so, the location of the user's eyes or face can be determined through a preset program.

[0270] If not, turn on the second light source to provide a larger illuminated area, thereby helping to determine the location of the user's eyes or face.

[0271] After determining the location of the user's eyes or face, query the database to see if the location information for that area is stored.

[0272] If so, the corresponding values ​​of k1, k2, and k3 are obtained directly from the location information of the area stored in the database, and the three adjustment units are controlled to drive the reflection module 30 to move based on the values ​​of k1, k2, and k3, so as to adjust the tilt angle of the reflection module 30.

[0273] If not, the first algorithm model is invoked to calculate the values ​​of k1, k2, and k3 based on the location information of the area. Then, the three adjustment units are controlled to drive the reflection module to move based on the values ​​of k1, k2, and k3, so as to adjust the tilt angle of the reflection module.

[0274] The first algorithm model is used to represent the relationship between the location of the user's eyes or face, or the target lighting area P0 that needs to be illuminated, and the values ​​of k1, k2, and k3.

[0275] The methods for obtaining the first algorithm model may include, but are not limited to:

[0276] After the three adjustment units perform displacements k1, k2, and k3 respectively, the spatial coordinates of specific points on the three adjustment units after the displacement can be expressed as:

[0277]

[0278] Among them, such as Figure 16 As shown, the three adjustment units 41 can move in the X, Y and Z directions respectively.

[0279] or,

[0280] p2(S1)=(a1x, a1y, a1z+k1);

[0281] p2(S2)=(a2x, a2y, a2z+k2);

[0282] p2(S3)=(a3x, a3y, a3z+k3).

[0283] Among them, such as Figure 17 As shown, all three adjustment units 41 can move in the Z direction.

[0284] Establish a mapping relationship between the coordinate position changes of specific points on the three adjustment units and the coordinates of the base point of the reflector surface shape equation, where the base point is the origin of the reflector surface shape equation.

[0285] Based on the changed base point coordinates, substitute them into the original surface shape equation of the reflecting surface to form a new surface shape equation of the reflecting surface.

[0286] The surface shape equation of the reflecting surface may include, but is not limited to:

[0287]

[0288] Furthermore, since the first light source illuminates the reflective surface at the new position and is reflected by the reflective surface to the target illumination area P0, the location of the target illumination area P0 illuminated by the first light source after reflection by the reflective surface can be determined based on the light source vector formed by the first light source illuminating the reflective surface, the surface shape equation of the reflective surface at the new position, and the law of reflection. This leads to the formation of a mapping relationship between the target illumination area P0 and the values ​​of k1, k2, and k3.

[0289] In other words, after determining the target illumination area P0 to be illuminated, the data (k1, k2, k3) that each adjustment unit needs to generate displacement based on the initial position can be correlated, and then the data that the adjustment unit still needs to move is determined based on the current position information of a specific point in the adjustment unit.

[0290] Furthermore, after controlling the three adjustment units to drive the reflection module 30 to move based on the values ​​of k1, k2 and k3, an image can be acquired by an image sensor, and a preset program can be used to determine whether the actual lighting area is correct, that is, whether the actual lighting area meets the lighting requirements of the target lighting area P0.

[0291] If so, eye-tracking analysis can be performed directly on the image to achieve eye-tracking functionality.

[0292] If not, the second algorithm model can be called to calculate the approximation value to obtain new values ​​of k1, k2 and k3. Then, based on the new values ​​of k1, k2 and k3, the three adjustment units are controlled to drive the reflection module 30 to move, and the actual lighting area is determined again until the actual lighting area meets the lighting requirements of the target lighting area P0.

[0293] The second algorithm model can also be used to represent the relationship between the location of the user's eyes or face, or the target lighting area P0 that needs to be illuminated, and the values ​​of k1, k2, and k3.

[0294] The difference between the second algorithm model and the first algorithm model lies in the fact that the input target illumination area P0 refers to the ideal illumination area.

[0295] If the deviation between the actual position of the user's eyes or face, i.e. the ideal target lighting area P0, and the actual illumination area is large, the second algorithm model can be invoked.

[0296] At this point, the second algorithm model first calculates the position of the ideal target lighting area P0. For example, if the image sensor identifies that the actual irradiated area to the user's eyes or face is to the left, then the ideal target lighting area P0 should be to the right of this irradiated area position to obtain the ideal target lighting area P0.

[0297] Specifically, based on the difference between the actual lighting area and the ideal target lighting area P0, the values ​​of k1, k2, and k3 of the three adjustment units are continuously adjusted to further optimize the position of the reflective surface until the reflective surface can reflect the light beam to the ideal target lighting area P0. The optimized actual values ​​of k1, k2, and k3 of the three adjustment units are obtained as the values ​​of k1, k2, and k3 corresponding to the ideal target lighting area P0.

[0298] Specifically, the mapping relationship between the ideal target lighting area P0 and its corresponding values ​​of k1, k2 and k3 can be stored in the database, and the first algorithm model can be iterated to automatically correct system errors, so that the actual lighting area can be closer to the target lighting area P0 and adapt to changes in the user's eyes or face.

[0299] It should be noted that, in this embodiment, the database can be obtained based on testing, and the methods of obtaining it may include, but are not limited to:

[0300] By continuously adjusting the values ​​of k1, k2, and k3 of the three adjustment units, the tilt angle of the reflective surface changes. The location of the area illuminated by the first beam emitted by the first light source after reflection by the reflective surface is determined. The location of this area is then correlated with the values ​​of k1, k2, and k3. A large number of adjustments and tests are then performed to form a preset database, thereby establishing a mapping relationship between the location of the irradiated area and the values ​​of k1, k2, and k3.

[0301] It is understood that the control method provided in the above embodiments is only illustrated by taking the example of setting three contact units on the carrier and connecting the three contact units to three adjustment units. When other numbers of adjustment units are used, the above control method can be adjusted to achieve the desired result of the technical solution of this application.

[0302] In this embodiment, the control method of the eye-tracking system can be applied to the eye-tracking system provided in any embodiment of this application. The control method can be executed by a control device, which can be implemented in hardware and / or software and can be configured in a control chip.

[0303] Furthermore, the control method provided in the above embodiments is only the logical basis for controlling the operation of the eye-tracking system. Based on this, the above process can be added, reduced, or modified according to actual needs.

[0304] For example, technologies such as user facial action prediction can be added to the control logic described above, which will not be detailed here.

[0305] In summary, the eye-tracking system provided in this application uses an adjustable-angle reflection module to reflect the first beam of a first light source to different positions within the head-mounted box, i.e., the target illumination area. In conjunction with the eye-tracking system's localization of the user's eyes or face, the target illumination area can be converted into the position and angle information of the reflection module in three-dimensional space, as well as the position information of the first light source. This allows for the acquisition of displacement vector information for multiple adjustment units. Based on this displacement vector information, the movement of the reflection module can be precisely controlled, enabling the reflection module to accurately reflect the first beam to the target illumination area, thus achieving accurate illumination of the user's eyes or face.

[0306] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0307] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An eye-tracking system, characterized in that, Includes a primary light source, a reflection module, and an angle adjustment module; The first light source is used to emit the first beam of light; The reflection module is located on the propagation path of the first beam. The reflection module includes a carrier and a reflecting surface located on one side surface of the carrier. The reflecting surface is used to reflect the first beam. The angle adjustment module includes at least one adjustment unit; The carrier includes at least one contact unit located on the side surface of the carrier opposite to the reflective surface; The adjustment unit is connected to the contact unit in a one-to-one correspondence. The adjustment unit is used to drive the contact unit to move in order to adjust the angle of the reflective surface so that the reflective surface reflects the first light beam to the target illumination area.

2. The eye-tracking system according to claim 1, characterized in that, It also includes at least one processor configured to: acquire facial and / or eye data, wherein the facial and / or eye data is used to determine adjustment information; and send the adjustment information to the angle adjustment module to control the movement of the adjustment unit.

3. The eye-tracking system according to claim 2, characterized in that, At least one of the processors is further configured to: The eye position is determined based on the facial and / or eye data; The target illumination area is determined based on the eye position; The adjustment information is determined based on the target lighting area.

4. The eye-tracking system according to claim 3, characterized in that, Determining the adjustment information based on the target lighting area includes: The target position of the adjustment unit is determined based on the target lighting area; The adjustment information is determined based on the target position of the adjustment unit.

5. The eye-tracking system according to claim 1, characterized in that, At least one of the adjustment units includes a first adjustment unit and a second adjustment unit; At least one of the contact units includes a first contact unit and a second contact unit; The first adjustment unit is connected to the first contact unit, and the second adjustment unit is connected to the second contact unit; The distance between the vertical projection of the first contact unit on the reflective surface and the center of the reflective surface is greater than 0; The distance between the vertical projection of the second contact unit on the reflective surface and the center of the reflective surface is greater than 0.

6. The eye-tracking system according to claim 5, characterized in that, The first adjustment unit is used to drive the first contact unit to move along a first direction; The second adjustment unit is used to drive the second contact unit to move along the second direction; The first direction and the second direction are parallel or intersecting.

7. The eye-tracking system according to claim 5, characterized in that, At least one of the adjustment units further includes a third adjustment unit, and at least one of the contact units further includes a third contact unit, wherein the third adjustment unit is connected to the third contact unit; The lines connecting the first contact unit, the second contact unit, and the third contact unit form a triangle.

8. The eye-tracking system according to claim 7, characterized in that, The center of the reflective surface is located within the region bounded by the vertical projection of the triangle onto the reflective surface.

9. The eye-tracking system according to claim 7, characterized in that, The first adjustment unit is used to drive the first contact unit to move along a first direction; The second adjustment unit is used to drive the second contact unit to move along the second direction; The third adjustment unit is used to drive the third contact unit to move along a third direction; The first direction, the second direction, and the third direction are parallel to each other.

10. The eye-tracking system according to claim 9, characterized in that, The first direction, the second direction, and the third direction are all parallel to the direction in which the carrier points to the reflective surface.

11. The eye-tracking system according to claim 7, characterized in that, The first adjustment unit is used to drive the first contact unit to move along a first direction; The second adjustment unit is used to drive the second contact unit to move along the second direction; The third adjustment unit is used to drive the third contact unit to move along a third direction; The first direction, the second direction, and the third direction are all different from each other.

12. The eye-tracking system according to claim 9, characterized in that, The angle between the first direction and the direction in which the carrier points to the reflective surface is the first angle; The angle between the second direction and the direction in which the carrier points to the reflective surface is the second included angle; The angle between the third direction and the direction in which the carrier points to the reflective surface is the third angle; The first included angle, the second included angle, and the third included angle are equal.

13. The eye-tracking system according to claim 1, characterized in that, The eye-tracking system also includes a light source adjustment module; The light source adjustment module is connected to the first light source and is used to drive the first light source to move along the propagation direction of the first beam.

14. The eye-tracking system according to claim 1, characterized in that, The eye-tracking system also includes a second light source; The second light source is used to emit a second beam of light, and the area of ​​the illumination region of the second beam of light is larger than the area of ​​the target illumination region.

15. The eye-tracking system according to claim 1, characterized in that, The reflective surface includes a curved surface.

16. The eye-tracking system according to claim 1, characterized in that, The adjustment unit includes a drive structure and a universal ball joint; The drive structure is movably connected to the contact unit via the universal ball joint, and is used to drive the contact unit to move.

17. The eye-tracking system according to claim 16, characterized in that, The drive structure includes a motor and a lead screw assembly; The motor is connected to the universal ball joint via the lead screw assembly.

18. The eye-tracking system according to claim 1, characterized in that, The angle adjustment module includes at least one support unit; The support unit is connected to the contact unit, and the support unit is used to support the contact unit.

Images