Eyeball tracking device and method, display device, apparatus and medium

Abstract

The present disclosure provides an eye tracking device and method, a display device, an apparatus and a medium, and belongs to the technical field of human-computer interaction. The eye tracking device comprises a distance detection unit, a light emitting unit, a light detection unit and a processing unit; the distance detection unit is configured to detect a first distance from an eye to a target object; the light emitting unit is configured to irradiate a target eyebox region of the target object with infrared light; the light detection unit comprises a plurality of sub-units arranged in an array, each of the sub-units comprises a plurality of receivers arranged in an array, and the plurality of receivers are configured to receive the infrared light reflected by the eye; the processing unit is configured to control a first receiver in the sub-unit to receive the infrared light reflected by the eye, and determine a position of the eye according to an intensity of the infrared light received by the first receiver; wherein the first receiver is a receiver corresponding to the first distance.

Description

Technical Field

[0001] This disclosure relates to the field of human-computer interaction technology, and in particular to an eye-tracking device and method, display device, equipment and medium. Background Technology

[0002] Eye-tracking devices are used to detect the position of a viewer's eyes so that the target object can react accordingly to the position of the eyes.

[0003] An eye-tracking device may include a light-emitting unit, an infrared camera, and a processing unit. The light-emitting unit emits infrared light towards the area where the eye is located. The infrared camera receives the infrared light reflected from the eye and forms an image. The processing unit determines the eye position based on the image formed by the infrared camera. Summary of the Invention

[0004] This disclosure provides an eye-tracking device and method, a display device, an apparatus, and a medium. The technical solution is as follows:

[0005] This disclosure provides an eye-tracking device, comprising: a distance detection unit, a light-emitting unit, a light detection unit, and a processing unit. The distance detection unit detects a first distance from the eye to a target object; the light-emitting unit illuminates a target eye region of the target object with infrared light; the light detection unit includes multiple sub-units arranged in an array, each sub-unit including multiple receivers arranged in an array, the multiple receivers receiving infrared light reflected from the eye; the processing unit controls a first receiver in the sub-unit to receive the infrared light reflected from the eye, and determines the position of the eye based on the intensity of the infrared light received by the first receiver; wherein the first receiver is a receiver corresponding to the first distance.

[0006] Optionally, the receiver includes an aperture and an infrared light sensor; the aperture is used to control whether infrared light can enter the corresponding infrared light sensor under the control of the processing unit.

[0007] Optionally, the aperture is a liquid crystal aperture or a microelectromechanical scanning mirror.

[0008] Optionally, the number and arrangement of receivers contained in the plurality of sub-units are the same; in at least two of the plurality of sub-units, the receivers corresponding to the same distance are arranged in the same or different positions.

[0009] Optionally, the subunit further includes a microlens for focusing the infrared light reflected from the eyeball onto a first receiver in the subunit.

[0010] Optionally, the sub-unit further includes a light-shielding structure located between any two adjacent receivers.

[0011] Optionally, the target eye box region includes multiple sub-regions arranged in an array; the processing unit is used to determine the position of the pupil in the eyeball based on the light intensity of the infrared light received by the first receiver; and to determine the target sub-region corresponding to the sub-unit to which the pupil position belongs as the position of the eyeball based on the correspondence between the multiple sub-units and the multiple sub-regions.

[0012] Optionally, the processing unit is used to determine the position of the pupil as the position of the first receiver that meets the following conditions: the intensity of the received infrared light is less than a first intensity threshold, and there are multiple first receivers in the vicinity where the intensity of the received infrared light is greater than a second intensity threshold.

[0013] In one possible implementation, the processing unit is further configured to control a second receiver in the subunit not to receive the infrared light reflected from the eyeball. The second receiver is a receiver other than the first receiver.

[0014] Optionally, the target object includes a display screen, the target object has multiple eye box regions, the multiple eye box regions are arranged along a direction parallel to the horizontal center line of the display screen, and the target eye box region is one of the eye box regions; the processing unit is further configured to determine the target eye box region according to the eye position corresponding to the eyeball.

[0015] This disclosure also provides an eye-tracking method, which includes: acquiring a first distance from an eyeball to a target object; controlling a first receiver in a plurality of sub-units to receive infrared light reflected from the eyeball, wherein the plurality of sub-units are arranged in an array, and any one of the plurality of sub-units includes a plurality of receivers arranged in an array, and the first receiver is the receiver in the sub-unit corresponding to the first distance; and determining the position of the eyeball based on the intensity of the infrared light received by the first receiver.

[0016] Optionally, the target eye box region includes multiple sub-regions arranged in an array; determining the position of the eyeball based on the intensity of infrared light received by the first receiver includes: determining the position of the pupil in the eyeball based on the intensity of infrared light received by the first receiver; and determining the target sub-region corresponding to the sub-unit to which the pupil is located as the position of the eyeball based on the correspondence between the multiple sub-units and the multiple sub-regions.

[0017] Optionally, determining the position of the pupil in the eyeball based on the intensity of the infrared light received by the first receiver includes: determining the position of the first receiver that meets the following conditions as the position of the pupil: the intensity of the received infrared light is less than a first intensity threshold, and there are multiple first receivers in the vicinity where the intensity of the received infrared light is greater than a second intensity threshold.

[0018] Optionally, the target object includes a display screen, the target object has multiple eye box regions, the multiple eye box regions are arranged along a direction parallel to the horizontal center line of the display screen, and the target eye box region is one of the eye box regions; the method further includes: determining the target eye box region according to the eye position corresponding to the eyeball.

[0019] This disclosure also provides a display device, which includes: a display screen and any of the aforementioned eye-tracking devices; the light-emitting unit and the light-detecting unit are both located on the periphery of the display area of ​​the display screen.

[0020] Optionally, the display device is a glasses-free 3D display, an augmented reality (AR) device, or a virtual reality (VR) device.

[0021] Optionally, the light-emitting unit includes a plurality of LEDs, which are arranged at intervals around the display area of ​​the display screen.

[0022] Optionally, the light detection unit is located at the middle position of the first side of the display screen.

[0023] This disclosure also provides a computer device including a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program in the memory to implement any of the aforementioned eye-tracking methods.

[0024] This disclosure also provides a computer-readable storage medium including at least one instruction, which, when executed by a processor, performs any of the aforementioned eye-tracking methods.

[0025] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements any of the aforementioned eye-tracking methods.

[0026] The beneficial effects of the technical solutions provided in this disclosure include at least the following:

[0027] Based on a first distance from the eyeball to the target object, a first receiver corresponding to that first distance is selected to receive the infrared light reflected from the eyeball. The processing unit determines the eyeball position based on the intensity of the infrared light received by the first receiver. Compared to all receivers simultaneously receiving infrared light, where the processing unit images based on the intensity of the infrared light received by all receivers and identifies the eyeball position through image processing, determining the eyeball position based on the intensity of the infrared light received by the first receiver requires less data processing, places lower demands on the processing unit's processing power, and helps improve the target object's response speed. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the structure of an eye-tracking device provided in an embodiment of this disclosure;

[0030] Figure 2 This is a schematic diagram of a light detection unit provided in an embodiment of this disclosure;

[0031] Figure 3 This is a schematic diagram illustrating the relationship between the eye box area and the display device provided in an embodiment of the present disclosure;

[0032] Figure 4 This is a cross-sectional schematic diagram of a partial structure of a subunit provided in an embodiment of this disclosure;

[0033] Figure 5 It is a diagram of the human eye;

[0034] Figure 6 This is a schematic diagram illustrating the working process of an eye-tracking device provided in an embodiment of the present disclosure;

[0035] Figures 7-9 A schematic diagram illustrating the working principle of the eye-tracking device provided in this embodiment at different distances;

[0036] Figure 10 This is a flowchart of an eye-tracking method provided in an embodiment of this disclosure;

[0037] Figure 11 This is a schematic diagram of the structure of an eye-tracking device provided in an embodiment of this disclosure;

[0038] Figure 12 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0040] Eye-tracking devices can detect eye position so that a target object can change accordingly based on the eye position. Here, the target object can be a product with display capabilities, such as a glasses-free 3D display, AR (Augmented Reality) device, VR (Virtual Reality) device, HUD (Head-Up Display), or mobile terminal (e.g., mobile phone, tablet, and laptop), which can control changes in the displayed content based on eye position. Alternatively, the target object can be a product capable of performing actions according to user commands, such as a robot, which can perform corresponding actions based on eye position. This disclosure does not limit the type of target object.

[0041] In related technologies, eye-tracking devices may include a light-emitting unit, an infrared camera, and a processing unit. The light-emitting unit emits infrared light towards the area where the eyes are located. The infrared camera receives the infrared light reflected from the eye and forms an image. The processing unit determines the eye position based on the image formed by the infrared camera. The photosensitive chip of the infrared camera includes multiple infrared light sensors arranged in an array. All infrared light sensors simultaneously detect the intensity of the infrared light reflected from the eye to form an image. The processing unit needs to process the data detected by all infrared light sensors; the data volume is large, requiring high processing power and a slow response speed.

[0042] Therefore, this disclosure provides an eye-tracking device. Figure 1 This is a schematic diagram of the structure of an eye-tracking device provided in an embodiment of this disclosure. Figure 1 As shown, the eye-tracking device includes: a distance detection unit 10, a light-emitting unit 20, a light detection unit 30, and a processing unit 40. The distance detection unit 10 is used to detect a first distance from eyeball a to target object b. The light-emitting unit 20 is used to illuminate the target eye box area of ​​target object b with infrared light.

[0043] Figure 2 This is a schematic diagram of the structure of a light detection unit provided in an embodiment of this disclosure. (Combined with...) Figure 1 and Figure 2 The light detection unit 30 includes multiple sub-units 30a arranged in an array, and each sub-unit includes multiple receivers 31 arranged in an array. The receivers 31 are used to receive infrared light reflected from the eye. The processing unit 40 is used to control the first receiver in each sub-unit 30a (e.g., ...). Figure 2 The black square in the image receives infrared light reflected from the eyeball, and determines the position of the eyeball based on the intensity of the infrared light received by the first receiver. The first receiver is the receiver corresponding to a first distance.

[0044] Based on a first distance from the eyeball to the target object, a first receiver corresponding to that first distance is selected to receive the infrared light reflected from the eyeball. The processing unit determines the eyeball position based on the intensity of the infrared light received by the first receiver. Compared to all receivers simultaneously receiving infrared light, where the processing unit images based on the intensity of the infrared light received by all receivers and identifies the eyeball position through image processing, determining the eyeball position based on the intensity of the infrared light received by the first receiver requires less data processing, places lower demands on the processing unit's processing power, and helps improve the target object's response speed.

[0045] In this embodiment of the disclosure, the eye box area refers to the range of eye movement when a user views or controls a target object. When the target object is a display device, the user can see the complete image displayed by the display device when the eye moves within the eye box area. When the target object is a robot or other action performer, the user only needs to receive the infrared light reflected from the eyeball when the eye moves within the eye box area.

[0046] For example, the eye box area is a three-dimensional spatial area, which can be a cuboid shape, a cylinder shape, or an elliptical cylinder shape, etc.

[0047] Figure 3 This is a schematic diagram illustrating the relationship between the eye-box area and the display device provided in an embodiment of this disclosure. Figure 3 As shown, when the eye box area A is rectangular, the length direction of the eye box area A is the same as the length direction of the display screen of the display device, such as... Figure 3 In the x-direction, the width direction of the eye box region A is the same as the width direction of the display screen, such as... Figure 3 In the y-direction, the height direction of the eye box region A is the same as the optical axis direction of the display screen, such as... Figure 3 The z-direction in the equation.

[0048] In some examples, the length direction is horizontal, the width direction is vertical, and the height direction is perpendicular to both the horizontal and vertical directions.

[0049] For some display devices with a small viewing area, a display screen has an eye box region, which is the target eye box region. For example, a head-mounted display device (such as a VR or AR device) includes two displays, one for the left eye and the other for the right eye, each display screen having an eye box region. Exemplarily, the vertical centerline of the eye box region, the vertical centerline of the display screen, and the optical axis of the display screen are coplanar.

[0050] For display devices with a large viewing area, a single display screen can have multiple eye-box regions, with the target eye-box region being one of these regions. For example, in glasses-free 3D displays, a single screen is used by both eyes, and the viewer has a relatively large range of movement relative to the screen, typically moving from the left to the right. In this case, the viewing area can be divided into multiple eye-box regions arranged along a direction parallel to the horizontal center line of the display screen. The eye-box region corresponding to the position of the viewer's left or right eye is the target eye-box region. Since the rotation of the left and right eyeballs is usually synchronized, one eye can be selected for eyeball position detection.

[0051] In this embodiment, the number of sub-units included in the light detection unit can be set according to actual needs, for example, it may include 24 sub-units, which are arranged in 4 rows and 6 columns. It should be noted that... Figure 2 The arrangement shown is merely an example, and this disclosure does not impose any limitations on it.

[0052] In this embodiment, the distance detection unit 10 includes a depth camera, a laser distance detector, or an infrared ranging sensor, etc. The distance detection unit 10 can be integrated onto the target object b. In some examples, the distance detection unit 10 can be located on the periphery of the display area of ​​the display device, for example, it can be integrated into the bezel of the display device. Exemplarily, the detection accuracy of the distance detection unit 10 can be at the millimeter level.

[0053] The eye box area is divided into multiple sub-eye box areas A1 along the direction from the eyeball to the display screen, that is, the eye box area is divided into multiple sub-eye box areas A1 according to the distance from the eyeball to the display screen. Each sub-eye box area A1 corresponds to a different distance.

[0054] To prevent the infrared light emitted by the light-emitting unit from directly entering the receiver and affecting the detection results, the light detection unit 30 can be located between the eye box area and the light-emitting unit 20 in the user's viewing direction, or in the optical axis O direction of the display device. Figure 1 As shown.

[0055] In some examples, the light-emitting unit 20 includes multiple light-emitting devices, such as infrared LEDs (Light Emitting Diodes). The wavelength of the infrared light emitted by the light-emitting devices can be between 800 and 1200 nm. Figure 3 As shown, when the target object b is a display device, multiple light-emitting devices can be arranged at intervals around the display area b1 of the display device.

[0056] In some examples, the light-emitting device is integrated into the display screen of the display device. For example, the light-emitting device is located in the non-display area b2 of the display screen and is manufactured synchronously with the light-emitting units in the display area of ​​the display screen. Another example is that the light-emitting device is mounted on the bezel of the display device. Yet another example is that the light-emitting device is located in other components surrounding the display screen (e.g., the stand of a head-mounted display).

[0057] When the target object is a display device, the light detection unit 30 can be located in the middle of the first side of the display screen, which can be the top or bottom edge. For example... Figure 3 In this configuration, the light detection unit 30 is located in the center of the top edge of the display screen. Since users typically view the display screen from the center of the display device, placing the light detection unit in the center of the first side makes it easier to detect the eye position.

[0058] In this embodiment, each subunit 30a contains the same number and arrangement of receivers 31; that is, multiple receivers 31 in one subunit 30a can be obtained by translating multiple receivers 31 in another subunit 30a. Exemplarily, all receivers 31 in each subunit 30a are arranged in an array. The array arrangement can be a single array, for example, 36 receivers arranged in 4 rows and 9 columns; or, the array arrangement can be formed by splicing together multiple subarrays, for example, as shown in... Figure 2 As shown, each sub-unit 30a includes 36 receivers. These 36 receivers are arranged in 5 rows, with 6 receivers in each of the first and last rows (each row is a subarray), and 8 receivers in each of the middle three rows (each row is a subarray).

[0059] In this embodiment of the disclosure, different receivers 31 in each subunit 30a correspond to different distances. In two subunits 30a, two receivers 31 corresponding to the same distance can be arranged in different positions. For example, in the first subunit, the receiver corresponding to the first distance is located in the first row, and in the second subunit, the receiver corresponding to the first distance is located in the second row, second in the second subunit. Alternatively, in two subunits 30a, two receivers 31 corresponding to the same distance may also be arranged in the same position. The arrangement of receivers corresponding to the same distance in different subunits can be determined by the optical path of the infrared light reflected from the eyeball, as long as it can be ensured that each subunit has a receiver capable of receiving infrared light reflected from the eyeball at different distances.

[0060] Figure 4 This is a cross-sectional schematic diagram of a partial structure of a subunit provided in an embodiment of this disclosure. For example... Figure 4 As shown, the receiver 31 may include an aperture 311 and an infrared light sensor 312. Each aperture 311 corresponds to one infrared light sensor 312. Each aperture 311 is used to control whether the infrared light reflected by the eyeball can enter the corresponding infrared light sensor 312. For example, the aperture 311 and the corresponding infrared light sensor 312 are arranged sequentially in a direction parallel to the optical axis of the display screen.

[0061] In some examples, the aperture 311 is a liquid crystal aperture. The processing unit is electrically connected to the electrodes of the liquid crystal aperture and generates an electric field by controlling the voltage applied to the electrodes. This electric field controls the deflection direction of the liquid crystal in the liquid crystal aperture, thereby controlling the transmittance of the liquid crystal aperture. When the transmittance of the liquid crystal aperture is at its highest, the aperture is in the open state, and the infrared light reflected from the eye can be detected by the corresponding infrared light sensor. When the transmittance of the liquid crystal aperture is at its lowest, the aperture is in the closed state, and the infrared light reflected from the eye cannot be detected by the corresponding infrared light sensor.

[0062] In other examples, aperture 311 is a microelectromechanical scanning mirror (MEMS). A MEMS scanning mirror comprises multiple micromirrors, each belonging to a receiver and driven by a micromotor. The processing unit supplies voltage to the micromotor to drive its rotation, thereby rotating the corresponding micromirror. When the micromirror is parallel to the display surface, it reflects infrared light from the eye to the light detection unit. In this case, the aperture is closed, and the infrared light reflected from the eye cannot be detected by the corresponding infrared sensor. When the micromirror is at an angle to the display surface, the aperture is open, and at least a portion of the infrared light reflected from the eye to the light detection unit can be incident on the corresponding infrared sensor and thus detected.

[0063] In practice, a liquid crystal aperture or a microelectromechanical scanning mirror can be attached to the glass cover of the display screen.

[0064] In some examples, the infrared light sensor 312 is fabricated directly on the display substrate of the display screen. For example, it can be fabricated simultaneously with the light-emitting devices in the display area of ​​the display screen, or it can be fabricated separately before or after the light-emitting devices in the display area of ​​the display screen. Integrating the infrared light sensor onto the display substrate facilitates the integration of the eye-tracking device with the display screen and helps to reduce the size of the display device.

[0065] In other examples, all the infrared light sensors of multiple sub-units 30a are integrated on a single chip and then fixed to the glass cover of the display.

[0066] In some examples, each subunit 30a also includes a microlens 32 for focusing infrared light reflected from the eye at a first distance onto a first receiver in the subunit 30a. Optionally, the microlenses 32 in the multiple subunits 30a can be an integral structure or a separate structure.

[0067] For example, the microlens 32 can be a solid lens or a non-solid lens. When the microlens 32 is a solid lens, it can be made of glass or resin materials. When the microlens 32 is a non-solid lens, it can be made of liquid crystal materials or liquid materials.

[0068] When the microlens 32 is a solid lens, the surface shape of the microlens 32 includes, but is not limited to, spherical, aspherical, Fresnel, and freeform surfaces.

[0069] Optionally, the microlens 32 can be a single-layer structure or a combination of multiple lenses. When the microlens is a combination of multiple lenses, adjacent lenses can be bonded together with adhesive.

[0070] In this embodiment of the disclosure, the microlens 32 can be at the micrometer level, while the infrared light sensor can be at the 0.1 micrometer level.

[0071] When the light emission direction of the light-emitting unit, the structure and related parameters of the microlens, the position of the eye box region relative to the display screen, and the structure and position of the light detection unit relative to the display screen are all determined, the light path direction of the infrared light after reflection at various positions in the eye box region can be simulated by computer. Combined with the light path direction of the infrared light after passing through the microlens, the position of the receiver corresponding to each distance in each sub-unit can be determined. For example, assuming the eyeball is at a certain distance, the propagation path of the infrared light at this time can be simulated, and the receiver incident on each sub-unit can be determined as the receiver corresponding to that distance. This process is repeated for all distances to determine the receiver corresponding to each distance.

[0072] If, under the current configuration, it cannot be guaranteed that each sub-unit has a receiver corresponding to all distances, then parameters such as the microlens shape need to be adjusted.

[0073] Optionally, each subunit 30a further includes a light-shielding structure 33 located between any two adjacent receivers 31. The light-shielding structure 33 prevents light entering the first receiver from entering the second receiver adjacent to the first receiver during propagation, which would cause the second receiver to also output a signal, affecting the accuracy of the detection result.

[0074] For example, the light-shielding structure 33 can be located at at least one of the following locations: surrounding the aperture, surrounding the infrared sensor, and surrounding the portion between the aperture and the infrared sensor. When the light-shielding structure surrounds the aperture, the infrared sensor, and the portion between them, the individual receivers can be completely isolated, avoiding crosstalk. However, to simplify the manufacturing process, the light-shielding structure can also surround only the aperture and / or the portion between the aperture and the infrared sensor.

[0075] It should be noted that, Figure 4 The diagram shows a portion of the cross-sectional structure of a sub-unit; therefore, in Figure 4 The light-shielding structures 33 surrounding the same receiver 31, as seen in the image, are separate.

[0076] Optionally, the light-shielding structure 33 can be made of a material with a wide infrared absorption band, covering the 800nm ​​to 1200nm band and even wider bands. For example, it can be made of the same material as the screen BM (Black Matrix) or other similar functional materials.

[0077] In this embodiment of the disclosure, the processing unit 40 pre-stores a correspondence between distances and receivers. Exemplarily, in this correspondence, each distance corresponds to a set of receiver identifiers; different distances correspond to different receiver identifiers. The receiver identifier can be obtained by combining the sub-unit number and the receiver's position within the sub-unit (e.g., row and column number); or, the receiver identifier can be obtained by combining the sub-unit number and the receiver's position within the sub-unit.

[0078] When the processing unit 40 acquires the first distance, it determines the first receiver corresponding to the first distance based on the correspondence, and controls the first receiver to work (i.e., to receive infrared light reflected from the eyeball).

[0079] Figure 5 This is a diagram of the human eye. (For example...) Figure 5As shown, the human eye 50 includes an eyeball, which includes a pupil 51 and an iris 52 surrounding the pupil 51. The pupil 51 and the iris 52 have different reflectivities for infrared light. The reflectivity of the pupil 51 for infrared light is much lower than that of the iris 52. In this embodiment of the present disclosure, the processing unit 40 uses this principle to identify the position of the eyeball.

[0080] In this embodiment, each eye-box region is divided into multiple sub-regions arranged in an array. Each sub-region corresponds one-to-one with a sub-unit. The resolution of the eye position detected by the eye-tracking device (i.e., the number of eye positions that can be detected per unit area) can be represented by the number of sub-regions contained in the eye-box region. The number of sub-regions contained in the eye-box region is limited by the manufacturing precision of the components contained in each sub-unit. With a fixed manufacturing precision for the sub-units, a one-to-one correspondence between multiple sub-regions and multiple sub-units maximizes the resolution of the eye-tracking device.

[0081] Alternatively, in other embodiments, multiple sub-units may correspond to one sub-region, for example, each pair of adjacent sub-units in each row or column may correspond to one sub-region.

[0082] Here, the correspondence between sub-units and sub-regions means that when the eyeball is located in a certain sub-region, the pupil position determined by the infrared light reflected from the eyeball is located within the sub-unit corresponding to that sub-region.

[0083] In this embodiment of the present disclosure, the processing unit 40 is used to determine the position of the pupil in the eyeball based on the light intensity of the infrared light received by the first receiver; and to determine the target sub-region corresponding to the sub-unit to which the position of the pupil belongs as the position of the eyeball based on the correspondence between multiple sub-units and multiple sub-regions.

[0084] For example, the processing unit 40 is used to determine the position of the pupil as the position of a first receiver that meets the following conditions: the intensity of the received infrared light is less than a first light intensity threshold, and there are multiple first receivers in the vicinity where the intensity of the received infrared light is greater than a second light intensity threshold. Here, the second light intensity threshold is greater than the first light intensity threshold, for example, the second light intensity threshold is 8 to 20 times the first light intensity threshold, such as 8 times, 10 times, etc.

[0085] Because the pupil reflects infrared light energy weakly, while the iris reflects infrared light strongly, the intensity of the infrared light received by the receiver that receives infrared light reflected from the pupil is weak (i.e., below the first intensity threshold), while the intensity of the infrared light received by the receiver that receives infrared light reflected from the iris is relatively high (i.e., above the second intensity threshold). By comparing the intensity data output by each receiver, data points with strong reflected light energy in the surrounding area and weak reflected light energy in the center can be determined; then, by tracing the receiver corresponding to that data point, the position of the pupil can be obtained.

[0086] When the distance between the eyeball and the target object changes, for example, to a second distance, the processing unit controls the first receiver in the multiple sub-units corresponding to the second distance to receive the infrared light reflected by the eyeball, and controls the second receiver in the multiple sub-units other than the first receiver not to receive the infrared light reflected by the eyeball; and re-determines the position of the eyeball based on the light intensity of the infrared light received by the first receiver, thereby enabling eyeball position tracking.

[0087] In one possible implementation, the processing unit 40 is further configured to control the second receiver in each subunit 30a to not receive infrared light reflected from the eyeball. Here, the second receiver is a receiver other than the first receiver, such as... Figure 2 The white square in the image. The processing unit 40 controls the second receiver to prevent it from receiving infrared light reflected from the eye by: not providing a driving voltage to the aperture of the second receiver, thus keeping the aperture of the second receiver closed and preventing the second receiver from receiving infrared light reflected from the eye. Since it is not necessary to drive the aperture of the second receiver, the power consumption of the eye-tracking device can be further reduced. Furthermore, closing the aperture of the second receiver prevents the influence of stray light.

[0088] In another possible implementation, the processing unit 40 is also used to control the second receiver in each subunit 30a to receive infrared light reflected from the eyeball. However, during data processing, the processing unit 40 first filters out the light intensity corresponding to the first receiver at the first distance based on the correspondence between distance and receiver, and then determines the position of the eyeball based on the intensity of the infrared light received by the first receiver.

[0089] Optionally, when the target object has multiple eye box regions, the processing unit is further configured to determine the target eye box region based on the eye position corresponding to the eyeball, so as to obtain the correspondence between multiple sub-regions and multiple sub-units in the target eye box region.

[0090] In this embodiment of the disclosure, the eye position can be determined based on images captured by a camera. In this case, the eye-tracking device further includes an eye position determination unit for determining the eye position. This eye position determination unit includes a camera or the like. The camera is used to capture images within the viewing area of ​​the display screen, and then, based on these images, determines the position of the viewer's eyes in a plane perpendicular to the optical axis of the display screen.

[0091] The above explanation uses the eyeball position within the eye box region as an example. In some examples, the eyeball position can also refer to the eye's gaze position on the display screen. Since there is a correspondence between sub-regions within the eye box region and sub-display regions within the display screen's display area—meaning that when the human eye is located in a certain sub-region within the eye box, it will focus on the image displayed in the corresponding sub-display region—the method can further include: determining the eye's gaze position on the display screen based on the correspondence between the sub-regions within the eye box region and the sub-display regions within the display screen's display area.

[0092] Alternatively, a correspondence between sub-units and sub-display areas can be established based on the correspondence between sub-regions and sub-units in the eye box area, and the correspondence between sub-regions in the eye box area and sub-display areas in the display area of ​​the screen. Then, based on the correspondence between sub-units and sub-display areas, the gaze position of the eyeball on the display screen can be determined directly based on the position of the pupil.

[0093] Figure 6 This is a schematic diagram illustrating the working process of an eye-tracking device provided in an embodiment of this disclosure. Figure 6 As shown, the working process includes: In S61, the distance detection unit detects the distance from the eye to the target object, for example, h1, h2...hn. In S62, the processing unit controls the aperture of the receiver corresponding to the target distance to open, for example, the receivers corresponding to distance h1 are 1-1, 2-1...mn; the receivers corresponding to distance h2 are 2-1, 2-2...2-n, and so on. In S63, the sensor of the receiver corresponding to the target distance receives infrared light, and a portion of the receivers corresponding to the target distance receive stronger infrared light (e.g.,...). Figure 6 The receiver shown in the black box receives one portion of infrared light, while the other portion receives weaker (almost none) infrared light. In S64, the processing unit determines the location where the surrounding infrared light is strong and the central infrared light is weak. In S65, the processing unit determines the location of the eyeball at the target distance based on this location.

[0094] Figures 7-9 This is a schematic diagram illustrating the working principle of the eye-tracking device provided in the embodiments of this disclosure at different distances.

[0095] like Figure 7 As shown, when the distance between the person and the display screen is L1, the distance detection unit outputs distance L1 to the processing unit 40. The processing unit controls the aperture of the receiver corresponding to distance L1 in each sub-unit to open and close the apertures of other receivers in each sub-unit. The sensor of the receiver corresponding to distance L1 receives infrared light. The processing unit determines the eye position based on the infrared light received by the receiver corresponding to distance L1.

[0096] like Figure 8 As shown, when the distance between the person and the display screen is L2, the distance detection unit outputs the distance L2 to the processing unit 40. The processing unit controls the aperture of the receiver corresponding to distance L2 in each sub-unit to open and close the apertures of other receivers in each sub-unit. The sensor of the receiver corresponding to distance L2 receives infrared light. The processing unit determines the eye position based on the infrared light received by the receiver corresponding to distance L2.

[0097] like Figure 9 As shown, when the distance between the person and the display screen is Ln, the distance detection unit outputs the distance Ln to the processing unit. The processing unit 40 controls the aperture of the receiver corresponding to distance Ln in each sub-unit to open and close the apertures of other receivers in each sub-unit. The sensor of the receiver corresponding to distance Ln receives infrared light. The processing unit determines the eye position based on the infrared light received by the receiver corresponding to distance Ln.

[0098] from Figures 7-9 As can be seen, for each distance corresponding to multiple receivers, some receive infrared light, while others do not. For example, Figure 7 In the middle, receivers 1-1 and 2-1 receive infrared light, while receiver m-1 does not receive infrared light; Figure 8 In the middle, receivers 1-2 and 2-2 receive infrared light, while receiver m-2 does not receive infrared light.

[0099] In addition, from Figures 7-9 It can also be seen that the receivers for different distances are different in each sub-unit. For example, Figure 7 In the diagram, the receivers corresponding to distance L1 are 1-1, 2-1...m-1; Figure 8 In the diagram, the receivers corresponding to distance L2 are 1-2, 2-2...m-2; Figure 9 In the diagram, the receivers corresponding to distances Ln are 1-n, 2-n...mn, where m represents the m-th sub-unit.

[0100] It should be noted that, Figure 7-9 The spacing between receivers in the image is for illustrative purposes only.

[0101] This disclosure also provides an eye-tracking method. This eye-tracking method can be executed by the aforementioned processing unit. Figure 10 This is a schematic flowchart illustrating the eye-tracking method provided in an embodiment of this disclosure. Figure 10 As shown, the method includes:

[0102] In S101, the first distance from the eyeball to the target object is obtained.

[0103] This first distance can be obtained through the aforementioned distance detection unit.

[0104] In S102, the first receiver in the multiple sub-units is controlled to receive infrared light reflected from the eyeball.

[0105] The array comprises multiple sub-units, and each sub-unit includes multiple receivers arranged in the array. The first receiver is the receiver in the sub-unit that corresponds to the first distance.

[0106] Optionally, S102 further includes controlling a second receiver in one of the multiple sub-units not to receive infrared light reflected from the eyeball, the second receiver being a receiver in the sub-unit other than the first receiver.

[0107] In S103, the position of the eyeball is determined based on the intensity of the infrared light received by the first receiver.

[0108] In some examples, S103 includes: first, determining the position of the pupil in the eyeball based on the light intensity of the infrared light received by the first receiver; second, determining the target sub-region corresponding to the sub-unit to which the pupil's position belongs as the position of the eyeball based on the correspondence between multiple sub-units and multiple sub-regions.

[0109] For example, in the first step, the position of the first receiver that meets the following conditions is determined as the position of the pupil: the intensity of the received infrared light is less than a first light intensity threshold, and there are multiple first receivers in the vicinity where the intensity of the received infrared light is greater than a second light intensity threshold.

[0110] When the distance between the eyeball and the target object changes, for example, to a second distance, the first receiver in the multiple sub-units corresponding to the second distance is controlled to receive the infrared light reflected by the eyeball, and the second receiver in the multiple sub-units other than the first receiver is controlled not to receive the infrared light reflected by the eyeball; and the position of the eyeball is re-determined based on the light intensity of the infrared light received by the first receiver, thereby enabling eyeball position tracking.

[0111] When the target object's display screen has multiple eye box areas, the method further includes: determining the target eye box area based on the eye position corresponding to the eyeball.

[0112] It should be noted that the eye-tracking method and the aforementioned eye-tracking device are based on the same concept. For relevant details, please refer to the aforementioned eye-tracking device. Detailed descriptions are omitted here.

[0113] This disclosure also provides a display device. For example... Figure 3 As shown, the display device includes: a display screen b and any of the eye tracking devices in the aforementioned embodiments.

[0114] Optionally, the light-emitting unit 20 includes a plurality of LEDs, which are arranged at intervals around the display area b1 of the display screen b.

[0115] Optionally, the display device can be a glasses-free 3D display, an AR device, or a VR device. AR devices include, but are not limited to, head-mounted AR devices (such as AR glasses or AR helmets) or HUDs. Alternatively, the display device can also be a terminal with display capabilities, such as a mobile phone, tablet, desktop monitor, or laptop.

[0116] Optionally, the light detection unit 30 is located in the middle of the first side of the display screen b.

[0117] For example, the present disclosure does not limit the type of display screen b, which may be a liquid crystal display screen, an OLED (Organic Light-Emitting Diode) display screen, an LED display screen, a Micro-LED display screen, a micro OLED display screen, a mini OLED display screen, etc.

[0118] This disclosure also provides an eye-tracking device. For example... Figure 11 As shown, the eye-tracking device 1100 includes: an acquisition module 1101, a control module 1102, and a determination module 1103. The acquisition module 1101 acquires a first distance from the eye to a target object. The control module 1102 controls a first receiver in a plurality of sub-units to receive infrared light reflected from the eye and controls a second receiver in the plurality of sub-units not to receive infrared light reflected from the eye. The plurality of sub-units are arranged in an array, and each sub-unit includes multiple receivers arranged in an array. The first receiver is the receiver in the sub-unit corresponding to the first distance, and the second receiver is the receiver in the sub-unit other than the first receiver. The determination module 1103 determines the position of the eye based on the intensity of the infrared light received by the first receiver.

[0119] In some examples, the determination module 1103 includes a pupil position determination submodule 11031 and an eyeball position determination submodule 11032. The pupil position determination submodule 11031 is used to determine the position of the pupil in the eyeball based on the intensity of the infrared light received by the first receiver; the eyeball position determination submodule 11032 is used to determine the target sub-region corresponding to the sub-unit to which the pupil's position belongs as the position of the eyeball based on the correspondence between multiple sub-units and multiple sub-regions.

[0120] For example, the pupil position determination submodule 11031 is used to determine the position of the first receiver that meets the following conditions as the position of the pupil: the intensity of the received infrared light is less than a first light intensity threshold, and there are multiple first receivers in the vicinity where the intensity of the received infrared light is greater than a second light intensity threshold.

[0121] When the display screen of the target object has multiple eye box areas, the determining module 1103 is also used to determine the target eye box area based on the eye position corresponding to the eyeball.

[0122] It should be noted that the eye-tracking device provided in the above embodiments is only illustrated by the division of the functional modules described above when performing eye tracking. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the eye-tracking device and the eye-tracking method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the eye-tracking device and method embodiments, which will not be repeated here.

[0123] Figure 12 This is a structural block diagram of a computer device provided in an embodiment of this disclosure. For example... Figure 12 As shown, the computer device 1200 includes a processor 1201 and a memory 1202.

[0124] Processor 1201 may include one or more processing cores, such as a 5-core processor or an 8-core processor. Processor 1201 may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). Processor 1201 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state.

[0125] The memory 1202 may include one or more computer-readable storage media, which may be non-transitory. The memory 1202 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1202 are used to store at least one instruction, which is executed by the processor 1201 to implement the eye-tracking method provided in the embodiments of this disclosure.

[0126] Those skilled in the art will understand that Figure 12The structure shown does not constitute a limitation on the computer device 1200 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0127] This disclosure also provides a non-transitory computer-readable storage medium, wherein when instructions in the storage medium are executed by a processor of a computer device, the computer device is able to perform the eye-tracking method provided in this disclosure.

[0128] This disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the eye-tracking method provided in this disclosure.

[0129] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” “third,” and similar terms used in this patent application specification and claims do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms “comprising” or “including” and similar terms mean that the elements or objects preceding “comprising” or “including” encompass the elements or objects listed following “comprising” or “including” and their equivalents, but do not exclude other elements or objects.

[0130] The above is not intended to limit this disclosure in any way. Although this disclosure has been disclosed above through embodiments, it is not intended to limit this disclosure. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this disclosure. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this disclosure without departing from the content of the technical solution of this disclosure shall still fall within the scope of the technical solution of this disclosure.

Claims

1. An eye-tracking device, characterized in that, The eye-tracking device includes: a distance detection unit, a light-emitting unit, a light detection unit, and a processing unit; The distance detection unit is used to detect the first distance from the eyeball to the target object; The light-emitting unit is used to illuminate the target eye box area of ​​the target object with infrared light, and the target eye box area includes multiple sub-regions arranged in an array; The light detection unit includes multiple sub-units arranged in an array, and each of the multiple sub-units includes multiple receivers arranged in an array, the multiple receivers being used to receive infrared light reflected by the eyeball; The processing unit is used to control the first receiver in the subunit to receive infrared light reflected by the eyeball, determine the position of the pupil in the eyeball according to the light intensity of the infrared light received by the first receiver, and determine the target sub-region corresponding to the subunit to which the position of the pupil belongs as the position of the eyeball according to the correspondence between the plurality of subunits and the plurality of sub-regions. Wherein, the first receiver is the receiver corresponding to the first distance.

2. The eye-tracking device according to claim 1, characterized in that, The receiver includes an aperture and an infrared light sensor; The aperture is used, under the control of the processing unit, to control whether infrared light can enter the corresponding infrared light sensor.

3. The eye-tracking device according to claim 2, characterized in that, The aperture is a liquid crystal aperture or a microelectromechanical scanning mirror.

4. The eye-tracking device according to any one of claims 1 to 3, characterized in that, The number and arrangement of receivers contained in the multiple sub-units are the same; In at least two of the plurality of sub-units, the receivers corresponding to the same distance are arranged in the same or different positions.

5. The eye-tracking device according to claim 4, characterized in that, The subunit also includes a microlens for focusing the infrared light reflected from the eyeball onto a first receiver in the subunit.

6. The eye-tracking device according to claim 4, characterized in that, The subunit also includes a light-shielding structure located between any two adjacent receivers.

7. The eye-tracking device according to claim 1, characterized in that, The processing unit is used to determine the position of the pupil as the position of the first receiver that meets the following conditions: The intensity of the received infrared light is less than a first intensity threshold, and there are multiple first receivers in the vicinity where the intensity of the received infrared light is greater than a second intensity threshold.

8. The eye-tracking device according to claim 7, characterized in that, The target object includes a display screen, and the target object has multiple eye box areas, which are arranged along a direction parallel to the horizontal center line of the display screen, and the target eye box area is one of the eye box areas; The processing unit is further configured to determine the target eye box region based on the eye position corresponding to the eyeball.

9. The eye-tracking device according to any one of claims 1 to 3 and claims 5 to 8, characterized in that, The processing unit is also configured to control the second receiver in the subunit not to receive the infrared light reflected by the eyeball, wherein the second receiver is a receiver other than the first receiver.

10. An eye-tracking method, characterized in that, The eye-tracking method includes: Obtain the first distance from the eyeball to the target object; The system controls a first receiver in a plurality of sub-units to receive infrared light reflected from the eyeball in the target eye box region of the target object. The plurality of sub-units are arranged in an array, and each of the plurality of sub-units includes a plurality of receivers arranged in an array. The first receiver is the receiver in the sub-unit that corresponds to the first distance. The target eye box region includes a plurality of sub-regions arranged in an array. Based on the intensity of the infrared light received by the first receiver, the position of the pupil in the eyeball is determined; and based on the correspondence between the plurality of sub-units and the plurality of sub-regions, the target sub-region corresponding to the sub-unit to which the position of the pupil belongs is determined as the position of the eyeball.

11. The eye-tracking method according to claim 10, characterized in that, Determining the position of the pupil in the eyeball based on the intensity of the infrared light received by the first receiver includes: The position of the first receiver that meets the following conditions is determined as the position of the pupil: The intensity of the received infrared light is less than a first intensity threshold, and there are multiple first receivers in the vicinity where the intensity of the received infrared light is greater than a second intensity threshold.

12. The eye-tracking method according to claim 11, characterized in that, The target object includes a display screen, the target object has multiple eye box regions, the multiple eye box regions are arranged along a direction parallel to the horizontal center line of the display screen, and the target eye box region is one of the eye box regions; the method further includes: The target eye box area is determined based on the eye position corresponding to the eyeball.

13. A display device, characterized in that, The display device includes: The display screen and the eye-tracking device as described in any one of claims 1 to 9; Both the light-emitting unit and the light-detecting unit are located on the periphery of the display area of ​​the display screen.

14. The display device according to claim 13, characterized in that, The display device is a glasses-free 3D display, an augmented reality (AR) device, or a virtual reality (VR) device.

15. The display device according to claim 13 or 14, characterized in that, The light-emitting unit includes multiple light-emitting diodes (LEDs), which are arranged at intervals around the display area of ​​the display screen.

16. The display device according to claim 13 or 14, characterized in that, The light detection unit is located in the middle of the first side of the display screen.

17. A computer device, characterized in that, It includes a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program in the memory to implement the eye-tracking method as described in any one of claims 10 to 12.

18. A computer-readable storage medium, characterized in that, The storage medium includes at least one instruction, which, when executed by a processor, implements the eye-tracking method as described in any one of claims 10 to 12.

Images