Near-to-eye display device and eye movement tracking method

Abstract

The embodiment of the invention provides near-to-eye display equipment and an eye movement tracking method. The near-to-eye display device comprises an optical waveguide, a light source assembly, an image acquisition unit and a processing unit, the optical waveguide comprises a plurality of first coupling-out structures, and the first coupling-out structures can be switched between a coupling-out state and a non-coupling-out state; the light source assembly is used for emitting infrared light towards the optical waveguide; the image acquisition unit is used for acquiring an eye image; the processing unit is electrically connected with the optical waveguide and the image acquisition unit, and is used for determining the position of a preset eye region according to the eye image; determining a target coupling-out structure in the plurality of first coupling-out structures according to the position of the eye preset area; the target coupling-out structure is controlled to be in a coupling-out state, so that the infrared light is coupled out of the target coupling-out structure to a pupil area of the eye; and determining the gazing direction of the eye according to the light spot formed by the infrared light in the image of the eye. The technical scheme of the embodiment of the invention aims to enlarge the eye movement tracking range, so that the infrared light is accurately transmitted to the pupil area of the eye, and the eye movement tracking precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of eye tracking technology, and in particular to a near-eye display device and an eye tracking method. Background Art

[0002] Near-eye display devices are usually equipped with an eye tracking function to achieve human-computer interaction by tracking the movement of the human eye. In the related art, infrared lamp beads are used to provide infrared light. The infrared light is reflected after irradiating the human eye, and the camera collects the image of the human eye, so as to judge the gaze direction of the human eye based on the collected image of the human eye to achieve eye tracking. However, the infrared light irradiation range provided by the infrared lamp beads is fixed. When the human eye looks straight ahead, the infrared light irradiates the pupil of the human eye and forms a clear bright spot on the cornea. The camera can collect a clear image of the human eye. When the human eye looks at the edge, the infrared light irradiates the sclera for diffuse reflection, forming speckles on the sclera, which makes it difficult to achieve high-precision eye tracking of the human eye image collected by the camera. That is, the range of eye tracking is limited. Summary of the invention

[0003] Embodiments of the present invention provide a near-eye display device and an eye tracking method, which aim to expand the range of eye tracking, enable infrared light to be accurately transmitted to the pupil area of ​​the eye, and improve the accuracy of eye tracking.

[0004] In a first aspect, an embodiment of the present invention provides a near-eye display device, including:

[0005] An optical waveguide, the optical waveguide comprising a plurality of first outcoupling structures, the first outcoupling structures being switchable between an outcoupling state and a non-outcoupling state;

[0006] A light source assembly, configured to emit infrared light toward the light waveguide;

[0007] An image acquisition unit, used for acquiring images of the eyes;

[0008] A processing unit is electrically connected to the optical waveguide and the image acquisition unit, and the processing unit is used to:

[0009] Determining the position of a preset eye area according to the eye image;

[0010] Determining a target out-coupling structure among the plurality of first out-coupling structures according to the position of the preset eye area;

[0011] Controlling the target outcoupling structure to be in the outcoupling state so that the infrared light is coupled out from the target outcoupling structure to the pupil area of ​​the eye;

[0012] The gaze direction of the eye is determined according to a light spot formed by the infrared light in the image of the eye.

[0013] Optionally, the preset eye area includes at least one of the following: a pupil area, a cornea area;

[0014] The distance between the position of the target outcoupling structure and the pupil area is smaller than the distance between other first outcoupling structures in the plurality of first outcoupling structures and the pupil area; and / or

[0015] The distance between the position of the target out-coupling structure and the corneal region is smaller than the distance between other first out-coupling structures among the plurality of first out-coupling structures and the corneal region.

[0016] Optionally, the preset eye area includes the pupil area, cornea area and sclera area;

[0017] The step of determining the position of the preset eye area according to the eye image comprises:

[0018] Determining the positional relationship between the pupil area, the cornea area, and the sclera area according to the image of the eye; and determining the target coupling structure among the plurality of first coupling structures according to the position of the preset eye area, comprising:

[0019] The target out-coupling structure is determined among the plurality of first out-coupling structures according to the positional relationship among the pupil area, the cornea area, and the sclera area.

[0020] Optionally, when determining a target out-coupling structure from among the plurality of first out-coupling structures according to the position of the preset eye area, the processing unit is configured to:

[0021] Based on the correspondence between the preset first coupling-out structures and the positions of the preset eye areas, a target coupling-out structure is determined among the plurality of first coupling-out structures according to the positions of the preset eye areas.

[0022] Optionally, the processing unit is further configured to control a coupling-out angle of the infrared light when the target coupling-out structure is in the coupling-out state.

[0023] Optionally, the first out-coupling structure comprises a first electrode layer, a birefringent material layer and a second electrode layer, and the birefringent material layer is located between the first electrode layer and the second electrode layer;

[0024] The processing unit is electrically connected to the first electrode layer and the second electrode layer, and controls the first out-coupling structure to switch to the out-coupling state or the non-out-coupling state by adjusting the voltage applied to the first electrode layer and the second electrode layer.

[0025] Optionally, the first out-coupling structure further includes an orientation layer, and the orientation layer is provided between the first electrode layer and the birefringent material layer.

[0026] Optionally, the light source assembly is further used to emit visible light toward the optical waveguide, and the optical waveguide further comprises:

[0027] The first waveguide body is provided with a second out-coupling structure and a plurality of the first out-coupling structures, wherein the second out-coupling structure is used to couple out the visible light coupled into the optical waveguide.

[0028] Optionally, the optical waveguide further comprises a first coupling structure, the first coupling structure is provided on the first waveguide body, and the visible light and the infrared light emitted by the light source assembly are coupled into the first waveguide body through the first coupling structure; or

[0029] The optical waveguide includes a first coupling structure and a second coupling structure, both of which are arranged on the first waveguide body, the visible light is coupled into the first waveguide body through the first coupling structure, and the infrared light is coupled into the first waveguide body through the second coupling structure.

[0030] Optionally, the first outcoupling structure and the second outcoupling structure are located on the same side of the first waveguide body, and a plurality of the first outcoupling structures are arranged around a central area of ​​the second outcoupling structure; or

[0031] The first out-coupling structure and the second out-coupling structure are located on opposite sides of the first waveguide body.

[0032] Optionally, the light source assembly is further used to emit visible light toward the optical waveguide, and the optical waveguide further comprises:

[0033] A first waveguide body, provided with a first coupling-in structure and a second coupling-out structure, wherein the visible light is coupled into the first waveguide body through the first coupling-in structure, and is transmitted to the second coupling-out structure and then coupled out through the second coupling-out structure;

[0034] The second waveguide body is provided with a second coupling-in structure and a plurality of the first coupling-out structures, and the infrared light is coupled into the second waveguide body through the second coupling-in structure.

[0035] In a second aspect, an embodiment of the present invention further provides an eye tracking method for a near-eye display device, wherein the near-eye display device includes a light source assembly, an optical waveguide, and an image acquisition unit, wherein the optical waveguide includes a plurality of first outcoupling structures, wherein the first outcoupling structures can be switched between an outcoupling state and a non-outcoupling state, wherein the light source assembly is used to emit infrared light toward the optical waveguide, and the image acquisition unit is used to acquire an image of the eye;

[0036] The eye tracking method comprises:

[0037] acquiring an image of the eye;

[0038] Determine the position of a preset eye area according to the image;

[0039] Determining a target out-coupling structure among the plurality of first out-coupling structures according to the position of the preset eye area;

[0040] Controlling the target outcoupling structure to be in the outcoupling state so that the infrared light is coupled out from the target outcoupling structure to the pupil area of ​​the eye;

[0041] acquiring the image of the eye again;

[0042] The gaze direction of the eye is determined according to the light spot formed by the infrared light in the image acquired again.

[0043] The embodiment of the present invention provides a near-eye display device and an eye tracking method, wherein an image acquisition unit acquires an image of the eye to determine the position of a preset eye area, and then determines a target coupling structure among a plurality of first coupling structures according to the position of the preset eye area, and controls the target coupling structure to be in a coupling state, so that the infrared light emitted by the light source assembly toward the optical waveguide is coupled to the pupil area of ​​the eye through the target coupling structure in the coupling state, so as to form a clear light spot on the cornea of ​​the eye, so that the gaze direction of the eye can be determined according to the image of the eye with the light spot acquired again by the image acquisition unit. Among them, the infrared light can cover a wider range after being transmitted inside the optical waveguide and coupled out from the plurality of first coupling structures, which is conducive to increasing the range of eye tracking, and the position of the preset eye area is determined according to the eye image acquired by the image acquisition unit, so that the optical waveguide can determine the target coupling structure according to the position of the preset eye area, so that the target coupling structure couples out the infrared light, which is conducive to improving the transmission position accuracy of the infrared light, and the coupled infrared light is accurately transmitted to the cornea of ​​the eye, forming a light spot on the cornea of ​​the eye, and realizing high-precision eye tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 A schematic diagram of a light spot formed on the eye in the related art provided by the present invention;

[0046] Figure 2 A structural block diagram of a near-eye display device provided by one embodiment of the present invention;

[0047] Figure 3A schematic diagram of a light spot formed on an eye according to an embodiment of the present invention;

[0048] Figure 4 A diagram of an application scenario of a near-eye display device provided by an embodiment of the present invention;

[0049] Figure 5 A diagram of an application scenario of another near-eye display device provided by an embodiment of the present invention;

[0050] Figure 6 A schematic structural diagram of a first coupling-out structure provided in one embodiment of the present invention;

[0051] Figure 7 A schematic diagram of the structure of a near-eye display device provided by one embodiment of the present invention;

[0052] Figure 8 A top view of a near-eye display device provided by one embodiment of the present invention;

[0053] Fig. 9 A top view of another near-eye display device provided by an embodiment of the present invention;

[0054] Fig.10 A schematic structural diagram of a near-eye display device provided by another embodiment of the present invention;

[0055] Fig.11 A schematic structural diagram of a near-eye display device provided by yet another embodiment of the present invention;

[0056] Fig.12 A schematic flow chart of an eye tracking method for a near-eye display device is provided for an embodiment of the present invention.

[0057] Description of main reference numerals:

[0058] 100, optical waveguide; 101, first coupling-out structure; 1011, first electrode layer; 1012, birefringent material layer; 1013, second electrode layer; 1014, orientation layer; 102, first waveguide body; 103, second coupling-in structure; 104, first coupling-in structure; 105, second coupling-in structure; 106, second waveguide body; 107, turning grating structure; 200, light source assembly; 300, image acquisition unit; 301, infrared camera; 400, processing unit. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0061] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0062] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0063] like Figure 1 As shown, in the related art, when the eye looks straight ahead, the infrared light directly enters the eye, and the light spot formed on the cornea of ​​the eye is relatively clear; however, when the eye turns to look in other directions, for example, when the eye turns to the inner corner of the eye and looks to the lower left, the cornea of ​​the eye turns, and the infrared light still directly enters the eye, and is diffusely reflected by the sclera of the eye to form stray light. At the same time, less or even no light spot is formed on the cornea of ​​the eye, resulting in the inability to perform eye tracking, that is, the range of eye tracking is limited, or the accuracy of eye tracking is poor.

[0064] In order to solve the above problems, an embodiment of the present invention provides a near-eye display device, which may include augmented reality (AR) glasses, virtual reality (VR) glasses, AR helmets, VR helmets, etc., without specific limitation.

[0065] like Figures 2 to 5As shown, the near-eye display device includes an optical waveguide 100, a light source assembly 200, an image acquisition unit 300, and a processing unit 400. The optical waveguide 100 includes a plurality of first coupling structures 101, and the first coupling structures 101 can be switched between a coupling state and a non-coupling state. The light source assembly 200 is used to emit infrared light toward the optical waveguide 100. The image acquisition unit 300 is used to acquire an image of the eye. The processing unit 400 is electrically connected to the optical waveguide 100 and the image acquisition unit 300. The processing unit 400 is used to determine the position of a preset eye area according to the image of the eye. According to the position of the preset eye area, a target coupling structure is determined among the plurality of first coupling structures 101. The target coupling structure is controlled to be in a coupling state so that infrared light is coupled from the target coupling structure to the pupil area of ​​the eye. According to the light spot formed by the infrared light in the image of the eye, the gaze direction of the eye is determined.

[0066] Optionally, the target outcoupling structure may include one or more first outcoupling structures 101. For example, the optical waveguide 100 includes six first outcoupling structures 101, and the target outcoupling structure may be one, three or four first outcoupling structures 101 determined according to the location of the preset eye area.

[0067] For example, when a user wears a near-eye display device, the image acquisition unit 300 acquires an image of the eye and transmits it to the processing unit 400. The processing unit 400 can determine the position of the preset area of ​​the eye according to the acquired image of the eye, for example, determine the position of the pupil area or cornea area of ​​the eye, and then determine the target coupling structure in the plurality of first coupling structures 101 according to the position of the preset area of ​​the eye, and control the target coupling structure to be in the coupling state. The infrared light emitted by the light source assembly 200 toward the optical waveguide 100 is coupled into the optical waveguide 100 and transmitted in the optical waveguide 100, and part of the infrared light transmitted to the target coupling structure can be coupled to the pupil area of ​​the eye through the target coupling structure to form a clear light spot in the cornea area of ​​the eye. After the image acquisition unit 300 acquires the image of the eye again, it transmits it to the processing unit 400. The processing unit 400 can determine the gaze direction of the eye according to the light spot formed by the infrared light in the image of the eye, so as to realize eye tracking.

[0068] It can be understood that the infrared light can cover a wider range after being transmitted inside the optical waveguide 100 and coupled out from the multiple first coupling structures 101, which is conducive to ensuring that when the pupil area of ​​the eye is at any position, the infrared light can be coupled out from the corresponding target coupling structure and transmitted to the pupil area of ​​the eye, thereby increasing the range of eye tracking. After determining the position of the preset area of ​​the eye according to the image of the eye, the target coupling structure is determined according to the position of the preset area of ​​the eye, so that the target coupling structure is in a coupling state and couples out the infrared light, which is conducive to improving the position transmission accuracy of the coupled infrared light, so that the coupled infrared light is more accurately transmitted to the pupil area of ​​the eye, avoiding the infrared light being transmitted to the sclera area and being reflected by the sclera to form stray light, and the infrared light can form a clear light spot in the cornea area of ​​the eye. Accordingly, the light spot in the image of the eye collected by the image acquisition unit 300 is also clearer, which is conducive to achieving high-precision eye tracking.

[0069] Figure 3 FIG. 1 is a schematic diagram showing a light spot formed in the cornea region of the eye when infrared light enters the eye during eye tracking in a near-eye display device in an embodiment of the present invention. Figure 3 As shown, when the eye looks in different directions, for example, when looking straight ahead, or when turning to the inner corner of the eye to look to the lower left, after determining the position of the preset eye area according to the image of the eye, the target coupling structure determined according to the preset eye position is controlled to be in a coupling state, and the other first coupling structures are in a non-coupling state, so that the target coupling structure can couple the infrared light to the pupil area of ​​the eye, and can form a clear light spot in the cornea area of ​​the eye, thereby realizing high-precision eye tracking.

[0070] Exemplarily, the image acquisition unit 300 may include an infrared camera 301 , and an image of a light spot formed on the cornea of ​​the eye by the infrared camera 301 is acquired.

[0071] Exemplarily, the image acquisition unit 300 may include an infrared camera 301, which acquires an image of the eye through the infrared camera 301, determines the position of a preset area of ​​the eye based on the image of the eye, and acquires the image of the eye again through the infrared camera 301 to determine the gaze direction of the eye based on a light spot formed by infrared light in the re-acquired image of the eye.

[0072] Exemplarily, the image acquisition unit 300 may include a visible light camera and an infrared camera 301, the visible light camera includes a CCD (Charge Coupled Device) camera and a CMOS (Complementary Metal Oxide Semiconductor) camera, the image of the eye is acquired by the visible light camera, and the position of a preset area of ​​the eye is determined based on the image of the eye, and the image of the eye is acquired again by the infrared camera 301 to determine the gaze direction of the eye based on the light spot formed by the infrared light in the again acquired image of the eye.

[0073] like Figure 4 As shown, Figure 4 The dotted line in the figure shows the transmission path of the infrared light. In some embodiments, the optical waveguide 100 further includes a turning grating structure 107. After the infrared light coupled into the optical waveguide 100 is transmitted to the turning grating structure 107, it can be reflected to different angles and paths through the turning grating, so that the infrared light can be transmitted to the multiple first coupling-out structures 101. This is conducive to expanding the range that the infrared light can cover and increasing the range of eye tracking.

[0074] For example, Figure 4 As shown, a plurality of first out-coupling structures 101 are arranged in an array along the left-right direction and the front-back direction. The turning grating is located on the left side of the plurality of first out-coupling structures 101. When the infrared light is coupled into the optical waveguide 100 and then transmitted to the turning grating structure 107, a portion of the infrared light is reflected by the turning grating structure 107 and transmitted to the plurality of first out-coupling structures 101 on the right side, and another portion of the infrared light is reflected by the turning grating structure 107 and transmitted to the rear side of the turning grating structure 107, and then partially transmitted to the plurality of first out-coupling structures 101 on the right side after passing through the turning grating structure 107 again, so that the infrared light can transmit each of the first out-coupling structures 101.

[0075] In some embodiments, the preset eye area includes at least one of the following: a pupil area, a corneal area. The distance between the position of the target outcoupling structure and the pupil area is less than the distance between other first outcoupling structures 101 in the plurality of first outcoupling structures 101 and the pupil area, and / or, the distance between the position of the target outcoupling structure and the corneal area is less than the distance between other first outcoupling structures 101 in the plurality of first outcoupling structures 101 and the corneal area. It can be understood that when the eye looks in different directions, there is a corresponding positional relationship between the pupil and the cornea of ​​the eye. After the position of the corneal area is determined according to the image of the eye, the pupil area is also determined. Therefore, the target outcoupling structure can be directly determined according to the pupil area, and / or the target outcoupling structure can be determined according to the corneal area.

[0076] Exemplarily, the distance between the first outcoupling structure 101 and the pupil area can be embodied as the distance between the geometric center of the first outcoupling structure 101 and the center of the pupil area. The distance between the first outcoupling structure 101 and the cornea area can be embodied as the distance between the geometric center of the first outcoupling structure 101 and the center of the cornea area.

[0077] For example, Figure 5 The dotted line in the figure shows the transmission path of infrared light. Figure 5 As shown in Figure (A), three first coupling structures 101 are arranged in sequence along the width direction of the eye. When the eye looks forward, the pupil area and cornea area of ​​the eye are located in the middle of the eye image. At this time, the distance from the middle first coupling structure 101 to the pupil area and cornea area of ​​the eye is smaller than the distance from the left and right first coupling structures 101 to the pupil area and cornea area of ​​the eye. The middle first coupling structure 101 can be used as the target coupling structure to be in the coupling state and couple infrared light to the pupil area. Figure 5 As shown in Figure (B), when the eye looks to the left, the pupil area and cornea area of ​​the eye are located on the left side of the eye image. At this time, the distance from the first coupling structure 101 on the left side of the three first coupling structures 101 to the pupil area and cornea area of ​​the eye is smaller than the distance from the two first coupling structures 101 on the right side to the pupil area and cornea area of ​​the eye. The first coupling structure 101 on the left side can be used as the target coupling structure to be in the coupling state and couple the infrared light to the pupil area. Figure 5 As shown in Figure (C), when the eye looks to the right, the pupil area and cornea area of ​​the eye are located on the right side of the eye image. At this time, the distance from the first out-coupling structure 101 on the right side of the three first out-coupling structures 101 to the pupil area and cornea area of ​​the eye is smaller than the distance from the two first out-coupling structures 101 on the left side to the pupil area and cornea area of ​​the eye. The first out-coupling structure 101 on the right side can be used as a target out-coupling structure to be in an out-coupling state and couple the infrared light to the pupil area.

[0078] In some embodiments, the plurality of first out-coupling structures 101 are located on the same plane, and the target out-coupling structure may be determined according to the position of the projection of the preset area of ​​the eye on the plane.

[0079] Exemplarily, the distance between the position of the target outcoupling structure and the projection of the pupil area of ​​the eye on the plane is smaller than the distance between the other first outcoupling structures 101 and the projection of the pupil area of ​​the eye on the plane. Alternatively, the position of the target outcoupling structure and the projection of the pupil area of ​​the eye on the plane at least partially overlap.

[0080] In some embodiments, the preset eye area includes a pupil area, a corneal area, and a scleral area. Determining the position of the preset eye area according to the image of the eye includes: determining the positional relationship between the pupil area, the corneal area, and the scleral area according to the image of the eye. Determining a target coupling structure among multiple first coupling structures according to the position of the preset eye area includes: determining the target coupling structure among multiple first coupling structures according to the positional relationship between the pupil area, the corneal area, and the scleral area. It can be understood that the position of the image acquisition unit in the near-eye display device is fixed. When the eye looks in different directions, the relative relationship between the pupil area, the corneal area and the sclera area in the image of the eye captured by the image acquisition unit is different. Therefore, based on the positional relationship between the pupil area, the corneal area and the sclera area, the gaze direction of the eye can be preliminarily determined, that is, the position of the pupil area where the infrared light needs to be transmitted is determined, so that the target coupling structure can be determined from multiple first coupling structures, so that the infrared light coupled by the target coupling structure can be accurately transmitted to the pupil area of ​​the eye, and the gaze direction of the eye is further determined, thereby achieving high-precision eye tracking.

[0081] For example, Figure 5 The middle figure (A) shows an image of the eye when the eye is looking diagonally toward the left front, in which the pupil region and the cornea region are located in the relative middle of the sclera region. Figure 5 The middle figure (B) shows an image of the eye when the eye is looking straight ahead, in which the pupil region and the cornea region are located to the relative right of the sclera region. Figure 5 The middle figure (C) shows an image of the eye when the eye is looking diagonally toward the right front. In this image, the pupil region and the cornea region are located on the relatively right side of the sclera region 23.

[0082] In some embodiments, when the processing unit 400 determines the target coupling structure among the multiple first coupling structures 101 according to the position of the preset eye area, it is used to: determine the target coupling structure among the multiple first coupling structures 101 according to the position of the preset eye area based on the correspondence between the preset first coupling structure 101 and the position of the preset eye area. It can be understood that according to any position where the preset eye area can be located, one or more first coupling structures 101 corresponding thereto can be pre-set in the processing unit 400, and after the position of the preset eye area is determined according to the image of the eye, the preset one or more first coupling structures 101 can be used as the target coupling structure.

[0083] For example, Figure 5As shown, assuming that three first coupling structures 101 are arranged in sequence along the width direction of the eye, the first coupling structure 101 on the left side may correspond to the position when the preset eye area moves to the left side in the width direction of the eye, the first coupling structure 101 in the middle may correspond to the position when the preset eye area moves to the middle in the width direction of the eye, and the first coupling structure 101 on the right side may correspond to the position when the preset eye area moves to the right side in the width direction of the eye. After the position of the preset eye area is determined according to the image of the eye, if the preset eye area is located on the left side in the width direction of the eye, the first coupling structure 101 on the left side may be used as the target coupling structure; if the preset eye area is located in the middle in the width direction of the eye, the first coupling structure 101 in the middle may be used as the target coupling structure; if the preset eye area is located on the right side in the width direction of the eye, the first coupling structure 101 on the right side may be used as the target coupling structure.

[0084] In some embodiments, the processing unit 400 is also used to control the coupling angle of the infrared light coupled out of the target coupling structure when the target coupling structure is in the coupling state. It can be understood that the coupling angle of the infrared light is adjustable, so that the coupled infrared light can be transmitted over a wider range, which is conducive to increasing the range of eye tracking, and the coupling angle of the infrared light can be adjusted according to the location of the target coupling structure and the location of the preset eye area, so that the infrared light falls more accurately on the preset eye area. In addition, the coupling angle of the infrared light is adjustable, which can not only reduce the number of the first coupling structure 101, but also reduce the limitation on the position of the first coupling structure 101, so that the first coupling structure 101 can be adjusted according to the design requirements of other structures, so as to facilitate the design and preparation of near-eye display devices.

[0085] For example, Figure 4 As shown in Figures (A) and (B), the target coupling structure may include multiple first coupling structures 101. By controlling the target coupling structure and adjusting the coupling angle of the infrared light, the multiple first coupling structures 101 in the target coupling structure couple the infrared light to different positions of the pupil area. For example, one beam of infrared light is coupled to the central area of ​​the pupil area, and the remaining multiple beams of infrared light are coupled to the peripheral area of ​​the pupil area, and the light spots formed by the multiple beams of infrared light are arranged in a ring to improve the accuracy of eye tracking.

[0086] like Figure 6As shown, in some embodiments, the first out-coupling structure 101 includes a first electrode layer 1011, a birefringent material layer 1012, and a second electrode layer 1013, and the birefringent material layer 1012 is located between the first electrode layer 1011 and the second electrode layer 1013. The processing unit 400 is electrically connected to the first electrode layer 1011 and the second electrode layer 1013, and the processing unit 400 controls the first out-coupling structure 101 to switch to the out-coupling state or the non-out-coupling state by adjusting the voltage applied to the first electrode layer 1011 and the second electrode layer 1013.

[0087] For example, Figure 6 As shown in Figure (A), the first electrode layer 1011 and the second electrode layer 1013 are not connected to voltage, and the birefringent material does not rotate. The first outcoupling structure 101 is in a non-outcoupling state and cannot diffract infrared light, so the infrared light cannot be outcoupled. Figure 6 As shown in Figure (B), the first electrode layer 1011 and the second electrode layer 1013 are connected to voltage, and the birefringent material rotates under the action of the electric field formed by the first electrode layer 1011 and the second electrode layer 1013, so that the first coupling-out structure 101 switches to the coupling-out state, thereby being able to diffract the infrared light to couple the infrared light out to the pupil area of ​​the eye.

[0088] Exemplarily, the birefringent material includes an electro-optic birefringent material, a magneto-optic birefringent material, or an acousto-optic birefringent material.

[0089] Furthermore, by adjusting the voltage applied to the first electrode layer 1011 and the second electrode layer 1013 , the coupling-out angle of the infrared light coupled out by the target coupling-out structure can also be controlled.

[0090] Furthermore, if Figure 6 As shown, the first out-coupling structure 101 further includes an orientation layer 1014, and the orientation layer 1014 is disposed between the first electrode layer 1011 and the birefringent material layer 1012. By providing the orientation layer 1014 to guide the birefringent material to align in a specific direction and angle, the birefringent material responds consistently to the electric field of the first electrode layer 1011 and the second electrode layer 1013, which helps to couple the infrared light to the pupil region of the eye.

[0091] Specifically, the alignment layer 1014 has a plurality of grooves arranged at intervals, and the birefringent material can be evenly arranged along the extending direction of the grooves.

[0092] In some embodiments, the diffraction period of different first out-coupling structures 101 can be changed by changing the refractive index of the birefringent material in the birefringent material layer 1012 in different first out-coupling structures 101, or the refractive index of the material of the orientation layer 1014, so that different first out-coupling structures 101 can couple out infrared light at different out-coupling angles when in the out-coupling state.

[0093] In some embodiments, the light source assembly 200 is further used to emit visible light toward the optical waveguide 100. The optical waveguide 100 also has a second outcoupling structure 103, which is used to outcouple the visible light emitted to the optical waveguide 100, and the outcoupled visible light can be transmitted to the eye or a predetermined component so that the user can see the displayed virtual image.

[0094] It can be understood that the first outcoupling structure 101 is different from the second outcoupling structure 103. The first outcoupling structure 101 can couple out infrared light, and the second outcoupling structure 103 can couple out visible light. The infrared light can be light with a wavelength range of 780nm-940nm, and the visible light can be light with a wavelength range of 400nm-760nm.

[0095] like Figure 7 As shown, Figure 7 The dashed line in the figure shows the transmission path of infrared light, and the dotted line shows the transmission path of visible light. In some embodiments, the light source assembly 200 is used to emit visible light and infrared light toward the optical waveguide 100, and the optical waveguide 100 also includes a first waveguide body 102, and the first waveguide body 102 is provided with a second coupling-out structure 103 and a plurality of first coupling-out structures 101, and the second coupling-out structure 103 is used to couple out the visible light coupled into the optical waveguide 100.

[0096] It can be understood that the first outcoupling structure 101 and the second outcoupling structure 103 are both arranged on the first waveguide body 102, and the visible light and the infrared light are both coupled into the first waveguide body 102 and transmitted in the first waveguide body 102, and then pass through the second outcoupling structure 103 and the first outcoupling structure 101 respectively. In this way, the structural multiplexing of the first waveguide body 102 is realized, the volume of the optical waveguide 100 is controlled, and the miniaturized design of the near-eye display device is realized.

[0097] It is worth noting that the near-eye display device can only obtain images of the human left eye / right eye and couple infrared light only to the human left eye / right eye, or it can simultaneously obtain images of the human left eye and right eye and simultaneously couple infrared light to the human left eye and right eye.

[0098] Exemplarily, the optical waveguide 100 includes a first waveguide unit and a second waveguide unit, each of which includes a first waveguide body 102, a second outcoupling structure 103 provided on the first waveguide body 102, and a plurality of first outcoupling structures 101 provided on the first waveguide body 102, the first waveguide unit and the second waveguide unit are respectively provided corresponding to the left eye and the right eye of the human body, and simultaneously couple out visible light and infrared light for the left eye and the right eye of the human body.

[0099] Combination Figure 8 and Fig. 9As shown, in an optional implementation, the first outcoupling structure 101 and the second outcoupling structure 103 are located on the same side of the first waveguide body 102, and a plurality of first outcoupling structures 101 are arranged around the central area of ​​the second outcoupling structure 103. It can be understood that the plurality of first outcoupling structures 101 are arranged around the central area of ​​the second outcoupling structure 103, so that the infrared light coupled out of the plurality of first outcoupling areas can be transmitted to the pupil area of ​​the eye when the preset eye area moves to any position.

[0100] Optionally, the first waveguide body 102 has a side facing the eye and a side facing away from the eye, and the first decoupling structure 101 and the second decoupling structure may be simultaneously located on the side of the first waveguide body 102 facing the eye, or simultaneously located on the side of the first waveguide body 102 facing away from the eye.

[0101] Optionally, the first outcoupling structure 101 may be a reflective diffraction structure or a transmissive diffraction structure, and the second outcoupling structure 103 may be a reflective diffraction structure or a transmissive diffraction structure, which may be specifically adjusted according to the setting positions of the first outcoupling structure 101 and the second outcoupling structure 103 .

[0102] An exemplary example is Figure 8 As shown, there are multiple second out-coupling structures 103, and the multiple second out-coupling structures 103 are arranged in a rectangular array. The multiple second out-coupling structures 103 form a central area and a peripheral area located outside the central area. The multiple first out-coupling structures 101 are interspersed in the multiple second out-coupling structures 103 to ensure the integrity of the image displayed by the near-eye display device. A plurality of first out-coupling structures 101 are arranged around a central area of ​​a plurality of second out-coupling structures 103, and when the preset eye area moves to an upper position in the height direction of the eye, the two upper first out-coupling structures 101 can couple out infrared light to the pupil area of ​​the eye, and when the preset eye area moves to a lower position in the height direction of the eye, the two lower first out-coupling structures 101 can couple out infrared light to the pupil area of ​​the eye, and when the preset eye area moves to a left position in the width direction of the eye, the two first out-coupling structures 101 on the left can couple out infrared light to the pupil area of ​​the eye, and when the preset eye area moves to a right position in the width direction of the eye, the two first out-coupling structures 101 on the right can couple out infrared light to the pupil area of ​​the eye.

[0103] Another exemplary example is Fig. 9 As shown, multiple first outcoupling structures 101 are arranged around the second outcoupling structure 103, and the multiple first outcoupling structures 101 can be closely arranged, and at this time, the multiple first outcoupling structures 101 are far away from the eye. The close arrangement of the multiple first outcoupling structures 101 can ensure that when the preset area of ​​the eye moves to any position, the infrared light can be coupled out to the pupil area of ​​the eye.

[0104] like Figure 8 As shown, in an optional implementation, the optical waveguide 100 further includes a first coupling structure 104 , which is disposed on the first waveguide body 102 , and the visible light and infrared light emitted by the light source assembly 200 are coupled into the first waveguide body 102 through the first coupling structure 104 .

[0105] Fig. 9 As shown, in another optional embodiment, the optical waveguide 100 includes a first coupling structure 104 and a second coupling structure 105, both of which are arranged on the first waveguide body 102, and the visible light is coupled into the first waveguide body 102 through the first coupling structure 104, and the infrared light is coupled into the first waveguide body 102 through the second coupling structure 105.

[0106] For example, the first coupling structure 104 and the second coupling structure 105 may be located at the same side of the first waveguide body 102 , or the first coupling structure 104 and the second coupling structure 105 may be located at opposite sides of the first waveguide body 102 .

[0107] For example, when the first coupling structure 104 and the second coupling structure 105 are located on the same side of the first waveguide body 102, the second coupling decoupling strand may be located at the periphery of the first coupling structure 104, and Figure 7 As shown, there can be a plurality of second coupling structures 105 , and the plurality of second coupling structures 105 are disposed around the periphery of the first coupling structure 104 and are disposed at intervals.

[0108] Exemplarily, the first coupling structure 104 may be a reflective diffraction structure or a transmissive diffraction structure, and the second coupling structure 105 may be a reflective diffraction structure or a transmissive diffraction structure, which may be specifically adjusted according to the setting positions of the first coupling structure 104 and the second coupling structure 105.

[0109] like Fig.10 As shown, Fig.10 The dotted line in the figure shows the transmission path of infrared light, and the dotted line shows the transmission path of visible light. In another optional embodiment, the first outcoupling structure 101 and the second outcoupling structure 103 are located on opposite sides of the first waveguide body 102. It can be understood that the first outcoupling structure 101 and the second outcoupling structure 103 can be located on the side of the first waveguide body 102 facing the eye and the side away from the eye, respectively.

[0110] For example, Fig.10As shown, the first outcoupling structure 101 is located on the side of the first waveguide body 102 away from the eye, and the second outcoupling structure 103 is located on the side of the first waveguide body 102 facing the eye. At this time, the first outcoupling structure 101 can be a reflective diffraction structure, so that after the infrared light is transmitted to the first outcoupling structure 101 along the direction away from the eye, it is diffracted by the first outcoupling structure 101, so that the infrared light is coupled out of the first waveguide body 102 and transmitted to the eye. The second outcoupling structure 103 can be a transmissive diffraction structure, so that after the visible light is transmitted to the second outcoupling structure 103 along the direction toward the eye, it is diffracted by the second outcoupling structure 103, so that the visible light is coupled out of the first waveguide body 102 and transmitted to the eye.

[0111] Furthermore, the first outcoupling structure 101 may be a structure for coupling out infrared light through visible light, and / or the second outcoupling structure 103 may be a structure for coupling out visible light through infrared light. This makes the position setting of the first outcoupling structure 101 and the second outcoupling structure 103 more flexible, and facilitates the first outcoupling structure 101 to couple out infrared light to the pupil area of ​​the eye.

[0112] Exemplarily, when the first outcoupling structure 101 and the second outcoupling structure 103 are located on opposite sides of the first waveguide body 102, the first outcoupling structure 101 may be partially projected onto the second outcoupling structure 103. Figure 8 As shown, the first outcoupling structure 101 is located on the side of the first waveguide body 102 away from the eye, and the second outcoupling structure 103 is located on the side of the first waveguide body 102 facing the eye. The first outcoupling structure 101 can be partially projected onto the second outcoupling structure 103. At this time, the second outcoupling structure 103 can be a structure that transmits infrared light and couples out visible light, so that the infrared light diffracted and coupled out by the first outcoupling structure 101 can be transmitted to the pupil area of ​​the eye after passing through the second outcoupling structure 103.

[0113] like Fig.11 As shown, in some embodiments, the light source assembly 200 is also used to emit visible light toward the optical waveguide 100, and the optical waveguide 100 also includes a first waveguide body 102 and a second waveguide body 106. The first waveguide body 102 is provided with a first coupling-in structure 104 and a second coupling-out structure 103, and the visible light is coupled into the first waveguide body 102 through the first coupling-in structure 104, and is transmitted to the second coupling-out structure 103 and then coupled out through the second coupling-out structure 103. The second waveguide body 106 is provided with a second coupling-in structure 105 and a plurality of first coupling-out structures 101, and the infrared light is coupled into the second waveguide body 106 through the second coupling-in structure 105.

[0114] Exemplarily, the first outcoupling structure 101 and the second outcoupling structure 103 are respectively arranged on different waveguide bodies, and the structures of the first waveguide body 102 and the second waveguide body 106 can be adjusted according to actual needs to change the transmission path of visible light in the first waveguide body 102 and the transmission path of infrared light in the second waveguide body 106.

[0115] For example, Fig.11 As shown, the first waveguide body 102 and the second waveguide body 106 are arranged at intervals along the visual direction of the eye, the second coupling structure 103 can be arranged on the side of the first waveguide body 102 facing the eye, or on the side away from the eye, and the first coupling structure 101 can be arranged on the side of the second waveguide body 106 facing the first waveguide body 102, or on the side away from the first waveguide body 102.

[0116] Please refer to Fig.12 , an embodiment of the present invention provides an eye tracking method for a near-eye display device. The eye tracking method can be applied to a near-eye display device, and the near-eye display device can be AR glasses, VR glasses, AR helmets, VR helmets, etc. Specifically, the near-eye display device includes a light source assembly, an optical waveguide, and an image acquisition unit, the optical waveguide includes a plurality of first outcoupling structures, the first outcoupling structures can be switched between an outcoupling state and a non-outcoupling state, the light source assembly is used to emit infrared light toward the optical waveguide, and the image acquisition unit is used to capture an image of the eye. The eye tracking method for the near-eye display device includes steps S101 to S106.

[0117] Step S101, acquiring an eye image.

[0118] Exemplarily, the image acquisition unit in the near-eye display device can capture the user's eyes and obtain an image of the eyes.

[0119] Step S102: determining the position of the preset eye area according to the image.

[0120] Exemplarily, the preset eye area includes at least one of a pupil area and a cornea area. Specifically, the range of the preset eye area in the image can be determined by performing image recognition on the image, and the position of the preset eye area in the coordinate system of the image acquisition unit or in the real coordinate system can be determined.

[0121] Step S103: determining a target out-coupling structure from a plurality of first out-coupling structures according to the position of the preset eye area.

[0122] Exemplarily, the target coupling-out structure may be determined based on the distance between the position of the preset eye area and a plurality of first coupling-out structures, or the target coupling-out structure may be determined based on the correspondence between the preset first coupling-out structure and the position of the preset eye area.

[0123] Step S104: Control the target out-coupling structure to be in an out-coupling state, so that the infrared light is coupled out from the target out-coupling structure to the pupil area of ​​the eye.

[0124] Step S105, acquiring the eye image again.

[0125] Exemplarily, the image acquisition unit may include an infrared camera, and an image of a light spot formed by infrared light on the cornea of ​​the eye is acquired by the infrared camera.

[0126] Step S106: determine the gaze direction of the eye according to the light spot formed by the infrared light in the image acquired again.

[0127] In the eye tracking method for a near-eye display device provided in the above-mentioned embodiment, the infrared light can cover a wider range after being transmitted inside the optical waveguide and coupled out from the multiple first coupling structures, which is conducive to ensuring that when the pupil area of ​​the eye is at any position, the infrared light can be coupled out from the corresponding target coupling structure and transmitted to the pupil area of ​​the eye, thereby increasing the range of eye tracking. After determining the position of the preset area of ​​the eye according to the image of the eye, the target coupling structure is determined according to the position of the area of ​​the eye, so that the target coupling structure is in a coupling state and couples out the infrared light, which is conducive to improving the position transmission accuracy of the coupled infrared light, so that the coupled infrared light is more accurately transmitted to the pupil area of ​​the eye, so as to form a clear light spot on the cornea of ​​the eye, and accordingly, the light spot in the image of the eye collected by the image acquisition unit is also clearer, which is conducive to achieving high-precision eye tracking.

[0128] In some embodiments, image recognition can be performed on the re-acquired image to determine the position of the pupil area in the image and the position of the light spot in the image, and based on the position of the pupil area in the image and the position of the light spot in the image, the position offset of the light spot relative to the user's pupil can be determined to determine the gaze direction of the eye and achieve eye tracking.

[0129] Furthermore, when the image includes multiple light spots, the position offset of each light spot relative to the pupil area can be determined respectively, so as to determine the gaze direction of the eye according to the average value of the multiple offsets. Of course, it is not limited to this. When the image includes multiple light spots, the average value of the positions of all light spots in the image can also be determined. According to the average value of the positions of all light spots in the image and the position of the pupil area in the image, the position offset of the light spot relative to the pupil area of ​​the user is determined, so as to determine the user's eye movement information.

[0130] In some embodiments, the image acquisition unit can obtain a visible light image of the eye to identify a preset eye area based on the visible light image and determine the position of the preset eye area. Exemplarily, the light source assembly is also used to emit visible light toward the optical waveguide, and the visible light is coupled into the optical waveguide and then coupled out to the eye, so that the user can view the image displayed by the near-eye display device. When the visible light is coupled out to the eye, the image acquisition unit can obtain an image of the eye under visible light, and after the processing unit obtains the visible light image, it determines the preset eye area through image recognition and determines the position of the preset eye area.

[0131] In some embodiments, it is determined whether the eye is in a fixation state, a motion state, or a scanning state by comparing images acquired at different time frames.

[0132] Exemplarily, when the difference between the position offsets corresponding to images captured by at least two adjacent time frames is less than or equal to a first preset difference, it is determined that the user's eyes are in a gaze state. For example, it can be determined that the user is gazing at or focusing on a certain position on the screen displayed by the near-eye display device.

[0133] Exemplarily, when the position offset corresponding to the images of at least two adjacent time frames is greater than a first preset difference and less than or equal to a second preset difference, it is determined that the user's eyes are in a moving tracking state. For example, it can be determined that the user is tracking or observing a moving object on the screen of the near-eye display device.

[0134] Exemplarily, when the position offset corresponding to the images of at least two adjacent time frames is greater than a second preset difference, it is determined that the user's eyes are in a scanning state. For example, it can be determined that the user is scanning different positions of the screen of the near-eye display device.

[0135] It is understandable that, when the gaze direction of the eyes is determined and eye tracking is achieved, the near-eye display device can be adjusted according to the user's eye state to achieve various interactive and perceptual functions. For example, the human-computer interaction interface of the near-eye display device is designed according to the user's eye state, the user's psychology of using the near-eye display device is learned through the user's eye state, and the driver's driving condition is monitored through the driver's eye state, etc., which are not limited here.

[0136] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0137] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the eye tracking method for a near-eye display device of the present application.

[0138] The computer-readable storage medium may be an internal storage unit of the near-eye display device described in the aforementioned embodiment, such as a hard disk or memory of the near-eye display device. The computer-readable storage medium may also be an external storage device of the near-eye display device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the near-eye display device.

[0139] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0140] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0141] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A near-eye display device, characterized in that: include: An optical waveguide, the optical waveguide comprising a plurality of first outcoupling structures, the first outcoupling structures being switchable between an outcoupling state and a non-outcoupling state; A light source assembly, configured to emit infrared light toward the light waveguide; An image acquisition unit, used for acquiring images of the eyes; A processing unit is electrically connected to the plurality of first coupling-out structures and the image acquisition unit, and the processing unit is used for: Determining the position of a preset eye area according to the eye image; Determining a target out-coupling structure among the plurality of first out-coupling structures according to the position of the preset eye area; Controlling the target outcoupling structure to be in the outcoupling state so that the infrared light is coupled out from the target outcoupling structure to the pupil area of ​​the eye; The gaze direction of the eye is determined according to a light spot formed by the infrared light in the image of the eye.

2. The near-eye display device according to claim 1, characterized in that: The preset eye area includes at least one of the following: a pupil area and a cornea area; The distance between the position of the target outcoupling structure and the pupil area is smaller than the distance between other first outcoupling structures in the plurality of first outcoupling structures and the pupil area; and / or The distance between the position of the target out-coupling structure and the corneal region is smaller than the distance between other first out-coupling structures among the plurality of first out-coupling structures and the corneal region.

3. The near-eye display device according to claim 1, characterized in that: The preset eye area includes the pupil area, cornea area and sclera area; The step of determining the position of the preset eye area according to the eye image comprises: determining a positional relationship among the pupil region, the cornea region, and the sclera region according to the image of the eye; The step of determining a target out-coupling structure from among the plurality of first out-coupling structures according to the position of the preset eye area comprises: The target out-coupling structure is determined among the plurality of first out-coupling structures according to the positional relationship among the pupil area, the cornea area, and the sclera area.

4. The near-eye display device according to claim 1, characterized in that: The processing unit is configured to: when determining a target out-coupling structure from among the plurality of first out-coupling structures according to the position of the preset eye area, Based on the correspondence between the preset first coupling-out structures and the positions of the preset eye areas, a target coupling-out structure is determined among the plurality of first coupling-out structures according to the positions of the preset eye areas.

5. The near-eye display device according to claim 1, characterized in that: The processing unit is further configured to control the outcoupling angle of the infrared light when the target outcoupling structure is in the outcoupling state.

6. The near-eye display device according to claim 1, characterized in that: The first out-coupling structure comprises a first electrode layer, a birefringent material layer and a second electrode layer, wherein the birefringent material layer is located between the first electrode layer and the second electrode layer; The processing unit is electrically connected to the first electrode layer and the second electrode layer, and controls the first out-coupling structure to switch to the out-coupling state or the non-out-coupling state by adjusting the voltage applied to the first electrode layer and the second electrode layer.

7. The near-eye display device according to claim 6, characterized in that: The first outcoupling structure further includes an orientation layer, and the orientation layer is arranged between the first electrode layer and the birefringent material layer.

8. The near-eye display device according to claim 1, characterized in that: The light source assembly is further configured to emit visible light toward the optical waveguide, wherein the optical waveguide further comprises: The first waveguide body is provided with a second out-coupling structure and a plurality of the first out-coupling structures, wherein the second out-coupling structure is used to couple out the visible light coupled into the optical waveguide.

9. The near-eye display device according to claim 8, characterized in that: The optical waveguide further comprises a first coupling structure, which is arranged on the first waveguide body, and the visible light and the infrared light emitted by the light source assembly are coupled into the first waveguide body through the first coupling structure; or The optical waveguide includes a first coupling structure and a second coupling structure, both of which are arranged on the first waveguide body, the visible light is coupled into the first waveguide body through the first coupling structure, and the infrared light is coupled into the first waveguide body through the second coupling structure.

10. The near-eye display device according to claim 8, characterized in that: The first out-coupling structure and the second out-coupling structure are located on the same side of the first waveguide body, and a plurality of the first out-coupling structures are arranged around a central area of ​​the second out-coupling structure; or The first out-coupling structure and the second out-coupling structure are located on opposite sides of the first waveguide body.

11. The near-eye display device according to claim 1, characterized in that: The light source assembly is further configured to emit visible light toward the optical waveguide, wherein the optical waveguide further comprises: A first waveguide body, provided with a first coupling-in structure and a second coupling-out structure, wherein the visible light is coupled into the first waveguide body through the first coupling-in structure, and is transmitted to the second coupling-out structure and then coupled out through the second coupling-out structure; The second waveguide body is provided with a second coupling-in structure and a plurality of the first coupling-out structures, and the infrared light is coupled into the second waveguide body through the second coupling-in structure.

12. An eye tracking method for a near-eye display device, characterized in that: The near-eye display device comprises a light source assembly, an optical waveguide and an image acquisition unit, wherein the optical waveguide comprises a plurality of first outcoupling structures, wherein the first outcoupling structures can be switched between an outcoupling state and a non-outcoupling state, the light source assembly is used to emit infrared light toward the optical waveguide, and the image acquisition unit is used to acquire an image of the eye; The eye tracking method comprises: acquiring an image of the eye; Determine the position of a preset eye area according to the image; Determining a target out-coupling structure among the plurality of first out-coupling structures according to the position of the preset eye area; Controlling the target outcoupling structure to be in the outcoupling state so that the infrared light is coupled out from the target outcoupling structure to the pupil area of ​​the eye; acquiring the image of the eye again; The gaze direction of the eye is determined according to the light spot formed by the infrared light in the image acquired again.

Images