Eye movement tracking device, near-eye display equipment and eye movement tracking method

Abstract

The invention relates to the technical field of near-to-eye display, and provides an eye movement tracking device, near-to-eye display equipment and an eye movement tracking method. The first coupling-in area, the first coupling-out area, the at least one second coupling-in area and the at least one second coupling-out area are arranged on the waveguide substrate; the second coupling-in region is arranged close to the first coupling-in region relative to the first coupling-out region; relative to the first coupling-in region, the second coupling-out region is arranged close to the first coupling-out region; the first coupling-in area is used for coupling incident light rays emitted by the projection module into the waveguide substrate, and the first coupling-out area is used for coupling the incident light rays propagating in the waveguide substrate out to human eyes; the second coupling-in area is convenient for receiving infrared light rays with different incident angles and coupling the infrared light rays into the waveguide substrate, and the infrared light rays are transmitted to the corresponding one or more second coupling-out areas in the waveguide substrate and are coupled out to human eyes through the second coupling-out areas; the eye movement tracking convenience and the eye movement tracking accuracy of the human eyes are improved when the eyeballs of the user are located at different positions.

Description

Technical Field

[0001] The present application relates to the field of near-eye display technology, and in particular to an eye tracking device, a near-eye display device, and an eye tracking method. Background Art

[0002] In the related art, when the infrared light provided by the infrared light source is used to track the user's eye movements, the infrared light provided by the infrared light source can form a corresponding light spot in the human eye, and the imaging device can capture the human eye to obtain a corresponding image for eye tracking of the human eye.

[0003] However, due to the limitations of the positions of the infrared light source, the human eye, and the imaging device, the image of the human eye obtained by the imaging device may easily show that the light spot corresponding to the infrared light in the image of the human eye is not located on the cornea of ​​the human eye, but on the sclera of the human eye. Figure 1a As shown in FIG. 1 , when infrared light is incident on the cornea of ​​a human eye, the cornea can reflect the infrared light to the imaging device, so that the image captured by the imaging device can include a clear light spot on the cornea. The clear light spot on the cornea can be used to track the eye movement of the human eye. Figure 1b As shown in the figure, when the human eye rotates to the inside of the eye socket, the cornea of ​​the human eye also rotates to the inside of the eye socket accordingly. Since the position of the infrared light source does not change, the infrared light provided by the infrared light source to the human eye will irradiate the sclera of the human eye. The sclera can only diffusely reflect the infrared light, so the image of the human eye obtained by the imaging device may contain unclear speckles located on the sclera. The unclear speckles located on the sclera cannot be used to track the eye movement of the human eye.

[0004] Based on this, under the limitations of the position of the infrared light source, the position of the human eye and the position of the imaging device, if the human eye image taken by the imaging device does not contain a clear light spot on the cornea, it will easily have an adverse effect on the accuracy of eye movement tracking of the human eye. Summary of the invention

[0005] The main purpose of the present application is to provide an eye tracking device, a near-eye display device and an eye tracking method, aiming to solve the technical problem that due to the limitations of the position of the infrared light source, the position of the human eye and the position of the imaging device, the human eye image taken by the imaging device does not contain a clear light spot on the cornea of ​​the human eye, which in turn leads to poor eye tracking accuracy of the human eye.

[0006] In a first aspect, the present application provides an eye tracking device, comprising:

[0007] waveguide substrate;

[0008] A first coupling-in region, a first coupling-out region, at least one second coupling-in region and at least one second coupling-out region are arranged on the waveguide substrate; relative to the first coupling-out region, the second coupling-in region is arranged close to the first coupling-in region; relative to the first coupling-in region, the second coupling-out region is arranged close to the first coupling-out region;

[0009] Among them, the first coupling-in area is used to couple the incident light emitted by the projection module into the waveguide substrate, and the first coupling-out area is used to couple the incident light propagating in the waveguide substrate to the human eye; the second coupling-in area is convenient for receiving infrared light with different incident angles and coupling the infrared light into the waveguide substrate, propagating in the waveguide substrate to the corresponding one or more second coupling-out areas, and coupling out to the human eye through the second coupling-out area.

[0010] In a second aspect, the present application further provides a near-eye display device, comprising:

[0011] An eye tracking device, wherein the eye tracking device is the aforementioned eye tracking device;

[0012] An optical-mechanical device, the optical-mechanical device comprising a projection module and an infrared module, the projection module is used to emit incident light to a first coupling-in area of ​​the eye-tracking device, the first coupling-in area is used to couple the incident light into a waveguide substrate of the eye-tracking device, and the first coupling-out area of ​​the eye-tracking device is used to couple the incident light propagating in the waveguide substrate to a human eye; the infrared module comprises an infrared light source and an angle adjustment member, the angle adjustment member is used to adjust the light angle of the infrared light provided by the infrared light source, so that the adjusted infrared light irradiates one of the second coupling-in areas of the eye-tracking device, is coupled into the waveguide substrate through the second coupling-in area, and then propagates to one or more corresponding second coupling-out areas of the eye-tracking device, and is coupled out to the human eye through the second coupling-out area;

[0013] An imaging device is used to capture an image of a user's eyes to achieve eye tracking.

[0014] In a third aspect, the present application further provides an eye tracking method for a near-eye display device, which is used for the aforementioned near-eye display device, and the eye tracking method comprises:

[0015] Controlling an infrared light source of a near-eye display device to provide infrared light;

[0016] The angle of the infrared light provided by the infrared light source is adjusted by the angle adjustment member of the near-eye display device to provide the adjusted infrared light to the user's eyes;

[0017] Acquire a human eye image of a user captured by an imaging device of the near-eye display device;

[0018] The eye movement information of the user is determined according to the human eye image.

[0019] The present application provides an eye tracking device, a near-eye display device and an eye tracking method for a near-eye display device. The eye tracking device includes: a waveguide substrate; a first coupling-in region, a first coupling-out region, at least one second coupling-in region and at least one second coupling-out region arranged on the waveguide substrate; relative to the first coupling-out region, the second coupling-in region is arranged close to the first coupling-in region; relative to the first coupling-in region, the second coupling-out region is arranged close to the first coupling-out region; wherein the first coupling-in region is used to couple the incident light emitted by the projection module into the waveguide substrate, and the first coupling-out region is used to couple the incident light propagating in the waveguide substrate to the human eye; the second coupling-in region is convenient for receiving infrared light with different incident angles and coupling the infrared light into the waveguide substrate, propagating in the waveguide substrate to the corresponding one or more second coupling-out regions, and coupling out to the human eye through the second coupling-out region.

[0020] Based on the setting of at least one second coupling-in area and at least one second coupling-out area, when the human eye is in different positions, such as when the cornea of ​​the human eye has not rotated to the inner side of the eye socket or when the cornea of ​​the human eye has rotated to the inner side of the eye socket, the eye tracking device can couple infrared light of corresponding incident angles through different second coupling-in areas, or couple infrared light of corresponding incident angles through corresponding positions of the same second coupling-in area, so that the infrared light coupled into the waveguide substrate can propagate through the waveguide substrate to the corresponding one or more second coupling-out areas and then couple out to the human eye. After the action of the eye tracking device, the infrared light can be coupled out to the cornea of ​​the human eye to form a corresponding light spot on the cornea of ​​the human eye. The light spot located on the cornea of ​​the human eye can be used for eye tracking of the human eye, which is conducive to improving the convenience and accuracy of eye tracking of the human eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1a to Figure 1b is a schematic diagram of a human eye image involved in the related art;

[0023] Figure 2 is a structural schematic diagram of an eye tracking device provided in an embodiment of the present application;

[0024] Figure 3 yes Figure 2 Wave vector diagram of the provided eye tracking device;

[0025] Figure 4is a schematic diagram of the structure of an eye tracking device involved in an embodiment of the present application;

[0026] Figure 5 yes Figure 4 Wave vector diagram of the provided eye tracking device;

[0027] Figure 6 is a schematic structural diagram of an eye tracking device according to another embodiment of the present application;

[0028] Figure 7 is a structural schematic diagram of a near-eye display device provided in an embodiment of the present application;

[0029] Figures 8a to 8c is a schematic diagram of an optical path of a near-eye display device involved in an embodiment of the present application;

[0030] Fig. 9 is a schematic structural diagram of a near-eye display device according to another embodiment of the present application;

[0031] Figures 10a to 10c is a schematic diagram of an optical path of a near-eye display device according to another embodiment of the present application;

[0032] Fig.11 It is a flow chart of an eye tracking method for a near-eye display device provided in an embodiment of the present application.

[0033] Explanation of the reference numerals: 10, near-eye display device; 100, eye tracking device; 110, waveguide substrate; 120, first coupling-in region; 130, first decoupling region; 140, second coupling-in region; 150, second decoupling region; 160, third coupling-in region; 170, third decoupling region; 200, optical-mechanical device; 210, projection module; 220, infrared module; 221, infrared light source; 222, angle adjustment member; 2221, reflector; 2222, adjustable liquid crystal grating; 230, beam combining assembly; 300, imaging device. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0036] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0037] See also Figure 2 , Figure 2 1 is a schematic diagram of the structure of an eye tracking device 100 provided in an embodiment of the present application. The eye tracking device 100 can be provided in a near-eye display device 10 to track the eyes of a user wearing the near-eye display device 10. The near-eye display device 10 can include augmented reality (AR) glasses, virtual reality (VR) glasses, mixed reality (MR) glasses, AR helmets, VR helmets, MR helmets, etc., without limitation.

[0038] like Figure 2 As shown, the eye tracking device 100 includes a waveguide substrate 110; a first coupling-in region 120, a first coupling-out region 130, at least one second coupling-in region 140 and at least one second coupling-out region 150 are arranged on the waveguide substrate 110; relative to the first coupling-out region 130, the second coupling-in region 140 is arranged close to the first coupling-in region 120; relative to the first coupling-in region 120, the second coupling-out region 150 is arranged close to the first coupling-out region 130; wherein the first coupling-in region 120 is used to couple the incident light emitted by the projection module 210 into the waveguide substrate 110, and the first coupling-out region 130 is used to couple the incident light propagating in the waveguide substrate 110 to the human eye; the second coupling-in region 140 is convenient for receiving infrared light with different incident angles and coupling the infrared light into the waveguide substrate 110, propagating in the waveguide substrate 110 to the corresponding one or more second coupling-out regions 150, and coupling out to the human eye through the second coupling-out region 150.

[0039] For example, the waveguide substrate 110 may include a plurality of substrate surfaces. The first incoupling region 120 , the first outcoupling region 130 , the second incoupling region 140 and the second outcoupling region 150 may be disposed on one of the substrate surfaces of the waveguide substrate 110 , which is not limited herein.

[0040] In some embodiments, the incident light may include visible light. For example, the wavelength range of the incident light may include 390 nanometers to 770 nanometers. When the incident light emitted by the projection module 210 irradiates the first coupling-in region 120, the first coupling-in region 120 may couple the incident light into the waveguide substrate 110. The incident light coupled into the waveguide substrate 110 may propagate at a preset angle until it propagates to the first coupling-out region 130 and then couples out to the human eye through the first coupling-out region 130. The preset angle may include an angle greater than the critical angle of total internal reflection of the medium and air interface of the eye tracking device 100. When the eye tracking device 100 can couple the incident light into the first coupling-in region 120, and couple out the incident light into the human eye through the first coupling-out region 130, the eye tracking device 100 may have a display function. Based on this, the eye tracking device 100 can be used for the subsequent preparation of the near-eye display device 10 to realize the display function of the near-eye display device 10. In an exemplary embodiment, when ambient light is irradiated onto the eye tracking device 100, the ambient light can be incident on the human eye through the first outcoupling region 130. When the user subsequently wears the near-eye display device 10 prepared by the eye tracking device 100, the user can view the surrounding environment, without limitation herein.

[0041] For example, the number of the second coupling-in regions 140 may include one or more, and the number of the second coupling-out regions 150 may also include one or more.

[0042] In some embodiments, when the number of the second coupling zone 140 is one, different positions of the second coupling zone 140 may have different incident angle ranges. For example, the second coupling zone 140 may be in the form of at least one of a ring and a semi-ring, so as to surround the first coupling zone 120. When the infrared light is irradiated onto the eye tracking device 100, and the incident angle of the infrared light is within the incident angle range of one position of the second coupling zone 140, the infrared light may be incident onto the corresponding position of the second coupling zone 140. Different positions of the same second coupling zone 140 have different incident angle ranges, so the second coupling zone 140 is convenient for receiving infrared light with different incident angles. Accordingly, the corresponding position of the second coupling zone 140 that receives the infrared light may couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 may propagate at a preset angle until it propagates to the corresponding one or more second coupling zones 150 and then couples out to the human eye through the second coupling zones 150. Accordingly, the number of the second outcoupling region 150 may also be one. For example, the second outcoupling region 150 may also be in the form of at least one of a ring and a semi-ring, so as to surround the first outcoupling region 130. When the infrared light is incident on one position of the second coupling-in region 140, the infrared light may be coupled out to the human eye through the second coupling-in region 140, the waveguide substrate 110, and the corresponding position of the second outcoupling region 150 in sequence. Of course, this is not limited to this, and is not limited here.

[0043] In some embodiments, the number of second coupling-in regions 140 may be multiple, and the number of second coupling-out regions 150 may also be multiple. The second coupling-in region 140 may have a preset incident angle range. When infrared light is irradiated onto the eye tracking device 100, and the incident angle of the infrared light is within the incident angle range of the second coupling-in region 140, the infrared light may be incident onto the second coupling-in region 140. Different second coupling-in regions 140 have different incident angle ranges, so multiple second coupling-in regions 140 are convenient for receiving infrared light with different incident angles. Accordingly, the second coupling-in region 140 that receives the infrared light may couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 may propagate at a preset angle until it propagates to the corresponding one or more second coupling-out regions 150 and then couples out to the human eye through the second coupling-out region 150. When the eye tracking device 100 can couple infrared light into the second coupling-in area 140 and couple infrared light out to the human eye through the second coupling-out area 150, the human eye can reflect the infrared light, and a corresponding light spot can be formed in the human eye. The light spot can be used to track the user's eye movements. Based on this, the eye tracking device 100 can be used for the subsequent preparation of the near-eye display device 10 to realize the eye tracking function of the near-eye display device 10.

[0044] Of course, the present invention is not limited thereto. The second coupling-in region 140 and the second coupling-out region 150 may have a one-to-one correspondence, a one-to-many relationship, or a many-to-one relationship, which is not limited here.

[0045] Exemplarily, the second coupling region 140 is distributed around the first coupling region 120, and relative to the first decoupling region 130, the second coupling region 140 is disposed close to the first coupling region 120. The second decoupling region 150 is distributed around the first decoupling region 130, and relative to the first coupling region 120, the second decoupling region 150 is disposed close to the first decoupling region 130.

[0046] In some embodiments, the second in-coupling region 140 is disposed around the first in-coupling region 120 , and / or the second out-coupling region 150 is disposed around the first out-coupling region 130 .

[0047] For example, in the case where a second coupling-in region 140 is disposed around the first coupling-in region 120, the second coupling-in region 140 may be in the shape of at least one of a ring and a semi-ring, so that the second coupling-in region 140 is disposed close to the first coupling-in region 120. In the case where a plurality of second coupling-in regions 140 are disposed around the first coupling-in region 120, the second coupling-in region 140 may be disposed close to the first coupling-in region 120. In the case where a second decoupling region 150 is disposed around the first decoupling region 130, the second decoupling region 150 may be in the shape of at least one of a ring and a semi-ring, so that the second decoupling region 150 is disposed close to the first decoupling region 130. In the case where a plurality of second decoupling regions 150 are disposed around the first decoupling region 130, the second decoupling region 150 may be disposed close to the first decoupling region 130.

[0048] Based on this, when preparing the near-eye display device 10 later, the optical-mechanical device 200 of the near-eye display device 10 can be set according to the respective setting positions of the first coupling-in area 120 and the second coupling-in area 140, for example, the projection module 210 and the infrared module 220 included in the optical-mechanical device 200 can be set to improve the compactness of the structural distribution of the optical-mechanical device 200, reduce the volume of the optical-mechanical device 200, and thus reduce the volume of the near-eye display device 10. Accordingly, when preparing the near-eye display device 10 later, the position of the imaging device 300 of the near-eye display device 10 can be set according to the respective setting positions of the first coupling-out area 130 and the second coupling-out area 150, so as to reduce the volume of the near-eye display device 10.

[0049] For example, since the first outcoupling region 130 can couple the incident light propagating to the first coupling-in region 120 to the human eye, the size of the first outcoupling region 130 can match the size of the human eye, so that the user can better view the display content of the eye tracking device 100, thereby improving the display effect of the eye tracking device 100 for the user. Based on the matching of the size of the first outcoupling region 130 with the size of the human eye, when one or more second outcoupling regions 150 are arranged around the first outcoupling region 130, the second outcoupling region 150 can be arranged close to the first outcoupling region 130. Accordingly, when the second outcoupling region 150 is arranged close to the first outcoupling region 130, it can be ensured that the infrared light coupled out by the second outcoupling region 150 can be coupled to the human eye, which is conducive to the subsequent improvement of the convenience of eye tracking of the human eye and the improvement of the accuracy of eye tracking of the human eye.

[0050] In some embodiments, the size of the second incoupling region 140 is smaller than the size of the first incoupling region 120 , and / or the size of the second outcoupling region 150 is smaller than the size of the first outcoupling region 130 .

[0051] For example, when the size of the second coupling-in zone 140 is smaller than the size of the first coupling-in zone 120, as many second coupling-in zones 140 as possible can be arranged around the first coupling-in zone 120 to more comprehensively receive infrared light of different incident angles. When the size of the second decoupling zone 150 is smaller than the size of the second decoupling zone 150, as many second decoupling zones 150 as possible can be arranged around the first decoupling zone 130 to more comprehensively provide infrared light of different exit angles to the human eye. Accordingly, when the number of second coupling-in zones 140 and / or the number of second decoupling zones 150 increases, the volume of the eye tracking device 100 will not be increased due to the smaller size of the second coupling-in zones 140 and / or the second decoupling zones 150, which is beneficial to improve the compactness of the structural distribution of the eye tracking device 100.

[0052] Since the incident angle ranges corresponding to different positions of the same second coupling-in area 140 may be different, and the incident angle ranges of multiple second coupling-in areas 140 may also be different, the infrared light irradiating the eye tracking device 100 at different incident angles may be irradiated to different positions of the same second coupling-in area 140 or different second coupling-in areas 140 respectively, and then coupled into the waveguide substrate 110, propagated through the waveguide substrate 110 to different positions of the same second coupling-in area 140 or one or more second coupling-out areas 150 corresponding to different second coupling-in areas 140, and then coupled out to different positions on the cornea of ​​the human eye. The cornea of ​​the human eye can reflect the infrared light coupled out by different second coupling-out areas 150, forming a corresponding plurality of light spots on the cornea of ​​the human eye. The plurality of light spots can be used to track the eye movement of the human eye in combination with the positions of the plurality of light spots on the cornea of ​​the human eye, which is conducive to improving the convenience and accuracy of eye tracking of the human eye. Accordingly, when the human eye rotates to different positions, infrared light can be coupled into the eye through different positions of the same second coupling-in area 140 or different second coupling-in areas 140, and infrared light can be coupled out to the human eye through the corresponding second coupling-out area 150, so that the coupled-out infrared light can irradiate the human eye, such as irradiating the cornea of ​​the human eye, which is beneficial to improving the accuracy of eye movement tracking of the human eye.

[0053] Exemplarily, the first coupling-in region 120 and the first coupling-out region 130 conform to a wave vector closure relationship.

[0054] See also Figure 3 , Figure 3 yes Figure 2 A schematic diagram of the wave vector of the eye tracking device 100 is provided.

[0055] Take the eye tracking device 100 including a plurality of second coupling-in regions 140 and second coupling-out regions 150 as an example. Figure 3 As shown, K1in is used to indicate the wave vector of the first coupling-in region 120. K1out1 and K1out2 are used to indicate the wave vector of the first coupling-out region 130, respectively.

[0056] Depend on Figure 3 It can be seen that the first coupling-in region 120 and the plurality of second coupling-in regions 140 are all in the inner circle, and the first coupling-out region 130 and the plurality of second coupling-out regions 150 are all in the outer circle. The radius of the inner circle can be expressed as n0 is used to indicate the refractive index of air. The radius of the outer ring can be expressed as n is used to indicate the refractive index of the eye tracking device 100. The annular area formed by the inner circle and the outer circle is used to indicate that the incident light coupled into the waveguide substrate 110 through the first coupling-in area 120 and the infrared light coupled into the waveguide substrate 110 through the second coupling-in area 140 propagate in the waveguide substrate 110 under the condition of total reflection. The rectangular area where the center of the circle (the origin of the coordinates) is located is used to indicate the field angle of the incident light emitted by the projection module 210, and the center of the circle is used to indicate that the incident light is vertically incident on the first coupling-in area 120. The incident light can be moved to the annular area through the K1in action of the first coupling-in area 120 to be coupled into the waveguide substrate 110. The incident light coupled into the waveguide substrate 110 can propagate in the waveguide substrate 110. Correspondingly, the incident light propagating in the waveguide substrate 110 can be moved back to the inner circle through the actions of K1out1 and K1out2 of the first outcoupling area 130 to be coupled out to the human eye.

[0057] Correspondingly, when a plurality of second coupling-in regions 140 and a plurality of second coupling-out regions 150 are disposed on the waveguide substrate 110 , the second coupling-in regions 140 and the second coupling-out regions 150 that have a corresponding relationship also conform to the wave vector closure relationship.

[0058] In some embodiments, the second incoupling region 140 and the second outcoupling region 150 may be in one-to-one correspondence.

[0059] like Figure 3 As shown, taking the case where the plurality of second coupling-in regions 140 include the second coupling-in region 1 to the second coupling-in region p, and the plurality of second coupling-out regions 150 include the second coupling-out region 1 to the second coupling-out region p as an example, the second coupling-in region 1 corresponds to the second coupling-out region 1 one-to-one, the second coupling-in region p corresponds to the second coupling-out region p one-to-one, and so on. K2in1 is used to indicate the wave vector of the second coupling-in region 1, K2out1 is used to indicate the wave vector of the second coupling-out region 1, K2inp is used to indicate the wave vector of the second coupling-in region p, K2outp is used to indicate the wave vector of the second coupling-out region p, and so on.

[0060] Depend on Figure 2 as well as Figure 3It can be known that the incident angle range of the second coupling-in area 140 at different setting positions on the waveguide substrate 110 can be different, and the infrared light has a corresponding field angle. When the infrared light is irradiated to the second coupling-in area 1, the infrared light can be moved to the annular area through the K2in1 action of the second coupling-in area 1 to be coupled into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 can propagate in the waveguide substrate 110. Correspondingly, the infrared light propagating in the waveguide substrate 110 can be moved back to the inner circle through the K2out1 action of the second coupling-out area 1 to be coupled out to the human eye. By analogy, when the infrared light is irradiated to the second coupling-in area p, the infrared light can be moved to the annular area through the K2inp action of the second coupling-in area p, and moved back to the inner circle through the K2outp action of the second coupling-out area p, so as to be coupled out to the human eye.

[0061] In some embodiments, the second incoupling region 140 and the second outcoupling region 150 may have a one-to-many correspondence.

[0062] See also Figure 4 , Figure 4 is a schematic structural diagram of an eye tracking device 100 involved in an embodiment of the present application.

[0063] like Figure 4 As shown, at least one second coupling-in region 140 and a plurality of second coupling-out regions 150 may be provided on the waveguide substrate 110 of the eye tracking device 100. When infrared light is irradiated to one of the second coupling-in regions 140, the second coupling-in region 140 may generate a plurality of diffraction orders. After the infrared light is coupled into the waveguide substrate 110 through the second coupling-in region 140, it may propagate through the waveguide substrate 110 along propagation paths corresponding to the plurality of diffraction orders to the corresponding plurality of second coupling-out regions 150, and then be coupled out to the human eye through the plurality of second coupling-out regions 150.

[0064] See also Figure 5 , Figure 5 yes Figure 4 A schematic diagram of the wave vector of the eye tracking device 100 is provided.

[0065] like Figure 5As shown, the relationship between the wave vector of the infrared light and the wave vector of the second coupling-in region 140 and the wave vector of the second coupling-out region 150 can be expressed as K2out=Kred+K2in. Kred is used to indicate the wave vector of the infrared light, K2in is used to indicate the wave vector of the second coupling-in region 140, and K2out is used to indicate the wave vector of the second coupling-out region 150. Taking the example that at least one second coupling-in region 140 includes a second coupling-in region 1, and a plurality of second coupling-out regions 150 include second coupling-out regions 1 to second coupling-out regions q, the second coupling-in region 1 and the second coupling-out regions 1 to second coupling-out regions q have a one-to-many correspondence. Among them, K2in may include K2in1, and K2in1 is used to indicate the wave vector of the second coupling-in region 1. K2out may include K2out1 to K2outq, and K2out1 is used to indicate the wave vector of the second coupling-out region 1, and K2outq is used to indicate the wave vector of the second coupling-out region q, and so on.

[0066] Depend on Figure 4 as well as Figure 5 It can be known that when the infrared light is irradiated to the second coupling-in zone 1, since the second coupling-in zone 1 can generate multiple diffraction orders, the infrared light can be moved to multiple sub-areas of the annular area through the K2in1 action of the second coupling-in zone 1 to be coupled into the waveguide substrate 110. Correspondingly, the infrared light of different diffraction orders coupled into the waveguide substrate 110 can propagate to the corresponding second decoupling zone 150 along different directions, such as from the second decoupling zone 1 to the second decoupling zone q. When the infrared light of one diffraction order propagates to the second decoupling zone 1, it can be moved back to the inner circle through the K2out1 action of the second decoupling zone 1 to be coupled out to the human eye. When the infrared light of another diffraction order propagates to the second decoupling zone q, it can be moved back to the inner circle through the K2outq action of the second decoupling zone q to be coupled out to the human eye. And so on.

[0067] In some embodiments, the eye tracking device 100 further includes at least one third coupling-in region 160 and at least one third coupling-out region 170 disposed on the waveguide substrate 110; relative to the first coupling-out region 130, the third coupling-in region 160 is disposed close to the first coupling-in region 120, and the location of the third coupling-in region 160 is different from the location of the second coupling-in region 140; relative to the first coupling-in region 120, the third coupling-out region 170 is disposed close to the first coupling-out region 130, and the location of the third coupling-out region 170 is different from the location of the second coupling-out region 150; wherein, at least one third coupling-in region 160 is used to receive infrared light and couple the infrared light into the waveguide substrate 110, propagate in the waveguide substrate 110 to the corresponding one or more third coupling-out regions 170, and couple out to the human eye through the third coupling-out region 170.

[0068] Exemplarily, the number of the third coupling-in regions 160 may include one or more, and the number of the third coupling-out regions 170 may also include one or more.

[0069] For example, when the number of the third coupling zone 160 is one, different positions of the third coupling zone 160 may have different incident angle ranges. The third coupling zone 160 may be, for example, in the form of at least one of a ring and a semi-ring, so as to be arranged around the first coupling zone 120. When the infrared light is irradiated onto the eye tracking device 100, and the incident angle of the infrared light is within the incident angle range corresponding to one of the positions of the third coupling zone 160, the infrared light may be incident on the corresponding position of the third coupling zone 160. The incident angle ranges corresponding to the different positions of the third coupling zone 160 are different from the incident angle range of the second coupling zone 140. Based on this, the second coupling zone 140 and the third coupling zone 160 included in the eye tracking device 100 are convenient for receiving infrared light with different incident angles. Accordingly, the corresponding position of the third coupling zone 160 that receives the infrared light can couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 may propagate at a preset angle until it propagates to one or more corresponding third outcoupling regions 170 and then is coupled out to the human eye through the third outcoupling regions 170 .

[0070] For example, the third coupling zone 160 may have a preset incident angle range. When the infrared light is irradiated onto the eye tracking device 100, and the incident angle of the infrared light is within the incident angle range of the third coupling zone 160, the infrared light may be incident onto the third coupling zone 160. The incident angle range of the third coupling zone 160 is different from the incident angle range of the second coupling zone 140. Based on this, the second coupling zone 140 and the third coupling zone 160 included in the eye tracking device 100 are convenient for receiving infrared light with different incident angles. Accordingly, the third coupling zone 160 that receives the infrared light may couple the infrared light into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 may propagate at a preset angle until it propagates to the corresponding one or more third decoupling zones 170 and then is coupled out to the human eye through the third decoupling zone 170.

[0071] Of course, the present invention is not limited thereto. The third coupling-in region 160 and the third coupling-out region 170 may have a one-to-one correspondence, a one-to-many relationship, or a many-to-one relationship, which is not limited here.

[0072] In some embodiments, the third coupling zone 160 is distributed around the first coupling zone 120, and relative to the first decoupling zone 130, the third coupling zone 160 is disposed close to the first coupling zone 120. The third decoupling zone 170 is distributed around the first decoupling zone 130, and relative to the first coupling zone 120, the third decoupling zone 170 is disposed close to the first decoupling zone 130. Since the incident angle range of the third coupling zone 160 is different from the incident angle range of the second coupling zone 140, the setting position of the third coupling zone 160 is different from the setting position of the second coupling zone 140. Accordingly, in order to facilitate the formation of light spots at different positions of the cornea of ​​the human eye, the emission angle of the third decoupling zone 170 to the infrared light and the emission angle of the second decoupling zone 150 to the infrared light can be different, and the setting position of the third decoupling zone 170 is different from the setting position of the second decoupling zone 150.

[0073] In some embodiments, the third coupling-in region 160 is disposed around the first coupling-in region 120, and / or the third decoupling region 170 is disposed around the first decoupling region 130. For example, the number of the third coupling-in region 160 may be one, and the third coupling-in region 160 may be in the shape of at least one of a ring and a semi-ring, and the third coupling-in region 160 may be disposed around the first coupling-in region 120. The number of the third coupling-in region 160 may also be multiple, so as to be disposed around the first coupling-in region 120. The number of the third decoupling region 170 may be one, and the third decoupling region 170 may be in the shape of at least one of a ring and a semi-ring, and the third decoupling region 170 may be disposed around the first decoupling region 130. The number of the third decoupling region 170 may also be multiple, so as to be disposed around the first decoupling region 130, which is not limited here.

[0074] In some embodiments, a size of the third in-coupling region 160 is smaller than a size of the first in-coupling region 120 , and / or a size of the third out-coupling region 170 is smaller than a size of the first out-coupling region 130 .

[0075] When the eye tracking device 100 can couple infrared light into the third coupling-in area 160 and couple infrared light out to the human eye through the third coupling-out area 170, and / or couple infrared light into the second coupling-in area 140 and couple infrared light out to the human eye through the second coupling-out area 150, different positions of the human eye can respectively reflect infrared light of different emission angles, and the human eye can form a corresponding light spot. When the light spot is located on the cornea of ​​the human eye, the light spot can be used to track the user's eyes. Based on this, the eye tracking device 100 can be used for the subsequent preparation of the near-eye display device 10 to realize the eye tracking function of the near-eye display device 10. Accordingly, when the user's eyes rotate to different positions, infrared light can be coupled into the user's eyes through at least one of the second coupling-in area 140 and the third coupling-in area 160, and infrared light can be coupled out to the user's eyes through the corresponding second coupling-out area 150 or the third coupling-out area 170, so that the coupled-out infrared light can irradiate the cornea of ​​the user's eyes, which is beneficial to improving the accuracy of eye movement tracking of the user's eyes.

[0076] Exemplarily, when at least one third coupling-in region 160 and at least one third coupling-out region 170 are disposed on the waveguide substrate 110 , the corresponding third coupling-in region 160 and the third coupling-out region 170 comply with the wave vector closure relationship.

[0077] In some embodiments, the third in-coupling region 160 and the third out-coupling region 170 may be in one-to-one correspondence.

[0078] like Figure 6 As shown, taking at least one third coupling-in region 160 including third coupling-in region 1 to third coupling-in region u, and multiple third coupling-out regions 170 including third coupling-out region 1 to third coupling-out region u as an example, the third coupling-in region 1 corresponds to the third coupling-out region 1 one-to-one, the third coupling-in region u corresponds to the third coupling-out region u one-to-one, and so on. K3in1 is used to indicate the wave vector of the third coupling-in region 1, K3out1 is used to indicate the wave vector of the third coupling-out region 1, K3inu is used to indicate the wave vector of the third coupling-in region u, K3outu is used to indicate the wave vector of the third coupling-out region u, and so on.

[0079] Since the incident angle ranges of the third coupling-in area 160 at different settings on the waveguide substrate 110 can be different, and the infrared light has a corresponding field angle. When the infrared light is irradiated to the third coupling-in area 1, the infrared light can be moved to the annular area through the K3in1 action of the third coupling-in area 1 to be coupled into the waveguide substrate 110. The infrared light coupled into the waveguide substrate 110 can propagate in the waveguide substrate 110. Correspondingly, the infrared light propagating in the waveguide substrate 110 can be moved back to the inner circle through the K3out1 action of the third coupling-out area 1 to be coupled out to the human eye. By analogy, when the infrared light is irradiated to the third coupling-in area u, the infrared light can be moved to the annular area through the K3inu action of the third coupling-in area u, and moved back to the inner circle through the K3outu action of the third coupling-out area u, so as to be coupled out to the human eye.

[0080] In some embodiments, the third incoupling region 160 and the third outcoupling region 170 may have a one-to-many correspondence.

[0081] like Figure 4 As shown, at least one third coupling-in region 160 and a plurality of third coupling-out regions 170 may be provided on the waveguide substrate 110 of the eye tracking device 100. When infrared light is irradiated to one of the third coupling-in regions 160, the third coupling-in region 160 may generate a plurality of diffraction orders. After the infrared light is coupled into the waveguide substrate 110 through the third coupling-in region 160, it may propagate through the waveguide substrate 110 along propagation paths corresponding to the plurality of diffraction orders to the corresponding plurality of third coupling-out regions 170, and then be coupled out to the human eye through the plurality of third coupling-out regions 170.

[0082] like Figure 5 As shown, the relationship between the wave vector of the infrared light and the wave vector of the third coupling-in region 160 and the wave vector of the third coupling-out region 170 can be expressed as K3out=Kred+K3in. Kred is used to indicate the wave vector of the infrared light, K3in is used to indicate the wave vector of the third coupling-in region 160, and K3out is used to indicate the wave vector of the third coupling-out region 170. Taking the example that at least one third coupling-in region 160 includes a third coupling-in region 1, and a plurality of third coupling-out regions 170 include third coupling-out regions 1 to third coupling-out regions v, the third coupling-in region 1 and the third coupling-out regions 1 to third coupling-out regions v have a one-to-many correspondence. Among them, K3in may include K3in1, and K3in1 is used to indicate the wave vector of the third coupling-in region 1. K3out may include K3out1 to K3outv, and K3out1 is used to indicate the wave vector of the third coupling-out region 1, and K2outv is used to indicate the wave vector of the third coupling-out region v, and so on.

[0083] Depend on Figure 4 as well as Figure 5It can be known that when the infrared light is irradiated to the third coupling-in zone 1, since the third coupling-in zone 1 can generate multiple diffraction orders, the infrared light can be moved to multiple sub-areas of the annular area through the K3in1 action of the third coupling-in zone 1 to be coupled into the waveguide substrate 110. Correspondingly, the infrared light of different diffraction orders coupled into the waveguide substrate 110 can propagate to the corresponding third decoupling zone 170 along different directions, such as the third decoupling zone 1 to the third decoupling zone v. When the infrared light of one diffraction order propagates to the third decoupling zone 1, it can be moved back to the inner circle through the K3out1 action of the third decoupling zone 1 to be coupled out to the human eye. When the infrared light of another diffraction order propagates to the third decoupling zone v, it can be moved back to the inner circle through the K2outv action of the third decoupling zone v to be coupled out to the human eye. And so on.

[0084] The eye tracking device 100 provided in the embodiment of the present application includes: a waveguide substrate 110; a first coupling-in region 120, a first coupling-out region 130, at least one second coupling-in region 140 and at least one second coupling-out region 150 arranged on the waveguide substrate 110; relative to the first coupling-out region 130, the second coupling-in region 140 is arranged close to the first coupling-in region 120; relative to the first coupling-in region 120, the second coupling-out region 150 is arranged close to the first coupling-out region 130; wherein the first coupling-in region 120 is used to couple the incident light emitted by the projection module 210 to the human eye; the second coupling-in region 140 is convenient for receiving infrared light with different incident angles and coupling the infrared light into the waveguide substrate 110, propagating in the waveguide substrate 110 to the corresponding one or more second coupling-out regions 150, and coupling out to the human eye through the second coupling-out region 150.

[0085] Based on the arrangement of at least one second coupling-in area 140 and at least one second coupling-out area 150, when the human eye is at different positions, such as when the cornea of ​​the human eye has not rotated to the inner side of the eye socket or when the cornea of ​​the human eye has rotated to the inner side of the eye socket, the eye tracking device 100 can couple infrared light of corresponding incident angles through different second coupling-in areas 140, or couple infrared light of corresponding incident angles through corresponding positions of the same second coupling-in area 140, so that the infrared light coupled into the waveguide substrate 110 can propagate through the waveguide substrate 110 to the corresponding one or more second coupling-out areas 150 and then couple out to the human eye. After the action of the eye tracking device 100, the infrared light can be coupled out to the cornea of ​​the human eye to form a corresponding light spot on the cornea of ​​the human eye. The light spot located on the cornea of ​​the human eye can be used for eye tracking of the human eye, which is conducive to improving the convenience and accuracy of eye tracking of the human eye.

[0086] See also Figure 7 , Figure 7 It is a structural schematic diagram of a near-eye display device 10 provided in an embodiment of the present application.

[0087] like Figure 7 As shown, the near-eye display device 10 includes an eye tracking device 100 , an optical-mechanical device 200 , and an imaging device 300 .

[0088] The eye tracking device 100 is the aforementioned eye tracking device 100. The specific principle and implementation of the eye tracking device 100 may refer to the eye tracking device 100 of the aforementioned embodiment, and will not be described in detail here.

[0089] The optical machine device 200 includes a projection module 210 and an infrared module 220. The projection module 210 is used to emit incident light to the first coupling-in area 120 of the eye tracking device 100. The first coupling-in area 120 is used to couple the incident light into the waveguide substrate 110 of the eye tracking device 100. The first coupling-out area 130 of the eye tracking device 100 is used to couple the incident light propagating in the waveguide substrate 110 to the human eye. The infrared module 220 includes an infrared light source 221 and an angle adjustment member 222. The angle adjustment member 222 is used to adjust the light angle of the infrared light provided by the infrared light source 221 so that the adjusted infrared light irradiates one of the second coupling-in areas 140 of the eye tracking device 100, is coupled into the waveguide substrate 110 through the second coupling-in area 140, and then propagates to the corresponding one or more second coupling-out areas 150, and is coupled out to the human eye through the second coupling-out area 150.

[0090] The imaging device 300 is used to capture the user's eye image to achieve eye tracking.

[0091] For example, corresponding to the user's human eyes including the left eye and the right eye, the near-eye display device 10 may include a left eye tracking device and a right eye tracking device. The left eye tracking device can be used to track the user's left eye, and the right eye tracking device can be used to track the user's right eye. The structure of the left eye tracking device may be consistent with or different from the structure of the right eye tracking device. For example, the left eye tracking device may be provided with a first coupling-in area 120, a first coupling-out area 130, at least one second coupling-in area 140, and at least one second coupling-out area 150, and the second coupling-in area 140 and the second coupling-out area 150 have a one-to-one correspondence. Correspondingly, the right eye tracking device may also be provided with a first coupling-in area 120, a first coupling-out area 130, at least one second coupling-in area 140, and at least one second coupling-out area 150, and the second coupling-in area 140 and the second coupling-out area 150 also have a one-to-one correspondence. Alternatively, the second coupling-in area 140 and the second coupling-out area 150 provided on the right eye tracking device 100 have a one-to-many correspondence. For another example, the left eye tracking device and the right eye tracking device can share the first coupling-in area 120 and at least one second coupling-in area 140, which is not limited here.

[0092] In some embodiments, the near-eye display device 10 can provide incident light and infrared light to the eye tracking device 100 through the optical device 200. For example, the optical device 200 may include a projection module 210 and an infrared module 220. The projection module 210 and the infrared module 220 may be integrated or separately provided, which is not limited here. The angle adjustment member 222 included in the infrared module 220 may be provided inside the optical device 200 or outside the optical device 200, which is not limited here.

[0093] For example, the projection module 210 can emit incident light to the first coupling region 120 of the eye tracking device 100, so as to couple the incident light into the waveguide substrate 110 through the first coupling region 120, propagate through the waveguide substrate 110 to the first coupling region 130, and then couple out to the human eye. Based on this, the near-eye display device 10 can have a display function.

[0094] For example, the infrared module 220 can provide infrared light to the eye tracking device 100. The infrared module 220 can include an infrared light source 221 and an angle adjustment member 222. The infrared light source 221 can provide infrared light. The infrared light provided by the infrared light source 221 can be irradiated to one of the second coupling-in regions 140 after being acted upon by the angle adjustment member 222. Accordingly, the second coupling-in region 140 that receives the infrared light can couple the infrared light into the waveguide substrate 110, propagate to the corresponding one or more second coupling-out regions 150 through the waveguide substrate 110, and couple out to the human eye through the corresponding second coupling-out region 150, such as coupling out to the cornea of ​​the human eye. The cornea of ​​the human eye can reflect the coupled-out infrared light, and a corresponding light spot can be formed on the cornea of ​​the human eye in the human eye image obtained by the imaging device 300. The light spot on the cornea of ​​the human eye can be used for eye tracking of the human eye.

[0095] In some embodiments, when the human eye rotates to different positions, the angle of the infrared light can be adjusted by the angle adjustment member 222, so that the adjusted infrared light can irradiate different positions of the same second coupling-in area 140 or different second coupling-in areas 140, thereby being coupled into the waveguide substrate 110 through different positions of the same second coupling-in area 140 or different second coupling-in areas 140, propagating through the waveguide substrate 110 to different positions of the same second coupling-in area 140 or one or more second coupling-out areas 150 corresponding to different second coupling-in areas 140, and then coupled out to different positions of the human eye, such as different positions of the cornea of ​​the human eye. Accordingly, in the case where the coupled infrared light irradiates the cornea of ​​the human eye, it is helpful to improve the accuracy of eye movement tracking of the human eye.

[0096] Exemplarily, the angle adjustment member 222 is also used to adjust the light angle of the infrared light provided by the infrared light source 221, so that the adjusted infrared light is irradiated to one of the third coupling-in regions 160, coupled into the waveguide substrate 110 through the third coupling-in region 160, and then propagates to the corresponding one or more third coupling-out regions 170, and is coupled out to the human eye through the third coupling-out region 170.

[0097] For example, when the third coupling-in area 160 and the third coupling-out area 170 are provided on the waveguide substrate 110 of the eye tracking device 100, the infrared light provided by the infrared light source 221 can be irradiated to one of the third coupling-in areas 160 through the action of the angle adjustment member 222. Accordingly, the third coupling-in area 160 that receives the infrared light can couple the infrared light into the waveguide substrate 110, propagate to one or more corresponding third coupling-out areas 170 through the waveguide substrate 110, and couple out to the human eye through the corresponding third coupling-out area 170, such as coupling out to the cornea of ​​the human eye. The cornea of ​​the human eye can reflect the coupled-out infrared light, and a corresponding light spot can be formed on the cornea of ​​the human eye in the human eye image obtained by the imaging device 300. The light spot on the cornea of ​​the human eye can be used for eye tracking of the human eye.

[0098] In some embodiments, when the human eye rotates to different positions, the angle of the infrared light can be adjusted by the angle adjustment member 222, so that the adjusted infrared light can irradiate different positions of the same third coupling-in area 160 or different third coupling-in areas 160, thereby being coupled into the waveguide substrate 110 through different positions of the same third coupling-in area 160 or different third coupling-in areas 160, propagating through the waveguide substrate 110 to different positions of the same third coupling-in area 160 or one or more third coupling-out areas 170 corresponding to different third coupling-in areas 160, and then coupled out to different positions of the human eye, such as being coupled out to different positions of the cornea of ​​the human eye. In other embodiments, when the user's eyes rotate to different positions, the angle of the infrared light can be adjusted by the angle adjustment member 222, so that the adjusted infrared light can irradiate different second coupling-in areas 140 and third coupling-in areas 160, thereby coupling into the waveguide substrate 110 through the second coupling-in area 140, propagating through the waveguide substrate 110 to the corresponding one or more second coupling-out areas 150 and then coupling out to the human eye, and coupling into the waveguide substrate 110 through the third coupling-in area 160, propagating through the waveguide substrate 110 to the corresponding one or more third coupling-out areas 170 and then coupling out to the human eye. Accordingly, when there is a coupled infrared light irradiating the cornea of ​​the human eye, it is helpful to improve the accuracy of eye tracking of the human eye.

[0099] The imaging device 300 can capture an image of the user's eyes, and the near-eye display device 10 can subsequently perform eye tracking on the user's eyes based on the image of the eyes.

[0100] In some embodiments, the angle adjustment member 222 includes at least one of a reflector 2221 and an adjustable liquid crystal grating 2222 .

[0101] like Figure 7 As shown, the angle adjustment member 222 may include a reflector 2221. The reflector 2221 may be disposed inside the optical-mechanical device 200. The setting position of the reflector 2221 in the optical-mechanical device 200 may be changed. The reflector 2221 at different setting positions may irradiate the infrared light provided by the infrared light source 221 to at least one of different positions of the same second coupling-in area 140, different second coupling-in areas 140, different positions of the same third coupling-in area 160, different third coupling-in areas 160, at least one second coupling-in area 140 and at least one third coupling-in area 160.

[0102] like Figure 8a as well as Figure 8c As shown, in the case where the reflector 2221 is set at the first position, the infrared light provided by the infrared light source 221 can irradiate one of the second coupling-in areas 140 after being acted upon by the reflector 2221 at the first position, and after being coupled into the waveguide substrate 110 through the second coupling-in area 140, it propagates through the waveguide substrate 110 to the corresponding one or more second coupling-out areas 150, and is coupled out to the cornea of ​​the human eye through the second coupling-out area 150.

[0103] like Figure 8b as well as Figure 8c As shown, in the case where the reflector 2221 is set at the second position, the infrared light provided by the infrared light source 221 can irradiate another second coupling-in area 140 after being acted upon by the reflector 2221 at the second position, and after being coupled into the waveguide substrate 110 through the second coupling-in area 140, it propagates through the waveguide substrate 110 to the corresponding one or more second coupling-out areas 150, and is coupled out to the cornea of ​​the human eye through the second coupling-out area 150.

[0104] Of course, the setting position of the reflector 2221 is not limited to the first position and the second position, and is not limited here.

[0105] like Fig. 9As shown, the angle adjustment member 222 may include an adjustable liquid crystal grating 2222. The adjustable liquid crystal grating 2222 may be disposed outside the optical mechanical device 200. The arrangement direction of the liquid crystal molecules in the adjustable liquid crystal grating 2222 is adjustable, and the adjustable liquid crystal grating 2222 may have different working states. For example, a voltage may be applied to the adjustable liquid crystal grating 2222 to adjust the arrangement direction of the liquid crystal molecules by the voltage, thereby changing the working state of the adjustable liquid crystal grating 2222. The reflector 2221 in different working states may irradiate the infrared light provided by the infrared light source 221 to at least one of different positions of the same second coupling region 140, different second coupling regions 140, different positions of the same third coupling region 160, different third coupling regions 160, at least one second coupling region 140 and at least one third coupling region 160.

[0106] like Fig.10a as well as Fig.10c As shown, when the adjustable liquid crystal grating 2222 is in the first working state, the infrared light provided by the infrared light source 221 can irradiate one of the third coupling-in areas 160 after the action of the first working state, and after being coupled into the waveguide substrate 110 through the third coupling-in area 160, it propagates through the waveguide substrate 110 to the corresponding one or more third coupling-out areas 170, and is coupled out to the cornea of ​​the human eye through the third coupling-out area 170.

[0107] like Fig.10a as well as Fig.10c As shown, when the adjustable liquid crystal grating 2222 is in the second working state, the infrared light provided by the infrared light source 221 can irradiate one of the second coupling-in areas 140 after the action of the second working state, and after being coupled into the waveguide substrate 110 through the second coupling-in area 140, it propagates through the waveguide substrate 110 to the corresponding one or more second coupling-out areas 150, and is coupled out to the cornea of ​​the human eye through the second coupling-out area 150.

[0108] Of course, the working state of the adjustable liquid crystal grating 2222 is not limited to the first working state and the second working state, and is not limited here.

[0109] For example, the angle adjustment member 222 may include a reflector 2221 and an adjustable liquid crystal grating 2222, and the multiple infrared rays provided by the infrared light source 221 may be irradiated to different positions of the same second coupling zone 140, different second coupling zones 140, different positions of the same third coupling zone 160, different third coupling zones 160, at least one of at least one second coupling zone 140 and at least one third coupling zone 160 after the combined action of the reflector 2221 at the corresponding setting position and the adjustable liquid crystal grating 2222 in the corresponding working state, thereby coupling out the adjusted infrared rays to the human eye, without limitation herein.

[0110] In some embodiments, the adjustable liquid crystal grating 2222 is respectively disposed adjacent to the projection module 210 and the infrared light source 221. The optical mechanical device 200 further includes a beam combining component 230, which is used to combine the incident light emitted by the projection module 210 and the infrared light provided by the infrared light source 221 and then incident on the adjustable liquid crystal grating 2222.

[0111] like Figures 10a to 10c As shown, the projection module 210 and the infrared light source 221 can be separately arranged, and the light direction of the incident light emitted by the projection module 210 is different from the light direction of the infrared light provided by the infrared light source 221. The projection module 210 and the infrared light source 221 are located on one side of the adjustable liquid crystal grating 2222, and the first coupling region 120 is located on the other side of the adjustable liquid crystal grating 2222. Then, the near-eye display device 10 needs to ensure that the incident light can be irradiated on the first coupling region 120, and that the infrared light can be incident on the adjustable liquid crystal grating 2222. Based on this, the near-eye display device 10 can control the light directions of the incident light and the infrared light through the beam combining component 230 included in the optical machine device 200. The beam combining component 230 can be located on the light-emitting side of the infrared light source 221 and the light-emitting side of the projection module 210. The beam combining component 230 can combine the incident light and the infrared light and then incident on the adjustable liquid crystal grating 2222. For example, the infrared light provided by the infrared light source 221 can be reflected to the adjustable liquid crystal grating 2222 by the beam combining component 230, so that the adjustable liquid crystal grating 2222 can adjust the light direction of the infrared light. Correspondingly, the incident light provided by the projection module 210 can be transmitted to the adjustable liquid crystal grating 2222 by the beam combining component 230, so that the adjustable liquid crystal grating 2222 transmits the incident light, so that the incident light is irradiated to the first coupling region 120.

[0112] In an exemplary embodiment, the beam combining assembly 230 may include a semi-reflective and semi-mirror lens. The semi-reflective and semi-mirror lens may be obtained by designing the reflector 2221 accordingly, and is not limited here.

[0113] The near-eye display device 10 provided in the embodiment of the present application comprises: an eye tracking device 100, the eye tracking device 100 being the aforementioned eye tracking device 100; an optical mechanical device 200, the optical mechanical device 200 comprising a projection module 210 and an infrared module 220, the projection module 210 being used to emit incident light to a first coupling-in region 120 of the eye tracking device 100, the first coupling-in region 120 being used to couple the incident light into a waveguide substrate 110 of the eye tracking device 100, and the first coupling-out region 130 of the eye tracking device 100 being used to couple the incident light into a waveguide substrate 110 propagating in the waveguide substrate 110. The incident light is coupled out to the human eye; the infrared module 220 includes an infrared light source 221 and an angle adjustment member 222, the angle adjustment member 222 is used to adjust the light angle of the infrared light provided by the infrared light source 221, so that the adjusted infrared light is irradiated to one of the second coupling-in areas 140 of the eye tracking device 100, coupled into the waveguide substrate 110 through the second coupling-in area 140, and then propagated to the corresponding one or more second coupling-out areas 150, and coupled out to the human eye through the second coupling-out area 150; an imaging device 300, the imaging device 300 is used to capture the user's human eye image to achieve eye tracking.

[0114] Based on the settings of the eye tracking device 100 and the infrared module 220, the angle of the infrared light provided by the infrared light source 221 can be adjusted by the angle adjustment member 222, so that the adjusted infrared light can be irradiated to one of the second coupling-in areas 140 of the eye tracking device 100, and then coupled out to the human eye through the second coupling-in area 140, the waveguide substrate 110, and one or more second coupling-out areas 150 corresponding to the second coupling-in area 140 in sequence. Accordingly, when the human eye is at different positions, the infrared light is irradiated to different second coupling-in areas 140 through the angle adjustment member 222, and then can be coupled out to different positions of the human eye, such as different positions of the cornea of ​​the human eye, through the different second coupling-in areas 140, the waveguide substrate 110, and one or more second coupling-out areas 150 corresponding to different second coupling-in areas 140, thereby forming light spots at different positions on the cornea of ​​the human eye. The light spots at different positions on the cornea of ​​the human eye can be used for eye tracking of the human eye, which is conducive to improving the convenience and accuracy of eye tracking of the human eye.

[0115] See also Fig. 9 , Fig. 9 1 is a flow chart of an eye tracking method for a near-eye display device 10 provided in an embodiment of the present application. The eye tracking method is used for the aforementioned near-eye display device 10. The specific principles and implementation methods of the near-eye display device 10 involved in the embodiment of the present application can refer to the eye tracking device 100 and the near-eye display device 10 of the aforementioned embodiment, which will not be repeated here.

[0116] like Fig. 9As shown, the eye tracking method of the near-eye display device 10 includes steps S101 to S104.

[0117] S101 . Control the infrared light source 221 of the near-eye display device 10 to provide infrared light.

[0118] For example, the near-eye display device 10 may provide infrared light through the infrared light source 221 of the infrared module 220. The infrared light may be irradiated onto the angle adjustment member 222 of the near-eye display device 10.

[0119] S102 . Adjust the angle of the infrared light provided by the infrared light source 221 through the angle adjustment member 222 of the near-eye display device 10 to provide the adjusted infrared light to the user's eyes.

[0120] In some embodiments, the angle adjustment member 222 can adjust the angle of the infrared light provided by the infrared light source 221 in response to the rotation of the user's eyeball, which is not limited here.

[0121] For example, at least one second coupling-in region 140 and at least one second coupling-out region 150 may be provided on the waveguide substrate 110 of the eye tracking device 100. The near-eye display device 10 may adjust the light angle of the infrared light provided by the infrared light source 221 to different light angles through the angle adjustment member 222. The adjusted infrared light may irradiate one of the second coupling-in regions 140, be coupled into the waveguide substrate 110 through the second coupling-in region 140, propagate through the waveguide substrate 110 to the corresponding one or more second coupling-out regions 150, and then be coupled out to the human eye through the second coupling-out region 150, thereby providing the adjusted infrared light to the human eye. Accordingly, in the presence of adjusted infrared light at different light angles, the infrared light at different light angles can be irradiated to different positions of the same second coupling-in zone 140 or different second coupling-in zones 140, thereby being coupled out to the human eye through the action of one or more second coupling-out zones 150 corresponding to different positions of the same second coupling-in zone 140 or different second coupling-in zones 140, the waveguide substrate 110, and different positions of the same second coupling-in zone 140 or different second coupling-in zones 140, thereby providing adjusted infrared light to different positions of the human eye, such as providing adjusted infrared light to different positions of the cornea of ​​the human eye.

[0122] For example, at least one second coupling-in region 140, at least one second coupling-out region 150, at least one third coupling-in region 160, and at least one third coupling-out region 170 may be provided on the waveguide substrate 110 of the eye tracking device 100. The near-eye display device 10 may adjust the light angle of the infrared light provided by the infrared light source 221 to different light angles through the angle adjustment member 222. The adjusted infrared light may irradiate one of the second coupling-in regions 140, be coupled into the waveguide substrate 110 through the second coupling-in region 140, propagate through the waveguide substrate 110 to the corresponding one or more second coupling-out regions 150, and then be coupled out to the human eye through the second coupling-out region 150, thereby providing the adjusted infrared light to the user's human eye. Alternatively, the adjusted infrared light can be irradiated to one of the third coupling-in regions 160, coupled to the waveguide substrate 110 through the third coupling-in region 160, propagated to the corresponding one or more third coupling-out regions 170 through the waveguide substrate 110, and coupled out to the human eye through the third coupling-out region 170, thereby providing the adjusted infrared light to the user's human eye. In the case of the presence of adjusted infrared light at different light angles, the infrared light at different light angles can be irradiated to at least one of different positions of the same second coupling-in region 140, different second coupling-in regions 140, different positions of the same third coupling-in region 160, different third coupling-in regions 160, at least one second coupling-in region 140 and at least one third coupling-in region 160, thereby being coupled out to the human eye through the action of the eye tracking device 100, thereby providing the adjusted infrared light to different positions of the user's human eye.

[0123] S103 : Acquire a human eye image of the user captured by the imaging device 300 of the near-eye display apparatus 10 .

[0124] For example, the imaging device 300 can capture the human eye to obtain the user's human eye image. The near-eye display device 10 can obtain the human eye image captured by the imaging device 300, so as to subsequently track the user's eye movement based on the user's human eye image.

[0125] S104: Determine the user's eye movement information based on the human eye image.

[0126] For example, the user's eye movement information can be determined based on the light spot in the human eye image. For example, the position offset of the light spot relative to the user's pupil can be determined based on the human eye image; and the user's eye movement information can be determined based on the position offset.

[0127] In some embodiments, image recognition can be performed on the human eye image to determine the position of the user's pupil in the human eye image and the position of the light spot in the human eye image. According to the position of the pupil in the human eye image and the position of the light spot in the human eye image, the position offset of the light spot relative to the user's pupil is determined. For example, the position offset in the X-axis direction can be obtained by subtracting the X-axis coordinate of the light spot in the coordinate system corresponding to the human eye image from the X-axis coordinate of the pupil in the coordinate system corresponding to the human eye image. The position offset in the Y-axis direction can be obtained by subtracting the Y-axis coordinate of the light spot in the coordinate system corresponding to the target image from the Y-axis coordinate of the pupil in the coordinate system corresponding to the human eye image. This is not limited here.

[0128] In the case where the human eye image includes multiple light spots, the position offset of each light spot relative to the user's pupil is determined respectively, and the user's eye movement information is determined according to the average value of the position offset of each light spot relative to the user's pupil. Of course, it is not limited to this. In the case where the human eye image includes multiple light spots, the average value of the positions of all light spots in the human eye image can be determined. According to the average value of the positions of all light spots in the human eye image and the position of the pupil in the human eye image, the position offset of the light spot relative to the user's pupil is determined, thereby determining the user's eye movement information, which is not limited here.

[0129] Exemplarily, the adjustment parameters of the angle adjustment member 222 are adjusted according to the first preset frequency; the angle of the infrared light is adjusted by the angle adjustment member 222 under different adjustment parameters.

[0130] In some implementations, the imaging device 300 is controlled to capture the user's eye image at a second preset frequency; the second preset frequency is lower than the first preset frequency.

[0131] For example, the angle adjustment member 222 includes a reflector 2221, and the adjustment parameter of the angle adjustment member 222 may include the setting position of the reflector 2221. Figure 8a As shown, at time t1, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to one of the second coupling-in areas 140 by setting the reflector 2221 at the first position, and couple it out to the human eye after the eye tracking device 100, thereby forming a light spot 1 at the corresponding position of the human eye. Figure 8b As shown, at time t2, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to another second coupling-in area 140 by setting the reflector 2221 at the second position, and couple it out to the human eye after the action of the eye tracking device 100, thereby forming a light spot 2 at the corresponding position of the human eye. Figure 8cAs shown, the near-eye display device 10 can adjust the setting position of the reflector 2221 according to the first preset frequency, and adjust the light angle of the infrared light by the reflector 2221 at different setting positions. When the imaging device 300 captures the user's human eye image according to the second preset frequency, the human eye image can include spot 1 and spot 2 for eye tracking of the user. The first preset frequency is greater than the second preset frequency.

[0132] For example, the angle adjustment member 222 includes an adjustable liquid crystal grating 2222, and the adjustment parameter of the angle adjustment member 222 may include the working state of the adjustable liquid crystal grating 2222. Fig.10a As shown, at time t3, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to one of the third coupling-in areas 160 through the adjustable liquid crystal grating 2222 in the first working state, and couple it out to the human eye after the action of the eye tracking device 100, thereby forming a light spot 3 at the corresponding position of the human eye. Fig.10b As shown, at time t4, the near-eye display device 10 can irradiate the infrared light provided by the infrared light source 221 to one of the second coupling-in areas 140 through the adjustable liquid crystal grating 2222 in the second working state, and couple it out to the human eye after the eye tracking device 100, thereby forming a light spot 4 at the corresponding position of the human eye. Fig.10c As shown, the near-eye display device 10 can adjust the working state of the adjustable liquid crystal grating 2222 according to the first preset frequency, and adjust the light angle of the infrared light through the adjustable liquid crystal grating 2222 in different working states. When the imaging device 300 captures the human eye image of the user according to the second preset frequency, the human eye image can include light spot 3 and light spot 4 for eye tracking of the user. The first preset frequency is greater than the second preset frequency.

[0133] The eye tracking method of the near-eye display device 10 provided in the embodiment of the present application includes: controlling the infrared light source 221 of the near-eye display device 10 to provide infrared light; adjusting the light angle of the infrared light provided by the infrared light source 221 through the angle adjustment member 222 of the near-eye display device 10 to provide the adjusted infrared light to the user's eye; obtaining the user's eye image taken by the imaging device 300 of the near-eye display device 10; and determining the user's eye movement information based on the eye image to improve the convenience and accuracy of eye tracking of the human eye.

[0134] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0135] Exemplarily, the above method and apparatus may be implemented in the form of a computer program, which may be run on the near-eye display device 10 to perform eye tracking on the user through the near-eye display device 10. Exemplarily, the near-eye display device 10 may include VR glasses, AR glasses, MR glasses, VR helmets, AR helmets, MR helmets, etc., which are not limited here.

[0136] Exemplarily, an embodiment of the present application further provides a near-eye display device 10. The near-eye display device 10 includes a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0137] The storage medium may store an operating system and a computer program. When the computer program is executed, the processor may execute any eye tracking method of the near-eye display device 10.

[0138] The processor is used to provide computing and control capabilities to support the operation of the entire near-eye display device 10.

[0139] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any eye tracking method of the near-eye display device 10.

[0140] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0141] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0142] Controlling the infrared light source 221 of the near-eye display device 10 to provide infrared light;

[0143] The angle of the infrared light provided by the infrared light source 221 is adjusted by the angle adjustment member 222 of the near-eye display device 10 to provide the adjusted infrared light to the user's eyes;

[0144] Acquire a human eye image of a user captured by the imaging device 300 of the near-eye display device 10;

[0145] The eye movement information of the user is determined according to the human eye image.

[0146] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of eye tracking of the near-eye display device 10 described above can refer to the corresponding process in the aforementioned embodiment of the eye tracking method of the near-eye display device 10, and will not be repeated here.

[0147] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the eye tracking method of the near-eye display device 10 of the present application.

[0148] The computer-readable storage medium may be an internal storage unit of the near-eye display device 10 described in the foregoing embodiment, such as a hard disk or memory of the near-eye display device 10. The computer-readable storage medium may also be an external storage device of the near-eye display device 10, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the near-eye display device 10.

[0149] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0150] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0151] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An eye tracking device, characterized in that: include: Waveguide substrate: A first coupling-in region, a first coupling-out region, at least one second coupling-in region and at least one second coupling-out region are arranged on the waveguide substrate; relative to the first coupling-out region, the second coupling-in region is arranged close to the first coupling-in region; relative to the first coupling-in region, the second coupling-out region is arranged close to the first coupling-out region; Among them, the first coupling-in area is used to couple the incident light emitted by the projection module into the waveguide substrate, and the first coupling-out area is used to couple the incident light propagating in the waveguide substrate to the human eye; the second coupling-in area is convenient for receiving infrared light with different incident angles and coupling the infrared light into the waveguide substrate, propagating in the waveguide substrate to the corresponding one or more second coupling-out areas, and coupling out to the human eye through the second coupling-out area.

2. The eye tracking device according to claim 1, characterized in that: The second coupling-in region is disposed around the first coupling-in region, and / or the second coupling-out region is disposed around the first coupling-out region.

3. The eye tracking device according to claim 1, characterized in that: The size of the second incoupling region is smaller than the size of the first incoupling region, and / or the size of the second outcoupling region is smaller than the size of the second outcoupling region.

4. The eye tracking device according to any one of claims 1 to 3, characterized in that: The eye tracking device further comprises at least one third coupling-in region and at least one third coupling-out region arranged on the waveguide substrate; relative to the first coupling-out region, the third coupling-in region is arranged close to the first coupling-in region, and the arrangement position of the third coupling-in region is different from the arrangement position of the second coupling-in region; relative to the first coupling-in region, the third coupling-out region is arranged close to the first coupling-out region, and the arrangement position of the third coupling-out region is different from the arrangement position of the second coupling-out region; Among them, at least one third coupling-in region is used to receive infrared light and couple the infrared light into the waveguide substrate, propagate in the waveguide substrate to the corresponding one or more third coupling-out regions, and couple out to the human eye through the third coupling-out regions.

5. A near-eye display device, characterized in that: include: An eye tracking device, wherein the eye tracking device is the eye tracking device according to any one of claims 1 to 4; An optical-mechanical device, the optical-mechanical device comprising a projection module and an infrared module, the projection module is used to emit incident light to a first coupling-in area of ​​the eye-tracking device, the first coupling-in area is used to couple the incident light into a waveguide substrate of the eye-tracking device, and the first coupling-out area of ​​the eye-tracking device is used to couple the incident light propagating in the waveguide substrate out to a human eye; the infrared module comprises an infrared light source and an angle adjustment member, the angle adjustment member is used to adjust the light angle of the infrared light provided by the infrared light source, so that the adjusted infrared light irradiates one of the second coupling-in areas of the eye-tracking device, is coupled into the waveguide substrate through the second coupling-in area, and then propagates to one or more corresponding second coupling-out areas, and is coupled out to a human eye through the second coupling-out area; An imaging device is used to capture an image of a user's eyes to achieve eye tracking.

6. The near-eye display device according to claim 5, characterized in that: The angle adjustment member is also used to adjust the light angle of the infrared light provided by the infrared light source, so that the adjusted infrared light irradiates one of the third coupling-in areas, is coupled into the waveguide substrate through the third coupling-in area, and then propagates to the corresponding one or more third coupling-out areas, and is coupled out to the human eye through the third coupling-out area.

7. The near-eye display device according to claim 5 or 6, characterized in that: The angle adjustment member includes at least one of a reflector and an adjustable liquid crystal grating.

8. The near-eye display device according to claim 7, characterized in that: The adjustable liquid crystal grating is respectively arranged adjacent to the projection module and the infrared light source; The optical-mechanical device further comprises a beam combining component, which is used to combine the incident light emitted by the projection module and the infrared light provided by the infrared light source and then make the combined beams incident on the adjustable liquid crystal grating.

9. An eye tracking method for a near-eye display device, characterized in that: For a near-eye display device as claimed in any one of claims 5 to 8, the eye tracking method comprises: Controlling an infrared light source of a near-eye display device to provide infrared light; The angle of the infrared light provided by the infrared light source is adjusted by the angle adjustment member of the near-eye display device to provide the adjusted infrared light to the user's eyes; Acquire a human eye image of a user captured by an imaging device of the near-eye display device; The eye movement information of the user is determined according to the human eye image.

10. The eye tracking method according to claim 9, characterized in that: The step of adjusting the angle of the infrared light provided by the infrared light source through the angle adjustment member of the near-eye display device comprises: According to a first preset frequency, adjusting an adjustment parameter of the angle adjustment member; Adjusting the angle of the infrared light by angle adjustment members under different adjustment parameters respectively; The acquiring the human eye image of the user captured by the imaging device of the near-eye display device comprises: The imaging device is controlled to capture the user's eye image at a second preset frequency; the second preset frequency is less than the first preset frequency.

Images